Hidden in plain sight: How the needs of the poor are being ignored

Britain has become a nation of ghettoes of the long-term unemployed – mocked and punished by three decades of incompetent and heartless government. Mrs Thatcher's blitz on the country's industrial heartlands; a strategy to wring the necks of noisy trade unions, created these barrios of suffering. Tony Blair's landslide victory in 1997 had the legions on the dole dancing in the streets, but their dreams were betrayed. Instead, New Labour institutionalised and buried the "dustbin" people who had sent nearly half the cabinet to London to give their children a future. David Cameron identified and condemned the Broken Society and promised to fix it. "Jam and Jerusalem on hallucinogens" was one description of his campaigning fever in the slums of Glasgow. But once Gordon Brown had sulked out of Downing Street nothing changed. Even now, as another polling day looms, the suits of Westminster have left whole communities of the damned to rot in the spring sunshine. Welcome to apartheid Britain and the death of pity in the Mother Hubbard of All Parliaments.

From the Rhondda to Hartlepool, Blackpool to Inverclyde, Wolverhampton to Hull, Glasgow and Caerphilly – the scrap heaps of political mediocrity – five million men and women remain on the dole. Of these, 2.6m – massaged from the official figures – are permanently on incapacity benefits – "on the sick". Half of those would be working in a stronger economy. New research by Professor Steve Fothergill of Sheffield Hallam University has revealed a state of hopeless atrophy across the country's 100 weakest economies. These Blightlands of third-world, third-generation unemployment embrace a third of the UK population. Even the tabloids' cherished bogeymen, "immigrants coming over here and stealing our jobs" have shunned the derelict mining villages and dockyard bombsites of the politicians' industrial clearance sales.

"Worklessness on this scale," Fothergill says, "is a colossal waste of talent and a waste of productive potential." The cost of these wasted lives – £16bn a year – is staggering. It dwarfs the £10bn real estate tag for the Olympic site in London.

Secure inside the Westminster village, a community gated now behind ramparts of bombers' blast walls – few politicians seem willing to confront the pandemic disease of working class poverty. Another academic, Danny Dorling, professor of human geography at Sheffield University, has an intriguing theory about this. Britain has become, in all but name, a one-party state. Labour and Tory have become politically and socially indistinguishable. Red Tory, Blue Labour (the latest wheeze of party stylists) add a touch of farce to the pretence of difference. Dorling's 2010 book, Injustice: Why Social Inequality Exists, shows the emergence of a new homogenous political tribe which has lost the nerve to debate. Why bother when all parties now court the swing votes of a single "squeezed middle class". To ingratiate himself with this group Labour leader, Ed Miliband, has announced his aversion to "irresponsible strikes" – meaning all strikes – ignoring the cries of "rubbish" from union leaders organising protest marches against public service cuts. Miliband, like David Cameron, has also sought the blessing of Rupert Murdoch. He has visited Wapping to fawn at the feet of News International boss Rebekah Brooks and told his backbench warriors to shut up about the mogul's £8bn bid for BSkyB. Poverty takes a back seat in the gruesome ritual of courtship to win the old man's approving nod.

The problems of sink estates, the inner-city tower blocks of Prozac-drugged children have been euphemised with the help of a new kind of visionary whose importance now exceeds even that of the politician's spin doctor thugs. Part-social philosopher, part-medicine man, their task has been to re-define poverty in a way that flatters the charitable sensitivity of politicians while subtly blaming the unemployed for not understanding how their communities could be improved. Cameron's crowd thought of it first. They discovered Phillip Blond, a Bunteresque theology academic and political conference fringe groupie from the University of Cumbria. Blond is a moral crusader who worries a lot about "the moral status of the foetus".

More importantly, for Cameron's people, he believes the welfare state, not capitalism, damaged the cohesive, self-help nature of working-class communities. The implication lurking in the jargon thicket seems to be that it's the workers' own fault if they haven't got jobs. They should have realised that redemption is "all about working class mutuality creating actual capital, building social capital". It all sounds a bit like a Disney medieval theme park, but even Tories who don't get it have queued to buy Blond's book on "Red Toryism". Not to be outdone, Ed Miliband's team tracked down a soothsayer of their own. They found him in Maurice Glasman, a north London poly lecturer. Glasman had helped Ed's older brother, David, write one of his speeches for the leadership election. Miliband senior had said that life was "about more than money and benefits" and had urged the party to "return to the values that used to be engraved upon the Labour heart". Leaving aside the bit about foetuses, Glasman's vision of a re-branded "Blue Labour" echoes the exotically-retro flavour of Blond's self-help Red Toryism. Any anxieties that Glasman's life-style – he lives above a shop in Hackney and rolls his own fags – might be thought a tad low-rent with the target audience of "squeezed middle classes" was soon resolved.

According to the New Statesman, Miliband junior's office rang him and said: "Would you like to be a lord?"

No-one seemed to see anything ironic in the Glasman/Miliband view about there being more to life than money. When David lost the leadership to Ed he consoled himself by supplementing his £67,000 salary as MP for South Shields with a £50,000-a-year contract with Sunderland Football Club. South Shields is one of the poorest town's in Britain. Some of its households haven't seen a wage packet for 20 years.

Not surprisingly, Professor Dorling's book hasn't attracted many fans among Blue Labour's well-heeled leadership. An Old Labour man himself he believes all the major parties – including Labour – share responsibility for bringing the country almost to its knees. Western countries like Britain, he says, have seen nothing like it since 1854, when Charles Dickens was writing Hard Times. He sees it as little short of a betrayal of Beveridge's post-war crusade against the social evils of the Thirties – ignorance, want, idleness, squalor and disease. Now a new credo, rooted in social apartheid that starts in schools, struts by the nation's slumdog neighbourhoods – elitism, exclusion, prejudice, greed and despair.

Yet pity was always one of the fundamental virtues of British life. It travelled easily, crossing class borders without a passport, informing every level of the nation's cultural life. In 1966 Jeremy Sandford's BBC film Cathy Come Home created an uproar about the homeless and made the charity Shelter into a major player which no politician dared ignore. In 1982 Alan Bleasdale's Boys From the Black Stuff reduced my generation of TV critics to tears of rage. Fleet Street's newspapers set the political agenda, keeping poverty high on news lists. Television's finest documentary makers stood on windy council estates, daring Parliament to flinch from its duty to the country's underclass.

Generations on and the decline of the old media, swept to the kerb by the new technology of the internet superhighway, has been a major factor undermining the accountability of the metropolitan political class. Old Fleet Street shed expensive reporters and joined the all-night party of vacuous celebrity, its newsroom survivors reduced to rifling Google for the latest tweet from Stephen Fry. In the worst recession since the Thirties, the repo men dump furniture on the lawns of countless provincial homes, unheeded by the newspapers in London.

In television, a new breed of programme controller clutters the schedules with lifestyle shows, house make-over shows, more shows featuring bad-tempered cooks, gloating "documentaries" shove cameras with eye-wincing intimacy into people's bodily deformities. Two of the most critically-acclaimed comedy shows, Paul Abbott's Shameless on Channel 4 and The Royle Family on BBC jeer at the poor with stereotypical portrayals of life on the dole as a joke. The latter focuses on an anally-obsessed brood of farting couch potatoes. The soap's admiring resume on Wikipedia herald's the family patriarch Jim, played by Ricky Tomlinson as "the man who made 'my arse' a national catchphrase". Was it a coincidence that David Cameron has dismissed unemployment as "a culture that pays people to sit on their sofa rather than go to work"?

The trouble with poverty is that it never sits comfortably on a sofa. A few weeks ago, on Question Time, the union leader Mark Serwotka described how 13 job vacancies in South Wales had attracted 1,000 applicants – vindication of a recent survey which showed that nine out of 10 people on the dole would actually prefer to be in work. Professor Fothergill's report, the culmination of a decade tracking the lives of people "on the sick" across the nation, has homed in on Merthyr Tydfil, where nearly a third of the population of working age is on the dole. Fothergill estimates that the little valley community needs 3,000 new jobs; across the wider valley community, the Rhondda and Cynon, the imperative is for 35,000. The scale of the task, replicated abroad in the US Rust Belt, almost defies solutions. Even Professor Dorling is stumped. When I asked him what governments should do to rebuild the UK employment market, he said something vague about looking after the aged properly; then he startled me by adding: "Also there's got to be depopulation of the Welsh Valleys."

"Sign-On" Valley is deeply symbolic to those who cherish the earliest days of socialism. It was the cradle and is now the graveyard of the Labour Party and its dreams of an alliance between intellectuals and working men's unions. Its birth was blessed there on a windy slope in February 1900 by its first parliamentary leader, Keir Hardy, the pacifist son of an unmarried Scottish farm servant. He called it "a poor little child of danger, nurseling of the storm". It's why I went there for The Independent in 1993, to ask Hardy's descendants what they thought about Labour's new plans to put on a suit and reach out to the voters of suburban England. Tony Blair was John Smith's shadow home secretary at the time, a pioneer – along with Peter Mandelson and Gordon Brown – of the scheme to create the "Savile Row Tendency".

Back in London I phoned Blair. I told him his valley supporters dreaded being dumped in the name of gentrification.

"Can we do this off the record?" he said. I got the sense that he couldn't wait to see the back of all those sooty, cloth-capped coal-shovellers.

Seven years later I had another taste of his steely resolve to keep Labour up-market. I was living in the North-East at the time. New Labour's adventures with the Dome had attracted widespread derision and Blair was panicking about not being elected for a second term. I wrote a long article for the New Statesman suggesting the party could rescue its fortunes by teaming up with Durham University to build a world-class IT city on its new under-developed campus site at Stockton by the Tees. Critically, given the region's chronic unemployment, it would reach out to young people on economically-ravaged estates like those in Peter Mandelson's seat, Hartlepool. I called it the Big Idea. At a stroke the PM could establish New Labour as a modern, yet caring party, anxious to retrieve a generation of children abandoned after the industrial car-crash of the Thatcher years. Blair rejected the plan; discussion with ministers was forbidden (none replied to my letters asking for support) and the party's MP in Stockton South, Dari Taylor, was told by Downing Street to back off. It was a spine-chilling demonstration of Blair's white-knuckle grip on the reins of government – and a portent of the kind of society he intended to build to secure his legacy. Blair wanted to put a grin on the face of the bankers and pour cheap money over the grateful voters of suburban England. The ragamuffins on the sink estates could wait.

Three years ago, now living in Dorset and still fuming, I threw the Big Idea into the back of my old Land Rover and took it to meet my MP, Oliver Letwin. The Tories has just started to build up an impressive head of steam denouncing the Broken Society; Iain Duncan Smith's genuine concerns about poverty were attracting attention – suggesting a government-in-waiting that wanted to get its head under the bonnet and fix things.

Letwin, a One Nation Tory in the Macmillan mould, did more than pick up the Big Idea and stroke it; he sat in my kitchen two years ago and constructed a dazzling vision. If the North-East project worked, he said, why stop there? Universities were all power stations of knowledge, energy, inspiration. They, too, could have their IT cities. The notion of creating a national grid of learning and job creation was breath-taking. And it made sense, Letwin said, to do this outside London because provincial land was cheaper.

Alas, post-election – to borrow the poet Yeats – "I, being poor, have only my dreams." It now seems as though regional Britain will have to whistle for its grid of superhighway cash cows. Earlier this year David Cameron and his chancellor George Osborne heralded the birth of "Britain's rival to Silicon Valley". Tech City, embraced by Google and Facebook, is to be built in London's East End in the sumptuous property environment of the Olympic Village. Cameron has relatives who work for Google; Osborne is a friend of Google's chief executive Eric Schmidt. An anonymous reviewer of the project on the internet – by a man who already runs a small IT company in the area's ironically-named Silicon Roundabout – has likened its influential founders to "a spreading family tree of mediaeval royalty". There doesn't seem to be much emphasis on the kind of broad social benefit envisaged in Oliver Letwin's provincial grid of academic powerhouses Cameron – anxious to encourage new ideas from abroad – signalled during the launch that Tech City will have a special place in its heart for foreign inventors.

A new category of travel document – Entrepreneur Visas – will guarantee a smooth passage through immigration control. Of course, any investment has to be a good thing. Sitting it in one of the richest cities on the planet will hardly resonate in the distant provinces of Broken Britain. Tech City confirms the haughty indifference of a political class which so often betrays the dreams of voters who send it to Parliament with a mandate to deliver. It scares people; makes them angry about their own sense of powerlessness. Nothing demonstrates this more than the email I received from New Labour MP Dari Taylor before she lost her marginal seat in Stockton South seat to her Tory opponent.

"I worked hard for your project with Tony Blair, but found no support," she said. "I was on my own and you have to accept that you either have the power or you don't. Sadly I did not."