Owen Jones: Here's the question... could you live on £67 a week?

In a church in Birmingham Ladywood, one of Britain's poorest communities, I recently sat next to Edwina Currie as she lectured poor people that they were not really poor if they owned a mobile phone. Currie, a former Tory minister, had been sent there by the BBC to explain her fact-free assertion that no one had to choose between heating their homes and feeding themselves.

She had form: in the 1980s, she claimed northerners were dying from "ignorance and chips". She refused to budge even when I pointed out the tragic case of Mark and Helen Mullins. Mark walked a 12-mile roundtrip every day to a soup kitchen; their desperate circumstances drove them to suicide. Here was a damning indictment of Britain in 2011, but still Currie demanded the poor pull themselves up by their bootstraps.

I'm sure Currie felt vindicated by last week's Social Attitudes Survey. Even as the ranks of Britain's poor and unemployed have been swelled by economic crisis, social contempt has increased. 54 per cent of those surveyed felt that unemployment benefits were too high, compared with 35 per cent in 1983 – another era of dole queues. If the question had been, "Could you live on £67.50 a week?" – the actual value of Jobseekers Allowance for those aged 25 or above – it's a safe bet the results would have been rather different. Nearly two-thirds believed that one factor for child poverty was parents who "don't want to work". Scroungers and work-shy freeloaders: these are Britain's poor as far as millions are concerned.

It's a triumph for the Tory governments of the 1980s in which Currie served. Margaret Thatcher once argued that "there really is no primary poverty left in this country", putting some people's lack of money down to a "really hard fundamental character-personality defect". Unemployment and poverty were not social problems, they were individual failings – or so Thatcherism drummed into the minds of the British public.

An integral part of the current Government's mission to turn a crisis of the market into a crisis of public spending has been to demonise benefit recipients. It was taken to its ugliest conclusion last week when a Government report suggested making chemotherapy patients prove they were too ill to work. Sections of the media happily fan this prejudice. When unemployed people make an appearance, it's invariably as the "scrounger". Channel 4's grotesque TV series The Fairy Jobmother featured the "feckless" being harangued back into work. The BBC's John Humphrys fronted The Future State of Welfare, condemning Britain's "dependency culture" and a "sense that the state owes us a living".

The reality of unemployed people desperate for work – like the 110,000 applications for Royal Mail's 18,000 temporary Christmas jobs – barely gets a mention. We don't even really talk about the unemployed any more: they're more likely to be "people on benefits", defining them not by lack of work, but by a reliance on taxpayers' money.

But it is Labour that must accept a large slice of responsibility for growing social contempt. Since the mid 1990s it has opted not to challenge the Thatcherite mantra of individual responsibility for social problems. Ed Miliband used his Conference speech to call for those in work to be given priority in social housing.

Forget the political rights or wrongs for a moment. It is a suicidal strategy for Labour to take. Failing to challenge – and even fuelling – social contempt simply increases potential support for the Conservatives. Those who hate "scroungers" most will never trust Labour to crack down ruthlessly enough: that's what the Conservatives are for. If Labour won't take on the consensus, the reality of unemployment and poverty will disappear from the public domain – and ever hardening social attitudes will cripple the party at the ballot box.

Social contempt can appeal to middle-class voters who resent spending their taxes on the undeserving, and low-paid workers who resent struggling to pay bills while others "milk the system". According to a survey by BritainThinks, it even includes benefit recipients who – belonging to a demonised group – are keen to distance themselves from "scroungers". A mass campaign revealing the reality of poverty and unemployment is desperately needed to shift public attitudes. Without such a crusade, there will be no political space to oppose benefit cuts or push for progressive policies like redistributing wealth. After all, if poverty is a product of individual failings, why have a welfare state at all?

The winners in all this will be the City firms who helped cause the crisis, and tax-dodging companies that deprive the Treasury of up to 60 times the amount lost through benefit fraud. How pleased they must be that those worse affected by their crisis are turning on each other, rather than on an ever-wealthier elite. As living standards plummet, misdirected anger could turn ugly. Unless there is a political game-changer, Canary Wharf will remain the glistening symbol of a City prospering while others suffer, a middle finger stuck up at the British public as they seethe with contempt for each other.

Owen Jones is the author of 'Chavs: the Demonisation of the Working Class'