The Conservatives face a battle of ideas and are not sure how to win it

For the first time since 1979 the Conservatives are facing a big ideological battle of unpredictable outcome. They do not quite know how to deal with it. Of course Tony Blair challenged them electorally, but at no point did he undermine their sense of ideological purpose. Indeed many agreed with Blair at key points or found that he agreed with them. David Cameron’s brilliant early insight as Tory leader was to recognise that by supporting Blair with genuine conviction he made him more vulnerable as a Labour Prime Minister.

Now Ed Miliband has laid down a rare ideological challenge. Opinion polls suggest voters are at least giving him an early “thumbs up” for doing so. Instead of adopting the Blairite strategy of advocating expedient policies broadly supported by the centre-right and its newspapers, he has advanced a big left-of-centre argument about markets, when they work and when they do not.

The battle presents substantial dangers for the Conservative leadership. When they were in opposition I remember vividly the first time I sensed Cameron and George Osborne would not win the following election. It was their response to the nationalisation of Northern Rock, one that brought home the limits of their self-proclaimed modernisation project. The duo held a rare joint press conference in which they could barely contain their excitement as they condemned the policy as a “return to the 1970s”. For all their early energetic originality as opposition leaders it was suddenly clear they were far more trapped by the past than they seemed, viewing the dauntingly new challenges of the present through the prism of the 1970s and 1980s. Revealingly Gordon Brown was similarly trapped, delaying the decision about nationalisation precisely because he feared accusations about a return to the 1970s.

Miliband is more daring and, in identifying the sense of helplessness felt by consumers, he made his move. In addition he was emphatically clear about what should happen when markets do not work, arguing that only governments can intervene. It is the first occasion in a very long time that a leader of any party has put the case so vividly, in what had been a largely non-existent debate, about the relationship between the state and markets. When he was Chancellor, Gordon Brown delivered a thoughtful speech on the theme, but he highlighted the limits of markets in education and health as part of his ideological battle with Blair. He would not have dared to touch the energy companies. Miliband has done so and, in the changed economic and political context, makes more waves than Brown did in the era when few questioned Thatcherite orthodoxy about the purity of markets.

Very quickly, and to their considerable surprise, Cameron and Osborne found themselves on the wrong side of the argument in relation to the nationalisation of Northern Rock, or rather discovered that they were swimming against the tide of history as the global financial crash demanded government intervention even from the likes of President Bush. The duo never recovered fully in terms of projecting a coherent message at the 2010 election.

The reason their evocation of the nightmarish 1970s did not work then is the same one that makes this a precarious moment for them now. The context is wholly different from that dark decade. When the likes of Roy Hattersley were assigned the absurd task of fixing the price of bread each month their powers were part of a much wider corporatist state, which was already gasping for breath. But the economic crises of the period led to such an over-reaction that British governments have been terrified to intervene on any front since, while administrations in equivalent countries have been far less fearful.

Miliband’s proposal is not an echo of the failed 1970s culture but a response to what followed, the growing number of markets that do not generate genuine competition and leave the consumer powerless, forced to choose between companies offering the same services at the same prices. I have yet to hear a convincing answer from the Conservatives at their Manchester conference to the question: What to do when markets do not work?

To their great credit the Conservatives are good at grown-up politics. Since the late 1970s they have been at ease talking about ideas, partly because their own, at least in relation to the economy and markets, have dominated for so long. At fringe meetings some speakers acknowledge Miliband has come up with a big idea and they must counter it with an equally large one. Others fear that their “Right to Buy” scheme rushed forward for this conference contradicts their attack on Miliband pointing out that an act of state intervention in the housing market might have more dangerous consequences.

At times Cameron and Osborne have lapsed into over-excitement, with Cameron describing Miliband’s ideas as “nuts” and Osborne comparing them with those espoused by Karl Marx, but the wider response has been more reflective. At the very least it has made some Tories reflect on previously unchallenged assumptions. This is an adult reaction, but conveys a sense of a party on the defensive rather than one marching confidently towards victory at the next election.

Osborne dealt with Miliband only briefly yesterday, in an elegantly constructed address that sought an elusive “big tent” of support. I liked his new more restrained style of delivery, the look of the statesman in the eye of the storm. Yes, the Conservatives had modernised and their policies were fair. Yes, they would be committed to increases in capital spending while being very tough on current spending, largely by being even more robust on welfare.

A policy for cutting spending on the long-term unemployed seems to be unveiled by every party every year. Even now there is not a huge difference between Labour’s “job guarantee” and Osborne’s announcement that the long-term unemployed will be compelled to do some form of work. In both cases it is not clear where this work will come from. I expect to hear the same proposal unveiled every year for the next two decades. But even from Osborne I heard no clear answer to the question about what to do with failing markets beyond, I suppose, putting up with them.

No doubt Cameron and Osborne would prefer an ideological contest to the one their party used to face when Blair was Labour leader. Nonetheless in 1991 Nigel Lawson prophetically highlighted the centrality of ideas. The former Chancellor predicted his party would win the forthcoming election because it was still winning the ideological battle. Sure enough they won with ease a year later, even though they had been miles behind in the polls. The battle of ideas is pivotal. More than 20 years on from that fourth successive victory, the Conservatives suddenly have much more of an ideological fight on their hands.