John Rentoul: Labour must read the Tories' book

A week is no time at all in politics. Some things take a long time to change, and political parties are among the most resistant. That is the lesson of a new book to be published in January. The Conservative Party from Thatcher to Cameron, by Tim Bale, a rising star of political science at the University of Sussex, sets out to answer two simple questions: why did it take the Tories so long to do what they had to do in order to regain power after 1997; and how were they able, after 2005, to put things right so quickly? These are questions that go to the heart of understanding what is happening in politics today.

For many Conservatives, the book will make painful reading, reminding them of what happened since their party abruptly ceased to be the most effective electoral machine in the history of democracy on Black Wednesday in 1992. There was John Redwood's challenge to John Major's leadership in 1995, with the brutal slogan, "No change equals no chance," which was the right analysis but the wrong candidate.

There was the doomed leadership of William Hague. Bale is rude about the idea that Hague's mistake was to revert to a right-wing, core-vote strategy after flirting with modernisation: "It could be argued that Hague had never really had a strategy worth the name." Then there was the even more doomed leadership of Iain Duncan Smith. Bale quotes an anonymous functionary who served under Major and Hague: "I can't think of a good thing to say about Iain. I mean, I really can't. He's not a bad bloke. He's not stupid but he couldn't be a Cabinet minister." Finally, after the party realised that Duncan Smith's leadership was not going to work, there was Michael Howard, elected unopposed because he could hold his own against Tony Blair, but fatally weakened, as he now admits, by his role in the Thatcher and Major governments.

The book is a brilliant analysis of why the party found it so hard to accept that election defeats suggested that it was doing something wrong, rather than that the electorate had made a terrible mistake. I remember Hague saying, towards the end of Blair's time as Prime Minister, words to the effect of: I told you the voters would see through his shallow appeal. Bales cites example after example of top Tories refusing to compromise with the voters. Most enjoyably, he quotes Michael Gove, now an ultra-moderniser as shadow schools secretary, as saying in 1999: "No location is as undignified as being 'in the centre' ... where the lowest common denominator and the highest possible public spending meet." Gove is now an enthusiastic exponent of the view that the Conservatives should take the centre ground once occupied by Blair, justifying schools reform with the sort of egalitarian rhetoric that would make Gove, the Times columnist of 10 years ago, blush with indignity.

So what was it that held the party back from doing the obvious things to win votes? Chance played its part, as when Michael Portillo missed the run-off against Kenneth Clarke in 2001 by one vote. Ideology played a role, in particular a disabling obsession with Europe that had even the young David Cameron, in his first attempt to be elected as an MP (in 1997), defying the leadership to declare his absolute opposition to Britain's adopting the euro.

But there were two more important factors. One is organisational weakness: the lack of a system of feedback from voters to force the pace of adaptation. Justine Greening, a new Tory MP with a private-sector background interviewed for the book, said: "You get from business to politics by taking your organisation structure, removing it so there's no direct reporting ... have every employee think that they could be the chief executive within quite a short period of time, and then have them not all quite agreed on what the product range is."

It is instructive that the party finally got its choice of leader right on the one occasion when a leadership contest was delayed after a general election defeat. In 1997 and 2001 the defeated leader stood down immediately; but in 2005 Howard stayed on, in a failed attempt to change the rules under which the new leader would be elected, and a successful attempt to allow David Cameron and George Osborne to shine. Crucially, though, the delay also forced the candidates to address in a more considered way why the party had lost.

Cameron's election four years ago also demonstrated the final lesson: the importance of personality. Bale observed, in an earlier book, that Cameron was the first Tory leader to be chosen for who he was rather than for who he wasn't, since leadership elections were introduced in 1965. Suddenly, in 2005, the Tories acquired a leader who had a basic plausibility and attractiveness that could have sold even the failed product of previous elections better than Hague or Howard; but they also acquired a strategy of detoxifying the brand and trying to appeal to swing voters. It is a strategy that is full of holes – Cameron's speech last week saying how much he really cares about poverty is still at odds with the party's plan to cut inheritance tax – but it is a strategy and broadly the right one.

The paradox is that the lessons of this book are less relevant to the Tories now. They have done a large part of what needs to be done – helped, of course, by the decline of their opponents. It is the Labour Party that needs to read this book and ask itself how it can get ahead of the long cycles of British politics.

It would be to defy all precedent for Labour to learn the lessons of its unpopularity before an election defeat. But there is no reason why it should go through years of introspection and four unsuitable leaders before it comes up with a good enough response to Cameron. It should know that the personality of the leader is not only critically important, but something that it could do something about in, well, about a week.

John Rentoul blogs at independent.co.uk/eagleeye