Alan Watkins: Poor Mr Davis – unloved and driven by pique

"Now what can he have meant by that?" Count Talleyrand is said to have remarked when another diplomat died suddenly in the middle of an international conference. The inhabitants of Westminster were asking the same question last Thursday about the resignation of Mr David Davis. Let me try to clear up the mystery.

On the previous day I watched the debate on the 42-day detention period. What first struck me – and it was an impression I've retained throughout the proceedings – was that the leader of the opposition, for the purposes of the debate, was not Mr Davis, the Shadow Home Secretary, but Mr Dominic Grieve, the Shadow Attorney General. Since then, Mr Grieve has been promoted to Mr Davis's former post, while Mr Davis is off to fight a by-election in his constituency of Haltemprice and Howden, which is near Hull.

Ms Jacqui Smith, the proper Home Secretary, made a short speech, really no more than filling in time, before the division at six. The broadcasting authorities somehow contrived to give the idea that the vote had taken place later in the evening. Likewise, they showed clips from Mr David Cameron and Mr Gordon Brown at Prime Minister's Questions earlier in the day, thereby falsifying what had happened. The star of the afternoon was Ms Diane Abbott, who spoke with passion against the Government.

Mr Grieve performed as competently as he usually does in his lawyerly way. It was he who was left to bear the heat and burden of the afternoon. This was a debate on the Report stage rather than on Second Reading. The Government was accordingly supposed to be reporting back to the House. This might be part of the explanation for Mr Davis's partial anonymity – that Mr Grieve was assumed to be dealing with what are called "committee points" rather than with great questions of general principle which are intended to be ventilated on Second Reading.

Even so, it should have been Mr Davis's big day or, at any rate, his big afternoon, despite the paucity of the attendance. But then, the broadcasters prefer to show pictures of crowded PMQs or divisions in the lobbies rather than of what is going on for most of the time. It would be fair to say that Mr Davis is taking his leave out of pique.

It would be equally fair to conclude that he feels unappreciated, even unloved. He has been a success as Shadow Home Secretary. He has seen off three Home Secretaries and a junior Home Office minister in Ms Beverley Hughes, though she has now returned to the fold in another capacity. He believes in capital punishment but does not believe in making an issue of it, any more than Margaret Thatcher did.

I should not have described him as a libertarian, in any sense except that the state has too much power. He is against identity cards and locking people up for long periods without charging them. Mr Davis thinks he should have been leader at some time in the early 2000s. So did Mr Kenneth Clarke, with rather more reason. Mr Clarke did at least take the precaution of standing, on several occasions, without success.

When Mr Michael Howard was elected unopposed in 2003, it was a measure of desperation. Mr Howard did his best at the 2005 election, but it was no good. He gave up – as Sir John Major and Mr Hague had given up before him – and Mr Davis was left to contest the succession with Mr Cameron. Mr Howard threw his influence, first, behind Mr George Osborne and later, when Mr Osborne had fallen out of the betting, behind Mr Cameron. Davis's friends had said earlier that the party should not be taken over by "the Notting Hill set". But so it was, at the 2005 party conference, in a free vote after a somehow contrived-sounding debate, though it was perfectly fair. Mr Hague, by the way, seems equally uncomfortable on the other side of Mr Cameron on the front bench. It can only be a matter of time before he too takes his leave, or has his leave taken on his behalf.

Mr Davis looks as if he will fight the by-election all on his own, apart from the collection of exhibitionists, eccentrics and plain lunatics who usually put in an appearance on these occasions. There is even a story that the former editor of The Sun, Mr Kelvin MacKenzie, may stand. The suggestion was apparently put to him at a party by the paper's proprietor, Mr Rupert Murdoch, who presumably believes in banging people up for as long as he can. Mr MacKenzie had previously unsuccessfully contested a local authority seat in leafy Weybridge.

The new candidate had previously confided in Ms Shami Chakrabarti, the director of Liberty, and Mr Nick Clegg, the leader of the Liberal Democrats. In similar circumstances, if such can be imagined, I should not have first sought the advice of the director of a pressure group usually opposed to my own views and also of the leader of a party with which it was in direct competition. After all, the Liberal Democrats came second at the general election. Mr Clegg could have put up a show. Labour is not standing, either; or so it seems. Ms Chakrabarti wisely advised Mr Davis to stay put; while Mr Clegg, by contrast, almost found himself in the position of being a co-conspirator.

In 1986 there was a mass resignation of Northern Ireland MPs who were all unopposed. In the post-war period, Sir Richard Acland moved from Labour to Independent at Gravesend in 1955 and was unsuccessful. Dick Taverne moved from Labour to Democratic Labour at Lincoln in 1973 and was successful. Bruce Douglas-Mann moved from Labour to the Social Democratic Party at Mitcham in 1982 and was unsuccessful.

Mr Quentin Davies moved from the Conservatives to Labour at Grantham last year and was one of the many MPs who have switched sides and did not feel it necessary to contest any by-election. Mr Davis has not changed sides and has no quarrel with his party. Admittedly, there is a subterranean dispute about whether the Conservative Party should promise to repeal the 42-day provision in office. But many such changes – for instance, the minimum wage – are accepted after a time.

The truth is that Mr Cameron does not really want to have Mr Davis in the team. If he had wanted to, he could have kept his place vacant until after the poll. As it is, he is heading for the back benches entirely through the sin of pride.