Robin Butler: The pillar of the Establishment

The leaking has been widespread and detailed. The Butler inquiry into why the secret services got the intelligence on Iraq so wrong in the run-up to war is due to report on Wednesday. But already, we have been prepared by the spin merchants for stern criticism of Britain's two top spymasters and the Government's chief legal officer. There will "be no comfort for anyone" is the verdict of one insider.

How can this be when Lord Butler of Brockwell was widely denounced as a government stooge as soon as his appointment was announced? Within days, 11 left-wing MPs signed a House of Commons motion charging that Lord Butler's record as the former head of Britain's civil service "undermines his credibility as a fair and impartial chairman". Newspaper columnists - from both left and right - joined in with excoriating articles about how the then Sir Robin Butler had, as Cabinet Secretary, repeatedly proved himself to the ultimate safe pair of Establishment hands.

Was this not the man who had told the Scott inquiry - into, ironically, covert British arms sales to Iraq in the days when we were all in favour of weapons of mass destruction - that it was OK for government ministers to dissemble with replies which were "selective about the facts" and give "an answer that is not the whole truth"?

Had he not investigated the sleazy Tory minister Neil Hamilton - who received brown envelopes full of cash from the Harrods owner Mohamed al-Fayed - and found him innocent? Did he not honourably acquit another sleazy minister, Jonathan Aitken, whose grasp on the truth proved so slender that he ended up in jail for perjury? Was it not Butler who had signed the controversial Order in Council which gave Alastair Campbell free reign with the Civil Service, with such disastrous consequences?

And now, the threnody continued, "this quintessential boy scout", who retired from government in 1998, saw it as his last public duty to return to dig his beloved Establishment out of one final hole.

The truth was never quite so simple. Frederick Edward Robin Butler was born in Lytham St Anne's, the son of the managing director of a paint manufacturing firm. It was only trade, of course, but his father was doing well enough to send young Robin to Harrow where the pattern of his life was set. He won a scholarship and became head boy before going up to Oxford where he took a double first and won two rugby blues. He was a wing-forward as befitted a young man with both speed and strength.

He came top in the Civil Service entrance exam and his golden career continued as he raced through its ranks. After joining the Treasury, in 1961, he progressed through the Budget Committee to the Cabinet Office before becoming, in 1972, private secretary to Edward Heath. It was just the first of five prime ministers he was to serve. He was so valued by Heath's successor, Harold Wilson, that Wilson's jealous political secretary, Marcia Williams, later Lady Falkender, tried to get him sacked.

Later, as principal private secretary to Margaret Thatcher, he faced a threat of a different kind. Shortly after 2:35am on 12 October 1984 the prime minister was interrupted in her room at the Grand Hotel, Brighton, by her principal private secretary, Sir Robin Butler, who needed her to look over "just one more paper". Moments later an IRA bomb exploded with devastating effect. The two narrowly escaped death, though five other people died. Four years later the man known in Whitehall as Ferb (because of his initials) was appointed, at the age of 50, to be a spectacularly young Cabinet Secretary and Head of the Home Civil Service - a post he held for 10 years.

It was under the next PM, John Major, that Robin Butler's serious discomfort began. He was plunged into what he has called "the long nightmare" of the Scott inquiry, which ran from 1992 to 1996. Lord Justice Scott was charged with unpacking a scandal over ministerial involvement with British businessmen supplying weapons material to Iraq in disregard of official sanctions. Called to give evidence Robin Butler achieved notoriety by defending Whitehall doublespeak with the words: "You have to be selective about the facts. It does not follow that you mislead people. You just do not give the full information ... It was a half answer. Half the picture can be true." Scott went on to let the Government off the hook, on a series of technicalities, much as Hutton was to do more recently.

But the nadir of Butler's career, as he was later ruefully to describe it, came in 1994 when John Major asked him to investigate allegations made by the Harrods owner, Mohamed al-Fayed, against several Tory ministers. One, Tim Smith, resigned after admitting that he had taken bribes. But Lord Butler cleared another, Neil Hamilton, who was subsequently disgraced when the truth came out in a libel action.

Next he cleared Jonathan Aitken, who had been accused of taking a bribe in the form of having his hotel bill at the Ritz in Paris paid by the Saudis at a secret meeting to carve up arms commissions. It later emerged that Butler's investigation had merely consisted of asking Aitken if he was lying and, when the minister said no, accepting his word.

Critics subsequently said that Butler's problem was that he should never have agreed to act as an arbiter in such matters. The job belonged to a politician or party official, not a civil servant. But he wanted to help Major, a man he liked - and with whom he shared a love of cricket. Butler was one of Mr Major's closest working companions in the dying days of his prime ministership.

Yet he remained the classic civil servant. He knew that Major was going to lose the general election and in the background prepared for transition. He spoke discreetly to Tony Blair and Paddy Ashdown, the Labour and Lib Dem leaders, about the possibility of a coalition that would have taken the Lib Dems into government for the first time since 1922, had Blair been left with a small - or no - majority.

In the event Blair got a landslide majority of 160. When New Labour arrived in office and civil servants crowded the windows of Whitehall, it was Robin Butler who orchestrated the clapping to cheer their new bosses into power. It was Butler who guided Blair and co through the unfamiliar territory of Whitehall protocol.

Initially Butler thought that Blair - who had worked his enormous charm on the mandarins - was going to be an easy run. "They were not wild men of the left," one insider said. "And civil servants wanted strong government after the paralysis of the Major years." So matey was Butler with the archetypal New Labour figure Peter Mandelson that he gave him a bottle of peach champagne to drink "somewhere under a tree" in the summer of their first year.

It was not to last. The Blairites were fiercely critical of Civil Service styles and structure. "They don't even carry mobile phones," one apparatchik noted witheringly. The incomers were more difficult, troublesome and truculent than Butler had expected. Within 18 months of Labour's victory all but two departmental heads of information were out of a job. "This lot are not nearly as nice as we thought," one of Butler's fellow mandarins noted.

Butler has told friends that his key mistake in that first year was to sign the Orders in Council that allowed Jonathan Powell, Blair's chief of staff, and Alastair Campbell, his press secretary, to give orders, despite being party political appointees, to civil servants. Such powers were unprecedented and violated the traditional line between elected politicians and an impartial Civil Service.

Events exposed by the process of the Hutton inquiry, if not condemned in its verdict, enable critics to draw a direct line from the Order in Council which gave Alastair Campbell the right to chair the planning meeting with civil servants and intelligence chiefs over the infamous "45-minute" dossier that took Britain to war in Iraq.

The former Conservative minister Virginia Bottomley once complained to Sir Robin Butler about the Government Information Service, asking why it couldn't be more committed to the government line. "Robin used to say that the rules I disliked in government, I would most value in opposition," she said. Critics complain that he failed to maintain this line under Blairite pressure.

When he retired Sir Robin defended himself on this point in a rare newspaper interview. His priority had been "trying to make government work". He defended himself against charges that he was a poor detective over Hamilton and Aitken, saying police work was not his job. Not everyone is unsympathetic on this. "If a minister, and a privy councillor, gave his word it's hard to see what else he could have done," says one insider. There are even those who defend his Orders in Council decision. "The weakening of the influence of the professional Civil Service, thanks to special advisers, was a prominent feature in the Thatcher years," says one constitutional observer. "Butler may well have felt he was just regularising a de facto reality."

But this seems overgenerous. In the early days of New Labour, when senior mandarins were expressing concern at the large numbers of special advisers, Butler tried to reassure them. Campbell, Powell and the rest were just a passing phenomenon. "What can a few special advisers do against thousands of civil servants?" he said to one colleague. "We'll swat them like flies." He clearly underestimated them.

Yet it may now be that the Blairites have underestimated Robin Butler, too. "If Blair thinks Butler is a patsy, then he's badly mistaken," says the constitutional historian Peter Hennessy.

Butler is said to have been "horrified", in taking evidence for his report, at the extent to which the line between politicians and civil servants has dissolved since he retired six years ago. He does not think much of Blair's practice of not having minutes taken at top-level meetings. And he is alarmed at the ambiguous relationship between John Scarlett, chairman of the Joint Intelligence Committee, and "his mate" Alastair Campbell.

Looking back on his career Butler has told friends that he is well aware that he made mistakes and feels anger at those who misled him. It may well be that he sees his report to be published next week as his final opportunity to put right the errors of the past.

A LIFE IN BRIEF

Born: St Anne's, 3 January 1938.

Family: Married Gillian Lois Galley 1962. One son and two daughters.

Education: Harrow School; University College, Oxford.

Career: Joined HM Treasury 1961; private secretary to Edward Heath MP 1972-74; private secretary to Harold Wilson MP 1974-75; principal private secretary to Margaret Thatcher as Prime Minister 1982-85; Secretary of the Cabinet and Head of the Home Civil Service 1988-98; master, University College Oxford since 1998; made a life baron (Lord Butler of Brockwell) 1988; member of royal commission on reform of Lords 1999.

He says...: "Very often one is finding oneself in a position where you have to give an answer that is not the whole truth."

They say...: "Robin Butler is a crown servant who knows the importance of keeping a proper demarcation between ministers, career civil servants and special advisers." - Peter Hennessy, professor of contemporary history, Queen Mary College, University of London