How to cure the Coalition

As Andrew Lansley's mighty NHS reform became law this year, the chief reaction among medics, and indeed among many Tory and Liberal Democrat MPs, was "never again". Never again must this institution – "the closest thing the English have to a religion" as the former Tory chancellor Nigel Lawson once put it – be subject to such a "train crash" of politics and policy making.

But there is another version of "never again". It is Andrew Lansley's. His aim was to establish what Labour used to describe as a "self improving" NHS. Doctors working through clinical commissioning groups would purchase health care locally. A national commissioning board would oversee their work, while buying the more specialised services. The system would operate within a formal framework of choice and competition that Labour itself had advocated in government. And ministers would stand back from the day-to-day management of the NHS – as many in the health service had long wished.

The core of Mr Lansley's vision remains, even if the amendments muddied the purity of the original design. Yet without the Liberal Democrats a Conservative minority government could not have got this bill through. Without them it would have been a very different bill, involving much less structural upheaval. It would also have been much harder for opponents within the Government to foster dissent.

There is plenty of dispute, of course, over whether any of this will work. It is too soon to tell. If it does not, a whole bunch of lessons will emerge.

Good policy requires a clear definition of the problem and a clear explanation of how the policy proposals will fix it – with the explanation set out in a way that maximises support and minimises opposition. All of that was missing. There was no clear narrative about how these changes, done in this way, at this time, would help the NHS meet its core challenge of saving £20bn over the next few years. Rather, in the short-term, with all the redundancy and other costs as managers lose their jobs and doctors learn new ones, the opposite is likely. And the changes are so complex that even Simon Burns, the Health minister, conceded: "You cannot encapsulate in one or two sentences the main thrust of this [reform]."

It is clear that the way the "programme for government" was stitched up as a purely political exercise in No 10 turned the reforms into something far bigger than they would otherwise have been. There is clearly a case for taking longer, and involving genuine policy expertise, in drawing up any future coalition agreement.

From the accounts drawn together for the Institute for Government and the King's Fund, it is clear that the health department warned Mr Lansley of the risks in what he was doing. It offered a consultation before the white paper to try to build support. And it offered the option of no, or much less, new law. As the Health Secretary himself remarked: "I could have done most of this without the legislation".

But Mr Lansley rejected all that. He was determined to get the reforms in well before the next election. And he was equally determined to cement them in law, so that subsequent health secretaries could not change them easily. After endless NHS re-organisations over the past 30 years, he wanted the bill to provide a permanent, or near permanent, solution to how the NHS is run.

The scale of the legislation, however, allowed the opponents of choice and competition to emerge from their trenches and refight all the battles they had engaged in 20 years earlier.

David Cameron's early decision to scrap the Prime Minister's policy unit and slash the number of special advisers left an overwhelmed No 10 unable to challenge the wisdom of Mr Lansley's plans when they ran into trouble. The decision to stop talking about the reforms before the election came at a price.

But for all that the act does settle some unanswered policy questions that have hung around the NHS for two decades. In all that time it has never been clear who, ultimately, was meant to be in charge of purchasing health care locally. Was it some form of health authority, or GPs? The answer, in the clinical commissioning groups, is an amalgam of the two. And if this were to conclude endless NHS "re-disorganisations", that would be a huge gain.

The odds remain that the big financial squeeze the NHS is facing will overwhelm all this. By 2015, the issue will be "the money, stupid".

But if, in the longer run, much of this turns out to work reasonably well, then Andrew Lansley will emerge as an unlikely hero of public service reform. And the lessons to be learned will be rather different. Stranger things have happened, though perhaps not many of them.

'Never Again? – The Story of the Health and Social Care Act 2012' – is available free at instituteforgovernment.co.uk and at kingsfund.co.uk. Nicholas Timmins is a senior fellow at both institutions

Policy points: A blueprint for further reform

Four lessons that need to be learnt from the chaotic NHS bill.

No surprises

Governments must prepare for policy long before formal legislation is published – ideally as a transparent part of an election manifesto.

Clarity

Good policy needs a clear definition of the problem, and a clear explanation of how the policy will solve it.

Slow down

Future coalition governments must take longer, and involve genuine policy expertise, in detailing their programme for government.

Sort out No 10

Downing Street must strengthen the policy unit and bring in more special advisers to challenge departments and anticipate problems.

Reaction: Civil servants 'must have power to challenge'

Britain's senior civil servants must be given the power to challenge ministers who make bad policy decisions, a leading think tank concludes today.

In its response to Nicholas Timmins' account of how the Government's heath reforms went so wrong, the Institute for Government says officials must be prepared to speak up against ill-thought out ideas.

It also calls on David Cameron to strengthen his policy team in Downing Street so it can effectively challenge departments and warns that "speed is not everything" when a government first comes to power.

The Independent this week has been serialising "Never Again? – The story of the Health and Social Care Act", which was written by Nicholas Timmins and commissioned by the Institute and The King's Fund.

In its response to the report, the Institute said it was clear there were many civil servants within the health department who had reservations about the legislation. But it added: "It is very difficult, in our system, for the Civil Service to do more than warn ministers and then accept ministers' judgement if those warnings are not accepted."

The Institute also claims not enough time was taken in the early days of the Coalition to set out a coherent programme for government – which led to many of the problems which later beset the bill.

"The programme was agreed too quickly and without sufficient consideration of whether what was agreed was workable," it said. "The Health Secretary's comments were ignored."

Oliver Wright