HS2, NHS reform and now prisons – why does Britain keep making a mess of its biggest projects?

Anyone casting even a cursory eye over the national news recently would have alighted on at least half a dozen examples of major government failings. This is my list; you probably have yours.

The Ministry of Justice is to take the privatised part of the probation service back in-house after it was judged to have been “systemically flawed”. This follows the announcement that it will end its contract with G4S to run Birmingham Prison, which inspectors found to be in a state of crisis.

Not only has violent crime risen over the past year, in some categories, sharply, according to the Office for National Statistics, but the rate of charging and convictions has fallen. When you hear your neighbours complaining that the police are not interested in domestic burglary, or car theft, or shoplifting any more, their impression is borne out by official figures. Only 8.2 per cent of recorded crime leads to a conviction.

At the other end of the scale, the head of the National Crime Agency, Lynne Owens, last week called for her budget to be doubled, on the grounds that the country is awash with organised crime – a term which appears to cover everything from money-laundering to drugs, through to human trafficking and cybercrime.

If you are still not worried, you may also have seen that, according to Home Office evidence to MPs, more than 700 people tried to cross the Channel illegally in the 14 months to February, of whom more than 400 are known to have succeeded plus an unknown number who may have gone undetected. This is despite the home secretary, Sajid Javid, having declared a “major incident” in December and the Ministry of Defence having deployed two warships in what is one of the world’s busiest shipping lanes. We do not know – because this information has not been released – how many of these have been allowed to stay, even temporarily, rather than being sent back to France, which the UK is entitled to do, given that it presumably regards France as a safe country.

Oh yes, and Crossrail, the new transport system intended to speed commuters into, out of and across London is now delayed until probably 2021 (when it should have come into service last autumn) and a House of Lords committee is the latest body to call into question the rationale for the high-speed rail link to the north of England, the sequencing of the work (with the southern section first), and – of course – its cost.

If there is agreement on anything it is that this major infrastructure project, like so many others, is destined to go massively over time and budget, only to be scrapped at the point when it leaves a large hole in the public purse, but not as large a hole as it would have done had it been completed.

And while scarcely a week goes by without UK researchers developing all sorts of exciting ways to treat cancer (minimum lead time, 10 years), it is still highly unlikely that your A&E doctor will be able to call up your records if you have an accident, because of the mistakes and delays in introducing any sort of functional IT system to the NHS.

The powers-that-be do seem quite good, though, at overlooking paper evidence, when it suits, if early testimony to the infected blood inquiry is anything to go by. That is an inquiry, by the way, which finally launched last month, after 30 years of being effectively hushed up.

Now, of course, these are all very disparate examples of what might be generalised as government incompetence. And they – doubtless – all have their own reasons, some more excusable than others. It is also true that, as a nation, we perhaps have a tendency to seize on the failures rather than the successes. But it also often seems that when things do work, especially in a crisis, it is less because the system operates as it was supposed to than because of people who go beyond the call of duty, a measure of enlightened improvisation, and individual acts of heroism (7/7, the London Bridge attacks, the Manchester Arena bombing).

Remember, too, that state-of-the-art emergency communications system were promised after 7/7, when the police and ambulance response was hampered by incompatible walkie-talkies that also could not operate underground. Well, it still has not materialised. In fact, as we heard last week, it is unlikely to be ready before 2022, in good time for the 15th anniversary.

The last time that I can actually remember anything to do with government really going right would probably be the 2012 London Olympics. But even that ran over budget (though proportionally nothing like Crossrail) and there was a serious last-minute hitch when G4S – yes, the same company that was failing to run Birmingham Prison – admitted at a late stage that they could not supply enough security staff – so the army was called in (and they did a great job).

London 2012, in fact, might offer a half a clue as to how government project-running might be improved. There was a very obvious and unmissable deadline. Progress (or not) could not be hidden from the public eye. There were named people in charge, whose reputations were on the line. And there was regular outside scrutiny – in this case from the International Olympic Committee. Someone else was watching.

The Olympics, though, were a one-off, the exception, perhaps, that proves the rule: that recent UK governments, not just this one, have really not been very good at what is generally called public administration. We are a relatively uncorrupt country, with a non-political civil service. But where public administration is concerned we are inefficient, low-tech and – as the plethora of examples above shows – serially incompetent.

Whether it is controlling our borders, where – almost uniquely among developed countries – we still have no exit controls; planning, monitoring and completing public infrastructure projects, or running a prison system or health service, there is a conspicuous shortage of professionalism. I have cuttings going back nearly 20 years that lament our lack of capacity to train nurses, yet here we still are, trawling much poorer countries for the staff they have trained.

The contrast with our near neighbours, the French and the Germans, is stark. They have their failings – Berlin’s new Brandenburg airport being a signal example, but that has attracted so much opprobrium precisely because it is an exception. When the French lay a new high-speed rail link, they plan it well in advance; they buy the land, they exploit the rail-laying to include communications cables. From cities to villages, plans for infrastructure and building are systematic and long term. France’s contaminated blood inquiry was held more than 15 years before ours, blighting reputations and securing convictions.

And competence matters. The cynicism and distrust in government that runs so deep today reflects the sense that government in general cannot do anything efficiently or well. It is not something we can be proud of. If that is to change, at least two things need to happen. The first is accountability. Every undertaking must be thoroughly transparent, with a clear chain of command and named individuals in charge. The second is more expertise. The well-educated generalist may well know the right questions to ask. But to negotiate a multimillion-pound contract, to plan and monitor a new rail line, or to cater for the nation’s health requirements 20 years hence, requires a lot more.