Hamish McRae: Why governments miss their targets

Why has the Government's system of setting performance targets produced such perverse results? Right now we have had two examples of failure. One is the way that the huge increase in pay to GPs has not resulted in them being open at more convenient hours, something that now has to be retrospectively tackled by the Health Secretary, Alan Johnson. The other is the way the police have focused on solving unimportant crimes rather than serious ones, as criticised by Sir Ronnie Flannigan's Review on Policing.

Mention "government targets" and people's eyes roll to the ceiling. Achieving a target has been elevated above providing a good service, indeed above doing the common sense thing. It has also resulted in people twisting statistics: hospitals trying not to mention MRSA on death certificates because that would mean they would miss their target to reduce such deaths.

My own favourite is an elderly aunt consulting a GP and then a few weeks later being rung to say there was some good news: they had got her onto the hospital waiting list. In other words, to get its waiting list down to the target, you had to go onto an informal waiting list before you could get onto the official one.

Yet the people who devised this system of targets, the politicians and their civil servants, mean well. The new GP contract, the one that has been causing all the trouble, had some sensible elements to it. One was that GPs should be paid extra if they met their targets for screening for various diseases. But the rates were set too generously, the GPs did what they were asked to do, but thanks to the financial incentives, more vigorously than the Government had expected. Result: the best-paid GPs in the world, but you still can't get a regular appointment on a Saturday morning.

The police targets were, on the face of it, sensible too. What is now recognised as a bureaucratic burden was a system, or rather a bundle of separate systems, designed to measure effectiveness and to protect individual rights. There are several elements that the report focuses on but to take one simple one, if you want to increase the proportion of crimes brought to a successful prosecution, a crude measure of success is going to push police into prosecuting unimportant crimes.

You can have two reactions to this. One is to say that we have a targets-obsessed Government, spewing out arbitrary top-down targets for everything, and that this is a result of Gordon Brown's relentless attempts at micro-management of every detail of the economy. That, coupled with Tony Blair's parallel obsession with presentation – an obsession that still lingers on – has resulted in a government culture where there are an endless series of headline-grabbing "initiatives" that are then publicly celebrated as being "achieved" two years later. Of course, no one believes this because we all assume that the figures are cooked and government ministers cannot understand why, even when they have made progress on something, no one trusts them.

I personally have some sympathy with this reaction but only some. I don't think the problem is particularly with this Government, though it is very unfortunate that so much of the once-in-a-generation increase in public spending should have been wasted. What worries me is that when the other lot get in they will make a similar set of mistakes unless they understand why outcomes have been so disappointing.

Are we so bad? Well we are not terrible. According to the OECD, which monitors government performance across the developed world, we are about the middle of the pack. But we are not good. I was talking to a senior civil servant the other day and he confessed he was depressed. He had spent 30 years trying to do jobs well, and believing that we were doing so, and he had to acknowledge that basically they had failed. I pointed out that we were not that bad and we certainly were not corrupt. Sure, he replied, but we weren't improving in the way some other countries were.

The conversation troubled me because I think he is right. We do screw up and I think the targets culture has helped us screw up bigger. But we are not alone in misusing targets. Other governments make similar errors. The private sector makes them too, sometimes spectacularly so. Look at executive bonuses. But the private sector also sometimes learns that you have to treat targets sensibly. Let me give you a couple of examples, both of which come from a new book on economics published this week: The Logic of Life by Tim Harford.

One of the world's great pole-vaulters was Sergei Bubka. He was paid a cash bonus every time he broke a world record. So he aimed to break his own previous record by the tiniest amount, rather than do his best jump. Gradually the bar crept upwards in successive tournaments until finally in the middle 1990s, when he past his best, he could no longer beat his previous performance. Had the incentives been structured differently he might well have produced even greater records early on. That is pretty much like the police being given the same reward for solving little crimes as for big ones.Now consider the case of a US supermarket, where two economists were given a complete run of data as to the speed at which different cashiers ran items through the tills. Some were fast, some slow, some accurate, some less so. Intriguingly, the slow ones speeded up if they were being watched by fast ones.

You might, in theory, have constructed a payment by results system, or perhaps put the fast ones into positions where they watched the slow ones and gave them a bonus for that. But the supermarket didn't. It reckoned that to do so would improve the scan rate but would result in more scanning mistakes and less time dealing with customer queries and complaints.

And there is a parallel with GPs. They do more scanning of patients because they are paid for that but they have no incentive to adapt their hours to suit their customers' needs.

There is a ray of hope here. I am not going to pretend that economists can fix everything and we have a great track record of getting macro-economic forecasts wrong. But economists have done a huge amount of work from around the world demonstrating how well-crafted incentives can improve performance, and conversely how badly-designed ones produce perverse results. Tim Harford's book goes far beyond targets and incentives. But there is surely a general message to policy-makers that governments should be very careful in their interventions, because they are liable to invoke the law of unexpected consequences. Look at the evidence; set broad objectives; then monitor progress by all means but basically let people get on with it. One other thing: don't lie about the statistics either.