Now the challenge is to prove that the 'war on terror' is not a war on Iraq

How could the the Americans - and some of the British - have so thoroughly underestimated the difficulties of occupying Iraq, when they first looked forward to being welcomed in the streets, and instead were faced by ambushes, car bombs and multiplying terrorists?

How could the the Americans - and some of the British - have so thoroughly underestimated the difficulties of occupying Iraq, when they first looked forward to being welcomed in the streets, and instead were faced by ambushes, car bombs and multiplying terrorists?

Lord Butler's commission, when it reports next week, should throw some light on the failures of intelligence as does yesterday's report by the US Senate Intelligence Committee. But they are about the intelligence leading up to war. Now that the occupation has ended, it is important to look at the more fundamental failure of the coalition: to foresee that the invasion would stimulate a resurgence of nationalism and patriotic loyalty, which would unite ordinary Iraqis against the occupiers.

It took a long time to sink in. When the Americans faced growing attacks which they had not expected, they too readily blamed them all on foreign terrorists, trained by al-Qa'ida or other fundamentalists groups, streaming into Iraq, without recognising they were being protected by many Iraqi civilians.

At first, they explained that this would really benefit the war on terrorism, since it was attracting all the perpetrators in one country, where they could more easily be tracked down and dealt with - what the Pentagon called the "fly-paper" theory.

But that argument was soon discredited, as the flies were not caught, while others were appearing and multiplying in other countries. "Now the flies are all over the Middle East, and also in Europe," one intelligence expert told me last week.

There was also a more worrying argument, with the same insect imagery: the "hornets' nest" theory, to which Tony Blair has referred, which argued that the occupation of Iraq would stir up trouble all through the Muslim world. And that theory looked much more credible as the violence increased.

And there was a still more worrying argument, the "tar baby" theory as it was called in Washington, which compared Iraq to the tar baby which Brer Fox made in the Uncle Remus story, to attract Brer Rabbit who hit it and then got stuck to it. And that is now the worst nightmare: the rebels in Iraq providing a continuing danger which can keep coalition troops in the country far longer than they intended.

Whatever the theory now, the occupation showed a serious lack of political understanding and information. It provoked a far wider resistance than the generals or politicians expected, and it enjoyed the support or protection of many quite ordinary Iraqis. Today, the leaders of the coalition still do not know who is really behind the well-organised resistance which is continuing to devastate Iraq, even now that it is being formally run by the Iraqis themselves.

Yet the extent of the resistance is hardly surprising. It is a common historical phenomenon that people who are invaded by a foreign power, even if they are liberated from a tyrant, are provoked into fierce patriotism.

As the UN representative Lakhdar Brahimi said to me when he left last month: "There are many Iraqis who are patriots, or alienated or outraged by the behaviour of the foreign occupiers - but who should not be called terrorists." And he added: "I've been arguing against foreigners thinking of themselves as saviours who think local people are a bunch of idiots waiting for clever solutions to their problems."

Why did the Americans and British not realise this simple reaction? The most obvious answer is that they lived in countries which - almost alone among Western nations - had never been occupied. The French, Germans or Dutch, not to mention most of the people in the Middle East, had vivid memories of occupation and their response to the domination and humiliation under a foreign power. But the experience was quite foreign to Americans and British who had only encountered their enemies outside their shores.

They could not easily imagine how an invasion by foreigners could stimulate national pride and anger among quite peaceable people, who would be prepared to give protection to rebels even if they did not fully approve of their methods, and would fight with extraordinary persistence and ingenuity once their own homeland was occupied.

If we try to visualise such an invasion - if Britain, for instance, had been invaded by the German army in the Second World War - we could begin to comprehend the ferocity of resisters and also the natural advantage of people who defend their own territory which they knew far better than foreigners.

The British in the time of the empire had learnt, at great cost, how fiercely their enemies would fight to defend their homeland. They learnt it most painfully during the Boer War a century ago, when Afrikaners knew exactly where to ambush British soldiers in a ravine, or hide in farmhouses on the veldt where ordinary farmers would risk their lives by protecting them.

Occupations, as in South Africa, could always provoke a nationalism and patriotism which was much less evident than before. The British empire would help to create nationalisms all over Africa and the Middle East among people who had previously had little sense of belonging to nations.

The Americans had many fewer historical memories of resistance to occupation. George Bush, when he tried to justify the war in Iraq, liked to quote the post-war occupation of Germany and Japan to show how the victorious powers could transform dictatorships into effective democracies, and win the gratitude of their people.

But there was no real comparison, for the German and Japanese armies had been decisively beaten in a long and devastating war which left little ambition for rebellion or resistance after the peace. I was a young naval officer in the post-war occupation of Germany, and I could see no signs of any impending revolt among the impoverished and half-starved people: after Hitler's suicide, there was little nostalgia for the Nazi ideology which had ruined their future.

The Allies soon appeared more benign as they helped to rebuild the German economy; having first insisted on "deNazification", to root out any Germans who had belonged to the Nazi party, they soon abandoned the policy, when they realised that the party had included many of the ablest Germans who had been in key positions, who were now crucial to the recovery - like Iraqis who had been sacked because they were members of the Baath party.

Saddam Hussein, like Hitler, was decisively defeated, but the Iraqi people could not have the same sense of defeat as the Germans; the occupation had increased their sense of a nation, with a distinctive Arab culture. They could not look towards Western democracy with the same hope as the post-war Germans.

This is the real challenge which faces the Americans and British as they help to rebuild Iraq before next year's elections: how to show ordinary Iraqis, after the miseries and mistakes of a year's occupation, that Western democracy means more than tanks patrolling the streets and prisoners being tortured; and that the "war on terrorism" has not been a war against their own nation.