Patrick Cockburn: Threats to Yemen prove America hasn't learned the lesson of history

We are the Awaleq

Born of bitterness

We are the nails that go into the rock

We are the sparks of hell

He who defies us will be burned

This is the tribal chant of the powerful Awaleq tribe of Yemen, in which they bid defiance to the world. Its angry tone conveys the flavour of Yemeni life and it should give pause to those in the US who blithely suggest greater American involvement in Yemen in the wake of the attempt to destroy a US plane by a Nigerian student who says he received training there.

Yemen has always been a dangerous place. Wonderfully beautiful, the mountainous north of the country is guerrilla paradise. The Yemenis are exceptionally hospitable, though this has its limits. For instance, the Kazam tribe east of Aden are generous to passing strangers, but deem the laws of hospitality to lapse when the stranger leaves their tribal territory, at which time he becomes "a good back to shoot at".

The Awaleq and Kazam tribes are not exotic survivals on the margins of Yemeni society but are both politically important and influential. The strength of the central government in the capital, Sanaa, is limited and it generally avoids direct confrontations with tribal confederations, tribes, clans and powerful families. Almost everybody has a gun, usually at least an AK-47 assault rifle, but tribesmen often own heavier armament.

I have always loved the country. It is physically very beautiful with cut stone villages perched on mountain tops on the sides of which are cut hundreds of terraces, making the country look like an exaggerated Tuscan landscape. Yemenis are intelligent, humorous, sociable and democratic, infinitely preferable as company to the arrogant and ignorant playboys of the Arab oil states in the rest of the Arabian Peninsula.

It is very much a country of direct action. Once when I was there a Chinese engineer was kidnapped as he drove along the main road linking Sanaa to Aden. The motives of the kidnappers were peculiar. It turned out they came from a bee-keeping tribe (Yemen is famous for its honey) whose bees live in hives inside hollow logs placed on metal stilts to protect them from ants. The police had raided the tribe's village and had damaged hives for which the owners were demanding compensation. The government had been slow in paying up so the tribesmen had decided to draw attention to their grievance by kidnapping the next foreigner on the main road and this turned out to be the Chinese engineer.

Yemen is a mosaic of conflicting authorities, though this authority may be confined to a few villages. Larger communities include the Shia around Sanaa in the north of the country near Saada, with whom the government has been fighting a fierce little civil war. The unification of North and South Yemen in 1990 has never wholly gelled and the government is wary of southern secessionism. Its ability to buy off its opponents is also under threat as oil revenues fall, with the few oilfields beginning to run dry.

It is in this fascinating but dangerous land that President Barack Obama is planning to increase US political and military involvement. Joint operations will be carried out by the US and Yemeni military. There will be American drone attacks on hamlets where al-Qa'ida supposedly has its bases.

There is ominous use by American politicians and commentators of the phrase "failed state" in relation to Yemen, as if this some how legitimised foreign intervention. It is extraordinary that the US political elite has never taken on board that its greatest defeats have been in just such "failed states"', not least Lebanon in 1982, when 240 US Marines were blown up; Somalia in the early 1990s when the body of a US helicopter pilot was dragged through the streets; Iraq after the overthrow of Saddam Hussein; and Afghanistan after the supposed fall of the Taliban.

Yemen has all the explosive ingredients of Lebanon, Somalia, Iraq and Afghanistan. But the arch-hawk Senator Joe Lieberman, chairman of the Senate Committee on Homeland Security, was happily confirming this week that the Green Berets and the US Special Forces are already there. He cited with approval an American official in Sanaa as telling him that, "Iraq was yesterday's war. Afghanistan is today's war. If you don't act pre-emptively Yemen will be tomorrow's war." In practice pre-emptive strikes are likely to bring a US military entanglement in Yemen even closer.

The US will get entangled because the Yemeni government will want to manipulate US action in its own interests and to preserve its wilting authority. It has long been trying to portray the Shia rebels in north Yemen as Iranian cats-paws in order to secure American and Saudi support. Al-Qa'ida in the Arabian Peninsula (AQAP) probably only has a few hundred activists in Yemen, but the government of long time Yemeni President Ali Abdulah Salih will portray his diverse opponents as somehow linked to al-Qa'ida.

In Yemen the US will be intervening on one side in a country which is always in danger of sliding into a civil war. This has happened before. In Iraq the US was the supporter of the Shia Arabs and Kurds against the Sunni Arabs. In Afghanistan it is the ally of the Tajiks, Uzbeks and Hazara against the Pashtun community. Whatever the intentions of Washington, its participation in these civil conflicts destabilises the country because one side becomes labelled as the quisling supporter of a foreign invader. Communal and nationalist antipathies combine to create a lethal blend.

Despite sectarian, ethnic and tribal loyalties in the countries where the US has intervened in the Middle East, they usually have a strong sense of national identity. Yemenis are highly conscious of their own nationality and their identity as Arabs. One of the reasons the country is so miserably poor, with almost half its 22 million people trying to live on $2 a day, is that in 1990 Yemen refused to join the war against Iraq and Saudi Arabia consequently expelled 850,000 Yemeni workers.

It is extraordinary to see the US begin to make the same mistakes in Yemen as it previously made in Afghanistan and Iraq. What it is doing is much to al-Qa'ida's advantage. The real strength of al-Qa'ida is not that it can "train" a fanatical Nigerian student to sew explosives into his underpants, but that it can provoke an exaggerated US response to every botched attack. Al-Qa'ida leaders openly admitted at the time of 9/11 that the aim of such operations is to provoke the US into direct military intervention in Muslim countries.

In Yemen the US is walking into the al-Qa'ida trap. Once there it will face the same dilemma it faces in Iraq and Afghanistan. It became impossible to exit these conflicts because the loss of face would be too great. Just as Washington saved banks and insurance giants from bankruptcy in 2008 because they were "too big to fail," so these wars become too important to lose because to do so would damage the US claim to be the sole superpower.

In Iraq the US is getting out more easily than seemed likely at one stage because Washington has persuaded Americans that they won a non-existent success. The ultimate US exit from Afghanistan may eventually be along very similar lines. But the danger of claiming spurious victories is that such distortions of history make it impossible for the US to learn from past mistakes and instead it repeats them by fresh interventions in countries like Yemen.