Patrick Cockburn: A crucial US ally against Middle East terrorism or a safe haven for al-Qa'ida?

The US is expressing worry that the struggle for power in Yemen will give opportunities to al-Qa'ida in the Arabian Peninsula (AQAP), which is based in Yemen. American officials say they fear that the enforced departure of President Ali Abdullah Saleh deprives them of a staunch ally against AQAP. Last year he was happily permitting the US to carry out air strikes in Yemen and claiming that the actions were by Yemen's armed forces.

There was an element of manipulation in this. The government gave the local franchise of al-Qa'ida room to operate, but used the AQAP threat to extract aid and money from the US. In reality, there has been a long-term understanding between Mr Saleh and Islamic extremists, if not AQAP. For instance, the government supported al-Iman University, which is a centre for violent Islamic fundamentalists.

Yemen is an ideal hideout for any small band of extremists. It is three-quarters the size of Texas, is mountainous and has a limited system of roads. The state's authority is variable and in many areas tribal authority is supreme. It is hardly surprising that AQAP, with an estimated 300 members locally, does not find it difficult to hide.

The problem for Washington is not that AQAP is particularly effective but that since 9/11 even the most unsuccessful plot, such as the Christmas 2009 attempted bombing of a plane over Detroit, produces a wave of anxiety in the US. The US government has to be seen to be doing something about it.

The fact that the would-be bomber, Nigerian student Umar Farouk Abdulmutallab, claimed that he received training in Yemen, meant that AQAP was, much to its delight, perceived internationally as a serious threat. Al-Qa'ida has long had a presence in Yemen. Osama bin Laden's family has roots in the Hadramaut, the part of southern Yemen that fronts the Indian Ocean. But it was only in 2000 that the organisation staged a spectacular attack there when a powerboat packed with explosives rammed the USS Cole, a naval destroyer, in Aden harbour, killing 17 American sailors.

In the years following 9/11, the US focus was on Afghanistan and Iraq and not on Yemen. But in 2006 some 23 al-Qa'ida suspects escaped from jail and started to rebuild the organisation. They were reinforced by militants fleeing from Saudi Arabia and the two groups merged in 2009 to form AQAP.

The number of attacks either in or launched from Yemen increased by the year. Some of these showed ingenuity, such as the attempt to assassinate the Saudi Prince Mohammed bin Nayef. There were bomb attacks on the US, British and Italian embassies in the capital, Sana'a. But it was the Yemeni connection to incidents in the US that attracted most attention. The American imam Anwar al-Awlaki – well-educated, persuasive and English-speaking – had been in touch with the US officer at Fort Hood who killed 13 soldiers.

The US responded with drone attacks and special forces units which targeted al-Qa'ida militants, but could not eliminate them. It is this type of operation that is now in doubt. But it is unlikely that Yemen will subside into war. Any supposed relaxation of state authority after Mr Saleh has stepped down will not make much difference because of the longstanding weakness of the government.