The rebel city Gaddafi says is full of recruits for al-Qa'ida

The brutal history of Derna is recorded in the portraits recently hung on the walls of its oldest mosque. Many of the faces staring down died in this year's uprising; some were conscripts in Colonel Gaddafi's African wars; others the victims of a notorious prison massacre in Tripoli 15 years ago.

A haunting collage freed from the files of internal security when it was ransacked two months ago shows young men who were shot, poisoned, hanged or hacked to death after an uprising against the repressive regime in 1996.

Ayman Burwag has his place on the wall of martyrs. He sits under rows of faces of the 1,200 men shot dead at Abu Salim prison on the orders of Colonel Gaddafi. His portrait shows a friendly young man in a check shirt superimposed on a scene from Mecca. He didn't die in the prison courtyard in Tripoli but thousands of miles away in Iraq in 2005. The 27-year-old was one of scores of young men who left this blighted port and went to die in Iraq.

When US soldiers stormed an insurgents' headquarters in the northern Iraqi town of Sinjar four years ago, Derna became notorious. Among the documents recovered was a list of 606 foreign fighters – 52 of them from this single Libyan town. Even more striking was that nearly nine out of 10 of them had volunteered for martyrdom missions. The "Sinjah list" has been used to portray Derna as an outpost of al-Qa'ida and contributed to concerns – stoked by Gaddafi – that Nato may be formenting a radical Islamic movement by supporting the rebels in Libya.

But Ayman's brothers – Sohel and Abdelgader – don't remember him as a radical and insist he had nothing to do with al-Qa'ida or Osama bin Laden. They remember a practical joker, a fan of Brazilian football, who had been engaged for three years and was saving money to get married. Ayman didn't tell his brothers that he was leaving. When he sold his car and didn't come home for dinner one night they guessed where he had gone.

"There were a lot of young men who went to Iraq from Derna," said Sohel. When he called his family from Iraq, the schoolteacher said that the pictures from Abu Ghraib had pushed him to go. "We begged him to come back but he said it was his duty as a Muslim." Several months later, an Iraqi man called Hussein rang to say that Ayman was shaheed; he had become a martyr.

The Burwags' cousins are now fighting on the front line against Colonel Gaddafi's forces. If they die, as many young boys from Derna already have, their faces will join Ayman's on the wall at the Asahaba Mosque. Derna has appeared regularly in the Libyan leader's wild rants against the rebels. His claims that they are drug-fuelled agents of al-Qa'ida may seem absurd but similar assertions about the city's population were, until recently, taken seriously by the countries now bombing the regime.

From a distance, Derna seems beautiful, stretched along the shore of the Mediterranean in the shadow of the Green Mountain. Up close, its decline is stark, the product of what locals call "systematic neglect". Apartment blocks crumble into derelict streets strewn with rubbish. Unemployment runs at 40 per cent.

At the entrance to the city, the giant concrete numerals 1805 remind visitors of what Colonel Gaddafi calls the "First US-Libya war" (the First Barbary War), during which pirates had seized the USS Philadelphia in Tripoli. The turning point in the war came when a small group of US marines and 500 mercenaries trekked from Egypt to assault Derna, raising the American flag over a foreign city for the first time.

Sitting in the Philadelphia cafe on the seafront, Abdulkarim Bentaher said the importance of those events had been overstated to provide a convenient narrative of anti-Americanism for the regime: "Derna has a longer history that Colonel Gaddafi doesn't talk about." An English professor who graduated from Columbia University, he points to the other tradition of education. "Under the Ottomans, we sent our children to Istanbul to be educated. Under King Idriss, most of the teachers in Libya came from Derna."

When a young army lieutenant called Muammar Gaddafi overthrew the king 42 years ago, the first protest against his "revolution" came from Derna. Bentaher remembers it well because his brother was paralysed by a bullet during the crackdown. It was the beginning of a bitter relationship with power that gradually transformed Libya's third largest city into a ghetto.

The blackened ruins of the five security agencies the regime used to control Derna, burned by residents in the current uprising, hardly look out of place among the run-down tenements where the people live. This is a town that knows it has an image problem. Since the 17 February uprising, university students have daubed the walls with slogans in foreign languages such as: "Yes to Pluralism", "We are freedom fighters not terrorists" and "No to Qaeda".

"Since 1970, the people here have said no to Gaddafi," said Abdelbaset Suleiman, a local leader of the Interim Transitional National Council set up by the rebels to govern Eastern Libya. "We have lived through oppression. We had no freedom. Our youth went to prison."

There have been no new buildings since 1970, he said. Businesses have been starved of government support and forced to close. "Those who went to Iraq were looking for any place to avoid this hell, even if it killed them."

It almost certainly killed Ashraf al-Hasadi, 18. The youngest of four brothers, he watched television coverage of Iraq for hours and started going more often to the mosque. His brothers, Sofian and Abdelhakim, became worried: "A lot of guys were going to Iraq and many of his close friends went," said Sofian. There were two phone calls, one from Damascus in Syria and then one from Baghdad asking for "forgiveness". Then nothing.

His brothers make no apology for Ashraf. "He went to fight the Americans in Iraq. This is jihad. They had no business being there," said Sofian. But the Hasadi brothers say they are moderate Muslims, that they don't mourn the death of Osama bin Laden and hope it will end the US war on terror.

For now, the family has new martyrs in what they say is a more important jihad on the home front: Abdulatif al-Hasadi, 24, and Moatez al-Hasadi, 25, both died last month fighting Colonel Gaddafi's forces in Bin Jawad and Ajdabiya. "The people who went to Iraq were in too much of a hurry. If they waited, the jihad was coming here," said Sofian.

Despite Colonel Gaddafi's tirades against al-Qa'ida in Derna, many here remember that it was the regime that opened offices at the local university offering to pay students' passage to go and fight the Americans in Iraq. And most children in the city were delivered by Catholic midwives. Half a dozen Italian nuns have lived here peacefully as long as anyone can remember. Mother Superior Celeste, 74, from Vicenza, snorts at the notion that the rest of the world should be afraid of Derna or that her adopted home was a hotbed of Islamic radicalism.

"Gaddafi speaks about al-Qa'ida. But he is the terrorist. He has attacked Derna many times, attacked the houses, taken the children in the night."

She rattles off a list of the city's ailments: no work, no schools, no proper healthcare, no sport, no money. Some of the young men had chosen to die, she said, because they thought: "We are dying anyway; let us go to die in Iraq."