It's the company you keep... the Duke's dangerous liaisons

Prince Andrew never was the smartest of the royals, so possibly it did not cross his mind that there was something weird about the photographs on the desk in Jeffrey Epstein's New York town house, or about the way he was left alone with a 17-year-old girl after being photographed with his arm around her waist.

But after his billionaire friend had admitted abusing a young girl, served a prison sentence and had been placed on the US sex offenders' register, you might think it would have penetrated the Duke of York's skull that this was a man to be avoided.

Now, belatedly, the duke is said to have acknowledged that it was "unwise" to maintain a friendship with a convicted paedophile, so there will be no more royal visits to Epstein's Florida mansion or New York town house – scenes of repeated abuse of under-age girls.

But whatever second thoughts the duke might have had, they are nothing compared with the outpouring of contrition from his ex-wife, Sarah Ferguson, over the £15,000 subsidy she accepted from Epstein last year.

"I personally, on behalf of myself, deeply regret that Jeffrey Epstein became involved in any way with me," she told the London Evening Standard yesterday. "I abhor paedophilia and any sexual abuse of children and know that this was a gigantic error of judgment on my behalf.

"I am just so contrite I cannot say. Whenever I can I will repay the money and will have nothing ever to do with Jeffrey Epstein ever again."

There is no suggestion the Prince knew about or was involved in Epstein's sordid sexual activities before the law caught up with him. But he flew to New York to spend four days as Epstein's guest as recently as last December, long after the law had caught up with the hedge fund manager. Andrew's office also knew that last year Epstein paid £15,000 to Johnny O'Sullivan, Sarah Ferguson's former personal assistant.

Accountants striving to sort out the financial affairs of the Duchess, whose lavish lifestyle had landed her £5 million in debt, were hoping to persuade her creditors to accept 25p in the pound, but Mr O'Sullivan was holding out for the full amount he was owed. Epstein helped the Duchess to restructure her debts by paying O'Sullivan £15,000.

It is difficult to see how the duke or his advisers could have failed to know by then what sort of man they were dealing with. The US authorities were alerted to Epstein's activities in 2005, when a woman contacted police in Palm Beach complaining that her 14-year-old daughter had been molested.

This triggered an investigation in which a large number of young women were asked about their encounters with Epstein. In June 2008, he pleaded guilty to procuring a person under the age of 18 for prostitution, and was sentenced to 18 months in jail and a year's house arrest. He was released in July 2009.

Another party in the story is Ghislaine Maxwell, the favourite child of the former publishing tycoon Robert Maxwell. After Maxwell drowned during a trip on his yacht The Lady Ghislaine in 1991, it emerged that he had looted pension funds under his control of about £500m.

His daughter moved to America to escape her father's notoriety, and became Epstein's inseparable companion. She introduced him to Prince Andrew, and on one occasion they were his guests at Sandringham.

Allegations of what Ghislaine Maxwell was prepared to do for her billionaire friend were detailed in an interview that 27-year-old Virginia Roberts gave to the Mail on Sunday, in which she claimed how, when she was 15, she was interviewed by Epstein, who was naked, and told by Ghislaine to massage his foot and then to strip. He then had sex with her, she claimed.

In the same interview, Ms Roberts described meeting Prince Andrew when she was 17, in New York in February 2001, when the duke was photographed with his arm around her waist. She claimed that behind Andrew was a desk covered in photographs of young women, including a an explicit one of her. "I don't think Andrew could have missed seeing it," she said. Later, the young woman was left alone with the duke at Ghislaine's house.

As details of the story emerged, it seems that people high up in the Government quietly decided that it was time for the Prince's role as a trade representative to be terminated.

It is not the only bad publicity that Prince Andrew has attracted recently. Among the many diplomatic cables published by WikiLeaks was one revealing how he shocked the US ambassador to Kyrgystan, Tatiana Gfoeller, by using "rude language à la British" to lay into the "idiocy" of the Serious Fraud Office and others who drew attention to corruption in some of the places he had visited to promote British trade links.

The duke has also had to deny that he was a "friend" of Colonel Gaddafi's son, Saif, although they have met. He also played host to the Tunisian dictator, Zine El Abidine Ben Ali, shortly before his overthrow. In 2007, the duke's marital home, Sunninghill Park, was sold for £15m – £3m more than the asking price – to an offshore company controlled by Timur Kulibayev, the billionaire son-in-law of the Kazakh president.

But by yesterday the Downing Street machine realised that if Prince Andrew is to continue acting as a UK special representative, he has to be seen to have the Government's backing. They pointed to the praise the duke has received from leading businessmen for his work.

The duke cannot be sacked because he was never appointed in the first place. When he left the Royal Navy in 2001, he volunteered to take over a role that his mother's cousin, the Duke of Kent, had performed for many years. He wants to carry on as if nothing has happened, and has a trip to Saudi Arabia planned. Ministers will be hoping the duke can keep out of trouble in the meantime.

Prince Andrew and friends

US

Jeffrey Epstein: A billionaire Wall Street financier and convicted child sex offender, Epstein struck up a close friendship with the Duke after the pair met 16 years ago. Epstein stayed at Sandringham and attended the Queen's birthday party in 2000, while Andrew paid visits to the Florida mansion where Epstein allegedly groomed girls for sex. The Duke only severed ties with Epstein at the weekend, despite his conviction in 2008. Epstein also paid £15,000 to help clear Sarah Ferguson's debts to her former PA.

Virginia Roberts: Now 27, Roberts was taken on by Epstein as a masseuse aged 15, and recently claimed to have been sexually exploited by the billionaire. The Duke was pictured with his arm around Roberts when she was 17, although there are no suggestions of any impropriety on his part.

Ghislaine Maxwell: The daughter of the disgraced media tycoon, Robert Maxwell, is Epstein's former girlfriend and the woman who introduced him to Andrew. She allegedly recruited Roberts to Epstein's employ and was a regular companion of the Duke's, apparently accompanying him to a 'hookers and pimps' party in New York as well as staying at Sandringham when Epstein visited. Born in Britain, she moved to New York after her father drowned off his yacht 'Lady Ghislaine' in 1991.

SAUDI ARABIA

Regular meetings with members of the Saudi royal family, who recently vowed to use all forces to quell public protests in the wake of demonstrations across the Arab world. According to WikiLeaks the Duke was overheard in Kyrgyzstan criticising the press over their investigation of a British arms deal with Saudi Arabia.

KAZAKHSTAN

Nursultan Nazarbayev: President of Kazakhstan, whose political relationship with Andrew became a friendship over their shared passion for goose-hunting.

Timur Kulibayev: The oil billionaire is the son in law of Nazarbayev. In 2007, he paid £15m - £3m more than the asking price - for Andrew's former marital home, Sunninghill Park, in Surrey.

Goga Ashkenazi: Stunning Kazakhstani-born Oxford graduate, ex wife of California hotel heir Stefan Ashkenazy and adored lover of Timur Kulibayev, Ashkenazi frequently plays host to Andrew on his trips to Kazakhstan, which range from golfing jaunts with Kilibayev to business trips in his official role as Britain's trade ambassador.

LIBYA

Tarek Kaituni: Convicted Libyan gun smuggler with whom Andrew enjoyed a four-day holiday in Tunisia in 2008 before visiting Colonel Gaddafi on a Foreign Office mission. The Duke stayed in a £480-a-night suite paid for by Kaituni, although he later reimbursed the cost.

Saif al-Islam: The son of Colonel Gaddafi, and one-time student at the London School of Economics, counts Prince Andrew as a friend. Andrew's contacts with the Libyan pair has caused former Foreign Office minister Chris Bryant to urge Cameron to 'dispense with the services' of the Prince.

AZERBAIJAN

Ilham Aliyev: Hardline President of oil-rich Azerbaijan, who in 2009 met Andrew with a view to establishing economic ties with Britain. Aliyev is accused by his opponents of electoral fraud, and has blocked media organisations including the BBC from broadcasting from his country. The Duke met Aliyev again at the Davos economic forum this year, while the Duke is also said to be friendly with his daughters, Leila and Arzu.

TUNISIA

Sakher el Materi: Son-in-law of recently deposed Tunisian President Zine al-Abidine Ben Ali.Materi attended a lunch held by the Duke at Buckingham Palace for British businesses keen to invest in Tunisia, just three months before the collapse of the regime. Materi has since been accused of money laundering.