When Joseph Ratzinger was elected Pope in 2005 the choice divided Catholics. Traditionalists cheered the appointment of a man who as a cardinal, had for two decades ruled the Vatican’s doctrinal watchdog with a mitre of iron. Liberals feared that the conclave had made the most divisive choice imaginable.

But Pope Benedict XVI turned out to be a pope of surprises – a tradition he maintained with his shock decision to resign, something no Pope has done for almost 600 years.

The surprises began when he issued his first major teaching document. The encyclical’s subject – love – was not what had been expected from a dogmatic hardliner. From the outset he understood as Pope he had to make a gear change to a much more pastoral and inclusive approach.

For the wider public that shift did not become evident until his visit to the UK in 2010. Shriller secularists predicted the man once known as God’s Rotweiller would not be well-received with his uncompromising views on society’s rampant materialism and moral relativism. But there was a gentleness about the way this German shepherd stated his religious certainties which showed a willingness to open up a dialogue with the secular world.

Benedict XVI’s most striking quality as Pope has been his thoughtful, cultured, deeply-read intellect which has enriched the dialogue with even those who disagree with his religious conservatism. He insisted faith and reason were not at loggerheads but that the history of Western Christianity showed that the world of secular rationality and the world of religious belief each add something to the other.

Early on he did not understand the impact he might have by such thinking aloud. One of his first public forays was to return to his old university at Regensberg with a lecture on Faith and Reason which carelessly quoted a medieval attack on Islam. Remarks which went unnoticed from an academic theologian, he learned, could cause riots and murder across the world when they came from a pontiff.

But by the time he spoke to Britain’s civic leaders in Westminster Hall there was a humility to his acceptance of the need for dialogue between church and state. Governments should balance the freedom of individuals with the best interests of the whole society, he warned. Short-term politically-pragmatic freedoms could have unintended and harmful consequences in complex social and ethical situations. That culture had led to the recent global financial crisis and social injustices in both the rich and poor worlds.

A society which judges right only by social consensus, without reference to moral absolutes, risks being enslaved by a dictatorship of relativism. The majority are not always right, as he learned living in a Germany which was complicit with Hitler’s persecution of the Jews. Politics needs the moral insights of faith, just as religion needs reason to prevent it from descending into sectarianism and fundamentalism, he said. There was a gentle wisdom to Benedict XVI which even attentive sceptics found thought-provoking.

The news headlines, however, concentrated on more sensationalist issues throughout Benedict’s eight years in office. The Church’s continued opposition to abortion and end-of-life issues, along with its intransigence on gay relationships, and its refusal to allow the distribution of condoms in Africa to married couples where one partner has Aids, alienated it from secular society. Benedict XVI was immovable on that. But on the issue of paedophile priests he received a more unfair press.

Though it emerged that when he was Archbishop of Munich in the 1970s a known molester was quietly re-assigned to duties that allowed him to resume contact with young people, behind the scenes at the Congregation for the Doctrine of the Faith he cracked down on what he privately described as the “filth” of priest abusers. He pressed his reluctant predecessor, John Paul II, to take a firmer line.

He introduced fast-track internal trials, extended abuse investigations to victims up to the age of 18 and penalised priests guilty of internet offences against children. He also removed the statute of limitations on offences from many years ago. When he became Pope he immediately took action against the founder of the Legionnaries of Christ whom Pope John Paul had protected.

But he did all this in private using canon law rather than reporting offenders to the police, compounding the culture of secrecy for which victims criticised the Church.

Perhaps Benedict’s greatest weakness was in this loyalty to the church’s old clandestine self-protective culture. For all his rhetorical embrace of the importance of the Second Vatican Council which revolutionised Catholicism in the 1960s, in practice he seemed intent on rowing back on its reforms. He upset the Church of England by creating an Ordinariate to poach disgruntled Anglicans. He has begun the appointment of a new generation of conservative bishops in the UK, and elsewhere. And he has created many cardinals of the same ilk who will now elect his successor.

Cardinal Carlo Maria Martini, the Jesuit who might well have been pope had Pope John Paul died earlier, gave an interview on his deathbed last year, which offered a withering critique of Benedict’s closed hierarchical Church. It was empty, bureaucratic, pompous and “200 years out of date”, he said. “The child sex scandals oblige us to undertake a journey of transformation,” he said. “The Church must admit its mistakes and begin a radical change, starting from the Pope”.

“After a fat pope, a thin pope,” the Italians say. After two conservatives, a more progressive pope? That is unlikely, though we may after centuries of white popes get a black one; two African cardinals and a Latin American are among the favourites.

Whoever succeeds, it will be to a papacy which Benedict XVI, through his unprecedented resignation, has subtly changed. By not dying in office the man who, at 78 was one of the oldest new popes in history when elected, has set aside the old idea that the papacy was a vocation unto death. It is now a ministry which can be set aside when appropriate. That will influence the way that Popes henceforth are viewed. The decision to resign may turn out to be the most modernising act Benedict XVI has ever committed.

Q&A: Life after the Vatican for Pope Benedict

Q: What now for Benedict XVI?

A: First off, he’ll have to find a new place to live. He currently resides in the Papal Apartment, inside the Papal Palace building in the Vatican. The residence comes with the job, so temporarily he will move to Castel Gandolfo, a small town 15 miles outside Rome that is the official summer residence of the Pope. But once the new man is appointed he will soon leave.

Q: So what about a new permanent home?

A: A Vatican spokesman said yesterday that: “When renovation work on the monastery of cloistered nuns inside the Vatican is complete, the Holy Father will move there for a period of prayer and reflection.” This would prove a problem without precedent for whoever takes over – having the previous incumbent casting a shadow over your tenure is not something any other pope has had to deal with. Longer term, some experts speculate he may move to a monastery in Switzerland or Germany.

Q: What will he be called?

A: Once he’s no longer Pope, Cardinal Ratzinger once again. He will also no longer be referred to as ‘his holiness’. But the prefix cardinal denotes someone who is active in the church, and it would be highly unusual for someone who was once the Pope to remain active once he has abdicated. What is for certain is that he will not be involved in the selection of his successor – only cardinals aged under 80 can participate, and Benedict is 85.

Q: Did anyone know he was going to quit?

A: The Archbishop of Canterbury, Rowan Williams, says he has known since Christmas. Yesterday the Archbishop of Rio, Orani Tempesta, revealed that when a date was fixed for a papal visit to Brazil this year for World Youth Day, Benedict XVI replied: “The Pope will go to World Youth Day, whether it is me or my successor.” A new pope often follows arrangements made for his predecessor, so a trip to South America could be on.