The Government is more anti-EU than at any time in the last 20 years. So why are Eurosceptics still so gloomy?

The more ardent Eurosceptics are in a state of near-permanent alarm. But their gloom is genuinely baffling. They have won. Their victory is overwhelming. They should be celebrating with good French wine every night. Instead they are tortured by imagined setbacks, and remain neurotically alert to future defeats.

This week’s visit to the UK of Angela Merkel is one sign of how far the Eurosceptics have triumphed. The German Chancellor seeks some significant reforms to the EU. She wants the UK to remain part of the EU and will do what she can to make sure that this happens. Having said that, she will only do what she also believes is right for the EU and what is politically feasible for her.

Like the leaders of the three main UK parties, Merkel agrees that the EU could be more efficient, that it could do less in some areas while focusing more on ways of achieving economic growth. There are plenty of other EU members that have reached the same conclusion. Indeed it is hard for any member to argue in favour of inefficiency and against measures that promote growth. Reform is on the table because it is in other countries’ interests as well as the UK’s.

What Merkel will not do is take big risks if it looks as if the UK is about to leave the EU, or if Cameron cannot win over most of his party to support a package of reforms in the context of the UK remaining a part of it. This is why the relationship between the two leaders is more ambiguous than Downing Street’s briefings suggest.

Merkel agreed with some of Cameron’s major speech on Europe delivered at the beginning of last year, but found parts of it baffling too – the elements that opened the door to withdrawal. Now Cameron has the nightmarish task of reassuring Merkel and other EU leaders that it is worth them going as far as they can to meet his demands, while reassuring his Eurosceptics here that the option of leaving remains open.

The Eurosceptics are in constant need of such reassurance, but it is the pro-Europeans who should be the ones who require comforting balm. In the mid‑1990s I watched the Maastricht debates in the House of Commons – brilliant, vivid exchanges and knife-edge votes that nearly brought down a Prime Minister. During the debates, Eurosceptics insisted that Britain must not join the euro and yearned for a wider EU so that the Franco-German dominance would be permanently undermined. They have achieved all that they wanted. Britain is in a strong position outside the euro but also part of a wider EU.

Step back a moment and the basic, much-loathed structure of the EU is unavoidable. Take the single market, which most Eurosceptics support. In order for the single market to work, some rules are necessary. Bureaucrats are required to administer the rules, but no one would want non-elected officials to decide what form the market should take. This is where elected politicians are required – figures who are accountable to electorates.

There are far too many bureaucrats, and arguably too many politicians with blurred lines of accountability. But here is the twist. Leaders like Merkel recognise this too. Reform of the bloated EU – to make it more nimble and accountable – is achievable. Britain is not alone in wanting this.

The Eurosceptics are on the verge of another victory. If that is not enough for Eurosceptics to raise a glass, consider what has happened within the UK since they argued successfully for their country to stay out of the euro and for a wider EU. In the mid-1990s Michael Heseltine was Deputy Prime Minister, Douglas Hurd was Foreign Secretary and Ken Clarke was Chancellor. All three were pro-Europeans. Now William Hague is Foreign Secretary and George Osborne is Chancellor, both of them lifelong Eurosceptics.

In the mid-1990s Tony Blair was the Labour leader who chose Europe as his mission. For a long time he saw it as his historic role to lead Britain into the euro. Now Labour is led by Ed Miliband and Ed Balls, two figures who were never dewy‑eyed about Europe but who were, and are, more forensically expedient reformers. Even the Lib Dems’ Danny Alexander, who once worked for Britain in Europe, openly admits he was wrong about the euro. The Eurosceptics have to make no such humble admission. Only the ardently pro‑Europeans have cause for apologetic agony.

But it is the victorious Tory Eurosceptics who agonise. Following on from the financial crisis they face a deep challenge to their strongly held assumptions – about the state, about regulation and about markets. On these matters they are relaxed. On Europe they have won but are still far from relaxed. Merkel’s supportiveness shows how many cards they have to play. But their obsessiveness makes it much harder for Merkel to be supportive.

Politics impacts on voters’ lives, but voters are disinclined to make the good and bad connections. It is more fun and fashionable to side with Russell Brand and damn the distant “Westminster village”, a vacuous but dangerous label. Last week’s upbeat reports in The Independent from Andrew Adonis about his bus tour across London were an implicit defence of politics.

Adonis celebrated the massive expansion of the bus network, but noted that more action was required. The improvements, and potential further improvements, arise because London has a Mayor accountable to the electorate. The previous government legislated for this change. The government before that one abolished the old GLC, and for years there was nothing the voters could do to bring about improvements to a dire transport system, one that made life hellish for many Londoners.

It might be tedious to follow the twists and turns of party politics but they are more central to voters’ lives than the fate, say, of their football teams. Even after 1997, when New Labour was fearfully timid on many fronts, there were profound differences between the parties. One was over whether a capital city needed its own elected representative. It got one. Ken Livingstone introduced the congestion charge that paid for the expansion of buses. Though not as pro-active, Boris Johnson has continued to push for improvements. There is a direct connection between the loathed Westminster village and every additional bus journey in London.

Twitter: @steverichards14