Caroline Lucas: There's light at the end of the tunnel

To find Dr Caroline Lucas, the first-ever official leader of the Green Party, you catch an environmentally friendly train or Tube to London Bridge station, and then take a carbon-neutral walk for the couple of hundred yards to the old Hop Exchange on Southwark Street.

Close up, from the pavement, the listed building seems past its best. But stepping through its iron gates reveals a breathtaking atrium of white walls, curved windows and green iron galleries, with a glass roof above the tiled floor where merchants traded the bitter fruit.

"What a wonderful place to work!" I greeted her early on Thursday, reaching her cramped offices off one of the galleries.

She is less enthusiastic. "It's charming, but impractical."

Just like the Greens?

"Not any longer," she retorts. And, indeed, after a day with her on the campaign trail, I am inclined to agree. Her party is changing, growing up, even – and Lucas is largely responsible. Partly as a result, it has its best chance of electoral triumph since exactly 20 years ago, when – to everyone's astonishment, including its own – it secured 2,295,695 votes, more than one in every seven cast, in the 1989 European elections.

It has dramatically changed its pitch; once unashamedly against economic growth, it now presents its policies as the best way to revive the economy. Its advertising campaign urges voters who believe they knew the party to "think again". And, unrepresented in Parliament, it is expected to reap dividends from public disgust at the MPs' expenses scandal.

Mind you, change was badly needed, for I was being too kind. For most of its 36-year history, the UK Green Party – Europe's first – has been not so much impractical as implausible, immature, frequently impossible and occasionally imbecilic. In truth, it has also been charmless, constantly fractured and repeatedly purging prominent members who achieved public appeal.

But more of that later. Now, an hour and a half before her European campaign launch, Lucas is predicting a new dawn. "It's a really exciting time for us," she says. "People are looking for alternatives. The three other parties are collected around an ever-smaller centre ground. A whole area of the political spectrum is completely empty, and it's an area where we feel we are the natural occupiers."

And that's not just on the environment, she adds ("though none of the other parties are getting anywhere close to dealing with the climate crisis"), but also on "social justice". Indeed, the manifesto blends drastic environmental measures – such as reducing Britain's carbon emissions by 10 per cent a year – with traditional left-wing policies such as building council houses. (See box.)

She goes on: "People are wanting something fresher and more authentic and a bit of passion in politics. When you see all these MPs from the other parties hanging their heads in shame over the expenses scandal, there's a very dangerous – and wrong – sense that you can't trust anyone in politics. So trying to reinvigorate the whole political system is important. There's a real new focus in the party. We do feel we are on the edge of a breakthrough."

Unfortunately, not a lot of people seem to have noticed. The surprisingly disciplined manifesto launch – with besuited candidates giving short and effective speeches – was held in a tiny, but still-half empty, room in Westminster, mainly peopled with Green Party activists. Lucas went on to appear on BBC2's The Daily Politics, but was squeezed into just a few minutes and not questioned about the party's main policies.

There was worse that evening when she campaigned in the Sussex village of Forest Row – natural territory, full of chiropractors, health food shops and superannuated children of the 1960s. The streets were almost deserted, and few stopped to chat, though she had addressed a packed meeting there only a couple of weeks ago. The campaigners ended up talking to each other in the shade of their "battle bus", an old Routemaster fuelled with recycled cooking oil.

Lucas gets frustrated, but at 48 she is well used to the ups and downs of Green politics. A doctor of literature, she read a book by Jonathon Porritt, himself a former luminary of the party, Seeing Green, in 1986. "It was one of those life-changing experiences, like a light coming on," she said. "I turned the book over and saw that the Green Party office was then based on the Clapham High Road. And I was living in Clapham at the time, so this seemed to be a sign.

"I marched out of my bedsit and up and down Clapham High Road looking for what I fondly imagined would be the large offices of the Green Party. I couldn't find anything, but eventually came across a sort of broom cupboard behind a Chinese restaurant. I walked in, volunteered and, over time, became their first paid press officer."

She then decided that she wanted to stand as a candidate, so left the party's employ and went to work for Oxfam, where she was elected to Oxfordshire County Council, the first Green ever to be elected to such a position. But in the 1992 general election the party garnered only 171,000 votes, and some prominent members suggested it should abandon electoral politics altogether. Membership – which topped 18,000 in the days of elderflower wine and organic roses after the 1989 vote – slumped to 5,000.

Yet in 1999, the introduction of proportional representation in the European elections enabled her to be one of the first two British Green MEPs. (She squeaked in for the South-east by 256 votes.) And now she has a fighting chance of being the first MP.

And therein lies a tale. Keith Taylor, a local councillor, got an unprecedented 22 per cent of the vote in Brighton Pavilion in 2005, and wanted to stand again at the next general election. Lucas challenged him for the candidacy, and won a closely fought vote. She rejects a charge of carpetbagging, saying that she was invited to stand by local members. But it says something about the party's new professionalism, even ruthlessness, in selecting its most electable candidate for its best prospect of victory.

Her election as leader last September was also, as she says, a "momentous" event for the party, as it had always rejected any idea of leadership. But she stresses that she will remain accountable, having to stand for re-election every two years, and has "no privileged position over policy-making". She adds: "The models that the other parties have used where you have a very powerful leader squashing any kind of independent ideas from the grass roots is not very attractive to the electorate."

Professor Paul Ekins, one of Britain's top Green economists and once a senior party figure, says: "Caroline has ridden the whirlwind." For attempting to lead the Greens had long been a suicide mission.

The party started, appropriately enough, with a bunny-hugger: Hugh Hefner. In 1974, he printed an article in Playboy by the doom-watching Professor Paul Ehrlich, predicting imminent disaster. A Warwickshire solicitor, Tony Whittaker, read it and was inspired. He advertised in the Coventry Evening Telegraph and got together a group to found the party, but most were against the idea. So it split even before it was established.

Another bitter schism developed a year later, and the party continued anarchic and intolerant, especially of anyone who seemed to get through to the electorate. It purged one leading figure after another, most notably Sara Parkin, whose popularity and political nous were largely responsible for the 1989 result. In September 1992, just two weeks after being hailed by the party as its "greatest asset", she was forced out of its chairmanship at its party conference. Immediately afterwards, a large proportion of the delegates gave a rapturous reception at a fringe meeting to David Icke, a former prominent Green, who proclaimed himself the son of God and denied global warming.

Lucas has not only survived but prevailed. "The Green Party is full of motivated, driven people who want to make change happen as fast as possible. Perhaps I have been slightly more patient (though I don't feel patient in the least), and have hung in there."

Policies have changed in that time: a party that started by rejecting economic growth, and long remained hostile to it, has entitled its latest manifesto "It's the Economy, Stupid". It now says without a blush, that "the old 'environment vs economics' argument was always false" and that its "central message" is that "we can tackle the recession and the climate crisis in one go". It wholeheartedly embraces the concept of a Green New Deal, rightly pointing out that environmental industries employ many more people.

The language has not entirely changed: it says it wants "to transform the economy to a new Green, stable, steady state", which seems to mean recycling and reusing resources rather merely consuming them. But the switch of emphasis is remarkable.

The mainstream environment movement now embraces Lucas's party, after years of hostility. After the 1989 results, I rang 10 of Britain's top environmentalists; only one had voted Green because, as one said: "We actually know what its policies are." Now the last three leaders of Friends of the Earth are all signed up.

And yet, I say, Greens in office have almost always disappointed; the Red-Green partnership that ran Germany for four years did much less for the environment than the conservative Merkel government that followed it. Lucas doesn't deny it, saying that Greens have paid too high a price to enter coalitions. But she vigorously protests when I add that – in one of the greatest own goals in political history – the Green Ralph Nader handed George Bush victory in 2000 by taking votes from Al Gore in Florida.

Against that, she and Jean Lambert, her fellow MEP and reformer, have built up an impressive record of initiatives in the European Parliament. And, with 119 councillors scattered across a 10th of the country's local authorities, the Greens are having an impact in local government. In Kirklees, West Yorkshire, for example, Green initiatives have provided 40,000 homes with free insulation.

The day I met Lucas, Oxford appointed a Green mayor – the first in any British city. And in the North-west, electoral mathematics mean that the last European seat will go either to the Greens or the BNP.

The first seat gained on any council is all important, she says. "Once you are through that credibility barrier, you get more candidates elected as people see your policies are effective and you deliver." Thanks to Lucas – and circumstances – the Greens now have a rare opportunity to persuade voters that, as at the Hop Exchange, crossing the threshold makes all the difference.

Eco-warriors' new agenda

* Green New Deal to create jobs, beat the recession and create an environmentally sustainable economy.

* Greenhouse gas emissions to be reduced by 90 per cent by 2030.

* VAT to be replaced with eco-taxes.

* Half of all energy from renewables by 2020.

* Four million homes insulated for free every year.

* Major expansion of public transport. Bus fleet to be doubled and railways renationalised.

* Development aid increased to 1 per cent of GNP by 2010.

* A £6bn programme to create 60,000 new low-carbon council houses Homes threatened with repossession to be bought by local authorities and rented to their former owners.

* Pensions increased to above the official poverty line for the first time.

* Everyone entitled to a basic citizen's income at jobseeker's allowance rate.

* Free and universal dental service.

* More community policing, more local police stations and the return of park keepers and bus conductors.

* Ban on GM crops; reduce pesticides.

* Ban on animal testing; zoos abolished.