Is this troubled nation ready for change?

Who, back then, could have imagined what would follow? On the glacially cold morning of 10 February 2007, I stood shivering in the crowd outside the Old State House in Springfield, Illinois, as Barack Obama formally declared himself a candidate to be the 44th President of the United States.

The very thought that, for the first time, an African-American and a relative political newcomer to boot, had a small but realistic chance of entering the Oval Office made the moment fascinating enough. There was also a small personal conceit. Back in October 1991, I had travelled to Little Rock to watch a young governor of Arkansas make a similar announcement. Bill Clinton went all the way. So, would lightning strike twice – might I have stumbled on another winner?

Three days before the end of this remarkable campaign of 2008, it looks that way. Just as Clinton caught the mood of the country 16 years ago, Obama has done so now. Having been present at the creation, I watched him this week as this political fairy tale for the ages moved towards its climax. Obama was addressing an overflowing audience of 10,000 or more on the campus of James Madison University at Harrisonburg, in Virginia's Shenandoah valley, the weather was as inhospitable as in Springfield almost 21 months before. Though it wasn't even November, and the trees were still resplendent in the scarlet, rust and orange of autumn, snow showers were forecast that night.

During what has been the most gruelling election campaign ever, the candidate has aged. His frame is as lean and athletic as when it all began, but flecks of grey are now to be seen in Obama's cropped black hair. The message, however, hasn't changed. The "closing argument" he delivered in Harrisonburg is the same as that opening salvo from the steps of the Old State House: unity, hope and, most important of all, change.

Rural and spiritually of the south, the Shenandoah valley is John McCain country, in a state which has voted Republican in every election since 1964. The very fact that Obama was there during the endgame merely underlined his status as front-runner, conducting the battle on his opponent's turf. "A few weeks ago, I was supporting McCain," a woman told me as we left, her eight-year-old daughter in tow. "But now I'm leaning to Obama, and he sounded really good today." Multiply that sentiment by thousands, maybe millions, and you have the story of this last six weeks of the campaign.

Whoever wins, the 2008 election is one for the history books. The beauty of American politics is that it can throw up the inconceivable as nowhere else. Obama's presence long ago meant the campaign would be unusual. That its climax has coincided with a financial crisis that is shaking US capitalism to its core, reducing the candidates on occasion to mere bit players, has only made it more so.

At times, you'd hardly guess so. Listen to the candidates, even as Wall Street took another stomach-churning plunge, and you were sometimes pressed to imagine anything truly serious was amiss. Joe the Plumber became a national celebrity, as byzantine arguments swirled about the impact of competing tax plans and the mechanics of healthcare reform – even as ordinary Americans worried about their jobs, their savings and whether they could meet their next mortgage payments.

But the financial and economic meltdown has transformed everything. Until mid-September the candidates were neck and neck, and some polls even put McCain ahead. Then came Lehman Brothers. The demise of a giant investment bank and the turmoil that ensued suddenly brought home how an old order, economic but also political, was dying as well.

Love it or loathe it, for better or worse, the US will remain the most powerful and influential country in the world, however this crisis unfolds. Only the US will be capable of projecting overwhelming military power to every corner of the globe. Perhaps it is no longer capable (if it ever was) of imposing a world order, but it remains the essential anchor of world order. And America is also the only country where a breakdown in an obscure corner of its mortgage market could trigger what might yet be a new Great Depression.

But this is a transformational moment in American politics. Watersheds in politics, unlike those in topography, only become clear in retrospect. No one, for instance, could have known in 1980 that Ronald Reagan would be the dominant figure in US politics for a generation. Few foresaw the breathtaking rise of China, and who 25 years ago would have predicted the current economic turmoil?

Now everything is in flux, starting with politics. If John McCain defies the odds and wins on Tuesday, it will be despite his being a Republican. Reaganite conservatism has run its course. The party will have to be re-invented if it is to become relevant again to the centre where elections, here as everywhere, are decided.

Barack Obama's good fortune has been to be his party's candidate at a moment of pent-up desire for change, like Britain in 1997 when the exhausted and broken Tory order fell to New Labour. Had Hillary Clinton or another leading Democrat won the nomination, she or he would surely be as far ahead in the polls today as Obama – perhaps further, given the uncertainty whether his race may be a handicap.

The changing geopolitical order, too, is weighing on Election 2008. Foreign policy is where an American president has his greatest power. In this campaign, it has inevitably been overshadowed by the country's economic woes, but voters are perfectly aware of the colossal human and financial cost of the war in Iraq, and the damage it has done to America's good name and moral authority, far outweighing any possible benefit of the conflict. Here too Obama is the beneficiary. Perhaps unfairly but inevitably, John McCain is identified with the old. Obama embodies change. And now the economic crisis, entwined with the crisis of the Republican Party and the damaged international standing of the US, yet far transcending them in its impact on the ordinary citizen.

You can measure it in the statistical indicators, such as this week's plunge in the consumer confidence index to its lowest level since they started to measure such things in the mid-Sixties – remember that consumer spending accounts for 70 per cent of the entire economy. And never before has 90 per cent of the population said the country is heading in the wrong direction: the real mystery is that 10 per cent of Americans still think everything is fine.

Or you can see it in the eerily quiet shopping malls (what a dreadful Christmas retail season this promises to be) and the empty homes with their changed locks and foreclosure sale notice in the front yard – nowhere more than in those new new exurban developments carved out of farmland and woods, that were supposed to be the new frontier of the American Dream.

Now the dream has gone sour. Yes, the rooted belief in American exceptionalism is still more or less intact. No Obama or McCain speech is complete without a reference to how the country is the most democratic/hardworking/productive/ inventive society yet conceived by man, and how it is somehow different (better?) from any other. That is how Americans see themselves and why the US remains the most optimistic country on earth.

But in this election, optimism is tempered by foreboding. Even before the crisis broke, the old 20th-century certainty that in America children would automatically be better off than their parents had been swept away by a decade of stagnation, even decline, in living standards for all but the very wealthy. It is no coincidence that the gap between rich and poor in the US is now wider than at any time since the market crash of 1929.

The assumption that prosperity is an eternal entitlement is no more. People are worried: not just about their jobs, their mortgages and the outstanding balances on their credit cards, but also about educational standards, the decrepit state of much of the country's infrastructure, the chronic failure to provide health care for all, and whether the money is there to pay for social security and other welfare benefits in 15 or 20 years' time. This week, a former General Motors employee undergoing job retraining in St Louis was interviewed on National Public Radio. According to Nadine Newberry, a 55-year-old black woman, this election is "the most important that I probably or anyone else would witness in a lifetime. This has always been the richest country on earth, yet something has gone terribly wrong".

A friend who is a senior aide for a Democratic senator on Capitol Hill made a similar point – how "some of our problems are now almost insoluble now, and that this is a last chance to act". Republicans might differ on the remedies but not on the diagnosis. Given the mountain of problems that await, many of which cannot be solved even in two presidential terms, and the painful, deeply unpopular measures needed to confront them, the wonder is that Obama, McCain or anyone else wants to be president at all.

I have covered four past US elections, but there is a seriousness about this one I've never seen before. With the Cold War safely won and Saddam driven from Kuwait, economic angst (with a little help from the third-party candidacy of Ross Perot) propelled Bill Clinton to victory over the elder George Bush in 1992. After 12 years of Republicans in the White House, why not take a chance on a clever, charismatic if rather flighty young governor from Arkansas?

The 1996 election was a sleepwalk for Clinton and most of us reporters, as he proclaimed himself "A Bridge to the 21st Century" and rolled over the hapless Bob Dole to win a second term. The 21st century then duly arrived, with the tedious contest between Al Gore and Bush the son, in which the post-election dramas in Florida were infinitely more exciting than the campaign itself. Hanging chads and butterfly ballots decided the outcome, and ushered in – though no one suspected so at the time – the most inept and disastrous presidency of the modern era.

Then 2004 did provide an election to raise passions, as the Iraq war foundered and Bush Jnr generated almost irrational hatred among Democrats. Both sides mobilised their bases as never before, and turnover was the highest since 1960. The Republicans, however, mobilised slightly better, while Bush retained just enough of his dwindling political capital to haul him over the finish line (had the vote come even three months later, I'm convinced he would have lost). But in 2004, the stock market was fine and house prices were rocketing. There was no sense of impending calamity; that bedrock notions of prosperity were at risk.

Not so in the extraordinary campaign of 2008, and the extraordinary choice that faces voters. Americans tend to elect presidents who in character are the opposite of their predecessors. Care-burdened Jimmy Carter was followed by sunny Ronald Reagan. The seemingly out-of-touch G H W Bush was succeeded by the clued-up and super-empathetic Bill Clinton. Having narrowly survived impeachment, "Slick Willie" was replaced by a younger Bush who promised to restore the honour and dignity of the country's highest office.

And now? One option is a war hero who suffered unimaginably as a prisoner of North Vietnam, a foreign policy specialist who has been one of the most engaging and interesting senators of his era. But unless all the polls are wrong, the US seems set to elect, at this most difficult moment in its modern history, the least experienced president in its modern history.

Obama's resumé, Hillary Clinton sneered during one of the primary debates, amounted to a single speech. She was being a bit unfair. Not one, but two speeches helped launch his candidacy: the speech he made as an Illinois state senator in late 2002, opposing the then looming Iraq war, and his breathtaking "one nation" keynote address to the Democratic convention two years later, that turned him into an overnight national sensation. And oratory is a vital part of the presidential arsenal. If ever Americans needed a politician with the power to uplift them, it is now.

At 47, Obama would not in fact be the youngest president; Teddy Roosevelt was a mere 42 in September 1901, and Kennedy and Clinton himself were both younger when they entered the White House. But all three were much better known; Roosevelt was vice-president when he took over, Kennedy had been a congressman and then a senator for eight years, while Clinton had served 12 years as governor of Arkansas.

More even than his race, Obama's lack of experience – especially in McCain's strongest suit of foreign affairs – was a problem for many voters. But the debates removed those doubts.

Franklin Roosevelt, who in 1933 also took over at a desperate economic hour, was famously said to have had a second-class intellect but a first-class temperament. In three presidential debates, Obama proved he was top class in both departments. Not only was he master of every subject, he was also also cool, smiling and unflappable, never rising to the bait of some vicious jibes from McCain. He didn't grandstand, he didn't produce slick one-liners and – miracle – you could see him actually thinking before he answered a question. If Americans were looking for the opposite of Bush, they had found their man.

McCain, it must be said, did not help his cause. The old John McCain had a temper, certainly, but also a refreshing irreverence and a great sense of humour. In the debates, however, he came across as a grumpy old man, rude, condescending and scornful of his opponent. Americans didn't like it when Al Gore got condescending with George W Bush in 2000, and they didn't like it now. Polls found Obama the victor in all three debates and, over and over again, people have told me: "I used to really like John McCain, but not any more, he just doesn't seem very nice." In US election campaigns, where a candidate's likeability is at least as important as his policies, there are few more damning judgements.

They also didn't much care for it when McCain, who in past campaigns made a point of taking the high road, took the low one against Obama. Normally, going negative works wonders (just ask Bush father and son) but not this year. The veiled racial attacks on Obama, Sarah Palin's charge that he once "palled around with terrorists", the insinuations that he was not a patriotic American but a Muslims, all turned people off. As has been pointed out above, in 2008 voters are serious.

The choice of Palin, at first a seeming masterstroke, has also backfired. The base adores her but many conservative commentators, centrist Republicans and all-important independents were appalled. Her selection, after a bare minimum of vetting, added to the image of the 72-year-old McCain as a reckless gambler. McCain the patriot seemed this time to be putting his career ahead of his country. In truth of course, nothing can prepare a person for the presidency. That indeed is one of the best arguments for the numbing length of campaigns. They are huge enterprises in their own right. At the very least, voters can get a sense of how the candidate performs under pressure, for the very highest of political stakes.

By common consent, Obama has run the best campaign in memory. He has made remarkably few blunders. He has had cringe-making moments – who wouldn't in a 21-month political marathon? In one primary debate, he unsmilingly told Hillary Clinton she was "likeable enough", and he was caught on tape at a high-end San Francisco fund-raiser talking patronisingly of how poor whites in places like Pennsylvania "clung to" guns and God because life was so tough for them. Not surprisingly, these voters went overwhelmingly for Hillary in the primaries.

But now those sins seem to have been forgiven. The Clinton supporters who once vowed to vote for McCain rather than Obama are now mostly back in the fold (driven by Sarah Palin). McCain has tried to brand Obama as a traditional tax-and-spend liberal, a charge that usually works in a country where self-described conservatives outnumber avowed liberals almost two to one and a third of voters call themselves independents. But it resonates less right now, when the hallowed free market has delivered a disaster, and when scared citizens want more government, not less, to see them through the tempest.

With Democrats set to make big gains in both the Senate and House which they already control, McCain warns of the perils of placing every lever of power in Washington in the hands of one party. But in times of peril, Americans are apt to do exactly that – as they did in 1932, a watershed election if ever there was one, and whose grim economic backdrop has uncanny similarities with today. This time, Americans seem to be saying, divided government is not the answer.

The real wonder is that, despite everything – his age, his daunting financial disadvantage, the ideological bankruptcy of his party, the record unpopularity of an incumbent Republican president, the meltdown of the economy and the tarnished image of the US abroad – John McCain is still in the game at all. But the ancient warrior is making a fight of it. Though Obama is well ahead in several of the swing states that will decide the election, his lead is shrinking at a national level. Democrats are petrified that once again their opponents will find some means, fair or foul, to deny them.

Ultimately, we are left with the questions that hung in the icy air of Springfield 21 months ago. Despite his fine showing in the debates, will the country really place its trust in a comparative novice, even one as gifted and charismatic as Barack Obama? And then there is the mystery that no poll yet has convincingly resolved. Is the US ready to send an African-American to the White House? The answer, I think, is yes – but even now I'm not certain.