Rupert Cornwell: Has America gone mad?

What on earth is going on? These days, a grounding in clinical psychology is probably of more use than a PhD in political science in determining the mood of the US voters ahead of November's mid-term elections. But whether or not the country is experiencing a collective nervous breakdown, last week's batch of primary results have established one thing beyond doubt – that America's electorate has never in modern times been angrier, more volatile and less predictable than now.

The most spectacular proof came in the normally inconsequential state of Delaware, where Christine O'Donnell – spectacularly unqualified but blessed by the Tea Party and Saint Sarah and making all the right ultra-conservative noises – defeated Mike Castle, the state's highly popular former governor, in the Republican primary for the Senate seat long held by Vice-President Joe Biden. A month or two ago, such a result would have been unthinkable. But then came a Palin tweet on behalf of Ms O'Donnell, and the rest is history.

She is the seventh outsider, no less, backed by the insurgent Tea Party movement to topple a Senate candidate endorsed by the Republican establishment in this tumultuous primary season. Ms O'Donnell has had run-ins with the tax man, failed to meet her mortgage payments and equates masturbation with adultery, and may be a bridge too far for the good citizens of Delaware in November's general election. But some of the Tea Party crowd will certainly win, in an election year shaping up as a rout for the Democrats.

Up to a point, such an outcome should come as no surprise. At mid-terms, the President's party is invariably punished, especially when that party controls not only the White House but Congress as well – and after thumping wins in 2006 and 2008, Democrats are now overdue for a hammering.

Republicans need to make a net gain of 39 seats to retake the House of Representatives. The consensus is they will, and probably with something to spare; some analysts are even predicting a repeat of the 52-seat swing in 1994, when a pugnacious minority leader named Newt Gingrich led Republicans back to command of the House for the first time in 40 years.

In the Senate, the outlook is cloudier. The Democrats currently have a 59-41 seat edge. At the time of writing, they seem set to lose seven at least, although thanks to the victory of the "unelectable" Ms O'Donnell, they are likely to hang on to a seat that was previously considered lost. Even so, if Republicans run the table, as they say in billiards, they can still secure an outright majority. The Democrats, after all, did so in not dissimilar circumstances in 2006, propelled by the unpopularity of George Bush; so why not the Republicans now?

But numbers tell only part of the story. For this is 2010, when even stronger emotions are loose in the land. Americans, as the whole world knows, are mad. They're mad about the economy, about high unemployment and a recession that feels like it will never end. They're mad at Wall Street, which got the country into the mess. They're mad at government, which, for all the colossal deficits it has run up, can't get them out of the mess. They're mad at big business for exporting jobs to China, at China which cheats so blatantly on its exchange rate, at unions that can't deliver for the workers, at the media who tell everything like it isn't.

And they're mad at that cool-mannered guy in the White House who appears to feel nobody's pain. Heck, isn't he a Muslim, and probably not even a native-born American at all? Finally, and most relevant to the upcoming election, they're mad at Congress and all those who sit and squabble there. And since there are more Democrats than Republicans on Capitol Hill, the Democrats stand to suffer the most.

But at this point, if you believe a poll by The New York Times just 24 hours after Ms O'Donnell swept all before her, you must abandon politics-as-usual and reach for the psychiatric manual. According to this poll, voters are fed up with Democrats. But they are even more scornful of Congressional Republicans, whose strategy for the last 18 months has consisted of the single word, "no", to everything President Obama has proposed.

The survey found that voters believe that Democrats are more likely to help the middle class, have better ideas for solving problems, are more likely to create jobs, and are more likely to help small businesses. Democrats, it is further believed, have the right ideas about immigration. Yet by every measure, Democrats face a thrashing. Go figure.

This is indeed a strange moment in the US, and our psychiatric manual might argue that deeper, secular forces and anxieties are at work. An American might very well conclude right now that a system that has effortlessly delivered prosperity and top-dog status for generations isn't working any more.

You have to be 80 at least to have a clear memory of the Great Depression, the last time America was on its knees. From the moment of their birth, its citizens are told they live in the greatest country on earth, like no other in history. But now fear gnaws that the nation is in decline, that other countries have caught up, and even overtaken it.

A spate of recent surveys leave little doubt that this is the case. Last week's Census Bureau figures – showing that there are record numbers living in poverty and without health coverage, and that average incomes have stayed flat at best over the past decade, despite the fortunes made by a few – have underscored how this recession is the deepest since the Depression.

You sense a flailing around for the vanished certainties once encapsulated in the notion of "the American Dream", and a venomous search for scapegoats in the establishment that has failed them. That is the mantra of the right-wing talk-show hosts who ceaselessly stoke the indignation, egging on the Tea Party, blaming defeatist Democrats, ungrateful foreigners, and Rinos – despised "Republicans in Name Only", like Delaware's Mike Castle – for the country's every ill.

This, in turn, illustrates another facet of the paradox. The Democrats are set for a beating; but it is the Republicans who are split. That second phenomenon is not new. Teddy Roosevelt's "Bull Moose" party split the Republicans in 1912, allowing Woodrow Wilson to become only the second Democrat to win the White House since the Civil War. Something of the same happened in 1964, when the Arizona outsider Barry Goldwater defeated Nelson Rockefeller, incarnation of traditional East Coast noblesse oblige Republicanism, for the party's presidential nomination and then went down to a landslide defeat.

But this Tea Party moment goes further. It combines the grassroots fervour that Goldwater unleashed (and whose lasting effects make him one of the most important figures in 20th-century Republicanism) with a populism and nativism that are equally recurrent strands in US history – think the anti-immigrant Know Nothing party of the mid-19th century, or red-baiting McCarthyism of more recent times. The fact that Barack Obama is the least populist, most cosmopolitan occupant of the White House since at least JFK, simply feeds the movement's ardour.

Where all this will lead, nobody knows. Maybe Ms Palin, the linking element between the Tea Party and the Republican party proper, will do a Goldwater, leading an insurgency to capture the party's nomination in 2012, only to go down in flames at the general election. Maybe she won't run; maybe both she and the movement will fizzle out. Or perhaps the Tea Party will take over the Republican Party: or again, it may be gently co-opted into it.

Whatever happens, however, the current state of affairs spells trouble for everyone in the short term: for President Obama who must cope with divided government, with Republicans less inclined than ever to compromise; and for the mainstream Republican White House aspirants in 2012 – Mitt Romney et al – who must somehow harness the Tea Party's energy while preserving an appeal to centrists and independents who ultimately decide presidential elections.

Above all, it will be bad news for ordinary Americans, who having voted out one bickering Congress, are seemingly about to vote in a replacement that is likely to be even more partisan and paralysed, one in which the centrists who forge the legislative compromises that actually get things done will be in ever shorter supply. Americans hate government – but they also know they need it. Go figure. Or, rather, go see a psychologist.