Three's a crowd: How the unexpected rise of a third contender broke the cosy two-party system

History doesn't repeat itself, but as the old adage says, it does rhyme. The rhyming is now clamorous and incessant. If the Liberal Democrat surge that followed Nick Clegg's victory in the first televised leaders' debate continues, we shall be in spitting distance of a hung Parliament. Of course, it may not continue. The Clegg boom may peter out; residual "Labservative" tribal loyalties may reassert themselves. But when all the caveats have been entered, there is no doubt that the two old parties, the ugly sisters of British politics, have had the fright of their lives. For old SDP-ers like me, it has been a moment to relish.

But it is also a moment for careful historical reconnaissance. There is no post-war parallel to the shock that seems likely to overwhelm the old parties when the votes are counted this May. There was a hung Parliament in 1974, but the circumstances then were so different from ours that it has no useful lessons for today. Yet there is a much older parallel: the election of 1923 and its sequel in 1924. In November 1923, Stanley Baldwin, the untried Conservative Prime Minister, suddenly called a general election to win a mandate for protective tariffs. Polling day came early in December, 1923. The Conservatives lost their overall majority, but they were still the largest party in the Commons. The rising Labour Party, led by the charismatic Ramsay MacDonald, came second. The recently reunited, but incorrigibly schismatic Liberal Party, led by the pre-First World War Liberal Prime Minister, Herbert Asquith, came third.

There followed a bizarre fever of speculation, anxiety and panic. MacDonald had never held office of any kind; nine years before, his opposition to the First World War had earned him a degree of loathing and contempt that few modern British politicians have encountered. Besides, the Labour Party was socialist. For many, it stood for expropriation and class war. A Labour Government, one commentator proclaimed, would be a disaster such as normally occurred only after defeat in war. All sorts of schemes were canvassed to avert it, the favourite being a Conservative-Liberal coalition, and the next a national Government of some kind.

But the principals kept their heads. Baldwin made it clear that he would have no truck with any anti-Labour deal. Asquith did the same. As the leader of the largest party, Baldwin rightly stayed on as Prime Minister until the new Parliament met in January 1924. His Government drew up a King's Speech. The Labour and Liberal parties joined forces to vote it down. Baldwin then resigned, and the King, George V, sent for MacDonald, who formed a minority Government – the first Labour Government in British history. It turned out to be spectacularly successful abroad and boringly competent at home, an ideal combination for a party whose chief task was to prove that the country and the constitution were safe in its hands.

In the end, the Conservatives and Liberals joined forces to defeat it. In the subsequent general election the Conservatives won a large majority, while Labour lost ground. But its defeat had a huge silver lining. The Liberals were crushed. Labour had become the Conservatives' opposite number in a new two-party system – the very system that Clegg is now trying to destroy.

What does this complex story tell us about the politics of 2010? What parallels are there between the political actors of 1923-4 and those of today? Three stand out. The most obvious is the parallel between David Cameron and Baldwin. Indeed Cameron sometimes seems to be trying, quite consciously, to act the part of Baldwin Mark II – emollient, inclusive, honourable and, above all, reassuring. His penchant for cycling to work (with photographers in tow) is a 21st century equivalent of Baldwin's penchant for striding along country lanes, clad in a baggy tweed suit (and also with photographers in tow). Before the First World War, the Conservatives had been the nasty party, using their majority in the Lords to throw out the Budget in 1909, and then flirting with armed rebellion in Ulster.

Like Cameron, Baldwin was trying to lay the ghost of Conservative nastiness and to speak in a carefully crafted contemporary idiom instead of the grandiloquent language of the past. His ostentatiously honourable refusal to countenance an anti-Labour manoeuvre in early 1924 was part of that strategy. But the parallel mustn't be pushed too far. Baldwin had far stronger cards in his hand than Cameron has. He was Prime Minister at the start of the story and leader of the largest party throughout. It was far easier for him to contain Ramsay MacDonald's challenge than for Cameron to contain Clegg's. All he had to do was wait for Labour to run into trouble, whereas Cameron has to strain every nerve to recapture the ground lost to the Liberal Democrats.

Which is where the second parallel comes into the story: that between Brown's Labour Party and Asquith's Liberals. Asquith had been a hugely successful Prime Minister. His Government had been the greatest reforming government since Gladstone's in 1868. Yet after 1918, and the arrival of universal male and partial female suffrage, all this availed him nothing. He appeared pompous, boring and hopelessly out of joint with the times. The causes that made him – taming the House of Lords, Irish Home Rule, Free Trade – no longer resonated. He was like a whale, beached by the tides of history. He seemed out of tune, a ghost from another age.

The same was true – in spades – of the Liberal Party. It embodied a great tradition of individual freedom and equal opportunity going back to John Locke and John Stuart Mill. The Whig ancestors of the Liberal Party had carried the great Reform Act of 1832. Liberal governments had broken the Anglican stranglehold on the ancient universities, reformed local government, abolished the sale of Commissions in the Army and almost secured Home Rule for Ireland. Gladstone, the greatest peacetime Prime Minister in our history, was a Liberal. So was Lloyd George, the founder of the welfare state and the second greatest wartime Prime Minister. Liberal thinkers such as TH Green, the philosopher, LT Hobhouse, the father of British sociology and JA Hobson, the precursor of Keynesian economics, had dominated the national conversation.

But the liberal ideal perished in the trenches and mass mobilisations of 1914-1918 and the class conflicts that followed. The age of the individual gave way to the age of the collective. The still, small voice of liberal individualism was drowned out by the big battalions of labour and capital. Despite occasional spurts of intellectual energy, the Liberal Party was hopelessly at sea in a mass society whose politics were dominated by the rival collectivisms of paternalist conservatism and state-centred socialism. It was consumed by factional disputes – backward-looking Asquithian grandees versus Lloyd Georgite adventurers – that meant nothing to the electorate.

If this sounds familiar, it is. In the intervening 90 years, the wheel has come full circle. The age of the collective is over. A new kind of individualism is in the ascendant. The mass society has disappeared; its preoccupations have disappeared with it. The great liberal issues that seemed quaintly archaic in the 1920s – citizenship rights, the devolution of power, individual freedom – have returned to the centre of the stage. The state-centred collectivism which the rising Labour Party offered in place of liberalism, and to which it still obstinately cleaves, is patently a busted flush, just as liberalism was in the 1920s. And, again like the Liberals of those days, Labour is consumed by personal bickering that means nothing to anyone outside its inner circle.

Against that background, the third of my parallels – that between the Liberal Party of today and the Labour Party of the 1920s – falls into place. The Liberal party of that era was sick, but it did not die of its own accord. It was Labour, and above all Labour's leader, Ramsay MacDonald, that despatched it to the knacker's yard. Labour's strategy had two prongs. First, it sought to re-draw the map of politics: to prove that the old battle between Liberals and Conservatives was over; that a new battle between socialism and capitalism had taken its place; and that in that new battle, Labour was ranged equally against both the old parties. Secondly, however, Labour also sought to show that it was the heir of the old Liberal Party, that what MacDonald had once called "advanced and sturdy radicals" now belonged in the Labour Party, that democratic socialism in fact encompassed liberalism.

The moral is clear. Clegg is absolutely right to pay no attention to Brown's shameless ogling. The much-touted suggestion that the Liberal Democrats are "really" on the same side as the Labour Party is both false and, from the Liberal Democrats' point of view, dangerous; and Clegg is both strategically wise and philosophically sound to reject it out of hand. But he also has to do something else. Like Labour in the 1920s, only in reverse, he has to show that the Liberal Democrats are the heirs of all that was best in the old Labour Party: that there was a libertarian strand in the Labour tradition, as well as a statist one, and that the libertarian strand is encompassed by the Liberal Democrat party of today. I suspect that the success of his project will depend on whether he can do the second as well as he is already doing the first.

Britain Since 1918: The Strange Career of British Democracy by David Marquand, is published in paperback by Orion (£14.99). To order a copy for the special price of £13.49 (free P&P) call Independent Books Direct on 08430 600 030, or visit www.independentbooksdirect.co.uk