The past is a cosy country

In 1898 a band of seriously eccentric gents called the British Israelites (who believed they were descended from the Biblical Lost Tribes) dug up the ancient burial mound at Tara, capital of Ireland's High Kings, searching for the Ark of the Covenant. So, at any rate, goes a widely believed, constantly retold story. The great Irish essayist Hubert Butler became intrigued, and quizzed his elderly cousin Synolda, who had lived by Tara. Her answer? "Well, there wasn't any band but just one young student, and he was not a British Israelite and he didn't excavate the mound." Apart from that, it was all true.

In 1898 a band of seriously eccentric gents called the British Israelites (who believed they were descended from the Biblical Lost Tribes) dug up the ancient burial mound at Tara, capital of Ireland's High Kings, searching for the Ark of the Covenant. So, at any rate, goes a widely believed, constantly retold story. The great Irish essayist Hubert Butler became intrigued, and quizzed his elderly cousin Synolda, who had lived by Tara. Her answer? "Well, there wasn't any band but just one young student, and he was not a British Israelite and he didn't excavate the mound." Apart from that, it was all true.

Cousin Synolda was the best kind of historian: overturning, with quiet authority, all the long-believed stories. Historians should give us the shock of the new, by looking with fresh eyes at the old. New patterns emerge, new ways of seeing are revealed. Sometimes, the result is disenchantment: no deranged mystics, no Lost Ark, just a solitary young twit on a wild goose chase. Just as often, though, the outcome can be a renewed sense of wonder.

That, no doubt, is part of the reason why history books are the only kind of serious non-fiction that can regularly challenge the popular novel and the lifestyle manual in bestseller lists. Right now, the big hitter is the first part of Simon Schama's History of Britain. Also shifting serious numbers of copies are Peter Ackroyd's London (Chatto & Windus), Roy Porter's Enlightenment (Allen Lane) and the second volume of Ian Kershaw's Hitler: Nemesis (Allen Lane).

This is not, obviously enough, an entirely new phenomenon. There has always been a mass, non-specialist readership for history. Edward Gibbon in the 18th century, Macaulay in the 19th, were bestsellers. There does, though, seem to be a special intensity about it this autumn. Works like Antony Beevor's Stalingrad and Niall Ferguson's The Pity of War showed the way last year. Today all the major commercial publishers, and their entrepreneurial commissioning editors, have been making bids for this market, with Simon Winder at Penguin by some margin the most active player. If the most visible fashion wave in non-fiction across much of the Nineties was for long literary biographies, today the historical blockbuster appears to have taken their place.

These are almost all big books. The typical specialist monograph is about 100,000 words, the standard length of the PhD theses from which they so often derive. Most of the current success-stories are at least twice that length. Equally evident is the taste for big issues, whether perennial favourites like the Nazis and the two world wars, revivified oldies such as Schama's new take on Our Island Story, surveys of a whole century like Richard Vinen's A History in Fragments (Little, Brown, £25), or even of all human history - as with Felipe Fernández-Armesto's Civilizations (Macmillan) and now Clive Ponting's A World History.

There is a problem here. History's capacity to shock and startle is increasingly restricted to the micro-level, to small stories about obscure things, like the British Israelites (not) at Tara. A wholly new view of a big issue, or a wide stretch of time, is ever harder to achieve. Historians of the big picture must instead find ways of defamiliarising well-worn stories; and do so in a fashion that will appeal both to the knowledgeable and to the newcomer.

Most academic historians, notoriously, flinch at that challenge. They have not, by and large, crawled as deep into the bunker of jargonised solipsism as their colleagues in the social sciences. Yet the typical university historian digs ever deeper in an ever narrower trench. And much of the profession has recently seemed obsessed with a small but ferocious civil war between postmodernists and traditionalists: a complete turn-off for non-combatants. The great bulk of academic history is far less interesting than it should be. One finds oneself echoing Jane Austen's puzzled comment that "I cannot think why it should be so dull, for much of it must surely be invention."

It is, surely, impatience with that claustrophobic world that has led so many academic historians to attempt broad-sweep books for general readers (though the decline in British academic salaries, which makes generous advances so desperately alluring, must also have something to do with it). Here, too, things have changed. Popular history used to be written mostly by amateurs (or, to use a more positive label, by "public intellectuals"), from Gibbon to Arthur Bryant. Almost all the writers mentioned here hold university posts.

The traditional mass-market history was also, nearly always, cast in the form of a chronological account of events, facts and dates. Very few recent Big Histories are in that mould. Instead, they are mostly organised around themes rather than events. Schama is a partial exception here. He includes numerous set-pieces of straightforward storytelling - mainly rather obvious ones, like the Battle of Hastings, but told with enormous verve and vigour.

Others, in trying to avoid the wearisome predictability of a chronological framework, involve themselves in other kinds of trouble. Thus Susan Brigden's New Worlds, Lost Worlds, the 16th-century volume in Penguin's History of Britain, is certainly livelier than most previous efforts to cover the same ground, and better at explaining why things happened. It also gives far more attention than such books once did to the "colonialist" dimension of British state-making. It falls short, however, on the basic task of telling the reader what happened, and in what order.

There are many ways of organising a complex story in an accessible way. Michael Burleigh, in The Third Reich, goes for a fiercely polemical, emotive stance, yet also - unusually among these "middlebrow" books - for an upfront theoretical argument. He views Nazism as both a "political religion" and a species of totalitarianism. Richard Vinen's overview of 20th-century Europe in A History in Fragments groups its story round themes like migration, generational change, and family life; but undermines itself by reaching too hard for a vividly epigrammatic style that can, and does, slip into smart-aleckism.

By contrast, Clive Ponting opts for a sobriety that can verge on dessication. In style, Ponting's is the driest of these books; though its range is the most ambitious. There have been many recent attempts to produce a truly global history which cuts European pretensions down to size and puts "the West" in its proper place. Ponting's is perhaps the most thoroughly revisionist ever: the bias, if anything, is Sinocentric rather than Eurocentric.

In his pre-academic life at the Ministry of Defence, Ponting became a hero to civil libertarians for his Falklands War whistle-blowing. His second career as historian had seemed until now a touch anticlimactic. A World History redeems the long-deferred promise, breaking more new ground than anything else mentioned here.

Yet all these books, so various in approach, still strike one as very conservative on one major level. The brilliant Indian historian Ranajit Guha once asked me why nobody wrote history in the style of Heidegger or Nietzsche. My flippant response was that no one would or should want to, since Nietzsche was completely mad, Heidegger both incomprehensible and a fascist.

It was a good question nonetheless. Novelists don't have to write like Dickens, or Jeffrey Archer, to be commercially successful. So why should historians still have to follow the tracks of Macaulay or Arthur Bryant? When a historian is praised, or blamed, for innovation, it invariably turns out that they have "experimented" only within very strict limits. Nothing in the current crop of mass-market histories even vaguely responds to Guha's call.

Stephen Howe's latest book is Ireland and Empire (OUP)