Leading commentators: What are their credentials?

Downing Street was hardly the most reliable source of information in late 2002, yet amid the fetid stream of dodgy dossiers and war propaganda, it did come up with one more or less reliable statistic. A Downing Street official estimated the British political commentariat - columnists writing regularly for the national press about domestic politics - numbered 120. This formidable force was apparently causing the Government more headaches than its opponents in Parliament.

Curiously, despite the vast importance of British commentary, academic research into the subject has been scanty, at least when compared with a distinguished tradition of research into the role of news.

Thus the Carnegie Trust for the Universities of Scotland did not take much persuading to hand me a generous cheque to investigate this so-called punditocracy. I approached 25 leading columnists, making sure, in order to secure a representative sample, to include men and women, left-wingers and right-wingers, and all species of national newspaper.

Eleven swallowed the bait, and many thanks to them. They were: Keith Aitken (Scottish Daily Express), Simon Carr (The Independent), Peter Hitchens (The Mail on Sunday), Simon Jenkins (The Guardian and The Sunday Times), Boris Johnson (The Daily Telegraph), Dominic Lawson (The Independent), Deborah Orr (The Independent), Matthew Parris (The Times), Henry Porter (The Observer), William Rees-Mogg (The Times and The Mail on Sunday), and Polly Toynbee (The Guardian). A good mixture.

I began by asking the columnists what qualifications they had for the job. Most cited reporting experience; some had also served as leader writers or editors. "An addiction to politics," said Simon Jenkins - this was a common sentiment.

Many could also claim direct political experience, as MPs (Johnson, Parris), as parliamentary candidates (Aitken, Rees-Mogg), or in setting up parties (Carr, Toynbee).

I found it intriguing that none mentioned their academic qualifications, despite many of them having degrees in relevant subjects, such as philosophy, politics and economics, Oxford University's flagship course for budding politicos. What Peter Hitchens calls "the University of Fleet Street" is apparently still regarded as the best preparation for column-writing.

Most thought that ongoing contact with politicians was useful. "You need to be plugged into the argument," as the well-connected Keith Aitken puts it. But all were emphatic that a good columnist keeps his or her distance. "One gets corrupted," said Simon Carr, who takes care not to write about the politicians he knows well. "I get charmed too easily," confessed Deborah Orr, although she recognised a place for "insiders" as well as "outsiders" in the columnist ranks. "A journalist should not be part of any political project," insisted Henry Porter.

Polly Toynbee is perhaps closest of the group to the present Government, but she too believes that a political columnist should not become "beholden" to politicians. Hitchens, a veteran of the Westminster scene, went further, however: "What is the point of contact with them? I don't like their gossip, and in any case the best among them do not tend to rise to the top." I am sure he speaks for many on that last point at least.

What then is the current role of the political columnist? Few see themselves as public intellectuals, but they do think they have a useful role in public affairs. "To engage people in politics," said Carr. "To be like the little boy in the story who says the emperor has no clothes," suggested Dominic Lawson. "To help to create and shape the debate," according to William Rees-Mogg.

Reassuringly, and contrary to what is often alleged about the commentariat, the columnists I spoke to were all strongly committed to factual information as part of their mission to engage. "Opinion not based on evidence," according to Simon Jenkins, "is worthless." His own columns are, of course, particularly well known for their meatiness.

Toynbee, whose work often displays a thorough grasp of sociological and policy detail, thinks that a column should contain at least three interesting facts. Boris Johnson sees the role of the columnist as that of "taking the available facts and arranging them so that people see things in a certain way". However, he adds, with characteristic wit, that some columns rest on a tiny plinth of fact.

Political savvy, in-depth knowledge and, of course, some literary ability are requirements for the task, but what impact do political columnists actually make? My interviewees were modest, expressing agnosticism or at least caution about their ability to change things.

Matthew Parris, despite being one of the most highly respected columnists, assessed his influence as only "slightly better than negligible". Deborah Orr writes some of the most vigorous pieces you will find in contemporary British journalism, yet she rated her impact at "0.001". Even Toynbee, a figure to be reckoned with on the centre-left, was far from lyrical about her influence.

Yet it is clear from what recently happened to Messrs Carr and Porter, whose columns in defence of civil liberty were denounced junta-style by senior members of the Government, that political columns, at their best, can make a very serious impact indeed. Porter, who also started getting personal e-mails from the Prime Minister, explains the secret: "If you bang on about a subject, if you do it properly [he spent countless hours acquainting himself with recent legislation before going public about the Government's assaults on freedom], if you are relentless as well as polite, then you can be influential." I believe that Porter is the most important columnist writing today.

Boris Johnson, who, rather like Churchill, successfully combines high-wire politics and journalism, confirms this from the other side of the fence. "Politicians," he told me, "tend to define themselves by how they are characterised in the press. If a speech is picked up by a columnist, that is a triumph. If they are always rubbished, they can feel crushed."

After leaving Johnson's comfortable office in Portcullis House, I spent the afternoon over the road at the House of Commons. Within half an hour of the opening of an important adjournment debate on human rights, someone was quoting an opinion column from that morning's Guardian.

One thing is clear: the commentariat is hugely influential. What of the future, though? Given that there are now in excess of 100 commentators, whereas you used to be able to count them on the fingers of one hand, can political columnists survive?

Most of those I spoke to detected a shift away from the idea of the sage commentator. "People are generally less deferential to superior knowledge," noted Carr. Jenkins identifies a related drift in journalism away from learned-essay columns (of which he is a master) to the attention-grabbing type, what he calls the "dictatorship of the first paragraph".

Yet none was pessimistic. Aitken and others pointed out that columnists - informed and trusted opinion - are what readers of newspapers are looking for now that news is largely broken by electronic media.

Lawson believes that there will be an increased role for columnists who can help people to understand issues in an independent way, instead of ones who just reflect the political culture of particular parties. It was partly his sense of a moving landscape in British and international politics that led this agile conservative thinker (famous in columnist country for having sacked Peregrine Worsthorne while editor of The Sunday Telegraph) to start writing for the left-of-centre Independent. Jenkins too has begun opining across traditional boundaries.

Of course, political bloggers are becoming a threat, but there will still be a place for the professionally crafted column, for the "balanced observer and explainer" (Parris). Indeed, Rees-Mogg, affectionately known by fellow journalists as "Mystic Mogg", predicts a vast expansion in the demand for well-researched, accurate comment.

While cynics might reply, well, you highly-paid columnists would say that, such forecasts are in fact very much in keeping with academic opinion on the growth of the "information society". Research indicates that as the quantity of data, statistics and news explodes, there will be a corresponding need for "information professionals" who can find patterns, navigate and make sense of it all.

And even if the market value of political columns were to collapse at some point in the future, that would still not spell the end. Political columnists might no longer be able to command five- or six-figure salaries for writing a few hundred words a week, but if they lower their price, they will surely find buyers. The political columnist of the future, Johnson muses, might have to be a road-sweeper by day. Now that really would be a democratic Opposition.

Yet, as Hitchens stressed to me, there is something fundamentally unhealthy in that whole scenario. If Parliament would just do its job properly, there would be less call for a political commentariat. Columnists can only propose; it is politicians who dispose.

Dr Alistair Duff is a senior lecturer in information and journalism at the School of Creative Industries, Napier University, Edinburgh