John Kampfner: Wikileaks shows up our media for their docility at the feet of authority

You should never shout "fire" in a crowded theatre. Once you have accepted this old adage, you accept that there are limits to free expression. The important word in the first sentence is not "fire", but "crowded". A crowded theatre would lead to a stampede. Where there is a real and identifiable danger, restraint should be shown. Context is everything in the free-speech debate; risk to life is an undeniable caveat. Most other caveats are, however, mere ruses by the powerful to prevent information from reaching the public domain.

It is within these parameters that the furore over Wikileaks and its exposures should be seen. The latest document dump is larger than the Iraq files and potentially more embarrassing, with its State Department assessments of governments and statesmen – from Hamid Karzai to Silvio Berlusconi to Nicolas Sarkozy. Diplomats have launched a frantic round of damage limitation. Oh to have been a fly on the wall during the excruciating conversation between the US ambassador and Downing Street. The Americans are entitled to put their side of the story, to seek to assuage any inconvenience caused.

The founder of Wikileaks, Julian Assange, believes the Americans are going much further than that, carrying out a concerted campaign to undermine and discredit him. I have no information on the cases that have led two Swedish women to press charges of rape and sexual harassment against him. Only those involved do. Mr Assange's legal representatives in the UK suspect that the Swedish authorities are playing the Americans' game, cutting corners in terms of correct legal procedures. So highly charged is the environment that it is extremely difficult to separate information from disinformation.

Mr Assange is an unconventional figure, a man who lives in the shadows and enjoys doing so. He is difficult to deal with and holds himself in high regard. When he contacted me through an intermediary two months ago, suggesting that Index on Censorship host him in a London event, I was happy to accept but made clear that I did not want to give him an open forum. I would engage him in debate with a detractor.

He accepted; then he disappeared for five days, not answering phone calls or emails. When he finally surfaced, he stipulated no cameras or photographers, and that we should sneak him in through the back door. I argued that this would not look great for a free-expression organisation. In the end we compromised, and the television crews were allowed in halfway through what turned out to be a fascinating debate with the columnist David Aaronovitch. The sell-out crowd did not give Mr Assange an easy ride. But there was a virtually unanimous presumption towards free speech, something that is woefully lacking in so much of British public life.

This must surely be the starting point. In the US, with its First Amendment, restrictions are seen as an exception to the rule. In the UK, free speech is regarded as a negotiable commodity. An interest group's right to be offended is seen as just as important as the right to air an opinion. A government's right to secrecy is seen as more important than the public's right to know.

The mainstream media in the UK are serial offenders. Newspapers that have no compunction about invasions of privacy or about shrill comment devote precious little time or energy to challenging authority through rigorous investigative journalism. Most political "scoops" are merely stories planted by politicians on pliant lobby hacks. Editors and senior journalists are habitually invited into MI5 and MI6 for briefings. These are affable occasions, often over lunch. There is no harm in that. What tends to happen, however, is that journalists are tickled pink by the attention. They love being invited to the "D-notice" committee to discuss how they can all behave "responsibly". It makes them feel important. Many suspend their critical faculties as a result.

Far from being "feral beasts", to use Tony Blair's phrase, the British media are overly respectful of authority. Newspapers and broadcasters tend to be suspicious of those who do not play the game, people like Mr Assange who are awkward outsiders. Some editors are quite happy to help the authorities in their denunciations of him, partly out of revenge for not being in his inner circle.

All governments have a legitimate right to protect national security. This should be a specific, and closely scrutinised, area of policy. Most of our secrecy rules are designed merely to protect politicians and officials from embarrassment. Documents are habitually over-classified for this purpose. The previous government made desperate attempts to stop legal evidence of its collusion in torture from reaching the public. Ministers argued, speciously, that this was to protect the "special intelligence relationship" with Washington. It will be intriguing to see how much information is allowed to be published when Sir Peter Gibson begins his official inquiry. Precedent suggests little grounds for optimism.

As with all free speech, as with Wikileaks, context is key. It is vital to know when governments collude in torture or other illegal acts. It is important to know when they say one thing in private (about a particular world leader) and do quite another in public. It is perturbing to know that aid agencies may have been used by the military, particularly in Afghanistan, to help Nato forces to "win hearts and minds".

These questions, and more, are vital for the democratic debate. The answers inevitably cause embarrassment. That too is essential for a healthy civil society. Good journalists and editors should be capable of separating the awkward from the damaging. Information that could endanger life, either in the short term or as part of a longer-term operation, should remain secret.

Once this latest flurry is over, prepare for the backlash. Mr Assange's industrial-scale leaking may lead to legislation in a number of countries that makes whistle-blowing harder than it already is. Perhaps the most curious aspect of the Wikileaks revelations is not that they have happened, but it took someone as mercurial as Mr Assange to be the conduit. Rather than throwing stones, newspapers should be asking themselves why they did not have the wherewithal to hold truth to power.

John Kampfner is the chief executive of Index on Censorship