Yasmin Alibhai-Brown: No democracy should declare free speech an absolute right

Too many states use brute force to quell and gag their people. In our western democracies, governments withhold information, stop legitimate protest, control speech and even thought. All wrong, must be resisted, agreed. Most of us, though, will not speak with one voice on the burning of the Koran by Sion Owens, a BNP candidate for the Welsh assembly. And what about the website that sells cheeky Jihadi, al-Qa'ida baby T-shirts and maternity clothes? Tory MP Robert Halfon is apoplectic and wants the site closed down. Are you with or against him? Do we teach children that words can wound or that their entitlement to speak trumps everything else?

Freedom of speech is endlessly discombobulating and testing. In the unspoiled meadows of ideals or unbound skies of philosophical postulations, it is easy to be unequivocal. Some in the real world, too, are enviable absolutists who believe the slightest tremor of concern is a concession and invitation to authoritarianism. Their God is Voltaire, who decreed that even when one hates what is being said by somebody, one must "fight to the death" for the right of that person to hold forth. (Noble rhetoric. Correct me if I am wrong, but I can't think of a single such martyr).

A protracted and violent struggle against mental tyranny was fought by Europeans and today in the Arab lands citizens are inspired by the same emancipatory, human impulses. However, Voltaire's spiritual children can be fundamentalist, thoughtless and irrational, blind and deaf, unresponsive to the complexities of modern life, of individual and group psychology, inequality and power. Freedom of expression is not black and white, but a thousand shades of grey. Its meaning and practice need to be unpacked. Each situation demands exhaustive and exhausting analysis before informed positions can be arrived at.

I was on a panel at the Oxford Literary Festival last week trying to do just that with journalist David Aaronovitch in the chair, and John Kampfner, chief executive of Index Against Censorship, and the blogger Guido Fawkes, who has (inexplicably) become an unaccountable and scary political force. For Fawkes anything goes. Easy, though not for those he picks on. Kampfner is an indefatigable campaigner against legal and official curtailments, the use of money by the rich to enforce censorship through the courts and unjust control. I agree with him most of the time.

When the powerful come down heavy on citizens or communities and vigilantes do the same, they must be resisted. It is intolerable that artists are inhibited, imprisoned or killed as just was Juliano Mer-Khamis, the exceptional Jewish, Israeli- Palestinian actor and founder of the Freedom Theatre in Jenin. Members of Hamas are allegedly behind this barbarism. And here our very own local religious hoodlums have threatened to kill Usama Hasan, a lecturer and London imam, because he refuses to reject evolution.

Come away from dramatic confrontations and the law to more intractable conflicts. Then it gets awfully complicated. The web is a wonderful liberator but also a nameless, shameless sniper. Professional blogger Lorraine Van Fossen rightly warns that when people express anything and everything, "... there are consequences, the right to react, that other freedom." That other freedom – disrespected by most libertarians. Saul Bellow complained much about the closing down of public discussion in the US: "We can't open our mouths without being denounced as racists, misogynists, supremacists, imperialists or fascists." He blamed the media. But those respondents were exercising their right to react, through verbal means. As I do, to the fury of many who would say they are righteous free speechers.

Frankie Boyle will, I expect, feel put upon by Ofcom, which lightly slapped his wrist for grotesque TV "jokes" about the disabled son of Katie Price. The FA is deciding what to do with Wayne Rooney, who swore horridly on TV. The footballer – who has apologised – must be crying into his champagne. I hope he gets his comeuppance. The public space is shared and most people watch what they say to make it less fraught and more liveable. We stop ourselves and our kids from saying rude and nasty things because we understand there have to be some social constraints on speech. And if you don't watch your mouth, you have to take what follows.

In 1919, the US Justice Oliver Wendell Holmes decreed that the only limits to freedom of speech were words that activate immediate danger, like a man shouting "fire!" in a crowded theatre. But what about when individuals set out calculatedly to provoke unrest and anger, which then happens? Like the burning of the Koran. Of course the offended should not rage and die for it – but that was the intention. The inciters are surely as culpable as the man in the theatre. They raise hatred, which eventually leads to violence. Jewish people, Tutsis, Bosnian Muslims, and millions of others were slain easily because words had taken away their humanity. The right-wing press has so demonised asylum seekers that today the UK Borders Agency presumes all applicants are liars unless they can prove otherwise. Words have institutionalised a grave injustice.

Young people bullying others through social network sites don't want the victims to try kill themselves, but many do. It is not immediate, but still evil. Internet abusers never have to pay for the breakages they cause. Kierkegaard worried that newspapers, "a dreadful, disproportionate means of communication", could send "any error into circulation with no thought of responsibility." How much more wanton is new technology. Those protecting the wild web from "regulation" should attend to the severe restrictions on free speech imposed by libel laws, confidentiality agreements, injunctions, and litigious individuals. We are not as free as we think, and to argue as if we are is disingenuous.

Another thing to consider is that most of us are biased. We want some words to be free, and others not. Will the Koran burner be backed by libertarians, atheists and Muslim bashers? Or will he face the same opprobrium as those Muslims who burnt Salman Rushdie's book? I await Fay Weldon and Ian McEwan's beautifully expressed outrage.

Buddha said: "The wise fashion speech with their thought, sifting it as a grain is sifted through a sieve." We need to be wise to use and preserve our precious freedoms. Sadly, we are not wise.

y.alibhai-brown@independent.co.uk