Yasmin Alibhai-Brown: Racism will go on rearing its ugly head until we shout stop

There I was in Hay-on-Wye, with the sun high, nature so vibrantly green, I imagined God in a smock, painting the scene before kindling life into his masterpiece. And better still was the How The Light Gets In festival of ideas, (partly sponsored by this newspaper), listening to supreme intellectuals sharing their quirky insights, seeing young people passionately engaged, meeting thoughtful punters – it really did feel like one of the best weekends of my life.

That bliss was unexpectedly shattered at around 4.30 pm on Saturday. I'd just been on a panel with two speakers discussing the web, the effects of travel and what home is. Then I went for a drink with two young black women. We found a table with a woman on her own, asked politely if we could sit there and she smilingly assented. She was in her early fifties, short, dark hair, wore smart-casual gear in neutral colours. Nothing about her seemed crazy or untoward.

We were talking about university students from Pakistan, most of whom are now refused visas. In a bitter voice, the woman cut in: "Good. We don't want them here. They are liars, forgers and scroungers." We stopped, shocked and silenced. She went on – I can only paraphrase. She claimed she was an academic at Cardiff University and they all knew this to be true. I felt outrage rising and told her I was a visiting professor at Cardiff and wondered if that is what she felt about her Muslim students. She discharged more projectile bile. They should go back and so should I. This country was not ours. Something burst out of me at this point, an accumulated sense of injustice that black and Asian people are supposed to take it, laugh it off, excuse the racism as madness or ignorance and agree that these are just small glitches in a racism- free kingdom. I reacted with reckless fury. Very un-British. And I have no regrets at all.

In our times, to complain about racism is a crime. When the highly respected, white journalist, David James Smith wrote last year about his black wife and bi-racial children and the prejudices they suffered when they moved to Lewes, in Sussex, his effigy was burnt in the streets by residents. The messenger pays for the message. Many will resort to invective and chastisement after reading this column. They won't stop me, inured as I am after experiencing both for so long. I hope the open-minded will see that racism is not extinct and is getting worse as therecession bites and the hopeless "war on terror" goes on and on.

Granted that Britain is not the ugly place it was in the Sixties and Seventies. After the London bombings Britons did not avenge themselves on all Muslims. For our younger citizens, mixing across race, religions and cultures is no big deal. More Britons of colour are in positions of power and influence than ever before and more than elsewhere in Europe. I also concede that today, racism against whites by Asian and black Britons is growing and not honestly addressed. Too many Muslims are hideously intolerant of mainstream values.

None of the above negates the evidence of overt and unconscious race discrimination across the UK. There are no offset deals here. Some of my best friends, including my tender husband and many mentors, are white. That can't wipe away the upset of what happened in Hay. I could fill ten pages of evidence and data proving racial inequality still affects our chances, sense of ourselves and the innate optimism which sustains humans. Disbelievers add insult to ongoinginjury. Some of them have been given positions of high authority – great dividends for denying the truth.

There is a line connecting the seemingly trivial through normalised unfairness to lethal violence. Naomi Campbell threatened to sue Cadburys for using her name in a "playful" advert for chocolate. She has just had a grovelling apology. It isn't just another tantrum by the black model – she knows black kids hate "chocolate" nicknames, that focus on the colour of their skins. Violet, a young, African girl I know only eats white chocolate because "brown will make me more darker". Last week Simon Parker, a primary school head was sacked for racially abusing teachers and children. "Muslim fucker", "black bitch" and "bongo-congo"were his favoured epithets. Recently, an academic at the LSE, Satoshi Kanazawa, used pseudoscience to "prove" black women were less attractive than any other women. Naomi Campbell, you're fired.

Then there was the obscene panic generated by the population figures from the Office of National Statistics. Numbers are going up alarmingly, quoth the media, and because of migration and the higher fertility of mothers born abroad. And here we were naively sure that we are fully British, not lesser beings because our hues are not pink. Some middle-class Chinese and Indian Britons are doing very well indeed, but a new Joseph Rowntree Foundation report warns that the ethnic minorities are more likely to be poor, unemployed and paid less than their white counterparts.

Politicians now set the poor against each other using race; it is a way of diverting responsibility. Labour's John Denham even declared racism was yesterday's issue and the focus should now be only on the white dispossessed. Why not both? We don't need an official report on advertising to know that there is "drastic under-representation" of black and Asian faces on TV. That means fewer jobs for the actors who might have earned some good cash and recognition. Meanwhile, a Business in the Community study found that no profession is devoid of racism; only 30 of the 21,600 state school heads are black or Asian; an NUS survey found that those with foreign sounding names paid a penalty when their work was marked; London Metropolitan University has almost as many black students as the whole of the elite Russell Group; only fifty professors of 14,000 are black and there have been a number of cases of police officers assaulting and intimidating innocent Muslims. It goes on and on.

In a boutique in London a few months back, I saw two young teens, one black and one white, thrown out by the owner: "We don't want trash here. Go, please, now". The black girl spoke back and was subjected to terrible racist swearing. I fled and said nothing. In Hay I didn't bottle out. A time comes.