Sarah Sands: Exit Andy Gray and Richard Keys. Enter the Men's Defence League?

There is always a touch of comic pathos about a bully getting his comeuppance. Richard Keys on TalkSport in his shirt, tie and new delivery – cautious, pompous – could not disguise the dry-throated trauma. It was not just that he had lost his reputation and was about to lose a highly paid job – there was also a horrible, dawning self-awareness.

Where were the blokes of Britain rising in his defence? Why so little support from the fans? The football websites were clogged with comments from men who said they might have said the same, but that they'd never liked Keys and that he deserved everything coming to him. Suddenly he had discovered how others saw him and his colleague Andy Gray – not as enviable, roguish masters of their universe, but as a pair of plonkers. As the 18th-century poet Thomas Gray – no relation, as far as I know – explained when no one came to the aid of the family cat drowning in the tub of goldfish: "A fav'rite has no friend."

Let us be clear about the cause of their downfall. It was not Hattie Harperson and her joyless friends waving pieces of legislation. It was not even a corporate conspiracy, though several have suggested an evil strategy by the Sky owner, Rupert Murdoch, to deflect attention from the News of the World scandal. The swaggering pair seem to have been destroyed by their colleagues, who had silently endured their monstrous egos and unfunny jokes for years.

I don't know if the YouTube assassin who released the off-camera video footage was a woman, but it was a superbly feline and devastating revenge. The evidence was released with cool logic. The jokes about the young assistant referee were, I thought, quite funny, particularly since Sian Massey was right and the know-all commentators wrong about a contentious offside decision.

But it didn't seem so funny as Gray invited a young female colleague to stuff his microphone connection down his trousers. And Richard Keys's "Did you smash it?" questions to Jamie Redknapp were deeply unpleasant. Redknapp corrects his host with a weak smile: "I used to go out with her." Keys, fully in his stride and king of the studio, does not pick up the mood: "You definitely smashed it. You could have gone round there any night and found Redknapp hanging out the back of it." I worry that you have to have quite a low opinion of women if you ascribe them even the pronoun "it". And this from a man with a wife and daughter.

Gray's joke was adapted at the National Television Awards by the Top Gear team, but the tone was different. The menace had gone. Dread of rape is primal in the female psyche and aggressive lewdness is not just offensive, it is intimidating.

Reassuringly, much of the early comment from football fans was broadly sensible. No one much liked the policing of private banter, but few liked Gray and Keys either. If they were television's football superstars, it was only in their own estimation. Television stars survive on popularity. Putting the men on public trial had a Roman sense of fairness about it.

But as the week wore on, the debate moved from the specific to the general, and men rallied to the flag. It was now a question of free speech, or, as Jeremy Clarkson put it, "heresy of thought". A deep pool of male resentment began to form. Male journalists on the posh papers as well as the red-tops muttered that it was one law for men and another for Loose Women. And if total equality in sport was obligatory, why weren't men and women competing against each other, hey? Was the whole of football on trial? Were men on trial? Perhaps we might have our own Tea Party movement. In America, the disenfranchised were those who loved guns and God. In Britain, where else would the revolution start but football?

Sport commentary is a traditionally male domain. It is not stereotyping to observe that, for very many men, it forms the architecture of their conversation and friendships. Sport is a male sanctuary. Yet the television sports reporter – whether football, rugby or cricket – is now likely to be a young woman.

As you would expect, the BBC presents this move as liberal equality. Sky, on the other hand, doesn't bother to disguise a predilection for what Gray would describe as "lookers", preferably blonde. This from the station that professed to be horrified by the comments of its own presenters. We are only a step way from the lads' mags – boys and blondes, mates and muff. Have you ever watched Sky's Soccer AM show on a Saturday morning? I have, over the shoulders of adolescent boys. Each week a young woman fan, pouting in her tight football shirt, parades in front of the studio fans, smiling at the wolf whistles.

Younger sports broadcasters have shrugged off Gray and Keys as the last of a passing generation, who were unenlightened about sexism, racism or homophobia. Yet television comedy and men's mags are awash with knowing sexism. If Soccer AM is all right, then Keys and Gray are entitled to wonder why a defence that their "banter" was ironic wouldn't wash. Call it postmodern sexism and you can say whatever you like. Frankie Boyle, in Friday's Sun had a succession of quick-fire gags about men, women and football. "Gray is the type of man who believes there is no place in football for women. Which is just plain wrong. Who are they supposed to roast?"

Because the current generation of young men are post-feminist, the assumption is that they cannot be genuinely sexist. It is a joke and the girls are in on it. Yes and no. I have seen girls described by boys on Facebook in a shocking way. I have interrupted teenage boys' conversations to chide them. My resetting of their moral compasses takes the form of a simple, headmistressy question: would you like to hear your sisters talked about in this way? The white working-class culture of football, now appropriated by so many of the middle classes, is also hugely sentimental. The football fan who talks of a "stupid tart" is likely to be reverential about his mother.

Yet moving to crush all sexist comment causes far greater damage than the original offence: human relations are far too intricate and delicate for blunt directives. Tone and intent are too subtle for legislation. I am not offended when a bright-eyed, cheerful plumber asks to speak to my husband. I am furious when an able young woman working in an all-male City office talks of throwing in the towel because she is worn down by humiliating "banter" from powerful men. Boys will be boys, shrugs the wife of Richard Keys – but you don't say "dogs will be dogs" as your Staffordshire tears into someone's leg or shits all over the pavement. There are times to be indulgent and times to knock it on the head.

The removal of Gray and Keys has unseen ramifications. Clarkson is a formidable entertainer with a large following and only his contract with News International standing between him and a call to arms. And while no one would suggest Clarkson represents 50 per cent of the population, he speaks for many who feel they have conceded too much. Is male culture to be wiped out? Is a pleasure in cars, sport, pubs and male company to be outlawed? Are men condemned to a life of rom-coms and holding hands?

As Keys and Gray reportedly enter talks with al-Jazeera – and let's see how lads-mag humour goes down in the Arab world – they could still end up being martyrs of a newly formed and vigorous Men's Defence League.