Eye witness: Little Joseph, 8, is a Millwall fan. Poor kid

So you're at the theatre, watching the panto with the kids, when half a dozen burly blokes start shouting from the stalls: "Who's the bastard in the frock?" Or worse. Incensed by the Widow Twankey's inability to find Aladdin they stand up in their seats and hurl abuse: "Fuck off, you blind cunt." Then the biggest, baldest, fattest one throws a bottle of beer at her.

Forgive the coarse language, but men are yelling exactly those words, very loudly, all around me at the New Den, home of Millwall Football Club. Except for the bit about the frock, obviously.

The swearing is no surprise to someone who has been going to matches all his life, but the energetic nastiness of this crowd is a shock. Whatever happened to football as cleaned-up family entertainment? Aren't the grounds supposed to be full of affluent new customers waiting quietly to be entertained, like in the theatre? Or rather, given the increasing number of children at games, like the panto?

Oh no they're not, it seems. In the last week football's much-hyped new image has looked in danger of disappearing quicker than a genie in a puff of smoke. Audience participation went too far on Sunday at Cardiff City, where the police used dogs and batons to control rioters. Missiles and punches were thrown. The Chelsea players were not laughing on Wednesday when they got hit with coins thrown from the stands at Stamford Bridge. Among them were John Terry and Jody Morris, the latest young England stars to find themselves in court for affray. The FA has censured Fulham and Everton for brawling on the pitch; the National Criminal Intelligence Service confirms football violence is on the increase.

Are these signs of a slide back into the dark ages? Doomsayers say so, most fans say no. Keith Ewing hopes not, as we take our seats at the Cold Blow Lane End with his eight-year-old son Joseph. This First Division contest between Millwall and Birmingham matters a great deal to Joseph but Keith is equally attracted by the chance to share an experience with his boy. They would probably have kept away from Millwall in the Seventies. Even now it has the most notorious supporters of all, which is unfair on a club that has worked hard to eradicate racism and expel the thugs. Slowly it has attracted people like the Ewings, who only moved within walking distance of the ground because it was a relatively cheap place to buy a flat.

When Joseph turned six, Keith decided to take him along to see the local team. A woman from their church already went with her son, so it seemed natural.

"In a season and a half of coming here I have not seen any trouble at all," says Keith, 41, who works for a relief and development charity. "I have never felt threatened and nor has Joseph."

Then he sings, with the crowd: "No one likes us," to the tune of "Sailing". "No one likes us, we don't care."

In 1993 Millwall built a new stadium. Unfortunately the board spent its money on four concrete sheds with less atmosphere than the industrial wasteland in which they are set. The flooded, overcrowded toilets are like a pre-war pigsty. Long queues make a traditional half-time Balti Chicken Pie impossible.

The pie can't be up to much, even at £2.30, because somebody throws one at the linesman. Soon there are plastic bottles, coins and lighters on the pitch. The tannoy warns the game could be abandoned. "Good fucking job!" says a loud voice. Millwall are a goal down. "I've never heard an announcement like that before," says Keith, sounding worried.

Both managers get red cards, as does a Birmingham player. His boss says the atmosphere got to him and I'm not surprised. All football crowds are partisan but this is the most humourless, mean-minded bunch I've ever been with. They even seem to hate their own side. Keith, a laid-back New Zealander, says the verbal violence is not a problem. "It is naive to think Joseph won't hear those words at school. This gives us a chance to talk about it."

For now Joseph is too young to know what players do in night-clubs, but he is influenced by what happens on the pitch.

The perception of players has changed dramatically in the last 10 years. When Gazza was treated like a genuine pop star after he cried at Italia '90 he was unusual. Now every club in the Premiership has a couple of those and markets them heavily to young fans.

"Players are such huge role models now and I get livid when I see their behaviour," says Keith. "They are top athletes who earn a great deal. The price is a responsibility to the community that pays their wages. But if clubs give teenagers huge sums of money they have got to help them understand how to conduct themselves."

The match ends in a draw. We move through the muttering crowd, through puddles and over mud, under a railway bridge, heading away from the few fans that have bothered to come down from Birmingham. There will be no fighting tonight, it's too cold and wet, but the next morning we hear six people have been banned for life. Sounds like a mercy to me.

"We won't accept anything being thrown on the pitch," says the chairman Theo Paphitis. "We will stamp on it hard. If you allow a meat pie today, who knows what they will be throwing tomorrow?"

The Ewings might not be around to find out. They and other new fans brought to the game in the football-friendly era of Fever Pitch will not accept violence as part of the package. "If this sort of thing starts to go on every week," says Keith, "I think we'll be staying away."