Dom Joly: I'm not racist, but ... those Brits should just go home

I'm worried that I might be racist. I've tried my best to get along with them, accept their cultural ways and live and let live, but I just can't. I cannot abide being on holiday near other Brits.

This is a particular problem for me this summer as I have forsaken the joys of Canadian lake-life for a holiday in the Dordogne – spiritual home of the middle-class Brit abroad. Every time I venture out from our isolated retreat I am surrounded by people called Jeremy and Emily who wear rugby shirts and espadrilles and believe that the secret to speaking a foreign language is simply to repeat yourself louder and louder until the "Froggy" surrenders and gives you what you want. "What you want" seems to be baked beans, Marmite and curry paste – there is a whole section in the local supermarket dedicated to English culinary tastes. Last week it was Tea and Sympathy in New York; this week it is the rather less glamorous Shop in Monségur.

Things ratcheted up a notch when we went shopping at night on Wednesday. This was quite a lovely occasion, with wonderful-smelling food stalls surrounding a central covered market. Once you had made your purchase you sat down at communal tables and listened to a fat man and his even fatter wife murder some rock 'n' roll classics from a small stage. As I joined the line for some succulent-looking pork and potatoes a rather louche-looking man with floppy hair and a tight purple jumper squared up to me. "Are you Dominic Joly?" he asked. I nodded. "I'm Giles Stiffwalker" (not his real name). "We were at prep school together."

My life flashed before me. It was like that Roald Dahl story "Galloping Foxley" where the protagonist sits opposite a man who had bullied him at prep school. I remembered Stiffwalker vividly, partly because he had barely changed in appearance from his nine-year-old self save for having acquired a taste for very thin, tight, tasteless jumpers. Mainly, however, it was because he had made four years of my life a living hell. I pictured him pinning me against the lockers in the coatroom and repeatedly punching me in the face. I remembered hiding behind a chest of drawers in an empty classroom block and hearing Stiffwalker and his partner in crime, an odious creature called Wigram, checking the building, room by room, like vengeful marines in Fallujah.

"Are you staying round here?" brayed Stiffwalker as he tried to herd a gaggle of mini Stiffwalkers towards the pizza stall. As it so happened, I was holding a semi-automatic machine gun at the time. Admittedly, it was only of the BB variety, just bought in a highly responsible fashion for my six-year-old son (it's not my fault, it's Stiffwalker's– he's responsible for everything). I fingered the safety catch. I could just blow him away here in the square and finish him off with the baguette I had in my other hand.

"I know the area really well. I summer here every year," Stiffwalker was droning on, and I hadn't killed him yet. Sadly, I already knew that I wasn't going to say anything. I was going to nod and smile and shake hands in that oh so English way, and then wander off and fester in a corner of this French town, fantasising about all the things I should have done.

I saw him only one more time. It was through the railings of the market and he was buying ice cream for his progeny of mini-bullies. I stroked the trigger of my gun longingly but I was neutered – cemented to a long table listening to the fat couple on stage attempt to perform "La Bamba", the international song of choice for the musically incontinent. Only 14 days to go.