Dom Joly: I make a great garden gnome, but a lousy house-seller

Trying to film a hidden camera show in this "summer" is starting to get me down.

The rain is relentless and drives the great British public (the essential element of my show) back into their houses to peer through their net curtains at the desperate little band of people driving around their town looking for people to confuse. Last week we were back in Weymouth. The previous time we were there, about a month ago, we didn't press record once. We were washed out and had to leave the south coast defeated. This time, however, we were back with a brief weather window in which we attempted to get as much as possible in the can.

On our first night, we headed off for a slap-up meal at a lovely seafood bistro in the harbour called Vaughan's. The food was superb but I was rather put off by a curious little plaque that was attached to the wall. It read: "The 'Black Death' entered England in 1348 through this port. It killed 30-50% of the country's total population." They should have added, "now wash your hands". It almost felt like Weymouth was proud of this achievement. I imagined sailors sitting around the docks going, "You know that Black Death thing? That was us, that was. One of my rats did that ..."

There were no such sailors that particular evening but there was "pool night" at the pub next door. This meant that as we exited the bistro there was an abundance of tough-looking fellows hanging around carrying pool cues. I panicked and took sanctuary in another wonderful place called "The Stable" which specialised in pizza and cider. The pizzas were incredible but we had already eaten so we concentrated on the rack of exotic ciders. This was a big mistake. Cider and filming are not a good mix. I remember little of the rest of the evening. The next thing I remember was being woken up, taken to the make-up room and turned into a rather evil-looking garden gnome. This could well have been a dream but I have photographs. As I wandered towards where we were going to be shooting, several kids spotted me and looked horrified. I fear that I might have kick-started an epidemic of gnomophobia (a condition that I am assured exists).

We were filming on the coast, in somebody's front garden which faced on to a pathway. I leant on the garden gate and engaged with whoever happened to walk past. Things went rather well, the weather held, and we got some good stuff. After about an hour or so we cut for a break and I sat in a chair in the garden drinking a very welcome cup of coffee. That was when I noticed an estate agent showing a family around the house next door which was for sale. They came into the seafront garden and the agent was busy describing the merits of the place. The family, however, were not listening. They were staring at me, a 6ft human gnome in the neighbouring garden. There was no crew about – it was just my potential new neighbours and me. I couldn't resist. "Hello," I said waving a little too frantically, "moving in are you? I've got a hot tub ..." They smiled politely and backed away into the safety of the house they would now never buy. Sometimes the funniest things are best unfilmed.