Alex James: 'Daphne's so posh, it is hard to believe she exists'

I'm building a castle in Oxfordshire, it's got everything. Trout farm, more or less, but I'm in London with no toothbrush and the same clothes on as yesterday, tramp-style. I'm no longer a Londoner. It hasn't been long, I'm still getting gas bills for when we lived in Covent Garden, but I can tell I'm definitely not a Londoner anymore because I stare at everything in wonder and the Tube is beautiful. A world of possibilities.

It's the best time to plant trees right now. I know, because I planted quite a few in spring and spent the summer watering them. They don't have any vigour of youth, trees. They have a rather forlorn, spindly appearance, rather depressing, unlike new grass, which is emerald green and so alive you can almost see it wriggle. A sunny morning makes our whole garden change colour, and where there were piles of scaffolding and concrete mixers is now a whispering lawn.

Since buying the farm, I've been cramming books on shrubs, flowers, meadows and all the aspects of garden design. Having gone from a first-floor balcony with a bit of lavender to 200 acres of weeds in a single bound, there has been much to discover. Reading gardening books really takes it out of you. People evidently pour their heart and soul out when they write them; they are often a life's work. The one I have come to trust is a chunky, no-frills tome called The English Formal Garden, translated from German. It was my constant companion on the road last year, as champagne might have been in the past. Thoughts of home, and the idea of a beautiful garden, kept me warm at night. At one point I even considered calling my son Lancelot, or Capability, or even Brown.

When you buy a farm, ab initio, you need someone who knows what's what to help you. These people are called "land agents" and they know everybody and everything. Apart from the architect, whom I talk to the most, Paddy, the land agent, is my closest companion. He says things like: "Nick's going to come and do the paths", and "Charlie's going to do the ditches and the splash", or "I've got a cheque from Fred". I know everything is taken care of.

But it was when Paddy said, "Daphne's coming about the garden", that we got really excited. Daphne is so posh it is hard to believe she exists. She was disgusted at the suburban-ness of our garden. Absolutely appalled. She took a lot of photos, like a photographer at a murder scene, and went away, furious. I was rather offended, having rather liked the garden, and didn't call her.

A few months down the line, after some ruthless decision-making, we realised we had removed everything that she had been nasty about and that maybe she was a genius, after all. She came over on Sunday, indignant, flabbergasted that hunting had been banned and we had a wander round. There are really just a couple of beds that need planting out. I suggested that I rather liked cardoons, a kind of purpley thistley cabbage. She completely ignored me, apart from pausing to narrow her eyes. The thing about hiring in expert consultants is that you have to find the point to which you're willing to defer to their greater knowledge. Daphne hasn't probably read that many books about gardening, but she's spent her life doing it, up to her elbows in earth. She knows what's going to work where. She definitely has me down for a whack job. I heard her mention the name "Capability" Brown in connection with some local formal gardens, and thought I'd try to make up some ground.

"Ah," I interjected, "Lancelot..." She gave me one of her looks, and stopped talking. "...laid out those gardens...?" I said, hopefully.

"They're not quite that early," she replied. She thought I meant the round-table fancy-pants jouster, not the man who designed the gardens at Blenheim Palace. What a mess. I just let it ride. The difference between people who've read books about things and people who do the things is a beautiful thing.