Peter York: I'll be the one with huge gypsy earrings sitting next to Lionel Blair

From today lunchtime I'm going to be with Lionel Blair in Sloane Square. No, stop messing about. We'll be right in the square, where normally there's just the flower-seller and the lavs and the taxi rank. Lionel and I will be available in booths, by appointment. You'll recognise me, I'll have the big gypsy earring and the old palmistry books. Come and see us between one and two. It's for three charities for the homeless.

It'll be weird, I can tell you. The whole of the centre's been covered with tents and booths and stages and stalls. It's the Fayre in the Square. It's Chelsea's new Elizabethan Jubilee event and the line-up is, well, surreal: a culture mash involving an Abba tribute band, a member of The Royle Family (Gran, Liz Smith), Sylvia Sims, Emma Forbes, George from George and Mildred (Brian Murphy) and me. As you can see the emphasis is on the healthier strand of British Empire family television entertainment. There are three bands, there's fireworks at 10pm.

And it's all about the Queen-and-Country side of Chelsea too, the cheery Sloaney, Cadogan Estate and Holy Trinity Sloane Street side (they're the organisers) and less about the greenery-yallery Tite Street artistic pallor aspect. That's all very well in its way I dare say. But not exactly Golden Jubilee material.

When the call came, from one of the organisers, my old friend and former flat-sharer Caroline, I had to go. Nearly 20 years ago Ann Barr and I wrote in The Official Sloane Rangers Handbook that, in Sloane-think, "Sloane Square is actually the spiritual and geographical centre of the known world". Peter Jones, the Sloane's own quartermasters' stores, was there and so was The General Trading Company, a shop so Sloane it either rang those inner bells or said "absolutely not for me" to radical modernists. The Square was where you met, if you'd come up from Sloanshire (one Tube stop from Victoria).

Neither Ann nor I lived in Chelsea in 1982 – the world had moved a bit west – but we had. I'd two Chelsea flat-share goes in the Seventies – first in the crotch of King's Road, just before World's End, above a hairdressers'; then later in Milner Street, opposite the Australian pub. The Australian was a crucial Sloane centre and a real notebook-filler with its braying boys and cheerful blondes. You had to have lived in Chelsea in your twenties. As long as I've known it Sloane Square has been a place of constant pilgrimage. Out of the Tube station they'd come, those tired and huddled masses, those French girls doing au pair in Highgate, those punks from Beckenham and those American kids who'd read about the punks, strap-hanging next to Caroline and Henrietta who were just going one stop to meet a girlfriend for lunch.

Even then there was a Sloane diaspora. Sloane aunts and grannies lived in symmetrically furnished flats on Sloane Street but the girls had started to fan out, not just along Fulham Road (remember Diana, Princess of Wales in Earl's Court) but way down south, south of the river to Battersea and Clapham and even edging towards Wandsworth. But the point about all these little ex-pat ghettos was how they continued to celebrate the styles and rituals of the old country, how all those colonies desperately aspired to the condition of Sloane Square. You don't think of Sloane Square as Georgian – waves of rebuilding means it presents as Edwardian – but it was. The original layout of 1771 had a village green with posts and chains.

It wasn't London then, not real Westminster and West End territory, but more Brighton-ish, with the old Ranelagh Gardens and its Pleasure Dome, the Rotunda and the original Chelsea Bun house by the entrance. (Today's Fayre has a Chelsea Bun house too. They're made by the Carlton Towers kitchen.) You get the picture: a raffish resort, a bit louche, for toffs and made-its, nothing so serious as big London. Annoying, a bit tarty, but constantly attractive and lucky. You get better weather in Sloane Square.

Anyway Lionel and I are hugely looking forward to seeing each other. It's been a while.