Howard Jacobson: I don't believe that if Eton was closed down then every school beneath it would improve

It was love at first sight. The first time I saw the young man in the odd silk socks – one foot red, the other blue – I fell in love with him.

I was at a dinner given by an Indian art collector, by which I mean a collector of art who happened to be Indian, not a collector of Indian art, though the house was full of Indian artefacts, perfumed by Indian incense, and Indian food was served. I stress the atmospherics to explain why I was going soft on a young man in odd socks, since going soft on young men isn't what I normally do whatever they're wearing.

Of course I didn't really or even nearly fall in love with him. But I was struck to an unusual degree by how well he handled meeting new people, by his manners and adroitness at a banquet where it would have been easy to slop food around and spill it down your shirt front (as I did), by how knowledgeably and confidently he spoke about contemporary art, and by his wearing odd socks in the spirit of a man who knew it to be a stylish thing to do so long as you could do it well.

Behold the benefits of a public school education. At my north of England grammar school you'd have been laughed at for wearing odd socks, called a nancy boy for wearing silk socks, and been expelled for wearing them of that colour. I won't, as a rule, hear a word against my education. In the grammar schools of the 1950s we addressed our teachers as "Sir", suffered light corporal punishment whether it was deserved or not (good to discover early that there's injustice in the world), read Wordsworth, learnt Latin, and didn't carry knives. End of discussion. But they didn't teach us style. For style you had to go to Eton.

I was no friend of Etonians in odd socks when I first encountered them at university. I hated Cambridge – hated it with an ardour that still burns – and that had much to do with the ubiquity of public school boys, whether they were aesthetes or hearties. It wasn't simply that they posed or rugger-tackled their way through Cambridge as though the university had been built for them alone, it was their obliviousness to me. I didn't count, they did. I wasn't anyone, they were everyone. And they knew one another even when they didn't actually know one another. They turned up acquainted in the abstract. Not just Cambridge but the universe was a private club for them, entry to which was assured by something as airily indefinable and yet as fiercely prohibitory as demeanour. Just that: bearing. They looked as though they belonged, and so they did. I didn't, and so I didn't.

So if I have a complaint, what is it? That their demeanour excluded me and that they should therefore not have had it? Or that I wish I had had it too? Is the virtue of a thing to be measured by what it's worth in itself, or by the misery of those who for whatever reason miss out on its benefits? Is good contingent on its general availability?

I ask these questions in the light, partly, of Alan Bennett's remarks about public schoolboys on Radio 4 recently. Remembering them as louts who, though admittedly better taught than he was, hogged the rolls and slurped the soup, Bennett wants to do away now with the institutions that bred them. I stumble over his admission that public schools provided better teaching. Isn't that a reason to keep them?

As for their being louts, weren't we all loutish in our way at 17? I slurped my soup and once threatened to throw a Jesus man down the steps of the Master's Lodge at Downing College because he wouldn't stop gatecrashing our seminars and provoking F R Leavis into telling the same stories about "that worm T S Eliot". "Mention Eliot again and you're dead meat," I told him. (The Jesus man, I mean, not Leavis.) Nothing to do with Eton. I was a grammar school Leavisite lout. And certainly the Etonian of my recent acquaintance, the one who wore odd socks, had nothing loutish about him. What he had been taught was how to appraise beauty, how to hold a conversation – in short, how to think and how to be a gentleman – and what's wrong with that?

Unfair that I hadn't been taught to be a gentleman, I agree. But wasn't that my school's fault? With all its virtues, it left out that: educating us to move with suaveness and aesthetic know-how through society. Is it the inevitable consequence of a two-tiered education system that the modern comprehensive sends its pupils ignorant, armed to the teeth, and full of curses into a world they consider hostile? Does inequality do this, or does the fault not lie, rather, in the social attitudes of those who administer and teach at comprehensives?

If we think good manners and assurance are tainted by association with wealth and privilege and are therefore not to be esteemed, we are to blame when only the wealthy and the privileged have them. I see nothing in the assumptions of egalitarian educationalists to make me believe that when Eton goes every school beneath it will move up a notch.

This is not a good time to be arguing the case for public schools. Though the disgraced Tory MP Derek Conway attended a comprehensive followed by a polytechnic (so not much of an advertisement for either), his munificently rewarded researcher-sons are both Harrovians, and the more we learn about their lifestyle the more it corresponds to that toffish loutishness Alan Bennett describes.

If Henry and Freddie Conway are the best Harrow can do, then, yes, firebomb it tomorrow. But anyone watching last week's television documentary A Boy Called Alex, about a gifted 17-year-old musician with cystic fibrosis, will have observed a different aspect of public school education. The film followed Alex's attempts to defy his illness and conduct his school – he is at Eton – in a full-scale performance of Bach's Magnificat. That's a lot of singers, a lot of musicians, a lot of musical know-how, and considerable patience and kindness, for a single school to lay its hands on. And behind it all, highly dedicated music teaching.

Few state schools have the money to provide that. But there are more modest works by Bach they could introduce their pupils to if they had the intellectual ambition. Thrown in with everything else the rich want for their children is cultivation. We shouldn't resent or wish to abolish that; we should emulate it.