Triumph at last for Julian Barnes (mixed with a touch of venom)

When Julian Barnes reached the Man Booker prize podium and snuck a sheet of paper out of his pocket, there was a vague sense that his winner's speech was a variation on words penned in hope decades ago, but which had since become curdled by the long wait.

He was first shortlisted in 1984 (Flaubert's Parrot) then in 1998 (England, England) and again in 2005 (Arthur & George). So when the speech finally got an airing, it was laced with the stewing frustration that had preceded his victory. He was, he said, "as much relieved as delighted to be receiving the prize."

The vindication bordered on self-righteousness, despite the surface humour. He offered an anecdote about the Argentine writer Jorge Luis Borges's near-misses with the Nobel Prize, which led him to imagine a conspiracy against him; "I started to think that perhaps there was a similar system in operation here."

There have always been levels of Booker distemper among writers who experience repeated near-misses, but Barnes's venom ranks highly. In 1985, he wrote derisorily about the prize, calling it a lottery akin to "posh bingo" and suggested it revealed far more about the psychology of the judges than it did the quality of the shortlisted books. This week, he maintained his "bingo" analogy, albeit in a more conciliatory tone.

"Without sounding like a self-correcting politician... Booker has a tendency to drive people a bit mad. For writers, it tends to drive them mad with hope and greed and expectation. The best way to stay sane is to treat it like posh bingo. I am relieved, yes, this was the fourth time I'd been shortlisted. I didn't want to go to my grave and get a Beryl [referring to the posthumous award for "Booker bridesmaid" Beryl Bainbridge]."

Barnes , 65, was equivocal when asked whether his winning novella, The Sense of an Ending, was his "best" work.

"[The best novel is] the one I'm about to write or the one I've just finished writing. If I thought I had written my best novel already, I'd find that depressing... I'm very attached to Flaubert's Parrot."

There are some who are as equivocal, and also those who feel this was Booker's way of rectifying past wrongs, with a "lifetime-achievement award" of sorts.

The day after the ceremony, Barnes did not want to discuss the victory nor his novella, as Man Booker winners have always traditionally done. Perhaps it was diffidence that led him to his silence and not churlishness. A source at Jonathan Cape, his publisher, explained that he has not given any lengthy interviews since his wife, the formidable literary agent Pat Kavanagh, died in October 2008. They had been married since 1979, though they separated for a short period when Kavanagh left Barnes for the writer Jeanette Winterson.

Touchingly, he had dedicated his books "to" Pat when she was alive, while the dedication in The Sense of an Ending reads "for" Pat, in commemoration of the loss.

In a Time Out interview just before Kavanagh's death, he described his own fearful moments, of being "pitchforked with terror" by the awareness of death. His moving meditation on mortality, Nothing to be Frightened of, which mixed essay and memoir, was published six months before Kavanagh was diagnosed with a brain tumour.

"I've had night terrors for decades... All bad things are exaggerated in the middle of the night. When you lie awake, you only think of bad things. The trouble with death is that it sometimes wakes you up!"

This is the first novel that Barnes has written following Kavanagh's death and reflections on history, memory and loss (repeated themes in Barnes's oeuvre) are here, particularly profound. The protagonist, a plodding antihero in his autumnal years, is also the unreliable narrator who reflects back on youthful friendship and a seminal first love. The novel explores philosophical arguments around history – both collective and personal – and the illusions and myths that become "fact" in our memories. As the narrator ponders: "How often do we tell our own life story? How often do we adjust, embellish, make sly cuts? And the longer life goes on, the fewer are those around to challenge our account, to remind us that our life is not our life, merely the story we have told about our life. Told to others, but – mainly – to ourselves."

Born in 1946 to two teachers of French, Barnes studied at Oxford and after a period as a lexicographer for the Oxford English Dictionary, he turned to journalism – he was a TV critic for five years and the New Yorker's London correspondent.

His first novel, Metroland, which like The Sense of an Ending, focuses on young male friendship, was published in 1980, but it was his third book, Flaubert's Parrot, that established his reputation among a set of writers including Martin Amis and Ian McEwan who would come to define a generation.

The novelist Toby Litt judged the Guardian First Book prize with Barnes in 2000 and remembers the writer to be a meticulous panelist. "He had taken tiny handwritten notes on 3x4ins cards. I got the impression he did everything like that. There was a list of eight books and the winner was Zadie Smith's White Teeth. There were two main contenders. Barnes was funny – he went through a list of the factual errors in White Teeth, including the fact that the (description of) the arrivals lounge at Heathrow was back-to-front. But beyond that, he wanted it to win. He had really analysed all the books and written notes in his tiny writing. He was an authoritative jury member."

Curiously, five years later, both Smith and Barnes found themselves on the Booker shortlist, but were pipped by John Banville, who won for The Sea. Professor John Mullan, an author and former Man Booker judge, said he would dispute Barnes's comparison between Booker and bingo, but conceded that Barnes had been unlucky in his losing years. "It's not so much that in the times before he had been done a terrible injustice, but that his books turned out to be unlucky enough to be shortlisted in a very good year. When he was selected for Arthur & George, it was an incredibly strong year [including books by Sebastian Barry, Kazuo Ishiguro and Ali Smith]".

Barnes may be following in the footsteps of those Man Booker winners who prefer temperance over open gleefulness. Or his muted sense of triumph could reflect a buttoned-up Britishness that this latest book encapsulates, and of which he has spoken in the past. In a 2006 BBC interview, he spoke of the bloating of emotions that abound on reality-TV shows. "I prefer emotional uptightness to opera-fication of emotions," he said.

In the same interview, he expounded on his "posh bingo" theory, in an anecdote featuring Ian McEwan. The two are friends who came up against each other on the 1998 Booker shortlist; McEwan won for Amsterdam: "You don't as a novelist write a book to win the Booker prize. You should treat it as a game. When my great friend Ian McEwan and I were both in for the Booker prize, he said, 'the judges get in and spin the bottle'. It came to Booker night and to my certain alarm, Ian McEwan won it. I gave him a big hug and said 'they spun the bottle'."

Perhaps McEwan is saying the same thing to him now.

Order 'The Sense of an Ending' for £10.99 with free p&p (normal price £12.99) from the Independent Bookshop 08430 600 030