IoS Books of the Year 2012: Literary fiction

The year began with the promise of big novels by major writers. Martin Amis, Ian McEwan, Zadie Smith, Richard Ford, and the Johns Banville and Lanchester all had new fiction in the offing. Just when you thought it was safe to go back to Hampton Court, Hilary Mantel was preparing the sequel to Wolf Hall. We braced ourselves for the opening of Will Self's Umbrella. And having sold half a billion books, J K Rowling conjured a grown-up pastiche of Anthony Trollope out of a local council election.

Looking back, all that anticipation feels faintly silly. Most failed to deliver in any obvious way. Zadie Smith's NW was ambitious and full of vivid sections that neither cohered into a whole, nor fragmented into a heap of broken images. Martin Amis's Lionel Asbo made me laugh (as any novel driven by incestuous relations with your grandmother should) and then feel ashamed for that laughter (as any novel driven by incestuous relations with your grandmother should).

I rather enjoyed JK Rowling's A Casual Vacancy, which at least dumbed up, not down. And while she can't write old people or anyone aged 20 to 45, she is brilliant on angry teens and parents. Nevertheless, perhaps only Self and Mantel emerged with reputations enhanced – something not even recognition by the Man Booker prize could tarnish.

Yet, none of the above make my Top 10. For some reason best known to itself, the Booker cocked a snook at Philip Hensher's glorious Scenes from Early Life (Fourth Estate, £18.99). Hensher's delicately measured but sensuous sentences captured a moving, closely observed and frequently funny family story set in Bangladesh in 1971. Nicola Barker's The Yips (Fourth Estate, £18.99), a bravura concoction of golf, social comedy and gloriously bonkers invention, did at least make the Booker longlist, but surely deserved to progress to the final group stages.

There were two bold, challenging works about South Africa written by novelists at opposite ends of their careers. Patrick Flanery's Absolution (Atlantic, £12.99) may have been a little too bashful about its thriller aspects, but this was a bold calling card from a major new talent. Nadine Gordimer's No Time Like the Present (Bloomsbury, £18.99) tackled similar subject matter: how those who risked their lives fighting Apartheid now struggle to adapt to life in "free" South Africa. As always, Gordimer's prose is a wonder of unsettling rhythms, vivid dialogue and striking imagery.

America provides its own May-September pairing. Anne Tyler's short, bittersweet The Beginner's Goodbye (Chatto and Windus, £14.99) was either a ghost story or a sideways portrait of a marriage. Meanwhile, Kevin Powers's debut, Yellow Birds, (Sceptre, £14.99) was inspired by his experiences as a soldier in Iraq. By turns shocking, philosophical, frightening and elegiac, it was deservedly nominated for a National Book Award.

2012 was a year of impressive first novels. Charlotte Rogan's The Lifeboat (Virago, £12.99) was a slippery but addictive read, propelled by one of the year's best premises: following the sinking of the Empress Alexandra, 39 survivors clamber aboard a lifeboat made for considerably fewer. The narrator, a deceptively cold fish called Grace, possessed one of the most striking and elusive voices of the year.

Alice Munro's inclusion in best-of-year lists is, I have been reliably informed, a legal requirement. Where Munro is concerned, the law is not an ass. From its finely weighted title to the four personal vignettes that end this exquisite collection, Dear Life (Chatto and Windus, £18.99) feels more plain-speaking than previous collections. I have already reached for it again to re-read these profound searches for lost time in small-town Canada. If someone has written 20 pages that are more unobtrusively powerful than the title story, then I am happy to eat them come Christmas. "There was quite a lot of killing going on, now that I think of it," is one of the lines of the year.

Finally, my two, somewhat unexpected favourite novels of the year: I have always enjoyed Jake Arnott's glam-rock gangster novels, but they hardly prepared me for The House of Rumour (Sceptre, £17.99). Confirming that the inter-linked short story is the coolest literary form du jour, Arnott shuffled narratives about science fiction, Scientology, Eighties pop stars, doomed love, nuclear physics and the occult into a knowing, clever and intricately woven collection that deserves to rain on Cloud Atlas's parade or accompany Jennifer Egan on a visit to the goon squad. Brilliant and oddly moving, The House of Rumour deserved to win every prize going, including Eurovision.

Thomas Keneally's The Daughters of Mars (Sceptre, £18.99) completed a strong year for Australian fiction. Keneally was inspired by the diaries of real Australian nurses and soldiers to craft a tour de force of storytelling that is both epic and intimate, experimental and traditional. This tale of sisters Naomi and Sally Durance leaving Australia to serve as nurses on the European battle front includes a shipwreck, visceral descriptions of the realities of hospital life, and morally complex meditations on empire, war, and the meaning of human life.

Reviewing this list now, it is curious to detect common themes to these diverse books. Shipwrecks. Ghosts. War, global and civil. Physics. Terrorism. Experiments with word, form and images. Short stories. And thanks to Nicola Barker and Alice Munro, rather more golf than expected. Novelists, it would seem, are partying like it's 1919. Long may they continue.