Andrew Davies: a festive lesbian drama

Affinity is the greatest of Sarah Waters's dark novels. It's also the least well-known of her books. Thanks to popular television adaptations of Tipping the Velvet and Fingersmith, people who would never usually have read niche lesbian fiction have been introduced to her writing.

But Affinity may have passed them by. Even though it earned her the 1999 Somerset Maugham Prize and the Sunday Times Young Writer of the Year award, it has a reputation as a forbidding and unhappy book. Yes, it's Victorian pastiche, like Tipping and Fingersmith. It's a ghost story; and a love story. But it's set in a women's prison. The heroine is a drug addict. And there isn't a satisfying "girl gets girl" ending.

But Affinity is Waters's masterpiece. And now with a new £2m ITV film of Affinity screening on 28 December, with a screenplay by master adaptor Andrew Davies, it will be a secret no longer.

Affinity is an ambitious novel in terms of genre and characterisation. The ending is like a slap in the face. You immediately have to go back to page one to see how you have been deceived and manipulated. It has a near cult following among Waters's fans. For fellow lesbian novelist Zoe Strachan: "Affinity is more psychologically complex than the other novels, exploring representations of female hysteria in a fantastically gothic setting, with hearty nods to Henry James and Wilkie Collins. Although Tipping the Velvet is more explicit, I think Affinity is sexier, again in true gothic fashion. Sapphic desires bubble darkly under the Victorian veneer, emotions are strictly repressed, and tension snakes around the reader in a way that's both exquisite and unbearable."

Mostly set at Millbank Prison, which once stood on the banks of London's Thames, the plot follows the heiress Margaret Prior who starts visiting Selina Dawes, a disgraced spiritualist whose previous patron died of a heart attack. Margaret, who is mourning the death of her father and recovering from a failed romance with her sister-in-law, falls under Selina's spell and becomes convinced of her innocence. Desperate to remake her life, she plots to free her. But is beautiful Selina a con-artist or a visionary?

It is this ambiguity that hooks us, argues novelist Joanna Briscoe: "Its linguistic virtuosity, plotting, and deep darkness astonished me when I first read it. Some complain of it being 'depressing', but I revel in its miseries, its choking atmosphere, and its handling of evil, because it's so superbly accomplished, and the darkness is shot through with romantic optimism and obsession. Affinity doesn't have Fingersmith's rollercoaster series of twists, but the massive deception on which it is built is more brooding and subtle and left me gasping. It's an astonishing novel."

The new film, starring Anna Madeley as Margaret, Zoe Tapper as Selina and Amanda Plummer (of Pulp Fiction fame) as a scary prison guard, is brilliant. Davies (who also wrote the screenplay for Tipping the Velvet in 2002) has been incredibly sensitive in this adaptation. Refined, repressed, it feels like the great lesbian novel Henry James never wrote.

There is still plenty of sexual yearning on screen, of course. And the scenes where Selina holds her seances – her body transported by spirits – have an orgasmic quality. There's clearly more than one reason why this medium is so popular with female clients. But it's also highly disturbing. In contrast to Tipping the Velvet, with its jolly group sex scenes and strap-on dildos, Affinity looks at the darker side of romantic obsession. Why is Margaret being fed huge amounts of laudanum? Is she being protected by her family, or held captive? Is she sane (at the start of the book she is recovering from a suicide attempt)?

The question of sanity matters because, as we read on, we begin to doubt our own grip on reality. As A N Wilson observed when it was first published: "You start by admiring the spooky excitements and end by feeling you have learnt something of 'the secrets of a woman's heart'."

The book is structured as alternate diaries, narrated by Margaret and Selina. This doubling goes to the very heart of the novel. As lovers, we all think we are unique. That no one has ever felt this before. And for Margaret, in the grip of obsession, to discover infidelity on a grand scale is hugely traumatic.

Waters has said Affinity fans are closest to her heart. They are prepared to confront that darkness. The novel is, of course, more than a bodice-ripper with lesbians in petticoats. It has real moral seriousness. We shudder at the cruelty of the Victorian penal system, the way even wealthy young women are treated as chattels. Waters says she wrote her first book, Tipping the Velvet, as a reaction against the worthy, small-scale ambitions of 1980s lesbian feminist fiction. But with Affinity she is tackling something more complex.

In many ways the book is a love letter to gothic fiction. But other influences include Tennyson's In Memoriam, and Great Expectations. As Waters points out, both Affinity and the Dickens novel feature an obsession with an enigmatic girl, the "return of the repressed" in the form of criminal sexuality, and a silent housekeeper with powerful hands, like a "wild beast tamed".

Directed by Tim Fywell, Affinity confirms that ITV can pull off daring drama. Of course, the plot-based novel is hugely cinematic – with its gallery of prisoners and warders, its closed rooms and "dark circles" with unruly spirits. The prison dominates the book like the Marshalsea in Little Dorrit. And there are great spooky set pieces – including a scene where Margaret finds Selina's plait has magically been transported from prison to her bedroom pillow.

As an adaptor, Davies openly acknowledges he has been handed a gift in the form of Waters's prose. At the very beginning, Margaret describes the prison in almost painterly detail: "The shadows there, flung from the jaundiced bricks, are the colour of bruises. The soil in which the walls are set is damp and dark as tobacco."

But daring as Affinity is, it's going out at Christmas when television needs a feelgood element. We can't go to bed feeling wretched. Without spoiling the ending, Davies has worked his magic to turn an unkind trick into something more romantic, if also more anguished.

The only wonder is no one has adapted Affinity before. In fact, award-winning playwright Shelley Silas has already tried. "When I read the novel I leapt up and down, and shouted: 'it has to be done on stage.' Because everything about it adds up to an utterly theatrical experience," she tells me. "Waters creates the atmosphere, offering us madness, secrets, the supernatural, love, and, of course, a good deal of delicious deception, making the pages turn by themselves. It is without doubt my favourite of Waters's novels and gothic to its core."

In fact, Silas got Waters's permission for a stage adaptation of this a couple of years ago. "But theatres just didn't go for it. Now I hope the film will make them sit up and realise what they have on their hands... I can't wait to see it."

'Affinity' is on ITV1 on 28 December at 9pm