Why is David Benioff swapping the big bucks of Hollywood for the genteel world of literary fiction?

Within minutes of introducing himself, David Benioff has given me an exclusive. On his flight over from the US he almost saw the demise of Harry Potter: the plane he and Daniel Radcliffe were on was hit by lightning. It's a pretty spooky incident considering Benioff's day job as a Hollywood screenwriter finds him chronicling the routine calamities of superheroes (his addition to the X-Men franchise is out this summer). He rounds it off with a sardonic twist atypical of Tinsel Town players. "I could just see the newspaper headlines," he smiles. "Harry Potter Killed – 300 also Dead."

Benioff is in London to promote City of Thieves, the long awaited follow-up to his rapturously received 2003 debut novel The 25th Hour (which he subsequently adapted for Spike Lee). Since then he has moved to Los Angeles, where he lives with his wife, the actress Amanda Peet, and balances the books through a combination of fiction and scripts. Sitting in the lounge of his Covent Garden hotel he has a transatlantic quality. The cargo pants and stripy T-shirt are all American. His looks are definitely not. With his sabre-slash features, Slavonic in their angularity, he looks like a Giacometti portrait of a wolf.

The settings for his projects are equally global in their influences. He focused on the New York boroughs in his debut, Greek mythology for the big screen take of Troy, and now the Russian wastelands. City of Thieves is a blackly comic tale of friendship and survival during the siege of Leningrad in the Second World War. It chronicles the alliance between Lev Beniov, a half-Jewish teenage thief, and Kolya Vlasov, a constipated, wisecracking deserter from the Red Army. They form an odd couple, forever bickering about the timeless obsessions of Russian men: women, literature and chess. Their story is told by Lev, narrated in the present day to his grandson (a thinly veiled version of Benioff). We learn that Lev mostly "talked about one week in 1942, the first week of the year, the week he met my grandmother, made his best friend, and killed two Germans".

In a city where the population is literally dying for a scrap of bread, their crimes put them in line for the firing squad. A colonel offers them a reprieve if they can procure a dozen eggs for his daughter's wedding cake. It might sound a strange focus for a quest narrative ("A thousand rubles? For a dozen eggs?" Lev groans. "Are they Fabergé?"). But then The Lord of the Rings centred on the pursuit of a piece of jewellery you can't even wear. These eggs act as the bond between Lev and Kolya. The focus remains on the relationship forged along the road and the imagining of the unimaginable. "You have never been so hungry; you have never been so cold. When we slept, if we slept, we dreamed of the feasts we had carelessly eaten seven months earlier," says Lev.

There is, claims Benioff, only a "tiny little bit" of family biography in the story. His family roots are Russian, but his own appearance in the book's prologue was simply "a framing device". The clarification between fact and fiction, he realises, is all the more important after the recent debacle of James Frey's "memoir", A Million Little Pieces, which was discovered to be largely made up. "I'm telling you straight off, I'm a liar," says Benioff. "There's

a great Isaac Bashevis Singer line where he says: when I was a child they called me a liar, now they call me a writer."

The lies he tells are, however, moulded by extensive research. In 2001, a guide took him round St Petersburg. "I could tell him the important places I wanted to see. But more importantly, he introduced me to some older people who'd been there and lived through it. Which was fascinating in how often their accounts differed from the official accounts. That whole period was completely whitewashed by the government."

There are some intriguing details in the book. Not least the wares sold in the city's Haymarket. Here "seven-layer sin" (wood alcohol requiring repeat sieving before consumption) and "library candy" (confectionary made from boiled book spines) trade for small fortunes. Also, while the French resistance movement is well documented, the Russian partisans, who consistently engaged in open combat and whose actions inform a major part of this novel, are now almost forgotten.

At this point in our conversation I'm distracted by a flurry of voices behind me. Benioff leans over, muttering that Harry Knowles, the corpulent founder of the film website aintitcool.com, and the actor Benicio del Toro have just come in. Knowles's power in the industry depresses Benioff: "He has terrible taste." Glancing over at the obese, garish figure – he looks like a parrot – I can understand the criticism. Benioff whispers that he has a good story about del Toro (of which more later). They, together with their entourage, disappear into the adjoining room and we return to talk of death and decay on the Russian steppes.

If anything, he tells me, he has underplayed the horrors of the 900-day siege. Still, at times the savagery witnessed by Lev and Kolya is enough to make you wince: the forced prostitution, random executions and, most disturbingly, the cannibalism. This is humanity gone feral.

"If you go into the detail of everything that happened it becomes this grim catalogue of atrocities," says Benioff. "After the war the Soviet Government declared it the city of heroes and it was meant to be about the valiant defiance of the Nazis. They absolutely were valiant and defiant but they were also eating each other.

"You grow up in the States thinking the war started with Pearl Harbour and the big turning point was D-Day," says Benioff. Of course, history tells us that the pivotal change for the Allies took place on the Russian front. I'm interested in whether this lack of insight stretches to the world of US publishing. Benioff acknowledges that, even with the success of books such as The Kite Runner (for which he wrote the BAFTA-nominated screenplay), there exists an unshakable prejudice in America against novels set abroad.

"One of my best friends wrote a book and the first chapter was set in Poland. He got a note from his editor saying: 'I wonder if we should move this opening chapter from Poland to the States? It's hard for me to imagine that anyone will be interested in a story that takes place so far away.' An editor at a major publishing house." Benioff shakes his head and says that he told his friend to send the editor a letter beginning: "In a galaxy far, far away..."

He is in no doubt which of his two paymasters is the more patient. "With the movies, people are not going to wait around. The deadline is a deadline. In publishing it's more a polite suggestion."

At the end of the 1990s, however, that gracious trade still seemed a long way off. Then in his late twenties, he had two unpublished novels in the drawer and was floundering as an untenured college lecturer. This is, it seems, what failed novelists do in America. "I'm just not a natural teacher," he admits. "I found it hard to wake up in the morning." What seemed like an instant success with The 25th Hour, the tale of a small-time drug dealer's last hurrah before going away for a seven-year stretch inside, was misleading. He has 34 rejection letters sitting in a pot.

So why the wait for the new novel? "It was deliberate laziness. I told the publisher what I was going to write back in 2001. I think the fiction-writing muscles had started to atrophy." He believes that the stylistic traits and recurring themes of his work, notably friendship and loyalty, were inculcated in his writing long before he moved to the West Coast. Comics, fairy tales and movies were the strongest influence, along with a lack of close friends at school. "That's when I started coming up with stories and liked the idea of imaginary friends. Once the characters take on a life, they keep you company."

Prose has allowed those friends to breathe. "A certain luxury when you get to writing a novel is to have the space to have your characters just banter. In film, you can't have a 20-minute banter about constipation." It does sound like the kind of thing that could put people off their popcorn. However, for all the sparky, rambling two-handers, there remains a keen understanding of pace in City of Thieves that is positively cinematic. He might think that the classic three-act structure is "a crock of shit" but he still "likes that ticking clock".

As we come to the end of our time, the Knowles/del Toro party passes back through. Once clear, Benioff recounts a meeting he once had with the Fight Club director David Fincher. The trio were discussing a scene involving hail, and suddenly del Toro announced: "I hear that hail is upside-down rain." Benioff laughs: "We didn't know what the fuck he was talking about. He wasn't joking; he was completely serious. I mean, wouldn't rain be upside-down rain?"

It's hard to imagine Hollywood taking to his arch manner as well as they do. However, as long as lightning doesn't strike twice, he can rest assured publishers will continue to welcome him into polite society.

The extract

City of Thieves, By David Benioff (Sceptre £12.99)

'... The clothes and boots had been stripped away. Their buttocks had been hacked off, the softest meat, easiest for making patties and sausages...Kolya and I continued along the Vologda line. He didn't tell any more jokes that morning'