A Serbian Film: Is this the nastiest film ever made?

At the American Film Market (AFM) this week in Santa Monica, there have been plenty of movies using shock tactics to attract the attention of the world's distributors. But gnarled old distributors and blasé film festival programmers alike seemed genuinely shocked by Srdjan Spasojevic's ultra-extreme thriller called A Serbian Film.

A Serbian Film is about retired porn star Milos (Srdjan Todorovic), a middle-aged man struggling to provide for his family who is lured back into the industry for one last film. He has been offered enough money to set him up for life but, in return, has signed a Faustian pact with the director Vukmir (Sergej Trifunovic). Milos will have no control over the scenes in which he appears.

The opening sequences shows Milos's young son innocently watching some of his father's greatest "hits" on the family TV. We see the doe-eyed kid looking innocently as Milos struts his stuff in some ludicrous Robin Askwith-style blue movies. The parents are shocked to discover that he has stumbled on such images and quickly turn it off. The scene is disorienting but also comic. It highlights the preposterousness of the world from which Milos has fled.

Gradually the film begins to darken. Once Milos accepts the role in Vukmir's film, the demands placed on him grow ever more extreme.

Publicists whispered to journalists that the film was truly "vile". Prior to its AFM screenings, the movie had already been yanked out of Frightfest in London when Westminster Council ruled it couldn't be shown in its uncut form and had started frenzied debates about censorship and freedom of speech. The British Board of Film Classification (BBFC) had asked for a staggering number of cuts in the film and for a full four minutes of footage to be excised in order for it to qualify for an 18 certificate.

Not since the heyday of the so-called "video nasties" in the early Eighties had a movie exercised the censors in quite such an extreme way.

Much of the imagery in A Serbian Film is indeed quite repellent. That, though, is not the same as saying that it is a repellent film. The film-making is stylised and self-conscious. The most notorious scenes (the rape of the new-born baby, the scene in which the star decapitates a woman and continues to have sex with her headless torso) are grotesque but very obviously contrived. In the film-within-a-film, Vukmir, the psychiatrist-turned-porn director, may be striving for the ultimate realism but Spasojevic heightens the absurdity. Forty years after A Clockwork Orange, audiences are surely too used to these kind of shock tactics to be affected by them – or so we might think. There is a knowing irony. As in Michael Haneke's films, the director seems to be challenging the audience to question their own voyeuristic instincts. As in Peter Greenaway's The Baby of Macôn, he is using extreme imagery for polemical purposes.

The problem is that the storytelling grows ever more intense. What begins as a self-reflexive formal exercise veers off in another direction altogether.

One US distributor fainted as he tried to leave a screening of A Serbian Film earlier this year, hit his head on the door and ended up needing stitches. The film's British sales agent was left hurriedly trying to clear up the pool of blood.

"He was getting really disturbed and he felt he was going to faint. At the time, we were both sitting on the floor because the theatre was completely full. He tapped me on the shoulder and said I need to go. He got up and ended up fainting and collapsing," recalls Thomas Ashley, the boss off Invincible Films, the US distributor for which the man worked.

What has proved alarming to censors isn't just the imagery. It's the fact that children are involved. Spasojevic clearly didn't expose these children directly to images of torture, rape and death. However, the juxtaposition of children with such exploitative imagery is itself deeply unsettling.

There is a feeling of nihilistic self-loathing that runs through the film. In some eyes, after the Balkan wars of the 1990s, Serbia is still a pariah state. The alleged war criminal General Mladic has never been arrested. The memory of Slobodan Milosevic hasn't been exorcised. Films like A Serbian Film and another equally extreme Serbian movie The Life and Death of a Porno Gang play on Western preconceptions about the country and can't help but reinforce them. The very title of A Serbian Film suggests that the director and his screenwriter Aleksander Radivojevic are making an allegory about their troubled and isolated homeland. The screenplay is full of references to the corruption and squalor of family life in the country. However, audiences have been responding to it in stubbornly literal fashion and haven't been slow to express their utter disgust.

Predictably, this disgust is now being harnessed to boost the film's profile in the marketplace. The film's British sales agent Jinga was quick to tell the press that following its withdrawal from Frightfest, A Serbian Film has been banned in Spain and withdrawn from three Spanish festivals – San Sebastián, Molins de Rei and FanCine Málaga.

As with any film that becomes a succès de scandale, A Serbian Film's notoriety risks stopping it from being judged on its merits. Even its fiercest critics concede that it's a film with a relentless narrative drive. The porn star is played with an unlikely crumpled charm by Srdjan Todorovic (a musician and veteran of Emir Kusturica's films.) He is (at least initially) a sympathetic figure: someone desperate to do the best for his family.

Invincible's Thomas Ashley, who ended up buying the movie for the US in spite of his colleague's fainting fit, captures well the strange mix of revulsion and admiration that it has been eliciting.

"Shocking and disturbing as it is, this is really a well-made film," he declares. "Everything that happens in the movie happens for a purpose, to get you to the next part of the story.... I've seen a lot of horror movies and a lot of exploitation movies and I've never had a movie affect me the way this film did."

In a market like this year's AFM, full of anaemic vampire movies pastiching Twilight and of "torture porn" of the Hostel or Saw variety, A Serbian Film can't help but stick out. It has a craftsmanship that these films lack. Its UK distributor Justin Marciano of Revolver believes it can find an audience among "intelligent fans of horror".

The movie will soon surface in some form (almost certainly in the cut version) in Britain before Christmas. When it does so, some are bound to condemn it as being beneath contempt. What A Serbian Film underlines, though, is that some pictures can still get under audiences' and censors' skins. If this was just another bad and grotesque horror film, nobody would be paying any attention to it. The fact that it has already provoked such ferocious debate suggests that it can't be dismissed that easily.

'A Serbian Film' will be released on 10 December