The Cabin in the Woods, Drew Goddard, 95 mins (15)

Why is horror cinema so obsessed with rules? Like the Scream series, The Cabin in the Woods is at once a grisly romp and a learned cogitation on how horror films work, and on the formal patterns so many of them follow.

It's a lot more fun than it sounds – and it made me think about horror and the fetishism of conventions. There's surely a parallel with sado-masochistic sex: in both horror and S&M, the potential of unimaginable, soul-consuming excess is invoked by a set of contracts that keep the unthinkable under control, making it relishable rather than agonising.

The pleasure in The Cabin in the Woods comes from its audacious tuggings-away of the dramatic rug under your feet. I'm honour-bound to reveal as little as possible about the plot, but this official tagline tells you all you need to know: "Five friends go to a remote cabin in the woods. Bad things happen." That may sound wearisomely familiar, but the blurb continues: "If you think you know this story, think again."

The Cabin in the Woods does indeed make you think again, and again, but it's also a highly efficient thrill machine. Co-written by the Buffy creator Joss Whedon and the Cloverfield/Lost writer Drew Goddard, who also directed, this is a horror film about the reasons we watch horror films, and the morally questionable positions they sometimes put us in. It's as intensely self-reflexive as Scream – but The Cabin, while playing with similar ideas, is significantly different, and considerably less pedantic.

From the start, the film places us on known ground and simultaneously undermines it (a visual metaphor throughout is the earth opening up and producing nasty surprises). We start with a group of overtly stereotypical college kids getting ready for a weekend away: a Nice Girl (Kristen Connolly); a blonde Bad Girl (Anna Hutchison); a Jock (Chris Hemsworth – Thor to you); a sensitive Swot (Jesse Williams); and a wisecracking Stoner Doofus (Fran Kranz, a real comic presence). Off they drive in their camper van – all that's missing is a talking dog.

But we know that the kids are being set up for some nasty fate. They're being watched, and manipulated, by a group of technocrats in a massively funded facility: the chief operatives are played by Richard Jenkins and Bradley Whitford, whose gravitas makes their badinage all the more chilling (imagine a ghoulish equivalent of The Muppets' Statler and Waldorf).

Weird things start to happen, bang on cue. The kids visit an eerie Abandoned Gas Station where a weird old man issues dark warnings. At the spooky cabin itself, a cellar door creaks open, and the kids react in time-honoured fashion – "The wind must have blown it open". They seem inexorably locked in a cycle of cliché. At first, we think we know where we are, watching a sort of hyper-ironised torture-porn scenario. One character even blurts, "Oh my God – I'm on a TV reality show!" – but what's at stake is far weirder and more ingenious.

Half-ghost train, half-graduate seminar, the film scores some basic points about horror spectatorship most unsettlingly by having the techies exult in bar-room fashion as they watch the ensuing slaughter on their screens. The scientists are presented as regular folks – glorified office workers – but they take connoisseur-like pleasure in death and pain administered efficiently and on schedule. They are the horror audience – but they're also the film industry that devotes formidable skills and resources to producing such thrills. With its many different departments (Zoology, Chem, Data, BioMed), the mysterious organisation resembles a Hollywood studio – NightmareWorks, perhaps.

Just as the film's diabolic machinery seems to be running its course, things suddenly veer off in an unexpected direction. And then, as we're scratching our heads in delighted mystification, the film explodes into delirious apocalyptic frenzy, a let-it-all-come-down ecstasy of overkill (suffice to say it's the most outrageous 'what-happens-if-I-press-this-red-button?' moment I've ever seen).

The Cabin is, as you'd expect, crammed to the rafters with genre references, among them The Shining, The Evil Dead, Cube, Hellraiser, The Blair Witch Project, the 1950s EC horror comics, Japanese horror (there's a delirious running gag about weird happenings in Kyoto), H P Lovecraft, even Michael Haneke's high-minded chiller-as-treatise Funny Games.

As in the Scream cycle, a lot hinges on our expectations of which kids will be killed and when (in keeping with Carol Clover's study Men, Women and Chain Saws and its theory of the "final girl"). There's a moral question at work, but it's to the film's credit that it handles the matter lightly and wittily – and then says, "What the hell – let's just bring on the fireworks."