Psycho Buildings: Artists Take on Architecture, Hayward Gallery, London

In May 2000, the Austrian art collective Gelitin lifted the window out of the wall of their studio, pushed a wooden balcony through the resultant hole and stood on it, waving, to be photographed. So far, so conceptually unchallenging, except for the studio's location: the 91st floor of the World Trade Center's north tower. (Photos of the incident, if it really happened – Gelitin are great pranksters – were taken from a helicopter.) A year later, the first jet hit the tower just at the point where the foursome had stood. Which makes you wonder what the Hayward Gallery can have been thinking of when it invited Gelitin to build a boating pond on its roof.

If Gelitin's commission signals a death wish on the Hayward's part, then it is hardly to be wondered at. Until the 1960s, the relationship of gallery to artist was, loosely, that of master to servant. Institutions bought art and then did what they wanted with it. In about 1960, though, all this changed. Artists, increasingly bolshie, began to chafe at the gallery's tyranny, and to find strategies for undermining it. Some made work from materials – dust, poo, rotting meat – that were impossible to conserve; others made sculpture on so vast scale a that no gallery could hold it. Even painters kicked over the traces. In 1965, Ed Ruscha made a picture called The Los Angeles County Museum on Fire, a piece of wishful thinking whose subject is just what it says it is. (Ruscha's painting does not, as you'd guess, hang in the Los Angeles County Museum.)

So the subtitle to the Hayward's 40th anniversary show – Artists Take on Architecture – sounds worryingly belligerent. You'd expect disrespect, and you'd be right: many of the artists in Psycho Buildings have used the gallery's architecture against it, like curatorial jujitsu. Gelitin's pond, Normally, Proceeding and Unrestricted With Without Title, is merely the most obvious example, playing on the idea of the modern art museum as a funfair for the masses. But, as with most of the work in this show, Normally ... is also oddly good-natured. Strike out from its pontoon in one of the wooden tubs provided and the South Bank is suddenly a different place, seen, physically and emotionally, through the eyes of a child. The piece is the inverse of Gelitin's World Trade Center balcony, the collective's genius lying in shifting our points of view.

In the épater le Hayward category also comes Atelier Bow-Wow's Life Tunnel, a Constructivist-looking sculpture tucked into the back corner of one of the downstairs galleries. The Japanese group's offering feels like a piece of Judd-like abstraction until you get up close, at which point it becomes clear that you are looking into a galleristic worm-hole. The parallel universe into which ABW's tunnel delivers you is simply the Hayward's next level up, but the means by which you get there – crouching down and clambering along – destroys any sense of the modern art gallery as a place of conceptual stillness and hush. (A warning to the childphobic: avoid Psycho Buildings during weekends and school holidays. The place is like an adventure playground.)

Broadly, I'd say that it is works such as these – the ones that respond to the Hayward's own architecture by cocking snooks at it – that are the most engaging in this show. Do Ho Suh's spectral rooms are as wonderful as ever – his red gauze re-creation of the staircase of his New York apartment block makes you want to weep – and Mike Nelson's reprised To the Memory of HP Lovecraft fills the Hayward's upper floor with the kind of ghosts you would definitely not want to meet on a dark night. But the story of this show is really the story of rebellion; and the clearer the artist/gallery tussle is spelled out, the more vivid the work becomes.

Given the history of the past 40 years, it seems big-hearted of the Hayward to have set itself up for a pratfall in this way. Actually, the joke is on the artists. The gallery's design, all sliding walls and anonymous spaces, long ago acknowledged their victory. How do you fight something that won't fight back? If Psycho Buildings had been held at Tate Britain, we might have seen fireworks. As it is, most of the artists seem to have ended up loving the gallery, as indeed they should. Happy birthday, dear Hayward, and many more.

'Psycho Buildings', Hayward Gallery (0871 663 2500) to 25 Aug