Party!, New Art Gallery, Walsall

It is ten years since the New Art Gallery in Walsall opened its doors beside the Walsall Canal. (You can see a narrow boat drawn up beside the café's window as you bite down on a panini.) Would this splashy, handsome gallery help to give a new vitality to this small Black Country town? Could there be a mini-Bilbao effect in the making? Ten years on, things are looking pretty good – there were 6,000 visitors during half-term week; kids seem to be dragging their parents back for a second look – and it's evidently time for a show on the theme of non-stop partying.

Start at the top, on the fourth floor, in the gallery which overlooks the town centre, and which gives you a pleasing, bird's-eye view of small-scale industrial dereliction and small-scale renewal, all set against the meandering canal network of the Black Country. The huge piece which dominates the gallery up here, and sets the tone for the entire show, is David Batchelor's Disco Mécanique, a strange, mechanised, nostalgia-soaked whirligig of plastic balls made from pairs of sunglasses, suspended and turning in the air. This piece was last seen at the Folkestone Triennial, where it dominated the ballroom of a grand, sea-front hotel. The mood is mesmerising and slightly sad, a reflection upon long-gone partying perhaps.

The theme of the show could have tempted the curators to be brash and noisy – as if the show were nothing more than a fizzy party itself. This temptation has been skillfully resisted, and where it is not entirely resisted, it is done well. The most pleasing audience-participation space is an activity room where giant purple balloons crowd against the window, wondering who has the gall to be staring in at them. When someone who knows the code to the door lets you in, you have to fight your way through a suffocating balloon-scape – they push up against your face, pummel your chest, batter at your knees. There's barely room to see or breathe! Help! The work itself is by Martin Creed and it's called, with more than a touch of comically clinical detachment, Work No. 965, Half the Air in a Given Space.

The range of works here is wide, and the theme is sufficiently baggy to accommodate all kinds of moods, from the headily crazed to the hungover and regretful, from hedonism to soulful reflections upon the sad aftermath of excess. Here is a statue of Michael Jackson by Yang Mian, teetering on the ends of his drainpipe blue legs. Much of the show feels like a raucous party with the sound turned down, so that you are suddenly shocked into an awareness that you are at a party at all. Stare down at Jim Lambie's soundlessly spinning, glittery record deck, needle poised above the track. Doesn't it engulf you in a near deafening silence? The works are well-paced and as various as you could wish – a nocturnal scene of ritualistic dancing by Paula Rego is set beside a Goya etching entitled Grotesque Dance, in which the dancers seem to be whirling away from each other like a mad frenzy of drunken puppets.

But visitors to the New Art Gallery need to be reminded of something else too. This is not only an entirely free-to-access public gallery with good changing exhibitions. It also has on permanent display one of the finest art collections to be seen outside the capital. I am referring to the Garman Ryan Collection, which not only includes an excellent span of works from the Egyptians to the late-20th century and first-rate works by Degas, Whistler, Picasso, Monet and many others – but also Jacob Epstein's archive. So when you tire of partying, you can soothe yourself by staring at Lucian Freud's lovely, bulbous-eyed Portrait of Kitty. Then, once fully rested, you can invite her on to the dance floor.

To 17 April (01922 654 400)