Art review: Curiosity- Oh, I do like to be beside the seaside

To say that Curiosity: Art and the Pleasures of Knowing is a strange show is only to praise it, since that is what it sets out to be. It succeeds, in spades. The exhibition, in Margate, also means to be wonderful, and it pulls that off, too, in the double sense of being both full of wonders and superbly well curated.

Let's start at the top of the stairs to the Turner Contemporary's upper galleries. On the landing is a video monitor playing Hippocampe (1933), a film about seahorses by Jean Painlevé. He was himself deeply odd, listed in the credits to Buñuel's Un Chien Andalou as "chef de fourmis", or "principal ant-handler". A Surrealist fellow-traveller, his movies on marine life – he was the Jacques Cousteau of his day – are at the same time mainstream and uncanny, documentaries that look like art or, just possibly, the other way about. Beyond the monitor on which Painlevé's film is showing is one of the Turner's huge windows, giving on to a rain-swept North Sea in which short-snouted seahorses will even now be greyly at play.

It is a great moment. If art is meant to jar you, then here is art. Part of the genius of Curiosity is that it was designed by Nick Grimshaw, the Turner Contemporary's architect. None knows the building's spare sight-lines, its low-budget dramas, better than he. There are similar juxtapositions everywhere in the show, of unalike things taken in inappropriate single glances. Which is as it should be, since, without saying so, Curiosity tells the story of Surrealism in British contemporary art.

At its heart, early Surrealism was about elisions – unlikely clashes, unintentional puns, slips of the Freudian mind. Mink is a respectable thing; so are teacups. Run yourself up a mink teacup, though, as Méret Oppenheim did, and you end up with something lawless. Likewise if you make a film about seahorses which is at the same time educational and weird, and yet again if you show it in front of a seahorse-filled sea.

A great deal – maybe most? – of British contemporary art has its roots in Surrealism: think of readymades (Tracey Emin's bed), jokes (David Shrigley's cartoons), messy-art (almost everything by nearly everyone), Postmodern appropriation. The Surrealists, in their turn, were fans of the Wunderkammer, or cabinet of curiosities, those rooms of oddities beloved of 17th-century scholars and kings. Thus Curiosity.

To see a stuffed walrus on its own – there is one in this show, from the Horniman Museum in south London – is one thing. To see it alongside a vitrine of split-hair works by Susan Hiller and Laurent Grasso's photograph of Pope Pius XII looking into the official Vatican telescope – who knew? – is quite another. Each object is individually odd, but it is the story we make out of their joint oddities – their meta-oddness, if you like – that is the point.

Appropriately for Margate, many of the things in traditional Wunderkammern came from the unknowable sea. Just visible from the Horniman walrus are the anatomically perfect (and lovely) glass snails and jellyfish made by the father-and-son team, Leopold and Rudolph Blaschka, in Dresden in the late 19th century. Beyond them is Aurélien Froment's jellyfish video, Pulmo Marina – "No head, no brain, no bones, nothing to distinguish front from back", its puzzled soundtrack says – and beyond that again, Anna Atkins's 1850s Photographs of British Algae, cyanotype studies of seaweed from the first book ever illustrated with photographs.

How you navigate your way around these things is up to you: to each man, his own Curiosity. Some of the works seem perversely un-odd. Richard Wentworth's Making Do and Getting By are shots of banal things – traffic cones, corks, pebbles – made notable only by the attention Wentworth brings to them. Record an object, put it in a glass case or impose a system on it and it will, willy-nilly, become strange.

Next to Making Do is a room in which Tacita Dean's characteristically slippery film, Manhattan Mouse Museum, plays. This shows the 84-year-old sculptor, Claes Oldenburg, at work in his New York studio. Oldenburg takes trivial things – buttons, bus tickets, book matches – out of shallow boxes stacked in the customised shelves which make up an entire studio wall. There is something insane about the degree of attention that he brings to these objects, but also about the obsessiveness with which Dean's camera watches him doing it.

Wonder is in Oldenburg, in Dean, in all of us. And it is very much so in this classy seaside show, which I strongly recommend – no, I insist – you see. Roll up! Roll up!

To 15 Sep (turnercontemporary.org)

Critic's Choice

Oxford's Ashmolean Museum shows off its impressive collection of Master Drawings (till 18 Aug), with works by Michelangelo, Titian, Rembrandt, Goya, Blake, Turner, Degas, Hockney…. At the Wellcome Collection in London, let yourself be surprised and entertained by Souzou: Outsider Art from Japan (till 30 Jun).