While you were sleeping: Tapping into the power of dreams

In the past few weeks, I've been propositioned by Ed Balls, lectured to by Danny Alexander and sneered at by Alastair Campbell. I've also travelled round New York with a colleague, just missed being blown up by a bomb, and swum in a river with Martin Amis. He didn't look great, and neither did I, but we weren't thinking about each other, we were thinking about the dinosaur that turned out to be a cactus. When you broke off a branch, and looked inside you found this milky liquid. We drank it and it was delicious.

It's lucky, perhaps, that it's not my dreams that feature in Christopher Nolan's new sci-fi thriller, Inception. Other people's dreams tend to make the eyes glaze over, and that's even when they don't involve the Chief Secretary to the Treasury. But there are, thank God, no politicians in Inception. Instead, there are thieves. Leonardo DiCaprio is one of them, a skilled (and very handsome) operator in the world of corporate espionage – a world, it turns out, with a twist. The twist is that he and his super-smart colleagues steal secrets not just from safes, but from the unconscious. To do this, they manufacture, and invite people into, dreams. At times, it's only their "totem" (a little talisman that each clutches) that lets them know if they're awake or asleep.

The film, which has garnered some rapturous reviews, is part detective story, part redemption story and part (this is Hollywood) love story. But the love story, too, has a twist. The beloved is dead and DiCaprio thinks he may have killed her. In order to return to his fantastically photogenic children he has to undertake one last heist. He has to plant an idea in someone else's mind, a process called, of course, "inception". I won't say what happens – and couldn't, because half an hour in I literally lost the plot – but it was certainly multilayered and visually spectacular. Whole dream cities sprang up and vanished. Roads suddenly turned perpendicular. There weren't, as far as I could tell, any dinosaurs or cactuses.

If DiCaprio (and indeed the audience) spent much of the film asking: "Was it a vision or a waking dream?" he was echoing not just Keats, but pretty much the whole of world literature and a great deal of its art.

The Bible, as I discovered when I played the butler in Joseph and the Amazing Technicolor Dreamcoat at primary school, is stuffed with dreams. In the musical, and its divinely inspired original, Joseph launches into accounts of the complex choreography of the sheaves of corn, stars, sun and moon which haunt his night-time hours. He later grills others for details of theirs. "Any dream," it seems, "will do."

In the Bible, and much of the literature that followed, the dancing sheaves, collapsing statues or mountains that – like Nolan's cities – suddenly sprang up after a roast goat, a flagon of wine and a nice lie-down, were assumed to be messages from God. Wise men were asked to pronounce on them, and were duly deemed prophets. And once the idea had, à la DiCaprio, been planted that God crept up on you in the night, and all you had to do was solve the symbolic sudoku to get the throne, or the girl, or (less excitingly) the wisdom, it proved remarkably hard to shake off.

For all the talk about God, the Old Testament dreamers seemed rather preoccupied with the stuff of this world. The followers of the religion that their last, and greatest, prophet inspired were, however, touchingly keen on the next one. "The Dream of the Rood", which dates back to the seventh century, is one of the first post-Biblical works to use the dream sequence as a devotional tool. With the marvellous matter-of-fact surrealism that we all take for granted in our non-waking hours, the narrator recounts a long conversation he has with the holy cross. The cross, he sees, is covered with precious gems but also blood. It tells a tale of pain and beauty and redemption that the narrator then feels obliged to pass on to others.

It was a good 600-odd years before the next major alliterative dream vision written in English, or something like it. In Langland's Piers Plowman, the narrator – dressed as a hermit and wandering the Malvern Hills – falls asleep, has a dream and launches a whole new genre of Utopian literature. Ostensibly a theological allegory about the quest for the pure Christian life, it has been treated by critics as social satire, but that's to forget the mystical tradition from which it sprang. Samuel Pepys apparently had a copy. It's a rather touching image: the metropolitan diarist, MP and man about town, poring, perhaps in a coffee house, over the visions of a ploughman. (Pepys's own dreams tended to be a little less spiritual. "Up," he writes on 5 December 1668, "after a little talk with my wife, which troubled me, she being ever since our late difference mighty watchful of sleep and dreams, and will not be persuaded but I do dream of Deb... I cannot deny that my thoughts waking do run now and then against my will and judgment upon her...")

Pepys's contemporary, John Bunyan, continued the tradition of Christian dream allegory in The Pilgrim's Progress. I'd forgotten, until I looked it up, that the full title is The Pilgrim's Progress, in the Similitude of a Dream. "As I walk'd," it starts, "through the wilderness of this world, I lighted on a certain place where was a Den, and I laid me down in that place to sleep; and as I slept, I dreamed a Dream." It's a traditional framing device – one that runs from Greek myths to Alice in Wonderland – and one that sets the narrator free. Anything can happen in a dream. If your cultural tradition is one of realism, then dreams are your way out. You can claim you're writing about God, as in much medieval literature, and actually write about knights and ladies and dragons. You can, in other words, claim you're writing about God and actually be writing about sex.

It was the Romantics who most fully embraced the dream as the purest pathway to the imagination. Coleridge claimed that "Kubla Khan" came to him in a dream (helped by a hefty dose of opium). When he woke up, he wrote it down, but after an unexpected visit from a man from Porlock, forgot the rest and had to stop. Keats's poems are dripping with dreams. His "into her dream he melted/ as the rose/ Blendeth its odour with the violet", from "The Eve of St Agnes", still reminds me, thrillingly, of my first kiss.

There's nothing like the erotic charge of a dream, nothing like the sense of unlimited possibility, nothing like its power to cheer or depress. Byron pretty much said it all when he said in "From the Dream" that "dreams in their development have breath,/ And tears, and tortures, and the touch of joy;/ They leave a weight upon our waking thoughts,/ They take a weight from our waking toils,/ They do divide our being; they become/ A portion of the past, – they speak/ Like Sybils of the future; they have power/ the tyranny of pleasure and of pain."

Which brings us, of course, to Freud. It was Freud, following in the footsteps of writers, mystics, saints, prophets, and tribal elders throughout history, who devoted his whole life to an examination of "the tyranny of pleasure and pain" and insisted that dreams had a value beyond the imaginative."We felt that we were the first," said his daughter, Anna, "who had been given a key to the understanding of human behaviour and its aberrations as being determined not by overt factors but by the pressure of instinctual forces emanating from the unconscious mind." At Freud's 80th birthday party, Thomas Mann praised him for discovering "a therapeutic method in the grand style". "Call this, if you choose," Mann said, "a poet's utopia."

Freud himself was more circumspect. It was, he replied, "the poets and philosophers before me who discovered the unconscious. What I discovered," he declared, "was the scientific method by which the unconscious can be studied." Actually, that was exactly what he didn't discover. Freud as a scientist has now been largely discredited, but as a writer and thinker he's in a league of his own. His theory that our dreams are not just mirrors of the psyche but the most powerful link we have with the fears, hopes and desires that drive us has permeated the entire culture of the 20th century. It still permeates our culture now. There could be no Dali without Freud. No Woody Allen, no Blade Runner and no Inception.

And there could be no Jung. After years of collaborating in the new science of psycho- analysis, Jung ditched Freud's assumption that certain images had fixed symbolic significance (a rather high proportion of which seemed to be phallic) and focused on "free association". Where Freud leant more towards the scientific, at least in his intent, Jung leant more towards the artistic, and the just plain weird. As keen on alchemy and astrology as sociology and anthropology, he has spawned some great poetic readings of the psyche, but also a fair bit of New Age nuttiness. In the US, you can even join "dream groups". Over wine and cheese, or coffee and cookies, you recount the details of your latest trip to work without your knickers, or your World Cup vic-tory. With the help of a "shaman", "dream-worker" or "dream coach" (to be found, apparently, on Craigslist), the group will tell you what it means.

We just can't stop this search for meaning. Even if we don't quite share the aboriginal view that our dream life is what's real, and the going-to-work-and-brushing-teeth stuff just a boring illusion, we still want to know that our adventures in the night are the key to a kind of truth that eludes us in the day. If we don't find God there, or Dan Brown's Lost Symbol, or Monty Python's Holy Grail, at least we might find some kind of secret. We don't know what we're searching for, but we'll know it when we see it.

What we don't want to know is that our dreams about the Queen (and there are whole books of them), or David Cameron, or Martin Amis, are just images we've lifted from the telly and chucked straight into the dustbin of our unconscious. We don't want to know, as some psychiatrists now insist, that our dreams are a dumping ground. And we definitely don't want to know that there's a particular bit of the brain that you could manipulate to wipe, or change, the content. In the great cyberspace of the unconscious, we all want to believe that we're the artists, not the mad contributors to the message boards.

Dreams, more than anything, represent the mystery at the heart of our existence. No one has yet found a way to control them. You can make people live like robots, but you can't change, or censor, the grand narratives they construct when they're asleep. Our dreams are the cauldron of our strongest emotions, our highest hopes, our deepest fears. In our waking hours, we tell ourselves stories to get through – stories with happy endings, stories of getting the guy or the girl, stories that bring order where once there was chaos. At night, what we get is the chaos. But it's the chaos, it turns out, that's cathartic.

On Monday night, I went out for dinner with a couple of writers. One of them, an academic, had been researching a book on dreams for 20 years. The other, a poet, was still haunted by a dream she'd had many years ago, a dream of "incredible happiness". Was it dreams, I wondered, that made us human? "No," said the academic. "Dogs dream. So do elephants." Was it art? "No," said the poet. "You can watch elephants painting on YouTube." Well, I'm damned if I know. But I still can't help thinking it has something to do with dreams, and something to do with language. Because Thomas Mann was right: dreams are a "poet's utopia". And in our dreams we're all poets.

'Inception' opens at cinemas nationwide on Friday