Neil Young: After the gold rush, the harvest

Next week, after two decades of fussing, fiddling, amassing and sifting, the first massive instalment of Neil Young's long-awaited career retrospective, The Neil Young Archives, finally becomes available.

It's a huge thing, both conceptually and physically: this first batch of 11 DVDs, covering the singer's career from his teenage origins in Winnipeg guitar band The Squires in 1963, through his time with Buffalo Springfield and Crosby, Stills and Nash, up to the point that he became a household name with the success of his Harvest album in 1972, arrives housed in a box the size of a couple of breeze-blocks, accompanied by a book of press clippings, lyric jottings, photos and ephemera. It is literally monumental: if the rest of Young's career is covered in similarly exhaustive fashion, his fans will not just be able to immerse themselves in it, they will probably be able to build a home from the boxes and live in it.

It's unlikely, however, that subsequent volumes will have the satisfying unity of the first set, which offers a detailed portrait not just of Neil Young but of the whole Los Angeles singer-songwriter counter-culture through the era in which it moved inexorably overground, rushing in to occupy the vacuum left at the end of the 1960s, when the Altamont and the Manson Family had effectively put the naive hippie dream to the sword.

Neil Young actually knew Charlie Manson a little, having been introduced to him at Beach Boy Dennis Wilson's house a few months before the Tate/LaBianca murders. Like many, he found the would-be songwriter a charismatic, powerful presence, with a curious gift – shared with Dylan, and Young himself – of being able to reel off lyrics apparently at will.

"Manson would sing a song and just make it up as he went along, for three or four minutes, and he never would repeat one word, and it all made perfect sense and it shook you up to listen to it," Young explained years later. "It was so good that it scared you." A few years later, he would write the song "Revolution Blues" from the standpoint of a homicidal psychotic clearly based on Manson. Back in the 1960s, however, Neil recommended Manson to his record company, Reprise, but nothing came of it. Had the label granted Manson his deepest wish and signed him up, who knows how things may have turned out? Sharon Tate might still be alive, Charles Manson might be a superstar rather than a bogeyman, and Neil Young might have been dumped by his label and retired from music altogether, as a car mechanic.

Cars have been one of the singer's enduring passions throughout his life. In one of the letters sent home by the teenage Young to his parents, reprinted in the Archives book, he appends a PS: "Dear Mother: Have purchased 1947 Buick Roadmaster Convertible. Will attempt homecoming in mid-September." Whether he made it back home is unknown, as is the fate of the vintage Buick; what is known is that shortly after, Young bought the hearse in which he travelled west to Los Angeles, where he formed the seminal Buffalo Springfield with his buddy Stephen Stills.

Since then, he has hardly stopped moving, criss-crossing America, and the world beyond, on regular tours; and on his most recent new album, Fork In The Road (released this March), he enthuses about his current passion, the LincVolt project. This involved the conversion of Neil's treasured 1959 Lincoln Continental to run on eco-fuels, the resulting vehicle then being driven across the USA without using gasoline.

It's but the latest of a constant string of frequently conflicting obsessions that have tugged the singer first this way, then that, throughout his career. Years before Dylan made his amorphous Renaldo and Clara movie, for instance, Young had a movie editing suite installed in his home on which he put together Journey Through The Past, a similarly shapeless mix of live footage, polemical counter-culture babble from David Crosby and other chums, a few poorly-paced set-pieces involving crudely-drawn archetype characters (a preacher, a general, a cardinal, etc) pursuing no discernible narrative thread, and some achingly tedious black holes like the one in which Neil and his partner Carrie Snodgress smoke fags on the fender of a vintage automobile for five minutes.

Made with $350,000 of his own money – a colossal amount in 1974 – the film was shown briefly in Boston, whose large student populace were deemed likely to be most sympathetic to its dubious charms, before being quickly withdrawn by the distributors. It has never been seen since, until its inclusion in the Archives box.

Neil, however, was undaunted by its fate, going on to spend a further $3m of his own money on Human Highway, an apocalyptic fantasy co-written and co-directed in 1978 with Dean Stockwell, and starring Young and Stockwell alongside Dennis Hopper, Sally Kirkland, Russ Tamblyn and new-wave band Devo, one of whose slogans – "Rust Never Sleeps" – would furnish Young with inspiration for his best album in years. More recently, he also made CSNY Déjà Vu, a documentary account of Crosby, Stills, Nash & Young's American tour in support of Neil's Living With War album, his furious indictment of the Iraq misadventure.

With typically unsparing candour, the film documents audiences' varied reactions to being asked to sing along with songs like "Let's Impeach the President" – some furiously giving the finger, others shouting along delightedly – while other scenes follow the stories of traumatised war veterans and liberal candidates trying to depose hawkish incumbents in local elections. It's a far more coherent and revealing film than his previous efforts, and served also to expose the growing shoots of dissent that would subsequently blossom into the election of President Obama.

Typically, the Living With War project was fought on a variety of fronts, Young also setting up an internet site – designed to spoof CNN and Fox News – on which songwriters could upload their own anti-war songs. His impetus for that album had been his frustration with what he viewed as the lack of young artists addressing the Iraq War with the same gusto with which an earlier generation had protested against the Vietnam War; but as the database of new anti-war songs on the website grew to over a thousand, it seemed that all that had been required was a platform for their presentation.

Young hasn't always been that politically engaged – certainly, not to the same extent as his partners in CSNY and many of their celebrity chums, who have consistently sustained a loudly leftist, pro-green presence throughout their careers. By comparison, Young is more prey to the passing whims of political circumstance. When unarmed demonstrating students were killed by National Guardsmen at Kent State University in Ohio in May 1970, his response was immediate and unequivocal: his song "Ohio", with its depiction of "tin soldiers and Nixon coming", was written, recorded and released by CSNY within days of the shootings. But in the mid-1980s, around the time he released an album pointedly titled Old Ways, Young seemed to have shifted significantly to the right, performing at the 1985 Live Aid concert a song called "Nothing Is Perfect", which lauded family values and mused hawkishly about the hostage crisis and the American military: "We got soldiers so strong they can bury their dead/ And still not go back shooting blind".

In interviews at the time, he railed against the national self-abasement of the Carter era, and explicitly supported Reagan's ramping-up of the Cold War arms race to oppose the Soviet misadventure in Afghanistan. "I stand behind Reagan when it comes to the build-up, to be able to play hardball with other countries that are aggressive towards free countries," he said. "I don't think there's anything wrong with that." A few years later he recanted, explaining, "I changed when I realised how wrong I was about a lot of things. The older you get, the more you realise that there's so many sides to every story, that right and wrong are not very easily defined."

Young's fans have long since learnt to expect the unexpected from him. Always headstrong and willing to follow his muse wherever it led him, he apparently took a perverse delight in confounding expectations both of fans and record company executives. Keen to hear the follow-up to the multi-million-selling country-rock album Harvest, the latter were deeply dismayed to be offered Tonight's The Night, a raw, ragged tribute to the deaths of two close friends from heroin overdoses. When Reprise refused to release it, he returned to the studio and recorded On The Beach, an equally bleak, rough album containing the aforementioned song about Charlie Manson alongside others offering Neil's gloomy prognoses on social change. Rolling Stone magazine called it "one of the most despairing albums of the decade", and they weren't far wrong.

And when Young played a rare concert in Britain around that time, he insisted – against vociferous dissatisfaction from an audience expecting the homespun hippie homilies and lovely melodies of After The Goldrush and Harvest – on playing instead the entire Tonight's The Night album, which hadn't yet been released. To assuage the restive crowd, he assured them that he really wanted them to hear this new material first, then after that he would play them something they'd heard before. But when he had finished Tonight's The Night in its entirety, he went right back and performed the title track again – which, of course, the audience had indeed heard before, albeit less than an hour earlier.

This contrarian spirit would be the key to Young's career through the next couple of decades, as he tacked back and forth between country-inflected albums like Comes a Time, Old Ways and Harvest Moon, harder rocking sets such as Zuma and the part-live, punk-inspired Rust Never Sleeps, and an increasingly baffling series of one-off excursions into uncharted and often unloved territories: if 1981's Re-ac-tor was an ill-received attempt to update Neil's Crazy Horse formula with new-wave rhythms, then the following year's Trans positively enraged fans with its new-wave synth-pop settings, computer-age lyrics, and most irritating of all, Neil's heavily vocoder-ed android-man vocals. In 1983, he had transformed himself again, this time into rockabilly throwback outfit Neil and the Shocking Pinks, for Everybody's Rockin', after which his record label Geffen actually tried to sue him for making "uncharacteristic, uncommercial records".

His response to their demand for a rock album was the lacklustre Landing On Water, possibly the worst release of his career; indeed, some observers contend it was a deliberate attempt to wind up David Geffen, an interpretation lent some credence by 1987's Life, whose cover portrayed the singer as imprisoned, the number 5 scratched on his cell wall supposedly a reference to the number of albums demanded by his contract with Geffen.

Neil proved no less quixotic upon his return to Reprise the following year: This Note's For You was a horn-laden R&B exercise whose title track railed against the increasing encroachment of corporate sponsorship into rock music; the video for the title-track was initially banned by MTV for portraying a Michael Jackson lookalike with his hair on fire, an example of Young's waspish sense of humour. Ironically, it subsequently won MTV's award for Best Video of the Year.

Despite this brief glint of success, however, by that time Neil Young appeared to be a spent force, both commercially and critically; but with characteristic obstinacy, he clawed his way back with 1989's tremendous Freedom, whose sardonic anthem "Rockin' In The Free World" became a double-edged soundtrack to the breakdown of communism. It hoisted the singer back to a position of enduring prominence which saw him feted as the godfather of grunge – Kurt Cobain's suicide note famously quoted the lines from Neil's "Hey Hey, My My (Into The Black)", "It's better to burn out than to fade away" – and proceeding with renewed vigour on a variety of fronts, albeit none too far removed from "classic" Neil Young territory.

He's unlikely to reach that stage of his Archives for several decades yet, of course, the first volume having taken some 20 years to compile. That's probably an indication of how important those early years were in establishing both Young's art and his character. It has been said that Crosby, Stills & Nash invited Young to join them because, nice guys as they were, their succulent harmonies didn't lie about their essentially laidback approach to life, and they knew they needed a top-notch songwriter and alpha-male personality to spur them on. And certainly, few rock stars can compare to Neil Young in terms of single-minded determination pursued with a ruthless disregard for others' wishes. More than one band line-up – even the beloved and faithful Crazy Horse – has been chagrined to find that they are suddenly out of a job because Young had decided to follow some creative whim in a completely different direction.

Not even stars like CS&N were immune: for many years, he refused point-blank to ever perform with the trio again until David Crosby had successfully beaten his drug addictions; and Stephen Stills only learnt that Neil had abandoned the duo's tour as The Stills-Young Band when he received a telegram mid-tour that read, "Dear Stephen: Funny how something that starts spontaneously ends that way. Eat a peach, Neil."

But it speaks volumes for the regard in which he's held by his peers that they never held such cavalier behaviour against him, nor his refusal to attend Buffalo Springfield's induction into the Rock'n'Roll Hall of Fame (a protest against being expected to stump up $1,500 apiece to be accompanied by more than one family member). And despite having participated willingly in the preparation of Jimmy McDonough's biography of him, Young subsequently took legal action to hold up its publication – later explaining that this was just a delaying tactic so that his teenage daughter wouldn't get to read it until she was 18.

This ornery, cussed aspect of Young's character has undoubtedly helped him surmount the various health problems that have assailed him through the years. As a child he was afflicted by polio, after which he was forced to learn how to walk again at the age of six; he has been troubled by epilepsy since his early 20s, has undergone several spinal surgeries, has been treated for a brain aneurysm, and still has trouble from a condition called hyperacusis, a hearing complaint that renders him overly sensitive to certain frequencies.

That could serve as an apt metaphor for Young's art, particularly the songwriting that seems to come so naturally to him. It's almost as if he's a human radio receiver, able to tune into events involuntarily, on an emotional level.

"I'm not sure that everything I write is mine," he has claimed. "I think some of it just comes through me. Writing is not like thinking: thinking involves logic, and when I'm writing a song, I try not to judge what I'm doing until I'm finished. I try to be open and follow the muse wherever it goes. And if it's not around, I don't push it. There's no sense in trying to fan a flame if there's no flame."

And though it may not burn as brightly now as it did in the era covered by volume one of his Archives, there's clearly enough still smouldering inside Neil Young to keep us warm for some years to come.

Five best albums

Everybody Knows This Is Nowhere (1969)

Young's second album was his first recorded with Crazy Horse, a rare combination of haunting melody, proto-grunge power and lengthy guitar excursions. Includes the wonderful 'Cinnamon Girl' and 'Cowgirl in the Sand'.

After the Goldrush (1970)

Probably Young's strongest all-round set, full of gentle melancholy and memorable melodies, including the classics 'Only Love Can Break Your Heart' and 'Southern Man'. No record better evokes the laidback LA hippie lifestyle.

Rust Never Sleeps (1979)

Rousing, ebullient proof that Young was just about the only Sixties rock musician to "get" punk-rock at the time. Includes the lovely "Pocahontas" and bristling affirmation of rock'n'roll "Hey Hey, My My (Into the Black)".

Freedom (1989)

Sterling return to form following the best part of a decade in the wilderness, finding Young once again engaged and focused. Features the prescient "Rockin' in the Free World", accidental anthem for the fall of communism.

Ragged Glory (1990)

Back with Crazy Horse again in unreconstructed hard-rock style, full of raw, spiky, feedback-drenched guitar bashes, with crude, croaky dabs of harmony vocals behind Young's characteristic weirdo drawl. It features the gloriously self- deprecating "Fuckin' Up".

My favourite Neil Young track

Chrissie Hynde: The Loner

His first solo album was hugely significant for me. I was 17 at the time and we'd just started to smoke pot. The image of the loner struck a chord with me. He's talking about the man who stands alone and doesn't conform. I love all the elements – rock guitar, almost melancholic tone, wistfulness, yearning, always tender, a little bit angry and hurt by a woman, clear, sweet, pure voice – that's never changed. He's one of the gods.

Jimi Goodwin, Doves: Broken Arrow

I first heard this on my Dad's 'Age of Atlantic' compilation and it still blows me away. I love the way it starts with a live version of another Neil Young song of Buffalo Springfield's, "Mr. Soul", then echomorphs into a marching band intro. It has completely different sections used between the verses and chorus, dream horror organ then waltz time and a jazz clarinet Ramsey Lewis-style outro. I learnt to play drums to it.

Beth Orton: Old Man

The first Neil Young song I heard. I was camping by the Norfolk coast and my friend had it on in the next tent. The music was like a calling to my soul. I had never heard a voice like this singing such tender words, almost at odds with one another, the music building to a heart-lifting singalong of fierce simplicity. Age-old wisdom from a 24-year-old which at the time seemed an eternity away from my 16 years. Capturing idealism in all its glory is one of Neil Young's gifts to the world. His songs unfold with the experience of years.

Paul Smith, Maximo Park: Birds

He's an awesome guitarist, but my favourite song is the piano-led "Birds". My Mam had 'After The Goldrush' on vinyl and this song stood out as a pure, melodic evocation of something that had been lost.

Dev Hynes, Lightspeed Champion: Violent Side

From my favourite album, 'Landing on Water'. A lot of people have either never heard of it or are not a fan because of its overuse of stadium-like (fashionable at the time) drums. To me the album is a perfect mix of 'Trans' and 'Rust Never Sleeps'. It has some of the best melody lines ever.

Alan Donohoe, The Rakes: Philadelphia

My sister lent me the soundtrack to Philadelphia and it had this song about the "city of brotherly love". I remember thinking it was pretty daring for an established singer-songwriter to write about gay rights. He has a soaring high range which really sticks in the brain.

Shingai Shoniwa, Noisettes: Old Man

He talks about being somewhere and feeling like there is so much more you could be doing with your life, "running around the same old town". Everyone has felt this at some point and this is what I feel Neil Young does perfectly. He can write a song that is introspective and about his own emotional despair yet relate it to 10 million other people.

Jeffrey Lewis: T-Bone

Minimal lyrics and maximum awesome effect, roughly one sentence for 9 minutes of music, shows that Neil is a songwriter of the first order. I'm somewhat playing devil's advocate in calling this my favourite Neil Young track of all time but in a certain way this song is a kind of masterstroke; in the 80s when many 60s artists were at the end of their creative energies, Neil throws down the gauntlet with "T-Bone" as if to say "I don't care, I'm gonna do whatever I want and keep on being awesome..."

Katie Melua: Philadelphia

I was completely blown away when I saw the film Philadelphia and heard this song. It is the most heartbreaking, wonderful song. As a result of hearing it in the film, I re-discovered 'Harvest' and "Harvest Moon". He's a great and passionate performer and his melodies are haunting.

Romeo Stodart, The Magic Numbers: Cowgirl In The Sand

A friend of mine was driving me home from band practice one night and we were listening to her cassette copy of CSN&Y's live album 4 Way Street. Neil's voice just sent shivers down my spine, it was so heartfelt though somewhat unsettling as it felt like I was listening in on someone's truth laid bare. Years later I discovered that it was one of three stone-cold classics that he'd written within a day of being bedridden with the flu.

Jon King, Gang of Four: The Needle & the Damage Done

Confessional, authentic, didactic. The voice of a simple guy who can't hide the pain, witness to a tragedy, telling it like it is, without irony: smack's bad, kills good people, don't do it. Neil's signature voice quivers with emotion, the looping melody circles in sadness. We listen and believe.

Graham Coxon

I don't have a favourite song. There is an LP I've listened to called 'Ragged Glory'. It's extremely American. I don't feel old enough. If I lived on a ranch, maybe, but I live in London. I don't know if it complements my life much, although I have some cowboy boots somewhere.

Win a box set

The Independent in conjunction with Warner Brothers are offering readers one DVD set of Neil Young's archives, worth £150, as well as five sets of his complete recordings on Warners.

To win, answer the following question: In which country was Neil Young born?

To submit your answer, go to independentpromotions.co.uk and input the reference code YOUNG. Closing date is 5pm on Friday 5 June. The winner will be drawn at random. The prize is non-transferable and non-refundable. The editor's decision is final. Only one entry per household.