You can take the boy out of Oldham...

This weekend, the Lancashire town of Oldham holds the eighth annual Walton Festival, which this year celebrates not only the life and works of its most famous civic son, Sir William Walton, but also one of his most important musical friends, Yehudi Menuhin. From Friday morning to Sunday night, Walton, who died in 1983, aged 80, makes a posthumous return to his roots, a place he avoided so assiduously for most of his life.

This weekend, the Lancashire town of Oldham holds the eighth annual Walton Festival, which this year celebrates not only the life and works of its most famous civic son, Sir William Walton, but also one of his most important musical friends, Yehudi Menuhin. From Friday morning to Sunday night, Walton, who died in 1983, aged 80, makes a posthumous return to his roots, a place he avoided so assiduously for most of his life.

Going first to Oxford on a scholarship to Christ Church Cathedral Choir School at the age of 10, then to London with his patrons the Sitwells, and later to the Italian island of Ischia, where he died, Walton's willed estrangement from his birthplace reveals a great deal about the man and his music, as well as illuminating the problems faced by the professional composer in an age when a private income was almost a prerequisite for the job. From the vantage point of Ischia, where Anthony Minghella filmed much of The Talented Mr Ripley, William Walton can even seem like a partial analogue of Tom Ripley; the stranger who swaps his lowly past for something more risky and exciting. Here, on the terrace of La Mortella, the home that he built with his Argentinian wife Susana (who will be in Oldham for the festivities), Walton recalled for Tony Palmer's 1979 film of his life, At the Haunted End of the Day, that: "I was determined never to go back to Oldham if I could possibly prevent it."

Despite Walton's apparent abandonment of his Lancashire roots, it's possible to see the legacy of Oldham inscribed as deeply in both the man and his music as the letters in a stick of Blackpool rock. Nor is the story of Walton and Oldham as straightforward as it might seem. Just as Walton's upbringing was far from a Monty Python-ish cliché of clog-bound poverty, so Oldham was a long way from the popular conception of a darkly satanic northern mill-town. Walton's family was as musically cultured as it could be given their lower middle-class status and his father's uncertain living as a music teacher with a day-job in the mill, and Oldham - according to Paul Barnett, the director of the Walton Festival - was at the time of the composer's birth a town where swagger and achievement were the order of the day.

"Walton was shaped by the place in the same way as lots of other Oldhamites who became notable in the arts or sports, like Eva Turner, the opera singer," says Barnett. "It was a new, self-made town, and in Walton's time it was known as King Cotton, with as much cotton spun in Oldham as in the rest of the world put together. For 40 years or so at the end of the 19th century, Oldham was the leading industrial town in the world, bristling with energy and a lack of fear. People in Oldham set up their own Stock Exchange in pubs, and pioneered the idea of co-operative mills. Also, the physical nature of the town, a hard place on the top of the Pennines, lent its character to the people who lived there."

Just in case Paul Barnett was doing a bit of spinning himself, last Sunday I went on a tour of the town with him to see what remained of the place the young Walton would have known. Perhaps symbolically, a thin drizzle of rain prevailed throughout. While the unsympathetic redevelopments of the Sixties and Seventies have more than taken their toll on downtown Oldham, whose now dowdy Union Street was once the scene of a regular evening promenade, the late Victorian suburb of Werneth, where Walton's family lived, is relatively untouched. It remains a comparatively ritzy place where the mill owners and their most favoured employees once lived, looking down on the town from the hill above.

The most persuasive theory about Oldham's influence on Walton is that it was the very same characteristics of the place that formed him, that consequently enabled him to leave it behind and look for something better. The town's pioneering, industrial-era spirit expressed itself in Walton's wish to get as far away from it as possible. In this sense, Walton's canny, craggy, unsentimental attitude to the business of making his way as a composer can be seen as Oldham through and through.

It was only after establishing relationships with wealthy patrons, first Christabel Aberconway, who left him a legacy, and then Alice Wimborne, whom he lived with and loved until her death in 1948, that Walton cut himself off from the Sitwells, with whom he had lived rent-free for eight years. Following Alice's death, Walton met and married Susana Gil, now Lady Walton, on a trip to Argentina. Having inherited from Alice a cottage in Belgravia, central London, Walton soon arranged the move to Ischia, which was financed by the sale of the property. Financially independent at last, he lost no time in leaving London for Italy and a new life with his new wife.

"It wasn't that he hated Oldham, but that he was aware of the need to move on," says Paul Barnett. "He realised early on that there was no future as a musician in the town unless he followed his father." It's a view with which the marvellously irrepressible Lady Walton, who now superintends her late husband's legacy from La Mortella in Ischia concurs. "He always acknowledged his upbringing, and in fact it was jolly lucky, because if he had not come from that part of the country he wouldn't have been so wily and as clever with his own life, and as determined to succeed, because conditions at home were so difficult," she told me in Ischia last month.

Lady Walton remembers her first trip to Oldham with Walton in the Fifties. "We went to visit the mother and the brother [Noel, who became a music teacher, and was something of a composer himself] and all that but it's difficult, because once you've grown out of that environment they look at you as if you came out of the zoo. The first thing was the meal, high tea, and I'd never encountered high tea before. I'd tucked in a bottle of whisky and William said, 'My God! If my sister-in-law sees that she'll destroy it.' So I had to hide it so that the brother could get the bottle of whisky that his wife, who was a non-drinker, disapproved of. God! And this high tea business, that was their supper. Well, habits are different for all kinds of reasons, and the light goes early there; it's dark and bleak and you either go to the pub or that's it. That was one of the reasons that determined William to live here. He remembered at 17 the shock of getting to Amalfi and finding this extraordinary light."

And for Walton, while his works might have come slowly, in a thin, Pennine drizzle of intermittent creativity, the notion of bright, transfiguring light became an aesthetic touchstone; as if, perhaps, he was determined to represent musically exactly those qualities that Oldham, which had made him, could never hope to achieve.

The Eighth Oldham Walton Festival: 6-8 October, tel: 0161 911 4072 for details