Keiren Fallon: Can he defy the racing establishment to stage a popular comeback?

They will be clearing up at Epsom Downs this morning. The Surrey racecourse was expecting a crowd of around 100,000 yesterday to watch the 229th running of the Derby, the most famous and prestigious flat race of the year. The victorious owner will have walked away with a cheque for £700,000 and the possibility of earning up to £10m more once their winning horse swaps his racing shoes for a life at stud.

The victorious jockey will have been celebrating too. Epsom, with its unique contours, uneven camber and nail-biting descent to Tattenham Corner, is widely recognised as the most difficult and demanding flat-racing track in the world, the turf's equivalent of the Monte Carlo Grand Prix, with its narrow streets and hairpin bends. Only the greatest names in weighing-room history have been able to ride Epsom with aplomb.

The ultimate Derby specialist was the incomparable Lester Piggott, who won the race nine times. Walter "The Choirboy" Swinburn was pretty good too, triumphing on three occasions, most famously as an angelic-looking teenager on the great but ill-fated Shergar in 1981. And back in the 1940s and 1950s there was no one like the charismatic Australian Rae Johnstone, another three-time winner and punters' hero. Twelve months ago the Derby headlines were dominated by an exuberant, and immensely relieved, Frankie Dettori, who finally clinched a victory at the 15th attempt.

The popular Italian, with his ready laugh and flying dismounts, brings racing exactly the kind of smiling publicity the sport's authorities crave. But if you ask around among the leading trainers you'll find there's general agreement that Epsom's master craftsman in the 21st century, indeed the strongest and most complete jockey since Piggott, is not Dettori but his great rival and polar opposite, Kieren Fallon, horse-racing's man in black.

The six-time champion jockey has won the Derby three times, including back-to-back for the Newmarket trainer Sir Michael Stoute in 2003 and 2004. He has also won the Oaks, the Epsom classic for three-year-old fillies, four times.

But the dramatic and poignant subtext to yesterday's Derby was that the 43-year-old Irishman, a man at the peak of his powers, was compelled to watch on television from the sidelines (this article went to press prior to the race). Fallon is not even half-way through an 18-month worldwide ban imposed last December for testing positive for cocaine after winning a race on a horse called, appropriately enough, MyboyCharlie, at Deauville last summer. It was the jockey's second offence. A year earlier he'd failed a similar test, also in France, leading to an initial six-month suspension that ruled him out of the 2007 Derby.

Fallon's mood, which can be dark at the best of times, will not have been improved by knowing that Stoute, his main patron, had three plausible contenders for yesterday's race: Tartan Bearer, Tajaaweed and Doctor Fremantle. Fallon has been working for the stable on the Newmarket gallops, teaching these immature thoroughbreds how to bridge the gap between promise and achievement. Yet it is some other jockey who will reap the benefit of that tuition, if not at Epsom then perhaps at the Irish Derby at the Curragh in three weeks' time, or in the Prix de L'Arc de Triomphe at Longchamp in October. "Kieren understands about the ban," says his former agent Dave Pollington. "But it will have hurt him bitterly not being involved. No other rider can hold a candle to him around Epsom."

As painful as the ban is for Fallon, he may have had to watch the Derby while enduring far worse than a drugs ban. From October to December last year he was one of six defendants on trial at the Old Bailey, accused of conspiring to defraud betting-exchange punters by deliberately "stopping" certain horses he was riding, on behalf of gamblers backing them to lose.

Many racing lovers following the court proceedings at the time, including plenty of Fallon supporters, feared that the legal drama might be the end of him. Others, among them some leading racing bureaucrats, journalists and commentators, implied that they hoped it would be.

The paradox of a sport's supposed advocates wanting to be rid of one of its biggest attractions was partly a reflection of an increasingly priggish and puritanical mood within sections of the racing media. But it was also a product of concern within the industry that neither the confidence of punters and politicians nor the future of the betting levy – a vital element of racing's finances – could be guaranteed if the sport appeared to be less than honest. There was exasperation too, from even the most unbiased observers, about the seemingly endless capacity of one of the sport's most talented riders for attracting controversy.

The realisation that the plasterer's son from Crusheen in County Clare was more a force of nature than a bland journeyman began in 1994. That was the season when Fallon pulled a fellow rider, Stuart Webster, from the saddle moments after they'd passed the winning post together in a race at Beverley, Yorkshire. The Irishman was incensed at what he felt was Webster's deliberate attempt to obstruct him.

In 1999, Fallon, now established in Newmarket, won the Derby for Henry Cecil on a horse called Oath. But only two months later he was dismissed from Cecil's employ over unfounded allegations that he'd had an improper relationship with the trainer's second wife.

In 2000 the jockey suffered an horrific shoulder injury after a fall at Ascot and in December 2003 he admitted to an alcohol problem; like the bulimic Swinburn before him, Fallon checked himself into the Priory for a 30-day treatment.

In 2005 Fallon began an alliance with Aidan O'Brien, the brilliant, soft-spoken trainer at the Coolmore Stud stable. Theirs seemed an unlikely marriage, pairing the wild boy with the teetotal family man. But the two got on famously, O'Brien's instinctive genius at training horses matched by Fallon's instinctive genius at riding them. They enjoyed numerous big-race successes, culminating in the unforgettable drama of last autumn's Prix de L'Arc de Triomphe in Paris. Fallon, 24 hours from his first appearance at the Central Criminal Court in London, gave an inspired ride to the O'Brien-trained and Coolmore-owned Dylan Thomas, landing the great prize by inches.

As the jockey returned to unsaddle, an Irish tricolour draped over his shoulders, many spectators were convinced they'd never see him trailing clouds of glory again. But others still couldn't understand how the sport of racing could ' somehow be a better place without him. "No other jockey would've won on that horse that day," believes Fallon's friend, the Yorkshire trainer Dandy Nicholls. "It was pure genius. He's so strong, he almost lifted him over the line."

Coolmore Stud had known about the corruption investigation involving their jockey when they took him on. And they didn't buy it. As one Coolmore insider puts it, "We all felt that while Kieren might've done a lot of things in his life, not all of them wise, he was innocent of these particular charges."

Coolmore's reservations didn't deter Fallon's critics from putting a different spin on the situation. Oblivious to the failure of another high-profile racing corruption case at Southwark Crown Court in 2000, they hinted that this time the authorities and the City of London police had compiled an unanswerable indictment. The termination of Kieren Fallon's riding career, which was expected to be one of the outcomes, would be the necessary sacrifice that would perhaps convince a modern audience that racing was a wholesome and responsible sport.

It was a scenario that smacked of attempts to find a frame big enough in which to fit the former champion, no matter what. The trainer Alan Jarvis certainly felt that some people "were out to get Kieren" at the time of the trial, and he stands by that statement now.

Fortunately for Fallon, the chief prosecution witness, an Australian steward imported especially for the occasion, was far from convincing. Fallon's barrister, John Kelsey-Fry, a consummate courtroom performer, exposed the flaws in the steward's testimony with forensic skill. He pointed out that the defendant had actually won five of the 17 races he was alleged to have fixed – a better-than-average strike rate for any jockey. He also reminded the jury that Fallon's efforts ended up costing his notional accomplices £338,000 in losing bets, suggesting a remarkably inept conspiracy.

A picture emerged of Fallon as a racing-obsessive, happy to trade texts and talk form and horses with everyone from friends such as the footballer Michael Owen to the humblest cabbie and bag carrier. But Kelsey-Fry contended that there was no evidence whatsoever to link his client with any of the other defendants in a criminal endeavour.

Mr Justice Forbes agreed and on 7 December threw the case out. The acquittal should have been the jockey's finest hour. But the very next day came the revelation of his second failed drug test, and those who wanted to see the back of him sensed that their wishes would be granted – albeit not quite in the way they had imagined.

Fallon had already had his licence to ride in the UK withdrawn by the British Horseracing Authority in July 2006, more than a year before he'd set foot in the dock at the Old Bailey. (He is still reluctant to discuss his experience on the record, partly because his legal advisers are considering civil compensation suits for his loss of income in Britain between 2006 and 2007.) Coolmore felt that was a wholly unjustified action and one that "cruelly disadvantaged" the jockey "at the peak of his career". The stable drew a comparison with the French and Irish authorities, who had allowed him to continue riding until the case was complete. But international flat-racing is a ruthlessly unsentimental business. Faced with the loss of their man's services until July 2009, Coolmore appointed Johnny Murtagh – another rider to have battled, successfully, with the bottle – as their new stable jockey.

There's no doubt that Aidan O'Brien felt personally let down by Fallon, having done so much to help him and keep his job open during the first drugs ban. "When Kieren joined us we all understood he had addiction problems," he says with painful sincerity. "Yet everyone at Coolmore always backed him 100 per cent of the way." What about the future? Does he really believe Fallon can come back, as he has done repeatedly in the past, and ride again at the highest level? "He may be able to," he concedes after a lengthy pause. "But it's going to be a long, hard road."

If you were to conduct a poll of the punters packed on to the Downs at Epsom yesterday or watching the Derby in their local betting shop, the vast majority would probably say they want to see Fallon riding again – and the sooner, the better. But then real racing fans, many of whose lives are far from being unblemished studies of tranquillity and perfection, are more sophisticated on this issue than some of the press-room judges who claim to speak on their behalf. They don't want jockeys to be cocaine addicts and they don't want them stopping horses for money. But they understand that in Newmarket, as in the world outside, most people are not all good or all bad but painted in various shades of grey.

Discovering that Kieren Fallon was faced with a range of temptations to which he sometimes succumbed hasn't made fans abandon the racecourses in droves. Neither did race-goers desert Epsom in the 1980s because of Lester Piggott's imprisonment for tax evasion or Walter Swinburn's eating and drinking disorders.

The earnest belief that, for the sake of its image, racing must be cleansed of all flawed and complex characters is anathema to a man such as Sir Peter O'Sullevan, wisest voice of an earlier generation. O'Sullevan, 90 this year, has consistently named Piggott and the fun-loving Aussie Johnstone as two of his greatest racing heroes; and Johnstone's problem wasn't drugs or booze but betting. "He gambled his way into poverty," wrote O'Sullevan elegantly in 2005, "and rode his way out again." Maybe like Piggott, Swinburn and Fallon, it was precisely that willingness to embrace danger and live life on the edge that distinguished Johnstone in a race from honourable, but less gifted, colleagues. Rae Johnstone died in 1964. Walter Swinburn retired eight years ago and has been building a successful second career as a trainer. And Lester Piggott emerged unbowed from his incarceration and went on to experience one of his finest hours in October 1990, winning the Breeders' Cup Mile on Royal Academy at Belmont Park, New York. Kieren Fallon is determined to emulate him.

The jockey certainly looks fit and well on the gallops and has been seen in the company of the talented and glamorous female rider Kirsty Milczarek. He's separated from his wife but remains close to his young children, whom he adores.

He's also started his own website offering subscribers regular updates on the good horses he's been exercising and his informed thoughts on the major races. He's kept a sense of humour, too, and has compared the internal enquiries conducted by the police and the racing authorities into their handling of his £13m trial fiasco with the episode of Blackadder Goes Forth in which Stephen Fry's General Melchett tries Rowan Atkinson's Blackadder for murdering his favourite carrier pigeon.

More seriously, he wishes to assure everyone that he's "a born winner... and losing is not a part of my genetic make-up. I always have and will always continue to give 100 per cent in everything that I do."

Not next summer, then, but maybe the one after that, Kieren Fallon will once more quicken the pulse and set hearts racing on Epsom Downs.