Who will have the final say in the clash of America’s TV chat kings?

There is something of the Alan Partridge about David Letterman, the grand old man of late-night chat who has been helping America digest its cocoa these past 27 years.

And there was something of the Partridge about his decision last week to take a metaphorical plate, and a metaphorical knife and a fork, and very publicly eat his own words.

The scene was the small New York studio where Letterman’s hour-long show is recorded before being broadcast at 11.30pm each weeknight. The issue in question: a spiralling and very public dispute that he’d kick-started by telling a joke about the sex life of the teenage daughter of Sarah Palin.

His mission: to make a grovelling apology. “It was a coarse joke, a bad joke,” he said, with regard to the gag, about 14-year-old Willow Palin being “knocked-up” by 30-something baseball star Alex Rodriguez. “The joke, really, in and of itself can’t be defended ... I need to do the right thing here, and apologise for having told that joke … I’m sorry about it, and I’ll try to do better in the future.”

The following morning, Palin accepted the ‘umble mea culpa, “on behalf of all young women.” And so ended a short, but explosive PR battle. For Letterman, who had initially attempted to defend his joke on grounds of artistic freedom, the decision to back down seemed, on the face of it, to represent an abject personal defeat.

Yet while the veteran Late Show presenter may have lost this battle, the affair which became known as “joke-gate” helped propel him to victory in a much, much wider war. Because, as Jonathan Ross would no doubt testify, nothing, on either side of the Atlantic gets an over-paid anchorman’s stock to rise exponentially like a good, old fashioned brush with

notoriety. Latest viewing figures reveal that the dispute with Palin helped lift viewing figures for Letterman’s CBS show to the 3.67 million mark. That’s just 100,000 less than its major rival Tonight, on NBC. And it marks a stunning turnaround: just last month, The Late Show boasted closer to three million viewers, while its counterpart bagged five.

In the dog-eat-dog world of US broadcasting, such developments are big news. Presenters command enormous salaries (a reported $31.5m, in Letterman’s case) because their shows make serious loot. NBC’s profits from late night slots are around $300-million-a-year. The prospect of one of America’s “big four” channels losing market-leading status in this most lucrative of slots could undermine its entire business model.

If the stuffed suits who run either major network wanted to work out the reason for this seismic shift, they wouldn’t have far to look. The sudden free-for-all may have enhanced by Letterman’s spat with Sarah Palin, but that wasn’t its root cause. The root cause can be explained in just two words: Conan O’Brien.

Three weeks ago, O’Brien took over at the helm of The Tonight Show from Jay Leno, who’d held the job since 1992. This was big news: the Tonight presenter is a kind of monarch among America’s chattering classes. His pulpit has for 50 years been one of the most influential in the land. It was, after all, home, for three decades, to the legendary Johnny Carson.

Since Conan’s arrival, however, America has been wondering, with increasing urgency, whether he’s up to it. Despite generally favourable reviews, and such star-stuffed guests as Will Ferrell, Tom Hanks, Pearl Jam and Gwyneth Paltrow, pictured, his first week in the hot-seat was an unmitigated commercial disaster: audiences declined steadily every single night.

Weeks two and three saw that decline level off a bit. But they nonetheless felt shaky. O’Brien now has just a short period of time left to stamp a mark on the nation’s viewing habits. On 14 September, Leno will return to NBC with his own, eponymous comedy show. It will be broadcast at 10pm, and is expected to compete, on a day-by-day basis, for his successor’s guests, laughs, and (depending on the stamina of viewers) audience.

The two men at the centre of this second great rivalry are polar opposites. Leno is a ballsy former stand-up comic who remains a traditional gag-man whose fans love the wise-cracking monologues that kick off his show each night, and laugh enthusiastically at his perfectly rendered punch-lines. His interviewing style, broadly, is to let silly people be silly. O’Brien, by contrast, is a more cerebral presence.

Slight and ginger-haired, Conan specialises in observational humour that revolves around life and its absurdities. He learned his trade as a writer for Saturday Night Live and The Simpsons, before being parachuted, more-or-less untried, into the presenter’s job on Late Night, the show that follows The Tonight Show and runs into the wee hours, in 1993.

At school, Leno might have been the class clown; Conan would be the class intellectual, whose a wit is an acquired taste. So it doesn’t take a rocket scientist to see how audiences for the two men might differ wildly, and how Leno’s loyal audience might have decided to look elsewhere for its giggles when O’Brien popped-up on the airwaves on 1 June.

“If McDonald’s is shut, you’d go to Burger King, not to KFC,” says Jacquie Jordan, a Los Angeles media consultant. “By the same token, when Leno goes missing, you tune into Letterman over Conan.” You might wonder, given this seemingly-obvious fact, why NBC allowed O’Brien to take The Tonight Show at all. The answer is that the station was powerless to prevent it: in 2004, Leno, who is now 59, signed a five-year contract extension, saying that by 2009 he’d be ready to retire. Anxious to ease the Tonight succession, and prevent O’Brien joining a rival, NBC promised him Leno’s job – and agreed to pay him $45m (£23m) if he didn’t get it.

When the handover deadline approached, Leno found himself ill-disposed to retirement. In fact, his stock was higher than ever: in March, he achieved the signal coup of enticing Barack Obama to his Burbank studio (having become the first serving President to appear on a chat-show, Obama made headlines with an ill-chosen joke about the Special Olympics).

NBC were desperate to prevent their star man Leno defecting to a rival such as ABC but could also not afford to pay O’Brien $45m compensation to maintain the status quo. As a result, they decided on a classic fudge: a new Jay Leno Showwill be broadcast five nights a week in the slot before O’Brien’s Tonight Show.

Leno’s return to TV will be recorded in a bigger studio than the deceptively small Burbank cubbyhole where he’s been based for the past 17 years, and is expected to be sold as light entertainment rather than straight chat. Leno will crack more of the jokes that his viewers love, and record more clips outside. He has a a four-year contract (though NBC have a get-out clause after two) and a salary which depends on ratings, but could reach $30m.

All of which leaves O’Brien in an awkward spot: losing market share to Letterman, and with his predecessor poised to come and remind viewers exactly what they have been missing. However he does have one saving grace: the demographic of his audience. While Conan may not pull the sheer numbers of Leno or Letterman, his viewers are smarter and younger, and therefore more attractive to advertisers. He dominates the important 18-49 bracket, and his audience boasts an average age of 45. For Letterman, the same figure is said to be 54.

Given the relatively small cost of producing chat-shows which works out at roughly $400,000 per hour, O’Brien can work out as a decent financial proposition for NBC. While mass-market audiences are ambivalent, critics are well disposed to him. He has started to cause the occasional stir (last Thursday, a well-lubricated William Shatner made headlines by making crude hand gestures on his show).

“Mass audiences have shifted, but that’s not so important so long as Conan connects with the 18-49 crowd” says Alex Ben Block, the Hollywood Reporter’s esteemed editor-at-large. “Conan will make a nice living and NBC, which is constantly talking about ‘managing for margins’ will think he’s doing just fine.”

In other words, despite the age-old rivalries that persist when the egos of competing chat-show hosts collide there is a scenario that suggests all three of the big beasts of American late-night television, Letterman, Leno and O’Brien, can emerge as long-term winners.

The loser, in an increasingly cost-onscious market is TV drama. An hour of scripted drama, at $3m, is eight times more expensive than chat. Already network dramas, for years one of the US’s greatest cultural exports, are being shunted from the early evening schedules by cheap reality programming. With the arrival of Leno’s new show, they now face creeping eviction from the 10pm slot.

“Leno is up against CSI, and will probably lose from a straight audience-number perspective, but it doesn’t matter because the show will make more money,” predicts Block. “For scripted drama, the nightmare has now begun.” But that, of course, is a whole other story.

The veteran: David Letterman

Career: The longest-serving late-night host since Johnny Carson has been propping up a walnut desk since 1982, winning 12 Emmys from 67 nominations.

Salary: $31m a year

Viewers: Currently between 3.5 and 4 million, up slightly from its historic low before Jay Leno moved. His highest ever mark was 7.2 million, in the early 1990s.

Finest hour: Too many to count, though his tete a tete with a monosyllabic Joaquin Phoenix was as memorable as any. Letterman ended with a perfectly-improvised zinger: “Thank you for not being here tonight!”

Lowest point: When Johnny Carson left The Tonight Show in 1992, Letterman was considered heir apparent. Not only was he overlooked, but Leno, the man who got the job, overtook him.

The newcomer: Conan O’Brien

Career: Former writer for The Simpsons and Saturday Night Live, he started presenting Late Night in 1992, and took over the reins at Tonight at the start of this month.

Salary: Reported at $8m 2003. Now thought to be nearer $15m.

Viewers: He built a comparatively large audience of 2.5 million at Late Night. But he’s reduced Tonight’s figures from five million to under four.

Finest hour: In 2004, he wrestled the late Steve Irwin in a paddling pool, in a famous sequence given heavy airtime after The Crocodile Hunter’s untimely death.

Lowest point: With no presenting experience, his arrival on Late Night in 1992 got a disastrous critical reception. Audiences were mystified, and one TV pundit dubbed him, “a nervous and largely unskilled noob”.

The comedian: Jay Leno

Career: Took over Tonight in 1992, after five years as an occasional stand-in for Johnny Carson. Launches a new show in August.

Salary: Between $20 and $27m. If his new show is a success, he could make as much as $30m. If it’s a failure, NBC have a chance to ditch him in 2011.

Viewers: Currently none. Before leaving tonight, he had 5.2 million, the biggest in his field.

Finest Hour: In 1995, Hugh Grant was caught with hooker Divine Brown. Days later, he appeared in Leno’s studio. “What were you thinking?” asked the host. The episode saw Leno leapfrog Letterman in the ratings battle for the first time.

Lowest point: In 2004, Howard Stern accused Leno (with some justification) of stealing comic material from his show.