How did the opinion polls in the US get it wrong – again?

We all have our own shortcuts for trying to understand uncertainty. I find it hard to grasp election predictions expressed as percentage probabilities. When Nate Silver’s 538 website issued its final forecast – an 89 per cent chance that Joe Biden would win – I did not know how to handle that information. It now looks as if Biden will win, but this forecast didn’t seem to capture how close the election would be.  

I prefer to look at state-by-state opinion polls, because it is a state-by-state election. As we learned painfully four years ago, the national vote does not decide the presidency – the votes in a handful of swing states do.  

So I was aware that the election was close. Biden’s lead in the opinion polls in the states he needed to win was on average less than five percentage points. In 2016, Hillary Clinton had a comparable lead of 3.6 points in opinion polls in the key states, and it turned out that those polls overstated her support. If Biden was in a better position, he was only marginally so.  

Here we come to my shortcut. I do not know enough about the intricacies of the US polling industry, so although I knew that pollsters had adjusted their methods since last time, I could not be sure how effective this would be. But I assumed that polling companies would not want to be caught out again, and would if anything overcorrect for their error last time.  

I thought we could be reasonably sure that the opinion polls wouldn’t get the result wrong – in the same direction – again. For that reason, I thought that the polls were likely to understate Biden’s support in the swing states, and therefore that he was likely to win.  

Oh well. If he wins, it will be despite the opinion polls overstating support for the Democratic candidate again. It turns out that US commentators don’t know enough about the intricacies of state polls either. The 538 website tried to produce averages of state opinion polls that were adjusted for the quality of different pollsters. But those averages turned out to be more wrong than the crude averages that included polls disparaged by the experts as being unacceptably partisan.  

Some of the states that I assumed were safe for Biden, such as Wisconsin and Michigan, where his (adjusted) average poll lead was 8 points, turned out to be on a knife edge. (Biden led in those states on the unadjusted averages of raw polls by 7 and 4 points respectively.)  

But most of the state polls were not far out. The unadjusted averages in Nevada (Biden lead of 2 points), Pennsylvania, Arizona, Florida (all 1 point), North Carolina (level) and Georgia (Trump ahead by 1) were very close to the results.  

One reason why the opinion polls seemed to have got it more wrong than they did – and why Donald Trump appeared to be winning for many hours overnight – is the order in which states declare, and the order in which they count their votes. Most of the early states were for Trump, including, importantly, Florida; and most of the early counts were of in-person votes, which were skewed towards the Republicans.  

We were warned that many states would be counting postal votes, which tended to be more Democratic, later. In the case of Pennsylvania, this was a deliberate political decision, possibly designed with the kind of outcome that has just happened in mind – so that Trump could claim victory and claim that it was being taken away from him. But it was difficult to be sure, as the results came in, that they would tend towards Biden.  

As ever, although we should always try to learn from mistakes and to improve our understanding of how opinion polls and elections work, we should be grateful that elections remain unpredictable. When politics loses its ability to surprise it will lose its humanity.