Were the polls wrong again?

Surprised about the 2020 election results so far? It seemed like Biden had a much larger lead in the polls than he is having in the election, especially in places like Florida. But aren’t wrong – they’re just uncertain.

Surveys are Samples

The best survey of a state’s opinion is a survey of the entire state – also known as an election. But it’s impossible for polling organizations to do that before an actual election, so surveyors have to use something called sampling: they survey a bunch of people (usually 500 to 1,500) who represent the state, and use those results to make an educated guess about what the state really believes.

If this seems weird, think of it like tasting restaurant food. How do you know if a restaurant is good or not? You can’t eat all of their food to see if any of it is bad, so you base your opinion on a sample of the food, which is the stuff you order. If your sample is really good, then you can guess pretty well that the restaurant is a good restaurant.

The only problem with sampling is that it’s very hard to be 100% certain. What if your food was the 1 plate out of 20 that had undercooked meat? Or what if the restaurant is really good at fries, but you ordered steak, which isn’t the best? You might conclude that the restaurant is bad when it’s really not.

The same thing happens in surveys; what if a survey just happened to sample a group of people that didn’t represent the population? Even the best surveys have a chance of this happening.

Margin of Error

Since surveys can be inaccurate in these ways, statisticians have come up with a nifty way to tell exactly how accurate they are, called the margin of error.

The margin of error gives a range around a polling number that is 95% likely to contain the true number for the state. So if your state polls at 51% for Biden, but has a margin of error of 4%, Biden’s real support could be anywhere from 47% to 55%.

Most surveys have a margin of error around 3.5% to 4%, and even the best ones have margins around 3%. To find the margin of error you’ll have to dig around in the actual survey – not the reports that make it into the news – and look for “Margin of Error” or “MoE” listed near the question you care about.

Estimating Probability of Winning

Using the margin of error, statisticians can predict the probability that a candidate really is ahead in a certain state. To do this they use a fancy rule called the Central Limit Theorem, which says that samples tend to converge on the true average. (Think of rolling two dice a bunch of times; eventually you’ll get more 7’s than any other number. The same is true for asking a whole bunch of people the same question – you’ll get to the true average eventually.)

The way that samples converge around an average is called a normal distribution. To see if one candidate is ahead in a survey, a statistician will measure the area between the normal distributions of the two (using a t-statistic), which gives a probability that the two averages are actually different. Here’s how to do that yourself if you’d like to.

If the normal distribution gives a 95% chance or greater that two survey results are different – for example, Trump getting 46% and Biden getting 51% – then it’s safe to say that the person in the lead is actually winning. Otherwise, it’s a complete toss-up; at that point, if you said one person was winning in the state, there’s a good chance you’d be completely wrong.

Why 95%? It’s just a good amount of certainty. Statisticians use different cutoffs depending on the situation – you could use 90%, or 99%, or anything in between. The point is that you don’t conclude that one person is winning unless you can be very sure that they actually are.

So what did the polls predict?

I scoured through 538’s election polls to find the best and most recent (A+ from a week ago beats A- from yesterday) for each of the battleground states, then used the normal distribution and t-statistics to test whether or not a candidate was really ahead for each state. Then I stuck it all on a prediction map for who could win each state.

In the map below (which originally comes from here), red states have a 95% or greater chance of voting for Trump, blue states have a 95% or greater chance of going for Biden, and the other states can’t be predicted accurately. (Note that I’m not including which way a state “leans” because that’s not statistically meaningful; if you can’t prove that two candidate’s ratings are different, then it doesn’t matter which one is ahead.)

(The dots in Maine and Nebraska are there because those states give electoral votes based on districts, not statewide.)

Compare this to the results so far, as of November 4 (straight from Google’s live election coverage):

So the polls were spot-on. Florida, Ohio, Texas, and Arizona were all too close to call before the election; Georgia, North Carolina, and Pennsylvania haven’t finished counting yet, as of 3:30 PM EST.

If the polls were not horribly wrong, we can expect Alaska to go for Trump, and Georgia, North Carolina, Nevada, Michigan, and Pennsylvania to go for Biden. This will give Biden a solid win of 321 electoral votes. Even if the polls were wrong, Michigan and Nevada are very likely to go for Biden, giving him 270 electoral votes and a tight, but safe, win.

So were the polls wrong in predicting a landslide win for Biden? Not yet.