Pre-election polling widely predicted a fairly sweeping victory of Joe Biden for US President over incumbent Donald Trump and the Democratic Party retaking the Senate. A day after the election, Joe Biden may have very narrowly won, but it is already obvious that the Democrats will not win control of the Senate. Consequently, even if Biden is ultimately certified after the inevitable recounts and court challenges, he will face a Senate hostile to confirming his cabinet and judge picks or passing any of his major legislative initiatives. This begs the question: why were polls so wrong again, as in 2016?
Polls are systematically wrong for the simple reason that random polling is impossible in practice. There are several ways polls are conducted, but every methodology is rife with errors that are hard to correct. First of all, we can ignore the most common problem: sample size error. It is well know in sampling theory that if you select a random sample of, let’s say, 1000 people, out of a population of one hundred million, there is a known random distribution by which your sample may be skewed. This is often reported as “the poll has an error of plus or minus three percent.” Whether whether any particular poll will be plus or minus and now much is itself random, so if you average a number similar polls, accuracy improves. This issue is well known and is not the source of systematic bias in polls.
Polls are biased because of several other factors. Ideally, a pollster would choose a sample randomly from among the entire population, but the “entire population” is not randomly and equally accessible. One problem is that an increasing number of people simply refuse to answer their phone when called. People who are busy or suspicious are more likely not to answer, so they are more likely to be skipped over. If busy and suspicious people are more likely to vote for one candidate over another, this polling method will fail to achieve a random sample.
Another problem is that many polling organizations choose numbers to call randomly, using a random number generator to pick phone numbers. That gives every possibly phone number an equal chance of being called, but it does not sample the population of voters equally because some phones will be shared among several potential voters. Other voters will have more than one phone. A few have no phone or rarely answer the one they have. Therefore people most likely to be polled have multiple phone numbers. Those least likely to be polled share a phone with others, have none, or rarely answer an unknown caller.
There may be other practices that polling organizations use that bias their sample. For example, the highly prestigious Pew Research Center asks for the youngest person over 18 when they call. This is likely to over-sample young people who are more likely Democratic and Biden voters. It is also possible that a parent will answer and simply hang up after this suspicious question to protect the privacy of their adult children. Suspicious people are less likely to cooperate.
There is another problem, which is much talked about, but I think less common. People may answer a survey, but lie. My guess is that most people not inclined to tell a pollster the truth would not waste their time and would instead hang up. The greater problem is gaining cooperation.
Because of the difficulty getting cooperation, another polling method is often used, especially when the target is “likely voters.” One might determine who is “likely” to vote by asking them, but such self-reporting might be wishful thinking or the lie you expect. Therefore databases of registered voters are non-randomly sampled to distribute people in the sample into categories similar to the population as a whole according to a “model” of who is likely to vote, often based on historic voting patterns. Then that well-crafted sample set is called, with understudies for those who do not respond to a call. For example, if 20% of people aged 18-25 tend to vote and 60% of people over 65 usually vote, your sample should include a larger proportion of old people. It is still difficult to imitate a random sample this way because there are so many different characteristics that might affect voting and these cross-cut each other. Furthermore, various types of people will be motivated to vote if the times or the candidate are atypical, changing who is a likely voter.
I suspect in this election, as in 2016, well-educated people were over sampled because they tend to have more phone lines on average and might more likely answer a random call. Suspicious or hostile people more likely to be Trump voters were under sampled. Trump voters are more likely to be antagonistic to the news media that often sponsors and reports polling. Polling organizations can try to “correct” for the inherent biases of their methods, but it is often no better than guesswork from one election to the next how much to correct and in what directions. These general problems with polling are global, not confined to the US. E.g., Brexit polls undoubtedly underestimated British opposition to EU membership for similar reasons.