The Peril and Promise of Election Polls

Daniel A. Cox October 31, 2022

A graphic of multiple election 2022 pins on the entire image.

Over the last couple of election cycles, election surveys have faced mounting criticism—much of it justified—for producing estimates that differed significantly from the eventual result. In both 2016 and 2020 the polls were biased in favor of Democratic candidates. (Although the same bias was not evident in 2018.) The 2020 election was a particularly bad year for pollsters, who produced some of the largest errors in the last 40 years

This year, Americans (and especially disappointed liberals) are much more circumspect about the accuracy of election polls and their ability to predict the outcomes of the midterm elections. And caution is warranted. Here are a few things to remember as we head down the home stretch: 

Sources of Error 

Because surveys are based on a sample of a given population, the estimates they produce may not exactly match the “true” population value. This variance is typically captured by a survey’s margin of sampling error that provides information about the possible range of outcomes we might expect.   However, margin of error should not be the only source of the error to be concerned about, even if it’s the most familiar. Pollsters have long noted that surveys are subject to possible errors due to framing or question wording, interviewer effects, and response bias. Election polls have additional potential problems. In a recent New York Times interview, Patrick Ruffini outlined how a poll’s margin of error is woefully insufficient at capturing the true extent of possible error, stating, “It doesn’t measure the totality of the error that could happen: people could change their minds or you could be surveying a completely different sampling universe of people that actually show up on Election Day.” These types of errors are very difficult to quantify and could easily double the size of the actual margin of error.  

Low-Quality Polls 

The Internet lowered the barriers to entry in many professions. Suddenly, anyone with an Internet connection could fashion themselves as an authoritative voice or expert on a particular subject. In the pre-Internet era, some combination of experience, training, and education were required to conduct surveys. However, as the cost of conducting traditional telephone surveys rose, inexpensive alternatives flooded the market. Cheap data sources allowed low-budget operators to produce work of dubious quality. To make matters worse, random digit dialing (RDD) telephone surveys conducted by live interviewers, once considered the of “gold standard” in survey research, were not immune to problems. Falling response rates and nonresponse bias raised legitimate questions about whether or not traditional methods were able to produce more accurate results. Today, most pollsters recognize that there is no longer a “gold standard” for survey research methodology. Without a set standard for survey quality it becomes more difficult to know what polls to trust. 

Polling Alone: Reading Polls in Context  

We have an abundance of polls to help us make sense of the evolving political landscape. This is good! Plentiful polling means we have more information instead of less to assess candidate quality, the public interest in particular campaigns, and the issues that are most salient. But it also means that treating any particular poll in isolation is both unnecessary and unhelpful. Any new election poll should be viewed as an additional clue as to what may unfold. It’s probably unrealistic to expect media companies and other public pollsters to highlight their competitor’s work, but analysts and commentators should help readers make sense of new findings by placing them in context of other previously published surveys. Polling aggregators are also helpful in this regard, but they are only as good as what goes in them, and typically they don’t offer much information about the voting preferences of important subgroups. This is especially important because most election polls are based on relatively small sample sizes where subgroup voting preferences can vary significantly from poll to poll.  

Errors abound in any endeavor that requires human beings to give precise and immutable responses about their future behavior. Although polls were not designed to predict future events, they nonetheless provide plenty of useful predictive data. Yet, like any tool, polls are most useful if you appreciate how they work and understand their limitations. There are a few things that will help. First, don’t trust any pollster who is not completely transparent about their methods and results. This means producing a full topline and methodology statement. Second, polls are most useful when they are analyzed together. Patterns that emerge across polls are more likely to reflect something real. And finally, realize that the margin of error is not the sole source of error and adjust expectations accordingly. 

Survey Reports

Generation Z and the Transformation of American Adolescence Cover Image

Daniel A. Cox, Kelsey Eyre Hammond, Kyle Gray
November 9, 2023

Generation Z and the Transformation of American Adolescence: How Gen Z’s Formative Experiences Shape Its Politics, Priorities, and Future

This report explores the foundational differences between American generations through their formative adolescent experiences.

Young man sitting in a dark room before a wall featuring various conspiracy theory-related items illuminated by a computer screen

Daniel A. Cox, M. Anthony Mills, Ian R. Banks, Kelsey Eyre Hammond, Kyle Gray
September 28, 2023

America’s Crisis of Confidence: Rising Mistrust, Conspiracies, and Vaccine Hesitancy After COVID-19

America is experiencing a crosscutting crisis of expertise and scientific distrust accelerated by the COVID-19 pandemic that poses significant challenges to democratic debate and public decision-making

A cartoon showing a vibrant office from the ceiling view.

Daniel A. Cox, Brent Orrell, Kyle Gray, Jessie Wall
September 14, 2023

The Social Workplace: Social Capital, Human Dignity, and Work in America, Volume II

The Social Workplace, Volume II examines Americans’ expectations and experiences surrounding work, the workplace, and key job-related priorities such as pay and interpersonal connections.

An empty debate stage featuring red and blue podiums below a stage light face an audience of nearly-empty seats.

Daniel A. Cox, Ruy Teixeira
June 29, 2023

The 2024 Presidential Election: Evolving Political Coalitions and Familiar Partisan Divisions

Ahead of the 2024 presidential election, the AEI Survey Center on American Life conducted a national survey of Americans that explored a wide range of political attitudes, current voting preferences, and perceptions of the political parties.