The Peril and Promise of Election Polls

Daniel A. Cox October 31, 2022

A graphic of multiple election 2022 pins on the entire image.

Over the last couple of election cycles, election surveys have faced mounting criticism—much of it justified—for producing estimates that differed significantly from the eventual result. In both 2016 and 2020 the polls were biased in favor of Democratic candidates. (Although the same bias was not evident in 2018.) The 2020 election was a particularly bad year for pollsters, who produced some of the largest errors in the last 40 years

This year, Americans (and especially disappointed liberals) are much more circumspect about the accuracy of election polls and their ability to predict the outcomes of the midterm elections. And caution is warranted. Here are a few things to remember as we head down the home stretch: 

Sources of Error 

Because surveys are based on a sample of a given population, the estimates they produce may not exactly match the “true” population value. This variance is typically captured by a survey’s margin of sampling error that provides information about the possible range of outcomes we might expect.   However, margin of error should not be the only source of the error to be concerned about, even if it’s the most familiar. Pollsters have long noted that surveys are subject to possible errors due to framing or question wording, interviewer effects, and response bias. Election polls have additional potential problems. In a recent New York Times interview, Patrick Ruffini outlined how a poll’s margin of error is woefully insufficient at capturing the true extent of possible error, stating, “It doesn’t measure the totality of the error that could happen: people could change their minds or you could be surveying a completely different sampling universe of people that actually show up on Election Day.” These types of errors are very difficult to quantify and could easily double the size of the actual margin of error.  

Low-Quality Polls 

The Internet lowered the barriers to entry in many professions. Suddenly, anyone with an Internet connection could fashion themselves as an authoritative voice or expert on a particular subject. In the pre-Internet era, some combination of experience, training, and education were required to conduct surveys. However, as the cost of conducting traditional telephone surveys rose, inexpensive alternatives flooded the market. Cheap data sources allowed low-budget operators to produce work of dubious quality. To make matters worse, random digit dialing (RDD) telephone surveys conducted by live interviewers, once considered the of “gold standard” in survey research, were not immune to problems. Falling response rates and nonresponse bias raised legitimate questions about whether or not traditional methods were able to produce more accurate results. Today, most pollsters recognize that there is no longer a “gold standard” for survey research methodology. Without a set standard for survey quality it becomes more difficult to know what polls to trust. 

Polling Alone: Reading Polls in Context  

We have an abundance of polls to help us make sense of the evolving political landscape. This is good! Plentiful polling means we have more information instead of less to assess candidate quality, the public interest in particular campaigns, and the issues that are most salient. But it also means that treating any particular poll in isolation is both unnecessary and unhelpful. Any new election poll should be viewed as an additional clue as to what may unfold. It’s probably unrealistic to expect media companies and other public pollsters to highlight their competitor’s work, but analysts and commentators should help readers make sense of new findings by placing them in context of other previously published surveys. Polling aggregators are also helpful in this regard, but they are only as good as what goes in them, and typically they don’t offer much information about the voting preferences of important subgroups. This is especially important because most election polls are based on relatively small sample sizes where subgroup voting preferences can vary significantly from poll to poll.  

Errors abound in any endeavor that requires human beings to give precise and immutable responses about their future behavior. Although polls were not designed to predict future events, they nonetheless provide plenty of useful predictive data. Yet, like any tool, polls are most useful if you appreciate how they work and understand their limitations. There are a few things that will help. First, don’t trust any pollster who is not completely transparent about their methods and results. This means producing a full topline and methodology statement. Second, polls are most useful when they are analyzed together. Patterns that emerge across polls are more likely to reflect something real. And finally, realize that the margin of error is not the sole source of error and adjust expectations accordingly. 

Survey Reports

Cartoon rendering of a series of different online dating app prospects, on phone screens

Daniel A. Cox
February 9, 2023

From Swiping to Sexting: The Enduring Gender Divide in American Dating and Relationships

The January 2023 American Perspectives Survey sheds some light on dating preferences, experiences, and perspectives. The national survey of more than 5,000 adults age 18 and older, including nearly 800 single adults, finds that Americans have strong dating preferences when it comes to living at home, being unemployed, and smoking.

Red leather-bound Qur'an on a wooden table with prayer beads and a light blue surgical mask draped over top.

Lindsey Witt-Swanson, Jennifer Benz, Daniel A. Cox
January 5, 2023

Faith After the Pandemic: How COVID-19 Changed American Religion

The Survey Center on American Life at AEI teamed up with researchers at NORC at the University of Chicago to measure religious affiliation and attendance both before the pandemic (2018 to March 2020) and again in spring 2022, revealing who remained at the pews, who returned to the pews, and who left.

A cartoon showing a vibrant office from the ceiling view.

Brent Orrell, Daniel A. Cox, Jessie Wall
October 25, 2022

The Social Workplace: Social Capital, Human Dignity, and Work in America

Why is work, more often than not, the center of life for Americans? Explore the social dimension of work and the role it plays in building human connections and strengthening social capital.

Photograph of pro-choice protestors holding signs

Karlyn Bowman, Daniel A. Cox
October 4, 2022

Gender, Generation and Abortion: Shifting Politics and Perspectives After Roe

Three months after the Supreme Court overturned Roe v. Wade, the issue of abortion continues to garner widespread public attention.