Toggle Menu

Why the Polls Were Wrong

Some debatable assumptions underlie their use by the press.

Anna Greenberg

December 14, 2000

Since election night, pundits and politicians, as well as academics and armchair commentators, have spent an enormous amount of time wringing their hands about the mistakes made in calling Florida and the role the exit polls may or may not have had in the process. Although understanding the decisions behind the calls is important, it obscures the equally important fact that most of the national polls were wrong for at least the final two weeks before the election.

Of the eleven national polls and trackers released between November 5 and 6, only two surveys showed Gore leading Bush (CBS and Reuters/MSNBC/Zogby), while one poll showed the candidates tied (Harris). The remaining eight polls showed Gore losing, one by as much as 5 percentage points, which would have been the equivalent of a landslide. While many of these surveys in the final analysis lay within the margin of error, all the polls were biased in the same direction. With the Bush lead firmly established in the media polls throughout October, pundits for the most part were ready to give the race to Bush, even though Gore won the popular vote and probably won Florida.

Polls play an increasingly central part in election coverage as the horse race becomes a greater and greater media preoccupation. With survey results, the press can breathlessly report the impact of events and gaffes, like Gore's performance in the debates or Bush's abuse of the English language. The twists and turns of the race are cloaked in the scientific mantle of survey research, like the results of the Gallup tracking poll used by CNN and USA Today, which once showed an eighteen-point shift in the electorate in the course of a couple of nights.

Although we understand a great deal about the science of polling, the methodology of these surveys relies on a set of debatable assumptions about who is most likely to turn out and vote on Election Day. This year most experts predicted low turnout, meaning that only the most motivated voters–normally the most highly educated and affluent citizens–would find their way to the polls. Generally, low turnout favors the Republicans, since high socioeconomic status is associated with both political participation and conservative political preferences.

To accommodate these predictions, the polls screened tightly for those most likely to vote, adjusted for predicted turnout and in some cases "weighted up" the GOP share in the sample. All those adjustments meant that most of the national polls going into Election Day showed a 2- to 5-point Bush lead, even when individual state polls showed Gore performing much better. In fact, one of the great mysteries during the campaign was how Gore could hold leads or remain tied in so many battleground states and trail Bush so consistently in the national polls. One answer was simply that those polls were wrong. In fact, internal Gore polling showed the race tied and stable for the last two weeks of the campaign.

Why did the media polls employ those methodological criteria even in the face of serious get-out-the-vote efforts by the NAACP and the labor unions to bring out the Democratic base? First, the polling outlets were burned in 1996 when they overstated the size of Clinton's lead over Dole throughout much of the campaign. Second, the media outlets are competing for market share in an ever-expanding universe of polling data. In the course of this competition, they develop "proprietary" models for determining the composition of samples, with a premium on having the most scientific method for predicting the outcome. Once those criteria are determined, it is difficult to alter them midstream, because changes in the polls may be ascribed to methodological adjustments rather than actual changes in the race.

There is no insidious conspiracy to rig the polls in favor of one candidate or another. The national pollsters who partner with media outlets are respected survey researchers. But turnout was higher than predicted, thanks to the mobilization efforts of interest groups and the closeness of the race. The polls could not accommodate a broader concept of who might vote, and this methodological choice had political consequences. The fact that Bush could be seen as the presumptive winner certainly is linked to the mistaken Florida second call on election night, but it was framed by a month of Bush polling ascendancy.

Anna GreenbergAnna Greenberg, vice president of Greenberg Quinlan Rosner Research, is also an assistant professor at Harvard University's Kennedy School, on leave.


Latest from the nation