spacer Share
Friday, November 9, 2012

Polling, Likely Voters, and the Law of the Commons

As our tradition has been in presidential election years, Gallup's focus this year was on producing an estimate of the national popular vote. We don’t “predict” the election, nor do we make estimates of the Electoral College. In the end, Gallup's national popular vote estimate was that the popular vote was too close to call, a statistical tie -- 50% for Mitt Romney, 49% for Barack Obama. When the dust settled, Romney got 48% of the popular vote and Obama received 50%, meaning that Gallup’s percentage-point estimate was within two percentage points for Romney and within one point for Obama. The “gap” difference was three points. All of these are well within the statistical margin of error and underscore the accuracy of random sampling today, even with all of the challenges provided by changing forms of communication (i.e., cellphones), changing demographics, lowered response rates, identifying likely voters, and a wide variety of other factors.

We would always like to make a final poll estimate that is exactly on target with the final popular vote percentage for both candidates. That is the goal. But our estimate (and almost all other national polls at the end), gave a broadly accurate picture of what was, in fact, a very close popular vote.

As we have always done, we here at Gallup will continue to review all aspects of our election polling, from sample development, to interviewing, to data processing, to statistical weighting, to final analysis. Already this year, our methodologists instituted a number of changes. As explained here, we increased our percentage of interviews conducted on cellphones to 50%, modified our weighting procedures, tweaked other aspects of our sampling, and changed some aspects of our likely voter procedures.

We will continue to examine our likely voter procedures, a real key to understanding the final popular vote. Our final estimate of registered voters was an unallocated 49% for Obama, 46% for Romney. The transition to likely voters moved that to the unallocated 49% Romney, 48% Obama. 

We have modified our likely voter procedures in a number of ways over the years since they were first developed by George Gallup and Paul Perry decades ago.  But I think it is clear that voting today is subject to new pushes and pulls, including, in particular, the highly sophisticated ground games employed by the Obama (and, to a lesser degree, the Romney) campaign this year. These methods may in the end affect voters who were not certain about voting at the time of a poll interview, but who were brought into the voting pool at the last minute by aggressive get-out-the-vote and late registration methods. Our traditional "bootstrap" method of identifying likely voters is self-weighting -- letting voters’ responses to questions determine their probability of voting. This bears investigation. We will use the government’s post-election data, along with internal evidence, to see if further assumptions, investigations, or changes might be necessary.

We do believe that the presidential campaign underwent significant changes as it progressed this year. Romney clearly gained as a result of the first debate in Denver, and he held onto at least a marginal lead position in our polling until the week before the election, when Superstorm Sandy hit. Obama gained five points on the gap between our last pre-storm polling and the final poll. It may be that he continued to gain on into Election Day.

Gallup uses a wide variety of CPS demographic weighting targets for all of our samples. We traditionally do not put governors or other controls on party identification, as someother polls do. We reported results using very large samples this year, which generally helps control for normal sampling variation, but we will continue to look at this process.

Changes are in the wind that may affect polling as we know it in the years ahead. I think it is likely that we could see significantly fewer polls conducted in the 2016 election -- at the state and national level -- than in this election. In fact, some reports already say there were fewer state polls conducted in this election than in 2008, so we may already be seeing the beginning of a trend.

Some of this is a result of budget cutbacks, and some will be a shift to the use of other technologies for assessing public opinion in the future.

But some of this will result from a variant of the venerable “law of the commons.” Individual farmers can each make a perfectly rational decision to graze their cows on the town commons. But all of these rational decisions together mean that the commons becomes overgrazed and, in the end, there is no grass left for any cow to graze. Many individual rational decisions can end up in a collective mess.

We have a reverse law of the commons with polls. It’s not easy nor cheap to conduct traditional random sample polls. It’s much easier, cheaper, and mostly less risky to focus on aggregating and analyzing others’ polls. Organizations that traditionally go to the expense and effort to conduct individual polls could, in theory, decide to put their efforts into aggregation and statistical analyses of other people’s polls in the next election cycle and cut out their own polling. If many organizations make this seemingly rational decision, we could quickly be in a situation in which there are fewer and fewer polls left to aggregate and put into statistical models. Many individual rational decisions could result in a loss for the collective interest of those interested in public opinion. 

This will develop into a significant issue for the industry going forward.

The U.S. government, of course, does not conduct political polls. Perhaps it will be necessary to develop a consortium of pre-election polls in the future, like the National Exit Poll -- one gigantic poll or set of polls for all interested organizations. Or, on the other hand, individual polling organizations may come together in the future and sponsor one consortium of analysts making sense of the individual polls.

But clearly the traditional process, by which individual organizations independently make decisions to conduct their own polls, may be challenged in coming elections.

Tuesday, November 6, 2012

Nine Things to Think About on Election Night

Nine interesting things to think about as we await the election results:
  1. Republicans are more enthusiastic about voting this year than are Democrats. This contrasts  with 2008, in particular, when the opposite was true. Here’s how we ask the question: “Compared to previous elections, are you more enthusiastic than usual about voting, or less enthusiastic?” Here are the numbers: In 2008, Democrats (including leaners): 76% more enthusiastic; Republicans (including leaners): 61% more enthusiastic. Now, in 2012, Democrats (including leaners): 61% more enthusiastic; Republicans (including leaners): 73% more enthusiastic. We don't know the precise impact of these attitudes on actual voting behavior, particularly in a world in which there are massive get out the vote efforts. Such "mechanical" processes can override or supersede prospective voters' usual patterns. 
  2. Phone status is a modest predictor of vote choice. The majority of Americans have both landlines and cell phones. We interview roughly half of our respondents on landlines and half on cell phones. Likely voters we interview on a landline skew slightly to Romney 51% to 46%, which is not too far off the overall sample average for our final Nov. 1-4 poll (Romney 49%; Obama 48%). Voters we interview on a cell phone skew slightly toward Obama (Obama 50%; Romney 46%). Those we interview on a cell phone who tell us they don’t have a landline are slightly more likely to vote for Obama (55% to 40%). These differences are no surprise but raise significant issues with polling methodology that we and other firms have been dealing with for years now.
  3. Other than party identification and ideology, the American electorate is more divided by race/ethnicity than any other variable. In short, the nation is very politically polarized along racial lines. Among all voters who identify as non-white, the race is 78% for Obama; 19% for Romney - among likely voters. Among all voters who identify as white (non-Hispanic), the race is 39% for Obama; 57% for Romney.
  4. Other predictors of vote choice, which the exit polls will confirm after this election, are age (the older you are, the more likely you are to vote for Romney), education (the biggest distinction being those with post-graduate education who swing significantly to Obama), and of course gender (women would elect Obama; men would elect Romney).
  5. Swing state voters have received more “love” from the candidates’ campaigns this year.  Whether or not that is a good thing from these voters' perspectives is a different question.  In 12 swing states, 52% have been contacted by the Obama campaign and 50% by the Romney campaign. Around the nation, the total is smaller, down to 34% by the Obama campaign and 35% by the Romney campaign.   
  6. If we take voters at their word, 70% had their minds set for whom they would be voting before the conventions; another 12% decided after the conventions -- leaving 11% who decided this month and 10% who were still not totally certain at the time we interviewed them last Thursday and Friday.
  7. Among registered voters, 50% say Obama deserves to be re-elected, while 49% say that he does not. It doesn’t get more divided than this.
  8. Satisfaction with the way things are going in the U.S. is still not highly positive, but at 33%, it is now at its highest level since June 2009. Economic confidence is also still negative, but as my colleague Alyssa Brown outlines, it is more positive now than it has been since 2008. This higher confidence is fueled by the very positive attitudes of Democrats and also an uptick among independents. Republicans remain very negative about the economy.
  9. Romney and Obama now have the same favorable scores -- 52% among likely voters. Obama, however, has a higher score among all national adults -- 55% favorable compared to Romney's 46% favorable. At this juncture, both candidates are very well-known; 3% of national adults do not have an opinion of Obama, while 7% do not have an opinion of Romney. This wasn't always the case. When we first asked about Obama in December 2006, 47% did not have an opinion of him. When we first asked about Romney in December 2006, 69% did not have an opinion of him. I conducted an exercise where I re-calculated each candidate's favorable opinion based on the base of those who gave an opinion. Obama started out much more favorably, with 79% of those with an opinion saying it was favorable.  For Romney, it was 61% favorable among those who had an opinion in his first poll. Using this scale, Obama's lowest point came in October 2010 with a 48% favorable opinion among those who had an opinion. Romney's low point came in August 2007 with a 42% favorable rating among those who rated him.  

Wednesday, October 31, 2012

Status Update on Gallup Election Polling Following Superstorm Sandy

Here's an update on Gallup’s thinking when we suspended our national daily tracking of the presidential election campaign as of Monday, Oct. 29. Basically, we reached the conclusion that Superstorm Sandy had compromised the ability of a national survey to provide a nationally representative assessment of the nation’s voting population.

An unprecedented estimated 8 million households have lost power because of the storm and many others have had their lives disrupted. Cell phone service has also been compromised in many areas. Additionally, those in the storm-affected regions who can be reached by phone may be unwilling or unable to be interviewed. Ascertaining likelihood to vote may also be problematic when many voters are uncertain of their access to the polls and may not be able to say with a high degree of confidence whether they will vote.

These problems are obviously not uniformly distributed across the country but are concentrated in parts of the Middle Atlantic and Northeast and in specific areas within these regions. Thus, while we could have achieved the targeted number of completed interviews in these regions, those interviews would not have been representative of the overall population of the area. New York state is a prime example. While it would be possible to complete the proportionate number of interviews in New York state, those interviews would be skewed disproportionately upstate, not in New York City and Long Island.

All survey data are weighted to match the demographic characteristics of the population as a whole, including weights for telephone use now that interviewing involves both land lines and cell phones. But it is impossible to adequately weight to compensate for large segments of the population who cannot be reached at all in a survey, or in very low percentages, and whose opinions may have changed from previous, pre-storm measures.

Gallup is now tentatively planning on conducting interviewing over the last four days of this week, Thursday through Sunday, to provide a final pre-election estimate of the election race is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.