New Hampshire Results Stir Polling Controversy

Why were the polls so miserably wrong on Tuesday's Democratic New Hampshire results, after Barack Obama held a double-digit lead on the eve of vote? Gary Langer at ABC News is on the big question of the morning:

There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong. We need to know why.

But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened - about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen - a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities - such as their own failings in sampling and likely voter modeling.

There have been previous races that misstated support for black candidates in biracial races. But most of those were long ago, and there have been plenty of polls in biracial races that were accurate. (For more on past problems with polls in biracial races, see this blog I wrote for Freakonomics last May.) And there was no overstatement of Obama in Iowa polls.

On the other hand, the pre-election polls in the New Hampshire Republican race were accurate. The problem was isolated to the Democratic side - where, it should be noted, we have not just one groundbreaking candidate in Barack Obama, but also another, in Hillary Clinton.

Langer's main hypothesis - assuming confidence in the poll findings - is that Clinton's voter turnout efforts were decisive, and perhaps the undecideds switched over to Hillary's banner at the last minute.

The Washington Post provides a deeper analysis:

There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong. We need to know why.

But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened - about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen - a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities - such as their own failings in sampling and likely voter modeling.

There have been previous races that misstated support for black candidates in biracial races. But most of those were long ago, and there have been plenty of polls in biracial races that were accurate. (For more on past problems with polls in biracial races, see this blog I wrote for Freakonomics last May.) And there was no overstatement of Obama in Iowa polls.

On the other hand, the pre-election polls in the New Hampshire Republican race were accurate. The problem was isolated to the Democratic side - where, it should be noted, we have not just one groundbreaking candidate in Barack Obama, but also another, in Hillary Clinton.

Here's more, on the racial angle and other possible factors:

Yesterday's result is sure to fuel debate among poll-watchers about the accuracy of polls in contests with African American candidates. In several well-known past examples, pre-election polls of such campaigns underestimated support for the white candidates. But a strong showing by polls in 2006 in elections with black candidates seemed to put that notion finally to rest.

Other factors that are more probable than the role of race include "likely voter" modeling, with pollsters perhaps over-counting the boost of enthusiasm among Obama supporters following his victory in Iowa last Thursday.

Independents may have opted at the last minute to participate in the Republican primary, depriving Obama of voters.

The New Hampshire ballot rules may also have played a role. In previous contests, the state rotated candidate names from precinct to precinct, but this year the names were consistently in alphabetical order, with Clinton near the top and Obama lower down. Stanford professor Jon A. Krosnick, a survey specialist, has estimated the impact of appearing high on the New Hampshire ballot at three percentage points or greater. Regardless, there were no immediate clear answers, and lots of data analysis ahead.

There's going to be more talk, throughout the campaign, of the racial vote.

But given the intensity of Obama's support in Iowa, there's little reason to support the hypothesis of a significant racial backlash in Tuesday's vote. More likely, the youth turnout in Iowa flowed to the Obama camp, but in New Hampshire older voters cast their ballots along more moderate to traditional lines.

Both Langer and the Washington Post mention Krosnik's "ballot order hypothesis," which sounds valid, as we know from 2000's "butterfly ballot"controversy that name-order and ballot-card design can have a large effect on the vote (although suggesting an Obama victory under rotating ballots requires deeping substantiation).

Clinton almost certainly benefitted from an "empathy vote," a surge of last-minute deciders influenced by Hillary's teary mini-meltdown, which flooded the media on Monday and Tuesday.

The Times of London looks at the data on "the crying game":

A rare moment of public emotion in a New Hampshire coffee shop was credited today with helping to bring Hillary Clinton back from the political dead and handing her victory in yesterday's crucial presidential primary.

Analysis of exit polls from New Hampshire showed that women voters, traditionally her most loyal supporters, flooded back after deserting her for Barack Obama in last week's Iowa caucuses. Mr Obama narrowly edged Mrs Clinton for the female vote in Iowa primary last week but yesterday she enjoyed a clear 13-point lead.

The psephologists will be poring over the New Hampshire results for some time to work out how the pollsters got it quite so wrong, but already it appears that a much-broadcast episode in which she welled up while talking to supporters in a Portsmouth restaurant could have earned Mrs Clinton a priceless sympathy vote in the state.

This is the "reverse Muskie effect," a reference to the 1972 campaign of Senator Edmund Muskie, which collapsed after he allegedly shed some tears in an emotional speech outside the headquarters of the Manchester-Union Leader.

But see Maureen Dowd's take over at the New York Times (via Memeorandum): "Certainly it was impressive that she could choke up and stay on message."

**********

UPDATE: Frank Newport, of the Gallup Organization, has an analysis of the New Hampshire polling fiasco, over at USA Today:

My best hypothesis is that Democratic voters in New Hampshire didn't cooperate with pollsters by maintaining their weekend voting intentions, but instead continued to evaluate candidates and to take new information into account right up until the time they went into the voting booth – and that a number in fact changed their minds about their vote at the last minute.

Newport's frankly blown away that voters changed their minds!

That's strange, coming from a polling expert of Newport's caliber - who's part of the Gallup organization no less - since one of the longest standing lessons of public opinion is that polls can miss late shifts in voter preferences. The key example: Gallup's own disastrous call of Thomas Dewey over President Harry Truman in the 1948 election:

I think Newport's right, though. Voters weren't exactly sure about their preferences when they spoke to pollsters over the weekend of January 4-6. Late-breaking events changed minds. That's actually to be expected from time to time, although a little humility is good in politics and public opinion.