(This piece was originally posted on November 4, 2012)
Aside from the principals themselves, their families, campaign managers and consultants, their staffs, their hangers-on and factotums, there is probably no one else with more at stake in this election than Nate Silver, who writes the New York Times’ FiveThirtyEight blog on political polling.
Silver’s fame rests on 2008, when he correctly predicted 49 of the 50 states in that year’s Presidential election. He uses a model that aggregates the data from all the other pollsters to come up with a consensus number, the theory being that one poll might be statistically biased or have a large margin of error, but combining them together minimizes these flaws.
For the last month, Silver has been like valium to Obama supporters, calming their rising anxiety that the race was slipping away. From Day One he has predicted that Obama had a better than 60% chance to win, and he’s currently at an 86% probability rate for a Democratic victory.
His analysis is very persuasive, boiling down to two points: 1) Obama is ahead in Ohio and Romney won’t win without Ohio, and 2) his model shows that the only way Romney can win is if it turns out the consolidated polling is statistically biased against him (read the full analysis here: http://tinyurl.com/ctsjzw3).
So he’s way out on a limb and will be hailed as a genius if he’s right and denounced as an Obama lackey if he’s wrong. Of course if Romney wins, he can still point out that under his predictions Romney did have a 16% chance to win and that this was one of those 16 out of 100 rolls of the dice that came up for him, but no one will really take that seriously.
He’s persuaded me that if the polls are right, Obama is the overwhelming favorite. But what if the polls aren’t right?
There’s something that just doesn’t feel right about them. In 2008, Obama got 51.5% of the vote in Ohio, yet many of the state polls have him higher than that now. Even taking into account Ohio’s better than average unemployment rate and the benefits they received from the auto bailout it seems unlikely to me that Obama would increase his share of the Buckeye vote at a time when he is down nationally and in almost every other state. After all, unemployment is still 7% there, which would normally be considered abysmal. More to the point, GOP registration has improved in Ohio and the early voting results show much more Republican voting than in 2008.
I’m reminded of two occasions when the polls were famously wrong. In 1948 the polls showed a big victory for Dewey over Truman, because, as it later turned out, the projections had been based on telephone surveys at a time when Republicans had more phones than Democrats. Then in 2004, the exit polls showed Kerry beating Bush; all through that Election Day Washington insiders with access to the exit polls knew that Bush was out, only to find out later that the pollsters had made some bad assumptions with their weighting.
At some point soon, if not already, the way polling is conducted is going to need to be completely overhauled. For polling to work, the first thing you need to do is produce a representative sample of the population. But because of the rise of cell-phone-only homes and caller ID, it is increasingly hard to draw a representative sample. Forty percent of homes now don’t have landlines (up from 20% four years ago), which automatically knocks them out of the sample and another large percentage have called ID and refuse to pick up the line if they don’t recognize the caller. Even Nate Silver acknowledges that only ten percent of the people that pollsters try to reach actually participate in the poll.
So before you even get into the question of margin of error for a 1,200-person sample, you’ve got a problem with the sample itself not being representative. To make the sample work, you’ve got to assume that the ten percent willing to answer questions are the same as the 90% you can’t reach. One way to do this is by weighting – giving more weight to hard-to-reach groups (young, minority, low-income). Then you have to assume that cell-phone-only homes are the same as landline phones. In other words, that a 40-year-old white woman with a cell phone is likely to vote the same way as a 40-year-old white woman without a cell phone; I’m not sure you can make that assumption.
But even if you can make the sample representative, you still face the problem of predicting turnout. Will 49% of young voters vote , as they did in 2008, or will it fall back to 36% as in 2000? What about African Americans, Hispanics and Evangelicals? I am willing to posit that African-American turnout will be the same, which will obviously help Obama, but I am not so sure about young voters.
For one thing, if you were an African American in 2008 you are still an African American this year, but if you were young in 2008 you are not so young now. All those college students from 2008 have since graduated (one hopes) and are dealing with this lousy economy. Current college students were in high school four years ago and you can’t assume they caught the Obama magic as 14-year-olds and have kept the flame alive since then.
Another polling challenge is that the people least likely to answer pollsters are those most hostile to existing institutions and most suspicious of the news media. And this year that would be Tea Partiers and Evangelicals – i.e., Romney supporters. Presumably pollsters try to take this into account when they sift through the raw data, but it’s hard to know how much to weight people who refuse to be polled in the first place.
Finally, I note that in the last few days, the popular vote lead that Romney maintained in the country as a whole, not just the battleground states, has shrunk and that he and Obama are now tied. Romney had a steady 2-3 point lead in the national polls prior to Hurricane Sandy, even as he was losing in the Electoral College, which doesn’t really make sense. Yes, it’s possible to lose the popular vote and win the Electoral College, as happened with Bush in 2000, but that was only a 500,000 vote gap, not a two-and-a-half million vote gap, which is what two percentage points would be.
My guess is that Romney’s national vote percentage is down not because his momentum was stalled by the lip lock that Chris Christie planted on the President, or because Hurricane Sandy made Obama look Presidential. Instead, I think the polling itself is off because the power is out in 4-5 million homes. Everyone is assuming that because the three most damaged states – Connecticut, New York and New Jersey – are solid Blue states, that Obama voters are the ones off the grid. But in fact, most of the areas without power (excluding lower Manhattan) are GOP enclaves within Democratic states. This includes tree-strewn suburbs like Fairfield County in Connecticut, and the only GOP borough in New York City, Staten Island.
Unlike some on the Right, I don’t think the polling firms themselves are biased, since it does them no good to give out bad data, but I do think the polls are unusually challenged this year, by the rise in cell phone only homes and an inability to predict turnout.
The dirty secret of polling is that most of it is based on educated guesses, and usually these guesses are pretty close. I know they all use brilliant measurement science techniques and sophisticated modeling, but in a razor-thin race, the difference between winning and losing can be less than a percentage point. In 2008, which was a clear victory for Obama, no one cared if a poll was a point or two off. This year it’s a very big thing to be off by one percent.
So who are you going to believe – the cold hard facts or your lying eyes? Your eyes tell you that GOP registration and early voting is well ahead of 2008, but the polls overwhelmingly say Obama is on track to win. We’ll know soon enough, but I’m betting that Nate Silver will have a very long night on Tuesday.