Saturday, October 27, 2012

Where are the National Polls?

We all see the divergence between the national tracking polls, which generally show Romney with a small lead on average, and the state-level polls, which generally show Obama with small leads in most of the key swing states on average.

How could these two things be true at the same time? There's actually two things going on here:

First, it is entirely possible that the President is winning by narrow margins in swing states like Ohio and Colorado but is losing by more in "solid Romney" states than he is winning in "solid Obama" states. That would translate into a national popular vote deficit but, potentially, an Electoral College victory. Most of the sites that create election-prediction models like Sam Wang's Princeton Election Consortium and Drew Linzer's Votamatic use state-level polls either exclusively or more heavily than national polls because the state-level polls provide more information that is useful. They are generally right to do this in terms of predicting what matters - the Electoral College outcome. But those national tracking polls are still unpleasant to look at ... which brings me to my second point ...

Second, it is really striking to me how few "full" national polls there have been in this campaign, especially in the final weeks of the campaign. There are at least 8 different national "tracking" polls that are released just about every day. But these are not the same as a regular poll. I can't claim perfect expertise on the impact (or not) of this phenomenon but it does seem to me there is some important effect of looking at only tracking polls as opposed to regular "full" national polls. Tracking polls take a smaller sample every night for 3 or 4 or 5 or even 7 (in the case of Gallup) nights. They put these together to create a national polling number. Then, when a new night comes on board, the sample from the oldest night drops out. What you end up with is a "rolling average" of the most recent nights. One potentially problematic aspect of this is that, because each night is a distinct sample and because the pollster wants to keep the demographics in the full sample correct from night to night, you have to weight some very small samples night to night before adding them to the sample. This would seem to introduce some error as opposed to doing a regular 2 or 3-night poll and then weighting the whole sample from there. In effect, you'd need to do less weighting as you'd have stronger sub-samples to work with.

Now, I'm just kind of spit-balling on all that and I could be wrong. But I do think that regardless, it is notable that there have been so few full national polls. Indeed, there has not been one full national poll that I can find since the third debate. I'd be interested to see one of those from a good pollster who does live interviews with cell phones included.

UPDATE: In a post today, Nate Silver speaks to the question of the split between the national tracking polls and the state-level polling. The bottom line is that he says the state-level numbers are suggesting leads for Obama in swing states like Ohio that are statistically very meaningful. Mark Blumenthal makes some similar arguments in Pollster.com

No comments: