1) My predictions were wrong / based on faulty assumptions, etc. or
2) The national polling numbers are not right
I'm generally skeptical when I hear other people making argument #2. It is not so much that pollsters are never wrong. Of course, there are bad polls and there are bad pollsters. But when all the pollsters are moving in one direction, there's only one way to argue they're all wrong and that is if they are all doing something systematically incorrect. That's possible but not very likely.
That said, there is one thing that jumps out at me when I look at the national polling numbers and the arguments I've made: I don't think many pollsters are good. This occurred to me as I was looking at the recent national polls. I've listed below all of the national polls recorded by Pollster.com in which either most or all of the sample has been since the first debate:
The pattern here is pretty clear and Obama's not doing well in these polls. If you average them, Romney is ahead by 0.8 points overall. I've said elsewhere though that pollsters who interviewed on 10/4 (the day after the first debate) showed huge bounces for Romney. If we include only polls that were in the field from 10/5 and after, Romney still leads though it is a slightly smaller average lead of 0.4 points overall.
So, what's going on? Am I wrong? Is Romney's bounce really this big and this lasting? Maybe. The one thing that jumps out at me when I look at that list of pollsters above is that there are only two pollsters there that I think do a good job. One of the pollsters I think is good is PPP though PPP generally has a bit of a Democratic lean. Their poll giving Romney a 2-point lean seems like really bad news then, right? True, except it does include some respondents from 10/4 and the poll is getting a little long in the tooth now. The other pollster I think is good is Pew. They have Obama down by 4 points ... but the same problems are there as with the PPP poll.
What's most striking about this list when we look at it this way is that I don't think any good pollsters have been releasing national polls this week. Who are the good pollsters in my view? Aside from PPP and Pew, I think NBC/WSJ, ABC/WP, CBS/NYT/Quinnipiac, and CNN are pretty good pollsters. None of them has any national poll that has been in the field since at least last Sunday. That's very strange to me.
Maybe my predictions are wrong. But I really want to see a national poll from one of those outlets before I apologize.
Posts
3 comments:
Good post. I'd love it if you could elaborate, though, on what makes a good poll/polling company vs. a less reliable one. Why do you like the ones you mentioned, in other words? Is it methodology, sample size, etc? In other words, going fwd., it'd be great to learn a little re: how to differentiate between a suspect vs. a solid poll, as well as I can identify a solid baseball metaphor, from one based on "sport."
There's a few things and maybe I'll write a follow-up post on this. But here's a couple of things:
One is track record. Gallup has been pretty bad in recent elections for instance. ARG and Zogby are pollsters that fall into this category.
Another thing is live interviewer vs. robo-poll and related to that is whether they call cell phones. Rasmussen and PPP are robo-poll outfits. PPP has a bit of a Democratic lean but I think they've generally been pretty good in recent cycles. I just sort of discount their numbers a small bit. Rasmussen has a fairly strong Republican lean, which would be fine if we could just make that mental adjustment but there's something else wrong with Rasmussen. They weight by party ID which is very problematic.
Robo-calling and the cell phone issue are related because it is illegal to call cell phones without a live interviewer. They try to get around this by weighting their sample in some way (Rasmussen uses party ID for one) but this gets very tricky very fast.
Finally, I like some of those network pollsters because they tend to use good sample sizes over a short number of nights. Gallup's tracking poll is over a 7-day period. That's problematic.
Anyway, those are some initial thoughts on all this.
Post a Comment