Forays in Bad Polling

A Marquette Law School Poll out today reports that Obama and Romney are tied at 46 among "likely voters."

Now, let me preface this by saying I could be wrong in how I'm interpreting this because the way the methodology of the poll is described is not entirely clear. And perhaps that is the first negative thing we can say about this poll. They are not clear about what they are doing. I'll include more specific caveats along these lines in a bit but, IF I'm reading this poll's methodology properly, it is really, really bad polling technique.

A recent Rasmussen poll of likely voters had Obama up by 4 in Wisconsin and Rasmussen is known to have at least a slight Republican lean. A more recent PPP poll, had Obama up by just 1 among registered voters. So it is possible the race is tied as this Marquette Law School poll indicates. But the methodological problem here has to do with the likely voter screen. As you know, there is a high-profile recall election coming up in just a few weeks and the poll also looked at that race. Just as an aside, the poll found that Scott Walker is up by 6 among likely voters so Obama is running ahead of at least Tom Barrett in the state and Barrett is a good proxy (sort of) for a generic Democrat.

So what's the problem here? The problem is that it seems that the likely voter screen is the same for the recall election as for the presidential race. In discussing the methodology at the bottom of the news release, they say:

The poll interviewed 704 registered Wisconsin voters by both landline and cell phone May 9-12, 2012. The margin of error is +/- 3.8 percentage points for the full sample. There are 600 “likely voters,” those who said they were certain to vote, with a margin of error of +/- 4.1 percentage points. Results for vote in the Governor, Lt. Governor and presidential races are reported for likely voters. All other results are for the full sample of 704 registered voters.

While we don't know for sure what the likely voter pool will be in November, I'm quite sure it will not be the same in terms of numbers or makeup as the recall election in a few weeks. Indeed, even if the recall election was going to be held in November on the presidential election day, we would have slightly different voter pools with more voters casting a voter for president. For a mid-year election day with few races on the ballot like the recall, the likely voter pool will be smaller and, more importantly, more Republican than Election Day in November. So I fear what we have here is a "likely voter" screen for the recall election that assumes the mix of voters in November will be the same as the mix of voters in the recall election. I think it is safe to assume that because we've got the same 600 voters in each pool. In addition, the statement indicates these voters said they are "certain" to vote. I have no doubt they will. But there will probably be some other voters voting who are currently "uncertain" they will vote and those voters are not randomly distributed. Good pollsters adjust for this with various methods including weighting sub-samples and other techniques. Here, we seem to have a very tight likely voter screen and that usually (not always) favors the Republican candidate.

In short, I think this may be a decent poll with respect to the Walker recall race but it is very problematic with respect to the presidential race in Wisconsin.