For those of us that have been complaining about the obvious Republican bias in Scott Rasmussen’s polls since Obama was sworn in, all while the mainstream media called us hyperbolic, we now finally have the definitive takedown by numbers guru Nate Silver following the 2010 elections.

The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight’s database, which includes all polls conducted since 1998.

Moreover, Rasmussen’s polls were quite biased, overestimating the standing of the Republican candidate by almost 4 points on average. In just 12 cases, Rasmussen’s polls overestimated the margin for the Democrat by 3 or more points. But it did so for the Republican candidate in 55 cases — that is, in more than half of the polls that it issued.

In the past, Rasmussen ‘s goal was to use his biased polls to help shape the media narrative for the Republicans (well, mission accomplished). The trick though was to fix his methodology as the actual voting approached so he wouldn’t look like such an outlier, but apparently he got too cocky this season. Can we all now agree to ignore Rasmussen forever?

Rupert Psmith filed this under:  
  1. Gherald says:

    | Can we all now agree to ignore Rasmussen forever?

    No… it's a bit more complicated then that…. Observe:

    <img src="http://www.538host.com/pollacc1.png"&gt;
    Notice the R 3.9 Bias?

    For the 2012 cycle, we can agree to adjust their topline numbers by 4 points, which is pretty much what Nate Silver's model implicitly does.

    And given the high 5.8 average error, we should weight them less than polls from other sources. (again pretty much what Nate Silver's model implicitly does)

    And given that they poll so prolifically (over twice as much as PPP, for instance) we should also apply some calculus that simulates what would happen if every pollster polled every race an infinite number of times (again, Nate Silver's model does this).

    Long story short, the bottom line is we should pay about 1/8 th as much attention to individual Rasmussen polls as we do to individual polls from other sources, and we should adjust their topline results by 4 points.

    But, this is what we have Nate Silver for! He crunches the numbers so we don't have to : )

  2. Gherald says:

    Hmm, my 1/8th is probably a bad estimate. I was thinking they're 1/2 as accurate and poll 4x as much than others ( so should be weighted 1/2 * 1/4 less).

    But, when looking at individual polls (rather than taking aggregates for pollster ratings as Nate does) it actually shouldn't be a mark against them that they poll so frequently. After all, they're giving us some information that we can't get from other pollsters.

    So, we still need to adjust their results by 4 points of course--but maybe paying 1/2 as much attention to them as other individual polls of that area/race/date is a better weighting.

    (though, of course, we won't have to look at individual polls much once Nate starts aggregating them in the summer of 2011 or so.)

  3. Gherald says:

    There's also this to note:

    This lends more credence to the view I expressed Wednesday: Rasmussen’s sample is biased because they’re polling on the cheap — using robocalls, which by law can’t dial cell phones, and otherwise cutting corners — rather than because of some agenda to propagandize for the GOP. The end result, however, is the same: Polls that can’t be trusted.

    I didn't know robocalls weren't allowed to reach cell phones. Interesting.

    But I don't think it excuses all of their 4 point bias. Maybe 2 points of it, IIRC some of the stuff Nate has written about polling cell phones.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>