Tuesday, September 21, 2004
And Speaking of Biased Samples
by Tom Bozzo
Same goes for Iowa and the Gallup national poll too.
There are a couple of issues. One is that the methods used to determine "likely" voters may systematically shift measured voter preferences. But the Gallup sample of registered voters is apparently Republican-heavy, too. Since voter registration status should be easier to measure in principle, it's probably necessary to know more about the fundamentals of the sample design and interview protocol. Getting a random sample of voters by dialing phone numbers is not a trivial matter. Issues like the contact scheduling, the method for selecting a respondent, etc. (regarding some of which the Gallup methodology FAQ is vague), can easily color the survey results by systematically departing from the ostensible (in-principle unbiased) sample design.
Meanwhile, the Gallup poll results should be treated with suspicion. Do not conclude from them that Bush has an insurmountable lead, or indeed any lead at all.
Update 9/21/04: Robert Waldmann, commenting on a large-sample poll by ARG (showing the race very close nationally), offers the related thought that most polling sample sizes, and hence reported margins of error, seem to be selected so as to hide small biases. That seems not implausible, as typical poll MOEs (3-4%) give low signal-to-noise in very close races.
A Gallup poll reports Bush up by eight points among Wisconsin "likely voters." The Left Coaster examines the poll internals and finds an oddity: apparently excessive inclusion of Republicans in the sample.
This poll was based on a sample which contained a party ID breakdown of
GOP: 38%
Dem: 32%
Ind: 32%
Yet an October 2003 study of the Wisconsin electorate by Harris Interactive, found the following party ID for the state:
GOP: 28%
Dem: 29%
Ind: 31%
Same goes for Iowa and the Gallup national poll too.
There are a couple of issues. One is that the methods used to determine "likely" voters may systematically shift measured voter preferences. But the Gallup sample of registered voters is apparently Republican-heavy, too. Since voter registration status should be easier to measure in principle, it's probably necessary to know more about the fundamentals of the sample design and interview protocol. Getting a random sample of voters by dialing phone numbers is not a trivial matter. Issues like the contact scheduling, the method for selecting a respondent, etc. (regarding some of which the Gallup methodology FAQ is vague), can easily color the survey results by systematically departing from the ostensible (in-principle unbiased) sample design.
Meanwhile, the Gallup poll results should be treated with suspicion. Do not conclude from them that Bush has an insurmountable lead, or indeed any lead at all.
Update 9/21/04: Robert Waldmann, commenting on a large-sample poll by ARG (showing the race very close nationally), offers the related thought that most polling sample sizes, and hence reported margins of error, seem to be selected so as to hide small biases. That seems not implausible, as typical poll MOEs (3-4%) give low signal-to-noise in very close races.
