Friday, May 12, 2006

The Devil's in the Survey Design

by Tom Bozzo

There's been lots of buzz over (and efforts to rationalize) the Washington Post-ABC poll reported as registering broad public support for the latest infringements of civil liberties revealed in the NSA's domestic spying program. It's evidently right-wing talking point #1, as our own Bryan Smith hauled it out in the comments section to imply that if the vox pop has spoken, it must be right.

I've mentioned this in the comments, and others have so commented on other blogs, but it bears mentioning that the wording of the poll should not be discounted as a factor behind the measured support. Here's the first of the poll's NSA-related questions:

45. It's been reported that the National Security Agency has been collecting the phone call records of tens of millions of Americans. It then analyzes calling patterns in an effort to identify possible terrorism suspects, without listening to or recording the conversations. Would you consider this an acceptable or unacceptable way for the federal government to investigate terrorism? Do you feel that way strongly or somewhat?

This would appear to be about as favorable of a description of the program as could be formulated. It makes no mention of the program's dubious legality, or the possibility for information-sharing among agencies or possibly NSA contractors. It suggests only the ostensible national security purpose, and possibly conveys a faulty impression that the data analysis is focused on the identification of "terrorism suspects."

Here are the detailed results:
------- Acceptable ------ ----- Unacceptable ------ No
NET Strongly Somewhat NET Somewhat Strongly opin.
5/11/06 63 41 22 35 11 24 2

I'm not saying it isn't distressing that 41% of the public considers it "strongly acceptable" for the government to secretly mine phone call records. But the other 22 percentage points of support characterize the program as no more than "somewhat acceptable" — again, despite wording that plays up the fighting terrorism angle and plays down the unlawful privacy violation angle. So there's hardly 63% of ringing endorsement out of this poll.

Here's the even curiouser item, showing greater support for NSA snooping around one's own phone records:

46. If you found out that the NSA had a record of phone numbers that you yourself have called, would that bother you, or not? IF YES: Would it bother you a lot, or just somewhat?

NET A lot Somewhat No No opin.
5/11/06 34 24 10 66 *

Again, look at the wording. The NSA is described as "having a record" of the phone calls. Just for safe-keeping, I suppose.

That the innocent have nothing to hide seemingly is a belief among a substantial subpopulation of law-and-order types. Some of these results are just saying that. Otherwise, the WaPo poll wording appears to be putting a positive spin on a story that's as yet not fully investigated. The results should accordingly be taken with an appropriate grain of salt.


Meanwhile, as the blogiverse has been very busy on the subject, I'll point out to this "what do you take us for" moment, picked up at Waxing America:
Sen. Wayne Allard, R-Colo., said NSA was using the data to analyze calling patterns in order to detect and track suspected terrorist activity, according to information provided to him by the White House. "Telephone customers' names, addresses and other personal information have not be[en] handed over to NSA as part of this program," he said.
Bravo, Sen. Allard, for your Brooklyn Bridge salesmanship!

It would appear to go without saying that trying to identify the terrorist-suspect nodes in a vast communications network goes from very hard to essentially impossible if the nodes are anonymous. To do any useful analysis, the call records must be linked with personally idenfying information (PII). As anyone who's ever hung up on a cold-calling telemarketer surely knows, obtaining identity information on telephone customers is not hard — it only takes the money to buy a sufficiently comprehensive commercial database. It's the call records that are sensitive because once you know what calls are made, linking that information (if unreliably) to PII on the phone subscribers is straightforward, esp. if you happen to be sitting on a secret but no doubt impressive supercomputing capacity.


In the event you need a comprehensive background, Gary Farber has, in a deserved I-told-you-so post, dozens of links to informative posts here.
Comments: Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?