Thursday, September 09, 2004
How Bad is Sex on TV for Teenagers? (Annals of Questionable Research)
by Tom Bozzo
Based on an analysis of survey responses from 1,581 teens, the authors estimate that a one standard deviation increase in the amount of sexual content viewed on TV by teens has the same effect on the likelihood of engaging in sexual intercourse of being 9 months older and the effect of being 17 months older on a range of lesser behaviors, some of which may constitute sex by the Seinfeldian definition. How much sex is in one standard deviation? We are not told. It's claimed, but not exactly shown, that this is not a lot.
My take? I don't expect to be hiding the DVD's of "A Mighty Wind" (rated PG-13 for "sex-related humor") or "Young Frankenstein" (rated PG, presumably in part for innuendo involving the good Doctor, the Monster, Inga, and Elizabeth) on the kids just yet -- once they're interested in something spicier than Elmo's World, that is.
I say this because the actual paper did not bowl me over with its airtight analysis.
Now, the suggestion that kids are influenced by TV is not exactly a bold claim requiring extraordinarily compelling evidence. However, this paper commits a basic analytical error of such a magnitude that I'm surprised it got past the Pediatrics referees: it fails to control for the respondents' exposure to sexual content in all other media.
Since it's hardly inconceivable that kids who are exposed to a lot of racy TV also listen to racy music, print, are influenced by the increasing sexualization of children's clothes (cf. Freese's adjacent coming-of-Apocalypse post), etc. -- classic "confounding" factors, in short -- there must be immediate doubt that the reported effect of televised sexual content is actually all from TV, whatever other methodological failings might be present in the study.
The non-economist commentators have noted the study's use of 'longitudinal' data (i.e., the same kids were surveyed at two points in time) as a means by which the authors can separate correlation between watching sex on TV and having it in real life from causation. Indeed, the paper's clever wrinkle is that it predicts behavior in the later period with TV viewing in the earlier period, which is at least a prima facie showing of causation in one technical sense of the word. However, the authors do not demonstrate the absence of causation in the other direction, from sexual behavior to TV viewing. So there may or may not be a "feedback loop" between the media exposure and sexual behavior.
As for the rest of the analysis, this is a pretty classic example of statistics writing that makes it very difficult to evaluate the data, methods, and results. (This paper would not do well by the standards applied by Stephen Ziliak and Deidre McCloskey (PDF) to econometric results published in the American Economic Review.) Among other things:
So, as usual, do not trust, and always verify.
Bad, says this paper from the September '04 Pediatrics (via Jeremy Freese, who appears to have discovered it via cnn.com). In the upper realms of the blogosphere, right-of-center political scientist Daniel Drezner reluctantly gives the social conservatives a point based on this study, conducted by RAND Corporation researchers (and Dale Kunkel of UCSB, perhaps best known for reasearch on TV violence and the effect of the V-chip).
Based on an analysis of survey responses from 1,581 teens, the authors estimate that a one standard deviation increase in the amount of sexual content viewed on TV by teens has the same effect on the likelihood of engaging in sexual intercourse of being 9 months older and the effect of being 17 months older on a range of lesser behaviors, some of which may constitute sex by the Seinfeldian definition. How much sex is in one standard deviation? We are not told. It's claimed, but not exactly shown, that this is not a lot.
My take? I don't expect to be hiding the DVD's of "A Mighty Wind" (rated PG-13 for "sex-related humor") or "Young Frankenstein" (rated PG, presumably in part for innuendo involving the good Doctor, the Monster, Inga, and Elizabeth) on the kids just yet -- once they're interested in something spicier than Elmo's World, that is.
I say this because the actual paper did not bowl me over with its airtight analysis.
Now, the suggestion that kids are influenced by TV is not exactly a bold claim requiring extraordinarily compelling evidence. However, this paper commits a basic analytical error of such a magnitude that I'm surprised it got past the Pediatrics referees: it fails to control for the respondents' exposure to sexual content in all other media.
Since it's hardly inconceivable that kids who are exposed to a lot of racy TV also listen to racy music, print, are influenced by the increasing sexualization of children's clothes (cf. Freese's adjacent coming-of-Apocalypse post), etc. -- classic "confounding" factors, in short -- there must be immediate doubt that the reported effect of televised sexual content is actually all from TV, whatever other methodological failings might be present in the study.
The non-economist commentators have noted the study's use of 'longitudinal' data (i.e., the same kids were surveyed at two points in time) as a means by which the authors can separate correlation between watching sex on TV and having it in real life from causation. Indeed, the paper's clever wrinkle is that it predicts behavior in the later period with TV viewing in the earlier period, which is at least a prima facie showing of causation in one technical sense of the word. However, the authors do not demonstrate the absence of causation in the other direction, from sexual behavior to TV viewing. So there may or may not be a "feedback loop" between the media exposure and sexual behavior.
As for the rest of the analysis, this is a pretty classic example of statistics writing that makes it very difficult to evaluate the data, methods, and results. (This paper would not do well by the standards applied by Stephen Ziliak and Deidre McCloskey (PDF) to econometric results published in the American Economic Review.) Among other things:
- Conventional descriptive statistics for covariates (apart from some sample demographics) are not provided.
- In particular, the TV sexual content variable is only described after standardization. Even though it has a range of -1.43 to +4.49, nearly six standard errors, it's described as "somewhat positively skewed" versus a normal distribution (which would also yield a much narrower expected range in this size of sample). As mentioned above, the practical, as opposed to statistical, significance of a standard error of viewing is left to the reader's imagination.
- The authors report extremely limited diagnostic statistics for their regression models: the sample size, coefficient estimates, asterisks for "significant" coefficients, and a 'concordance' index as a measure of goodness-of-fit. My econometrician prejudice is for other types of statistics (scaled likelihood ratios) for this purpose, but setting that aside, it's irksome that the concordance index (which measures the extent to which the model predicts differences in outcomes among the respondents, so the concordance index is better to the extent that, if subject A had sex and subject B doesn't, the model says A had a higher probability of having sex than B) is described via a nonspecific reference to an multi-thousand-page statistics encyclopedia that is not on the Marginal Utility bookshelf.
So, as usual, do not trust, and always verify.
