Tuesday, September 13, 2005

Economics of Self-Experimentation

by Tom Bozzo

Sunday's NY Times Magazine Freakonomics column mostly drops the "onomics" to discuss Seth Roberts's career of conducting psychological experiments on himself (described at much greater length in this postprint). I was less than impressed a few posts ago with Roberts's ability to recognize complex versus pseudo-complex political discourse, and I wonder how the wacky solutions to weight loss and an array of other personal problems generalize, if at all. For instance:
Stranger yet was the fix he discovered for lifting his mood: at least one hour each morning of TV viewing, specifically life-size talking heads - but never such TV at night. Once he stumbled upon this solution, Roberts, like many scientists, looked back to the Stone Age for explication. Anthropological research suggests that early humans had lots of face-to-face contact every morning but precious little after dark, a pattern that Roberts's TV viewing now mimicked.
Perhaps my best social psychology pal can let me know if I should give the after-the-fact evolutionary explanation any more credence than I'm instinctively inclined to give it, which is nearly none (*).

What grabbed me more from the column was the Dubner and Levitt gloss on the self-experimentation methodology. Curiously, they don't mention the first stated purpose of Roberts's research in the postprint, which is using self-experimentation "to generate plausible new scientific ideas." This actually does not strike me as very freaky at all, since I'd guess that navel-gazing of one form or another inspires a lot of real research (**). Roberts's research program would look more coherent if he carried out conventional experiments related to his "discoveries;" if he does so, he's quiet about it in the postprint. It doesn't especially bother me if it's not the most efficient means of idea-generation, though it would have been freakonomically interesting if Dubner and Levitt had shown that it was.

Rather, after noting the absence of experimental controls and the inability to conduct double-blind trials (***), Dubner and Levitt wonder if
the not-quite-scientific nature of self-experimentation [might] also be a good thing? A great many laboratory-based scientific experiments, especially those in the medical field (****), are later revealed to have been marred by poor methodology or blatant self-interest.
This is a big leap. Why not just promote better experimental methodology, or scrutinize research funded by interested parties more closely? It also doesn't obviously follow that the "extreme" self-interest inherent in self-experimentation necessarily increases the "transparency" of self-experimenters' methods. Would you trust the safety of a "safe" cigarette "discovered" through a "simple" method such as a brute force search because of the naked self-interest of the tobacco company?

Moreover, as self-experimentation is elevated beyond its not unreasonable role as a means for generating testable ideas, it risks running into the buzzsaw of the counterfactualist critique a la Robert William Fogel. Discovering that treatment X is associated with outcome Y in an uncontrolled or eccentrically self-directed self-experiment says bloody little about whether there exists an alternative Z that's no more (or maybe not much more) costly than X and that can give you at least Y. For his own part, Roberts doesn't seem to be positing any "axioms of indispensability" for his discoveries, but by traditional standards they aren't even shown to be effective let alone indispensable.

I'm also more than a little astonished by this bit of association:
In some ways, self-experimentation has more in common with economics than with the hard sciences. Without the ability to run randomized experiments, economists are often left to exploit whatever data they can get hold of.
It is true enough that few economists have the luxury of using data from randominzed experiments. But in applied econometrics, clever exploitation of data sources means finding data, and employing appropriate statistical methods, that under some economic model can be shown to estimate (*****) the quantities of interest as if one had data from nicely controlled experiments; Dubner and Levitt call this relying on "creative proxies" for well-controlled experimental data. The unavoidable leaps from econometric theory to most applied implementations don't change this basic scheme.

At least Levitt knows why he got the John Bates Clark medal. But surely he knows that it's not sufficient for self-experimentation just to produce variation within its "samples." Self-experimentation must produce data that are amenable to analysis equivalent to controlled experimental data — which it seemingly is not well suited to do by the column's own account. As a result, there's little demonstrated about Roberts's solutions beyond the freakiness. Stick to the physics diet in the meanwhile.

--------------------------

(*) How many generations after the invention of the campfire would it take to make this effect go away?

(**) My dissertation was inspired in part by a desire to predict whether I'd could expect to be able to buy Macs indefinitely in an increasing-returns-to-technology-adoption world.

(***) In some areas of study, this might actually not be much of a practical impediment because of the difficulty of developing "placebo" controls — what's a placebo diet? — but Dubner and Levitt don't distinguish areas where this is the case from experimental research in general.

(****) Maybe they're still smarting over criticism of their column on the efficacy of car restraints for children, which I've resisted discussing pending Levitt making the July 2005 working paper upon which the column was based available. (Otherwise, the statistical claims made in the column are unintelligible.) I suppose I could write him if curiosity gets the better of me.

(*****) E.g., identify and estimate consistently and/or "efficiently" with respect to a specified loss criterion.
Comments:
It's an interesting evolutionary logic for him to suggest that just becuase something happened to be the prevailing pattern of life in the Pleistocene, we've developed not just a taste for it but even an affective dependence on it. Even so, it says something that this wouldn't even make my Top 100 of weird invocations of evolutionary logic in supposedly highbrow America media.
 
I am "Seth's friend Tim" (Timothy Beneke) who lost 100 pounds pictured with a "before" (March 1999) and "after" (September 1, 2005) picture in the photo gallery at the freakonomics site. (Before: http://www.freakonomics.com/2gallery7.html After: http://www.freakonomics.com/2gallery8.html) I made use of Seth's basic principle that calories with weak or even zero taste reduce hunger, and developed a new method. Consuming calories with weak taste, and more recently, with zero taste, practicing what I call "taste celibacy" has enabled me to lose the weight.

Some weight details:
November 2, 1999 -- weight: 280. Began eeating weaker tasting low glycemic index foods -- eating fruits instead of juice, cutting out strong tasting desserts; no bread or potatoes; eating more low GI fruits and vegies (I used the book "The Glucose Revolution" as a guide to glycemic index.)

September 2000. weight: 250.

July 2003. Weight: 250. Began using roughly 350 calories of Star light-tasting olive oil a day scattered between meals, and continued to eat somewhat weaker tasting/low GI foods.

June 2004. Weight 210. Began experimenting with a mush, composed of liquified fruits and vegetables, mixed with a powder made of brown rice, almond meal, flax seed meal, dry non-fat milk, garbonzo powder, potato flour, and soy protein powder. I cooked it all together in water until it reached a moderately hard consistency. Then I take a tiny spoon, take some mush, and place it in my mouth, and take a big gulp of water and float it down my throat bypassing taste. I wash my my mouth out with water if I notice any lingering taste residue -- which is rare. Doing 25% of my calories with mush and olive oil only kept me at 210 for 10 months.

April 22, 2005. Weight 210. Then I began experimenting with total taste celibacy -- getting, initially for a few days 100% of my calories taste free. Between April 22, and today, September 17, averaging about 75% of my calories taste free, I've gone from 210 to 177, and am confident I can lose a bit more. I plan to lose another 8 pounds. I found, to my surprise that while taste celibacy deprives me of a certain pleasure, it's also liberating because eating has been such a source of worry, guilt, anxiety and ambivalence for so long.

There is a great deal more I could say but will stop for now.
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?