Marginal Utility

by Tom Bozzo

As a follow-up from the earlier post on the topic, in the Crooked Timber thread following Quiggin's post, commenter RB points to a letter to Nature's editor from 1994 by Johns Hopkins biostatistician Steven Goodman making the case that Gott's reasoning is an example of an old statistical fallacy. (Goodman posted it as a comment to Tierney's NYT blog.) Goodman's general thrust — 'lies, damn lies, statistics' — is correct, but he maybe goes a bit too far in deploying the f-word.

Simply put, the principle of indifference [i.e., the fallacy] says that it you know nothing about a specified number of possible outcomes, you can assign them equal probability. This is exactly what Dr. Gott does when he assigns a probability of 2.5% to each of the 40 segments of a hypothetical lifetime. There are many problems with this seductively simple logic. The most fundamental one is that, as Keynes said, this procedure creates knowledge (specific probability statements) out of complete ignorance.

Actually, there is a more charitable version than this, which is how I'd previously set up the problem, and how Monton and Kierland characterize Gott's original argument. In my account, the uniform distribution of the observation point is explicitly part of the (assumed) information set; I've packed my free lunch as it were. If that doesn't sound like much, it's not. However, I would submit that the more useful thing to argue over is whether the uniform distribution assumption is warranted. As it happens, I said before that the assumption is strong before, and what I mean is that in practice it seems unwarranted for the array of amusing social applications that Gott can't seem to resist.

But for a little more damnation by faint praise, let's just remember what's being promised by the method: a prediction within a factor of 39 of the start-to-present interval. As a practical matter, the real problem in many cases is not that too much fabricated information is being brought to bear, especially at the upper bound.

Labels: Philosophy, Science, Social Science, Statistics

by Tom Bozzo

Just going to show what happens when you drop off even the post-paywall NYT op-ed page, reaction to John Tierney's report that we have 46 years to colonize Mars Or Else Civilization is Dooooomed has been relatively muted over the Intertubes. Prof. Bainbridge quotes the Ole Perfesser without comment (see Roy at Alicublog for the omitted analysis) but also Charlie Stross's excellent post on the grim case for space colonization.

So how do you get that 46 years?

Suppose you're observing an Event that occurs during a fixed time interval (potentially a strong assumption). Suppose also that Baldrick is flying your space-time conveyance and drops you at a random point in the interval (potentially a very strong assumption). Suppose third that you know nothing else about the event. Your "best guess" as to where you've landed, in the expected value sense, is the midpoint of the interval. So if you then get your bearings and figure out how long ago the event started, which is all the information you have, your best guess is that the event will end the same amount of time in the future. That isn't a very good guess, though, in the sense that there's a 50% chance that the "true" end will be sooner or later than that.

Applied to the human spaceflight program, dated to 1961 (questionable [*]), then by advanced mathematics about 46 years have elapsed since then and the information you have and the assumptions above lead to the result. QED.

What the astrophysicist J. Richard Gott did, in a short paper, was to construct interval estimates with high confidence levels -- statements that the unknown end date for the event should fall between A and B 95 percent of the time. For the spaceflight case, A is 2008 (next year) and B is AD 3,801. But saying that you're 97.5 percent confident that the human spaceflight program will end in the next 1,800 years or so doesn't have the same sense of urgency. More generally, 95 percent confidence results in a range from 1/39th the age of the event on the low side to 39 times the age of the event on the high side. Call this the "Copernican formula" if you will. The proof methodology (see this paper [PDF], helpfully linked by Tierney) uses only undergraduate-level mathematical statistics, so read it yourself if you're so inclined.

This leads me to strongly endorse John Quiggin's conclusion:

The real lesson from Bayesian inference is that, with little or no sample data, even limited prior information will have a big influence on the posterior distribution. That is, if you are dealing with the kinds of cases Gott is talking about, you’re better off thinking about the problem than relying on an almost valueless statistical inference.

Indeed, if observing the passage of a year and nothing else, the upper bound of the interval moves out 39 years. That can be a big deal in many applied circumstances! For example, here's Gott himself writing in the New Scientist in 1997. A subhead of "Living proof" suggests he isn't engaged in deliberate leg-pulling as he recounts:

As another test, I used my formula on the day my "Nature" paper was published to predict the future longevities of the 44 Broadway and off-Broadway plays and musicals then running in New York; 36 have now closed - all in agreement with the predictions. The "Will Rogers Follies", which had been open for 757 days, closed after another 101 days, and the "Kiss of the Spider Woman", open for 24 days, closed after another 765 days. In each case the future longevity was within a factor of 39 of the past longevity, as predicted.

In this application, a prediction within a factor of 39 of past longevity conceivably covers the range from total flops to huge hits to productions that will eventually be performed by automata in Wisconsin Dells. The "prediction" for the "Will Rogers Follies" is that it will (likely) close within the next 82 years. That's out on a limb. (And certain philosophers inclined to bash social scientists for theories with weak predictive value might put this in their pipe and smoke it.)

Reinforcing Quiggins's point on how posterior distributions may be influenced, had Gott's paper appeared a week earlier, he'd have missed on "Kiss" to the tune of 100 days, since the previous week of running time adds some 9 months to the prediction's upper bound. If it matters whether the production runs another week or another year, searching for information is not unlikely to be rewarded.

Meanwhile, if you wanted to make some inference on whether both "Kiss" and "Will" would be playing at some future date, forget about it. The most interesting contribution comes from Brian Weatherson (at CT and Thoughts, Arguments, and Rants), who derives a neat result showing that if you infer the probability of both plays running at a future date based solely on the length of time they've run together (the information the method admits), it follows that if "Kiss" (the shorter-duration event) is still playing at that date, then "Will" (the longer-running event) will also be playing with probability 1. Weatherson concludes that there must be "something deeply mistaken with the Copernican formula."

My own little gloss, pending peer review in the self-correcting blogithingy, is here in the CT comments. What seems to be happening in this case is that (1) Gott's method throws away the information on how long "Will" has been running, and (2) sneaks in an additional assumption that the "Kiss" and "Will" events must be dependent or correlated. There may be circumstances under which these extremely strong assumptions may be justified, Weatherson maybe goes a bit too far in suggesting that these but they strike me as implying more than diffuse information on anything other than the elapsed times of the events.

Last, since you are by definition still with me here in the unlikely event you are reading this, here's the brief rant portion of the post: How the frack did Gott get 5 frackin' pages in Nature for this, which looks a lot more like it merits a paragraph of Mathematical News of the Weird?! I've been turned down cold — not even this 'reject and resubmit' stuff Drek writes about for stuff a hundred times harder and at least somewhat more relevant, if I don't say so myself. (If you really have time to kill, you may note that part of what I'm talking about eventually came out via other researchers' efforts as part of this IIASA working paper a few years later.) And if that's happened to me, then so too must everyone except Nick Bostrom, as I infer from Tom's Anti-Copernican Principle. W. T. F.

And BTW, Nature, what's up with US$30 for an e-print? Surely the revenue-maximizing price — which given the approximately $0 marginal cost, is also profit-maximizing — is not set at levels that make the likes of me think about sending junior staff to the library (were there a business case for actually obtaining the paper). Just saying.

[/rant]


[*] It's not like Yuri Gagarin's rocket just materialized on the pad and blasted off. And remember, going back even a few years into the preflight stages of human space programs puts a century or two on the upper bound.

Labels: Philosophy, Science, Social Science, Statistics

by Tom Bozzo

Brad DeLong points to Adam Kotsko, who not only liked Stanley Fish's "Atheism and Evidence," but indeed lamented that the Times Select paywall keeps it from a broader audience. So let me expand on my previous reaction to Fish.

Fish criticizes Sam Harris and Richard Dawkins for their confidence that natural explanations will be found for currently not-well-understood phenomena of human behavior and consciousness. He invokes Francis S. Collins to name a scientist who would

argue that physical processes cannot account for the universal presence of moral impulses like altruism, “the truly selfless giving of oneself to others” with no expectation of a reward. How can there be a naturalistic [i.e., evolutionary] explanation of that?

Fish, let alone Collins, shouldn't need an economist to answer, "easy." Behaviors that don't seem to maximize individual fitness but may improve the population fitness aren't a problem for evolutionary explanations. (Elaboration of this concept, I gather, is Dawkins's major contribution to evolutionary theory.) Taking the politically charged subject of human behavior out of the picture, evolutionary accounts explain how, for instance, the gene that causes sickle-cell anemia can persist in populations at high risk for malaria despite the fatal consequences for individuals who get two copies of it.

(If I wanted to be snarky, I would say that writers inclined to lofty phraseology like "the universal presence of moral impulses like altruism" should read more anti-"death tax" polemics. I'd also wonder why Kotsko's postmodern allergy to overarching meta-narratives isn't aggravated by such questionable assertions of universality in human motivation.)

What Fish's argument really does is lays down a bet against future achivement of science:

Of course one conclusion that could be drawn [from hitherto limited progress in obtaining naturalistic explanations of human behavior] is that the research will not pan out because moral intuitions will not be reducible to physical processes. That may be why so few of the facts are in.

It's not good when you're trying to make the case that others are making logical leaps to leap to a conclusion that purportedly limited progress in a relatively new field of scientific research implies a problem beyond naturalistic explanation. Fish may offer the argument in the (not totally unreasonable) expectation that there will remain uncertainty over the physical processes that might be responsible for "moral intuitions" for the remainder of his life and thus that he won't be around to suggest that this explanation for the limitations of present knowledge is facially foolish.

Younger folks might not want to risk too much of their wealth on the anti-materialist position, for there's already evidence suggesting that behavior not totally unlike "moral intuitions" are in fact emergent properties of physical processes. For example, many people who are more-or-less miserable find themselves not as miserable while taking SSRIs. This suggests that "misery" is, at least in part, a property that's mediated by the chemical reactions SSRIs interfere with. A non-materialistic alternative explanation would seem to imply that SSRIs have some mystical effect on the "soul" or "spirit" despite being the products of scientific research that makes no appeal to mysticism, not to mention being manufactured in non-magical labs by secular corporations.

Yet this is likely a pillar of Kotsko's affection for Fish's essay, since Kotsko dislikes "reductionism." It is Kotsko's own business if he finds the set of all explanations from the in-principle effable world inadequate. But labeling "naturalism" in this sense as "reductionism" of the bad sort does some violence to much-less-innocent forms of reductionism, such as reducing people to reified utility functions and enacting policies that are sensitive to the assumptions one places on H. Economicus. (Cf. Waldmann's Wager.)

Kotsko starts quoting Fish in what is little more than a "past performance is no guarantee of future results" argument:

[Fish:] A very strong assertion is made – we will “undoubtedly discover lawful connections between our states of consciousness [and] our modes of conduct” – but no evidence is offered in support of it; and indeed the absence of evidence becomes a reason for confidence in its eventual emergence.

I'm inclined to call this as the first of a couple of flagrant fouls, insofar as I don't think this fairly characterizes the basis for confidence in future scientific progress. First, there is plenty of evidence of "lawful connections" between natural processes and "states of consciousness" and/or "modes of conduct" (q.q.v.) which frankly are obvious enough that it's inappropriate to criticize the Harrises and Dawkinses for not reciting them. Second, it takes something like willful blindness to suggest that science doesn't have an excellent track record in developing naturalistic explanations for natural processes. Third, also on the obvious side, the toolkit available to would-be students of the brain-ethics link has been rapidly expanding — think of the prospects for a computational biology research program based on 1980s technology. Last, the system under consideration is rilly rilly complex and it stands to reason that such "facts" as may be teased out of naturalistic explanations will take time to develop.

Kotsko also quotes what, to me, is Fish at his most infuriating:

[Fish:] [Dawkins says there] are “good Darwinian reasons for individuals to be altruistic, generous or ‘moral’ towards each other.”

So there's the answer to the "how can there be a naturalistic explanation" question.

[Fish, continuing directly:] Exactly! They are good Darwinian reasons; remove the natural selection hypothesis from the structure of thought and they will be seen not as reasons, but as absurdities. I “believe in evolution,” Dawkins declares, “because the evidence supports it”; but the evidence is evidence only because he is seeing with Darwin-directed eyes. The evidence at once supports his faith and is evidence by virtue of it. [Emphasis added.]

That's flagrant foul #2. Note the demotion of "natural selection" to a "hypothesis" as opposed to a natural mechanism that can be demonstrated empirically in the wild and/or simulated in a variety of lab-type settings (not least, the human body). The Darwinian explanation is that the behavior makes the group better off despite (maybe) having cost to some individuals, which frankly doesn't sound facially absurd under, say, a Divine Selection Hypothesis where "good works" facilitate more pleasant after-lives. (An economist might argue that it's not necessarily true that altruism necessarily is "costly" to the individual; at a minimum, I would argue specifically that it narrows the real scope of source-of-moral-behavior conundrums.) More to the point, Dawkins makes no claims that obviously can't be explained in terms of neuron interconnections and brain chemistry.

Fish carries this idea of circular reinforcement of belief systems to the point of gross misrepresentation:

The reasoning is circular, but not viciously so. The process is entirely familiar and entirely ordinary; a conviction (of the existence of God or the existence of natural selection or the greatness of a piece of literature) generates speculation and questions, and the resulting answers act as confirmation of the conviction that has generated them.

Even if you believe that the exsistences of God and of natural selection are "convictions" of equal stature — I doubt you'd get buy-in from either the theist or the atheist directions — the claim that answering "speculation and questions" necessarily reinforces the foundational convictions is just so much bullshit. Kotsko (presumably with Kuhn and/or Feyerabend in mind) criticizes falsificationism as the "Newtonian mechanics" of the philosophy of science, suggesting that scientists should better represent how the process of science really works. But science does not tell us that Newtonian mechanics are useless. Neither Kuhn nor Feyerabend is correctly read as demonstrating that scientific theories are inherently self-reinforcing. The actual dynamics might not be "maverick researcher proves the establishment wrong to universal acclaim," but convictions leading to scientific theories that ultimately explain stuff badly aren't renowned for their social-Darwinistic fitness. Falsification doesn't have to be the whole story to be a useful concept.

Part of the problem seems to be that Fish and Kotsko go at least a bit off the anti-empirical deep end. This is especially evident in Kotsko's claim that "[t]heological claims are also falsifiable within any given theological community -- it's not as if people can just say any old thing and be accepted." [Emphasis added.] Since theological claims aren't empirical, it could be argued that he really means something other than "falsifiable." To swipe a thought from Robert Waldmann, theological "facts" may be derived in logically correct ways from theological axioms, but since those facts not only are non-empirical, but often claimed not to be subject to empirical validation (i.e., they constitute "articles of faith" independent of empiricism). This renders them something other than testable theories in the scientific usage.
Kotsko makes a valid point that it's wrong to treat theological dogma as immutable.

[D]ogma does change over time. If everything was unequivocally "set" for all time in some indisputable set of revealed propositions, then the history of Christianity, with its many controversies and many moments of genuine uncertainty as to which side would win, would literally make no sense at all.

But this, too, undermines another contention of Fish's:

[Fish:] Asking that religious faith consider itself falsified by empirical evidence is as foolish as asking that natural selection tremble before the assertion of deity and design. Falsification, if it occurs, always occurs from the inside.

At best, this depends on what you mean by "religious faith." Looking at a document such as the 1950 encyclical Humanae Generis (a Ground Zero for religion-science interactions), it's clear enough that a core of Catholic faith is put beyond the reach of empirical falsification. But it would seem to demand evidence that there isn't pressure on aspects of religious faith from emprical science. It seems beyond credulity that the processes by which many religions dropped (or diminished) tenets that the solar system is geocentric, that mental illnesses are not caused by demonic possession, or that the creation of the universe was according to the accounts in Genesis were generated "from the inside."

Ultimately, Fish warns that his own beliefs can't be inferred from his arguments and he may think the entire preceding argument is total bullshit and he won't say. ("Despite what some commentators assumed, I am not taking a position on the issues raised by the three books; readers of this and the previous column have learned nothing about my own religious views, or even if I have any.") So maybe the whole exercise has been an extended masturbation or devil's advocacy session; Fish isn't telling. My guess is left as an exercise for the reader.

Labels: Atheism, Evolution, Philosophy, Religion

by Tom Bozzo

George Akerlof invokes your discipline a few times in "The Missing Motivation in Macroeconomics", in the AER deposited in my mailbox today (contents page with viewing options), but adapted from the Presidential Address at this past January's AEA meetings. Based on joint work with Maryland's Rachel Kranton (who taught me some I-O back in the day), "norms" are the "missing motivation" in microfoundations-based macroeconomics (*).

I would imagine few sociologists would find this shocking. Critique of Economics Methodology 101 is that economists love themselves their H. economicus beyond reason (often true). An advanced (and also largely true) version is that some economics — especially microfoundations-based macro — doesn't so much describe how the economy works but rather how hypothetical economies comprised of fully rational actors work, and at least some economists fail to notice the difference. See, e.g., Wisconsin's Daniel Hausman for additional details.

Part of the problem is with the mathematical methods that are the bread-and-butter of "neoclassical" economics. The "classic" microfoundations-of-macro models were cleverly set up so that equilibria existed and could be characterized with the use of math that may look easy or difficult depending on your frame of reference. That gives way to extremely difficult or impossible math if, say, consumption is subject to social externalities — e.g., Jane Mangement Consultant drives a BMW in part because Bob Management Consultant drives a Mercedes. (This also has implications that right-leaning economists find politically unpalatable). There's a tendency to push off really problematic stuff (in the sense of not being susceptible to proving "theorems") to the "fuzzier" social sciences.

Update 4/18/07: Brayden King picks up the gauntlet here, at Orgtheory.net, showing why Orgtheory got its 'thinking blogger' award; see also the contributions from Brayden's co-bloggers in the comments thread.


(*) I.e., behavior of the economy as a whole derived from models of optimizing or rational agents' behavior.

Labels: Economics, Philosophy, Sociology