Marginal Utility

by Ken Houghton

Drek has invaded Scatterplot, starting with the claim that he can write posts "generally beneath the dignity of sociology."

In short, he's supposed to do for Scatterplot what I'm doing for AB.

I believe this only leaves Kim here. Or have I missed something with you as well?

Lego and family pictures should resume eventually.

Labels: blogging, Meta, Scatterplot, Social Science, Total Drek

by Ken Houghton

Note the progression here:

The essential story there was one of hard-science arrogance: Forrester, an eminent professor of engineering, decided to try his hand at economics, and basically said, “I’m going to do economics with equations! And run them on a computer! I’m sure those stupid economists have never thought of that!” And he didn’t walk over to the east side of campus to ask whether, in fact, any economists ever had thought of that, and what they had learned. (Economists tend to do the same thing to sociologists and political scientists. The general rule to remember is that if some discipline seems less developed than your own, it’s probably not because the researchers aren’t as smart as you are, it’s because the subject is harder.) [italic his; emphasis mine]

Labels: Economics, Social Science

by Ken Houghton

UPDATED: Jeremy spoke in class today, 14 Nov 2007

I want to reassure Jeremy (oops; make that Jeremy and Others)* that Economists don't Run Everything.

Nor, apparently, do they have exclusive rights to using the word "economic" in their papers. But, possibly to his regret, it isn't the sociologists who wrote this paper, either.

I want to see the expense reports.

(h/t Bad Science)

*Now we know why Tom is thinking of moving.

Labels: Social Psychology, Social Science

by Drek

During a recent fit of productivity I happened to read an article by Stan Kaplowitz and his colleagues* dealing with opinion change. Specifically, the article reports on a series of experiments that examine the impact of two factors on subject opinions: disconfirmation and discrepancy. For those who don't have a comprehensive library of social psychological terminology filed away in their brains** disconfirmation simply means "the opinion advanced by a second individual differs from what you would expect." So, for example, if a Republican politician advanced strong support of abortion rights, equal opportunity, and environmental protection, we would experience an amount of disconfirmation. Discrepancy, on the other hand, represents the size of the difference between one's own opinion and another person's. So, if I think taxes should be raised by 2% and my Republican father believes that they should be lowered by 2%, we are experiencing discrepancy.

Part of what is interesting here is that these two factors can vary independently of one another. So, if my Republican father also said that taxes should be raised by 2% I would experience disconfirmation, since this differs from his usual stance, but not discrepancy, since our positions are identical. Likewise, if the aforementioned Republican politician expressed a distaste for abortion rights, equal opportunity, or environmental protection I might experience discrepancy, since I don't agree with his stances, but not disconfirmation, since those are the positions I anticipate he holds. The question then becomes, how do disconfirmation and discrepancy mutually influence opinion change?

Well, as it happens, they both have interesting effects. Discrepancy, it appears, tends to produce opinion change. So, if I am exposed to arguments greatly at variance with my own, the size of the variance appears to have an impact on the size of my opinion change.*** Disconfirmation, likewise, has a substantial impact- a person who takes an opinion contrary to what you would expect has a greater influence on opinion change than one whose opinions fall into line with expectations. Particularly interesting, however, is why this effect appears to occur. To quote Kaplowitz et al:

Our results show that when the source takes an unexpected position, this position need not suggest that the source is unbiased. Rather, it may suggest that the case at hand merits a comparative evaluation which is extreme enough to overcome any bias on the part of the source.

So, in other words, if my Republican father agrees that a 2% tax hike is necessary, I'm more likely to assume that this is because the severity of the economic situation warrants the change than I am to conclude that he is more impartial than I previously thought. Why is this interesting? Well, for a simple reason: it suggests that the more unexpected a position an authority takes, the more likely individuals are to assume that the authority has a good reason for taking it. It is, effectively, a social psychological explanation for what is commonly known as the big lie doctrine. Domestically we may have seen this play out following September 11th, when the populace reacted to a government that was unexpectedly trampling due process by assuming that there must be a good reason for it. The genesis of Iraq War II may well have been similarly influenced. What we see is an effective reversal of Carl Sagan's argument that extraordinary claims require extraordinary evidence. Apparently, to many people, extraordinary and unexpected claims may imply that extraordinary evidence exists, whether they have personally seen it or not.

And all of this, oddly enough, makes me think of Michael Behe and his cohort Wild Bill. They are, respectively, a biochemist and a mathematician**** and both are advocates of Intelligent Design creationism. Consider, for a moment, how their presence at the forefront of the "movement" impacts those who are exposed to them. Most people probably expect those of high education, particularly Ph.D.s and especially Ph.D.s from technical fields like biochemistry and mathematics to be proponents of evolution. It is, after all, the perspective that seems to dominate in the academy***** and academics are often expected to be atheists or atheist sympathizers. If you don't believe me about that, read Conservapedia for a week or two. Yet, suddenly, here come Mickey and Wild Bill- two academics who claim that evolution is a crock and this new "intelligent design," a transparent disguise wrapped around creationism, is the truth. Whether or not there is any discrepancy in this argument, certainly a great amount of disconfirmation results. It is obvious why, therefore, Behe and Dembski have been so popular and effective as messengers for creationism.

Yet, what happens as time goes by? The longer Behe and Dembski keep up their tirades against evolution, the more we learn about their theological commitments, the weaker that disconfirmation becomes. Eventually, we expect what they have to say and are not surprised by it in the least. It appears, strangely enough, that ideologues like Dembski and Behe have a limited period of usefulness, almost as though there is a freshness date stamped upon their rumps. "Lousy arguments best if advanced by..." or something to that effect. In order to maintain its effectiveness, Intelligent Design creationism must therefore continually introduce new scientists and academics who are equally unlikely to support intelligent design and yet do so- something that ID has been manifestly unable to do. And as this failure drags on, we might well expect what little momentum ID has acquired to peter out.

It's all quite speculative but perhaps in this basic research on opinion change what we actually have is a partial explanation for the trajectory of ID, as well as a promise for the future. The next iteration of ID will be back just as soon as they dress it up in new clothes and find another pair of unexpected hacks to advance in into the public eye.

Or, then again, maybe I'm deliberately wasting your time. Your call.


* Kaplowitz, Stan, Edward L. Fink, James Mulcrone, David Atkin, & Saleh Dabil. (1991). "Disentangling the Effects of Discrepant and Disconfirming Information." Social Psychology Quarterly. 54(3). 191-207.

** e.g. Me.

*** Other scholars are invited to correct me if more recent literature suggests otherwise.

**** As well as a sort of amateur theologian, which may explain why his understanding of statistics is so often criticized. Not that theologians can't do math, but rather that Dembski has a lot of motivation to make the math say what he wants it to say.

***** Not to mention in evidence-based institutions everywhere.

Labels: Evolution, intelligent design, Social Psychology, Social Science, Sociology

by Drek

In a blog post a while back Brad Wright helpfully posted a list of cognitive biases common to human thought. For those who are not familiar, cognitive biases are shortcuts or pseudo-flaws* in human reasoning that can lead us to incorrect or unsupportable conclusions. Brad raises the question of how these biases affect discussions of religion, but I'm not really interested in that just now. Since all humans that we've checked (and we've checked a lot) appear to be susceptible to these kinds of biases, it's most likely the case that both the theist and the atheist are equally vulnerable. It may be that each group has its own "preferred" kind of bias, but that's not really an improvement.

I bring this up because of an excellent article that recently appeared in the Washington Post that deals with a particularly disturbing cognitive bias. This article reports on, among other things, some research performed by the Centers for Disease Control that came to a rather disquieting conclusion: it appears that efforts to contradict false information may actually end up reinforcing it. To quote from the article:

The federal Centers for Disease Control and Prevention recently issued a flier to combat myths about the flu vaccine. It recited various commonly held views and labeled them either "true" or "false." Among those identified as false were statements such as "The side effects are worse than the flu" and "Only older people need flu vaccine."

When University of Michigan social psychologist Norbert Schwarz had volunteers read the CDC flier, however, he found that within 30 minutes, older people misremembered 28 percent of the false statements as true. Three days later, they remembered 40 percent of the myths as factual.

Younger people did better at first, but three days later they made as many errors as older people did after 30 minutes. Most troubling was that people of all ages now felt that the source of their false beliefs was the respected CDC.

So, not only was the incorrect information retained, and not only was it retained as "accurate" knowledge, but it had somehow acquired the prestige of being supported by the Centers for Disease Control. Given that this was a study dealing with influenza, readers can be pardoned for not being too concerned, but what if this were instead dealing with information about HIV, tuberculosis, or anthrax? Would we feel as sanguine if citizens were coming to believe false information about those much more serious diseases? I suspect not. The unfortunate truth here is that this tendency for contradictions to reinforce that which they seek to discredit is a serious problem for our society. The medical implications are obvious- and may help to account for continued hysteria about vaccines- but the problems do not stop there.

We could talk about the political implications of this research. Indeed, the article itself does so, observing that this tendency to continue to believe discredited information, even to believe it more strongly, may account for a number of persistent myths surrounding the 9/11 attacks. For example:

This phenomenon may help explain why large numbers of Americans incorrectly think that Saddam Hussein was directly involved in planning the Sept 11, 2001, terrorist attacks, and that most of the Sept. 11 hijackers were Iraqi. While these beliefs likely arose because Bush administration officials have repeatedly tried to connect Iraq with Sept. 11, the experiments suggest that intelligence reports and other efforts to debunk this account may in fact help keep it alive.

Similarly, many in the Arab world are convinced that the destruction of the World Trade Center on Sept. 11 was not the work of Arab terrorists but was a controlled demolition; that 4,000 Jews working there had been warned to stay home that day; and that the Pentagon was struck by a missile rather than a plane.

...

A report last year by the Pew Global Attitudes Project, however, found that the number of Muslims worldwide who do not believe that Arabs carried out the Sept. 11 attacks is soaring -- to 59 percent of Turks and Egyptians, 65 percent of Indonesians, 53 percent of Jordanians, 41 percent of Pakistanis and even 56 percent of British Muslims.

In more general terms, this research may also help explain why the political right in the U.S. seems to so consistently kick the ass of the political left. With its reliance on soundbites and fireworks from the likes of Rush Limbaugh, Ann Coulter and Bill O'Reilly, the right is ideally structured to keep punching out assertions of often dubious accuracy. And when the left attempts to combat them, it may end up simply strengthening its opponents. Heads I win, tails you lose. Perhaps if you are on the right wing you won't find this idea disturbing but you should, if only because it implies that the only way to carry out politics is to reduce it to the level of a deranged shouting match.

I mean, we're more or less there already, but it would sure be nice if we could at least try to make use of reasoned debate and discussion.**

Finally, this research has some fairly significant implications for those of us who teach, and especially those of us who teach sociology. One of the greatest problems we face in sociology is in leading our students to question their own society. We have to guide students into accepting the idea that their own society is not the only way, and probably not the best way, of living. Often this involves contradicting things our students believe or helping them to see that their own beliefs are, themselves, contradictory.*** Unfortunately, this may not actually be the best way to go about it, and in demonstrating how a previous belief is incorrect, we may do little more than reinforce it in our students' minds. It is, perhaps, no surprise then that many adults look back on their sociology classes as having been silly, obvious, or a waste of time. With the hazy perspective of years, they have forgotten all the things sociology tried to teach them, and perhaps remember only those things we sought to contradict. Only now, they remember them as being true.

I'm not sure what is to be done about this. Remaining silent won't work as silence is often taken as tacit approval. Nevertheless, we can perhaps avoid some of the consequences of this cognitive bias by spending less time contradicting bad ideas, and more time arguing for the good ones. This may, of course, be less satisfying sometimes but in the final analysis, do we want to feel good, or do we want to be effective? I prefer to think we want to be effective. And, if nothing else, we should be sure to talk about these cognitive biases whenever it's appropriate. We're all vulnerable to the mistakes they lead us towards, and our only real defense is being aware that they exist.

It's never easy to defeat your own brain but, from time to time, it's the very best thing you can do.


* I say "pseudo-flaws" because these biases were probably very useful in our evolutionary environment where the idea wasn't to reach the best conclusion, but rather the one that was good enough to keep you alive. So, for example, given a choice between alpha error and beta error, alpha error is the one to make. It's better to think you see a predator that doesn't exist than to miss the one that does. If you want to think more about this, I have pondered the matter at least once before.

** Sorry, folks. Sometimes my zeal for democracy as envisioned by political philosophers leads me to say some pretty naive things.

*** My favorite example being that common sense tells us both that "Birds of a feather flock together" and that "Opposites attract." It's pretty easy for common sense to appear to be correct when it covers all of the bases like that.

Labels: cognitive biases, Health Care, Politics, Science, Social Science, Sociology, teaching

by Ken Houghton

As noted previously, I spent the post-July 4th weekend at Readercon in Burlington, MA. And while the panel discussing Karen Joy Fowler's work was adequate (as noted below, this is mainly due to Maureen of EOB fame, the other panelists, and the participation of Ms. Fowler herself), I can honestly say that the other panel I moderated, "See It Like Saruman: Reconciling Fantasy and Progress" was well worth hearing. And you may well get a chance, as it is one of two panels of which the convention team plans to post an audio recording.

The panel participants were Judith Berman, John Crowley, James Morrow, and Michael Swanwick. (The panel begins with my reading an excerpt from this essay (PDF), so it also included a de facto cameo by The Most Dangerous Perfesser.)

It was, in part, an economics panel, and—though I managed not to use the phrase "creative destruction"*—the sense of utility leading to choices, and the overriding theme of changing perspectives due to the Enlightenment, development, and the categorization of Karl Marx's work as "a Utopian Fantasy" probably makes it worth your attention from that perspective.

The Megan McArdles among H. economus will be horrified that we "didn't think everything through."** The rest of the world knows that Tolkien didn't either:

History is written by the winners. That explains why Tolkien never mentions that the destruction of Fangorn Forest and other efforts towards industrialization by Saruman significantly raised the standard of living for the wild men of Dunland, in fact creating (for the first time in Middle Earth) a comfortable middle class. While there is a natural opposition between the romantic and pastoral ideal embodied in traditional fantasy and the Enlightenment ideal of progress (especially in its modern industrial and technological modes), we don't believe they are completely incompatible. What works of fantasy have attempted to accommodate both? What interesting new direction might the heroic fantasy novel be taken if the true positive effects of modernization were acknowledged?

Anyone else wonder why the ten rules of Heterodoxy *** (see Figure 2 at bottom) are essential, especially when discussing Macroeconomic issues?****


*This is not really a Good Thing.
**English translation: share her obsessions. What is the price of eggs in Bolivia, again?
***Max presents a, er, more effusive version here.
****I personally don't consider it coincident that the Nation piece on Heterdoxy closes with George Akerlof's AEA keynote address, since it is Akerlof's discussion of the effect of informational asymmetries that is given short shrift in introductory Economics courses, despite having occurred a couple of generations ago.

Labels: Economic Development, Economics, heterodoxy, sf, Social Science

by Tom Bozzo

As a follow-up from the earlier post on the topic, in the Crooked Timber thread following Quiggin's post, commenter RB points to a letter to Nature's editor from 1994 by Johns Hopkins biostatistician Steven Goodman making the case that Gott's reasoning is an example of an old statistical fallacy. (Goodman posted it as a comment to Tierney's NYT blog.) Goodman's general thrust — 'lies, damn lies, statistics' — is correct, but he maybe goes a bit too far in deploying the f-word.

Simply put, the principle of indifference [i.e., the fallacy] says that it you know nothing about a specified number of possible outcomes, you can assign them equal probability. This is exactly what Dr. Gott does when he assigns a probability of 2.5% to each of the 40 segments of a hypothetical lifetime. There are many problems with this seductively simple logic. The most fundamental one is that, as Keynes said, this procedure creates knowledge (specific probability statements) out of complete ignorance.

Actually, there is a more charitable version than this, which is how I'd previously set up the problem, and how Monton and Kierland characterize Gott's original argument. In my account, the uniform distribution of the observation point is explicitly part of the (assumed) information set; I've packed my free lunch as it were. If that doesn't sound like much, it's not. However, I would submit that the more useful thing to argue over is whether the uniform distribution assumption is warranted. As it happens, I said before that the assumption is strong before, and what I mean is that in practice it seems unwarranted for the array of amusing social applications that Gott can't seem to resist.

But for a little more damnation by faint praise, let's just remember what's being promised by the method: a prediction within a factor of 39 of the start-to-present interval. As a practical matter, the real problem in many cases is not that too much fabricated information is being brought to bear, especially at the upper bound.

Labels: Philosophy, Science, Social Science, Statistics

by Tom Bozzo

Just going to show what happens when you drop off even the post-paywall NYT op-ed page, reaction to John Tierney's report that we have 46 years to colonize Mars Or Else Civilization is Dooooomed has been relatively muted over the Intertubes. Prof. Bainbridge quotes the Ole Perfesser without comment (see Roy at Alicublog for the omitted analysis) but also Charlie Stross's excellent post on the grim case for space colonization.

So how do you get that 46 years?

Suppose you're observing an Event that occurs during a fixed time interval (potentially a strong assumption). Suppose also that Baldrick is flying your space-time conveyance and drops you at a random point in the interval (potentially a very strong assumption). Suppose third that you know nothing else about the event. Your "best guess" as to where you've landed, in the expected value sense, is the midpoint of the interval. So if you then get your bearings and figure out how long ago the event started, which is all the information you have, your best guess is that the event will end the same amount of time in the future. That isn't a very good guess, though, in the sense that there's a 50% chance that the "true" end will be sooner or later than that.

Applied to the human spaceflight program, dated to 1961 (questionable [*]), then by advanced mathematics about 46 years have elapsed since then and the information you have and the assumptions above lead to the result. QED.

What the astrophysicist J. Richard Gott did, in a short paper, was to construct interval estimates with high confidence levels -- statements that the unknown end date for the event should fall between A and B 95 percent of the time. For the spaceflight case, A is 2008 (next year) and B is AD 3,801. But saying that you're 97.5 percent confident that the human spaceflight program will end in the next 1,800 years or so doesn't have the same sense of urgency. More generally, 95 percent confidence results in a range from 1/39th the age of the event on the low side to 39 times the age of the event on the high side. Call this the "Copernican formula" if you will. The proof methodology (see this paper [PDF], helpfully linked by Tierney) uses only undergraduate-level mathematical statistics, so read it yourself if you're so inclined.

This leads me to strongly endorse John Quiggin's conclusion:

The real lesson from Bayesian inference is that, with little or no sample data, even limited prior information will have a big influence on the posterior distribution. That is, if you are dealing with the kinds of cases Gott is talking about, you’re better off thinking about the problem than relying on an almost valueless statistical inference.

Indeed, if observing the passage of a year and nothing else, the upper bound of the interval moves out 39 years. That can be a big deal in many applied circumstances! For example, here's Gott himself writing in the New Scientist in 1997. A subhead of "Living proof" suggests he isn't engaged in deliberate leg-pulling as he recounts:

As another test, I used my formula on the day my "Nature" paper was published to predict the future longevities of the 44 Broadway and off-Broadway plays and musicals then running in New York; 36 have now closed - all in agreement with the predictions. The "Will Rogers Follies", which had been open for 757 days, closed after another 101 days, and the "Kiss of the Spider Woman", open for 24 days, closed after another 765 days. In each case the future longevity was within a factor of 39 of the past longevity, as predicted.

In this application, a prediction within a factor of 39 of past longevity conceivably covers the range from total flops to huge hits to productions that will eventually be performed by automata in Wisconsin Dells. The "prediction" for the "Will Rogers Follies" is that it will (likely) close within the next 82 years. That's out on a limb. (And certain philosophers inclined to bash social scientists for theories with weak predictive value might put this in their pipe and smoke it.)

Reinforcing Quiggins's point on how posterior distributions may be influenced, had Gott's paper appeared a week earlier, he'd have missed on "Kiss" to the tune of 100 days, since the previous week of running time adds some 9 months to the prediction's upper bound. If it matters whether the production runs another week or another year, searching for information is not unlikely to be rewarded.

Meanwhile, if you wanted to make some inference on whether both "Kiss" and "Will" would be playing at some future date, forget about it. The most interesting contribution comes from Brian Weatherson (at CT and Thoughts, Arguments, and Rants), who derives a neat result showing that if you infer the probability of both plays running at a future date based solely on the length of time they've run together (the information the method admits), it follows that if "Kiss" (the shorter-duration event) is still playing at that date, then "Will" (the longer-running event) will also be playing with probability 1. Weatherson concludes that there must be "something deeply mistaken with the Copernican formula."

My own little gloss, pending peer review in the self-correcting blogithingy, is here in the CT comments. What seems to be happening in this case is that (1) Gott's method throws away the information on how long "Will" has been running, and (2) sneaks in an additional assumption that the "Kiss" and "Will" events must be dependent or correlated. There may be circumstances under which these extremely strong assumptions may be justified, Weatherson maybe goes a bit too far in suggesting that these but they strike me as implying more than diffuse information on anything other than the elapsed times of the events.

Last, since you are by definition still with me here in the unlikely event you are reading this, here's the brief rant portion of the post: How the frack did Gott get 5 frackin' pages in Nature for this, which looks a lot more like it merits a paragraph of Mathematical News of the Weird?! I've been turned down cold — not even this 'reject and resubmit' stuff Drek writes about for stuff a hundred times harder and at least somewhat more relevant, if I don't say so myself. (If you really have time to kill, you may note that part of what I'm talking about eventually came out via other researchers' efforts as part of this IIASA working paper a few years later.) And if that's happened to me, then so too must everyone except Nick Bostrom, as I infer from Tom's Anti-Copernican Principle. W. T. F.

And BTW, Nature, what's up with US$30 for an e-print? Surely the revenue-maximizing price — which given the approximately $0 marginal cost, is also profit-maximizing — is not set at levels that make the likes of me think about sending junior staff to the library (were there a business case for actually obtaining the paper). Just saying.

[/rant]


[*] It's not like Yuri Gagarin's rocket just materialized on the pad and blasted off. And remember, going back even a few years into the preflight stages of human space programs puts a century or two on the upper bound.

Labels: Philosophy, Science, Social Science, Statistics

by Tom Bozzo

At Decision Science News (another h/t to Brad DeLong), Dan Goldstein prints a comment from J. Scott Armstrong who has "concluded that tests of statistical significance should never be used." [Emphasis mine.] He is not conducting statistical performance art, and I substantially agree with the conclusion. A couple random remarks:

• There are results which lead to a conclusion that social science researchers tend to tweak their statistical models to cross significance thresholds so they can produce positive results with (presumably) greater probability of publication. But,
• To do so invalidates the published inferences. Because,
• The "classical" statistics reported by most software packages are invalid under any pretesting (i.e., deciding on a model specification based on results from preliminary estimation). And,
• The prospects for computing or simulating correct statistics are as good as the quality of the researcher's choice trail. But,
• A lot of social science "theories" don't determine the full set of explanatory variables, making the lure of statistical model diagnostics attractive. Though,
• There are families of models (e.g., the 'flexible functional form' cost models in economics, which I work with) where individual coefficients have no theoretical interpretation, in which case the researcher has no direct basis for evaluating the consequences of a restriction. More broadly,
• Properties of social science data often mean we need to use consistent but inefficient estimators; sometimes "better" significance from inappropriate estimation methods has little if any meaning. (*) Last,
• Some researchers (not least many who publish empirical results in top economics journals) tend to focus excessively on statistical significance to the detriment of more interesting discussions of the non-statistical significance of their results. (Views differ.)
  

Armstrong's reasonable recommendations are:

Authors... instead... should report on effect sizes, confidence intervals, replications/extensions, and meta-analyses.

For those of you with institutional access, links to the International Journal of Forecasting article are at the Decision Science News link.

(Cross-posted at Total Drek.)


(*) This sometimes leads to wacky advice being given to everyday applied researchers from econo- or sociometricians, of the "if a result from an inconsistent esitmator goes away with a consistent (but inefficient) procedure, be suspicious [or vice-versa]." Armstrong's bottom-line recommendations address the reasonable suspicions that might arise.

Labels: Econometrics, Economics, Social Science, Statistics