Jonah Lehrer has a fascinating piece in the New Yorker about the tendency for scientific results (i.e effect sizes) to decline over time. For example, Jonah talks about the decline in the therapeutic powers of anti-psychotic medicine. While initial studies showed a dramatic decrease in symptoms, a more recent study showed that the drugs therapeutic effect was less than half of what was first documented. In fact, it’s not just anti-psychotic medicine that sees a decline with repetition; it’s all of science.
This got me thinking back to all the great research I saw get presented at the annual conference for the Society for Judgement and Decision-Makings (SJDM), which was hosted last month in a very caliginous St. Louis. As I was sitting and listening to back-to-back presentations, I couldn’t help but notice how the researchers presented data that only confirmed their hypotheses. I know what your thinking; why would researchers go to an important conference to present a bunch of data to support the null hypothesis; after all, they are representing their academic institution in addition to getting noticed in the hopes of receiving future grants. I can’t say I blame them; published studies and citations are a form of currency.
But this got me thinking about our instinctive tendency to gather evidence that only confirms our beliefs. And how this bias even plagues social scientists. Take, for example, this simple experiment done with playing cards. Participants were given a set of four cards, each with a letter or a number on the side facing up-A, B, 2, 3- and told that each card had a letter on one side and a number on the other. The researchers then instructed the participants to determine, by turning over the proper card, whether “all cards with a vowel had an even number on the opposite side. The results were not surprising; most participants turned over only the A and 2 cards to find evidence consistent with the hypothesis. Ignoring the B and 3 cards, even though both cards were just as informative. Scientist think one of the reasons why we tend to shy away from disconfirming evidence is that its harder to deal with cognitively. In other words, its not efficient; non-confirming evidence is ambiguous and therefore we need to take additional steps to put this information to use. Same goes for negatively framed information; our brains have a harder time conceptualizing negative assertions. For example, which sentence makes you think twice “ All greeks are mortal,” or “All non-greeks are non-mortal.” I assume this additional cognitive step is one reason why social scientists naturally steer away from looking at opposing evidence. With little time and the sizable amount of information to sift through, an additional step is inefficient. But, as the saying goes, “the truth ain’t easy.”
Reference: Thomas Gilovich: How we Know What isn't So
Reference: Thomas Gilovich: How we Know What isn't So
