Monday, June 26, 2006

Confirmation bias


Scientific American carries a wonderful article by Michael Shermer on confirmation bias, which we all suffer from and which makes us "seek and find confirmatory evidence in support of already existing beliefs and ignore or reinterpret disconfirmatory evidence." The article starts with this quote from Francis Bacon:

The human understanding when it has once adopted an opinion ... draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises ... in order that by this great and pernicious predetermination the authority of its former conclusions may remain inviolate.

The article recounts the results of recent brain imaging studies, and their implications. You can get all the details there, but here's a section where Shermer talks about the 'self-correcting' machinery used in science that allows us -- biased mortals -- to avoid pitfalls such as confirmation bias:

In science we have built-in self-correcting machinery. Strict double-blind controls are required in experiments, in which neither the subjects nor the experimenters know the experimental conditions during the data-collection phase. Results are vetted at professional conferences and in peer-reviewed journals. Research must be replicated in other laboratories unaffiliated with the original researcher. Disconfirmatory evidence, as well as contradictory interpretations of the data, must be included in the paper. Colleagues are rewarded for being skeptical. Extraordinary claims require extraordinary evidence.

1 Comments:

  1. Anonymous said...

    I commented earlier and if both come through, please delete one. I think that there is much truth in it and it is some thing one should be constantly aware of. Moreover, the sort of opinions that one forms, I think, depend on one's personality and may be testable. See, for example,
    http://www.physorg.com/news69951399.html
    for one type of classification. I think that group thinking may to some extent help. Of course, forming a group itself involves some similarity in thinking and cohesiveness. However, initially one can formulate and discuss important isues and then slowly explore through sites for more diverse opinions, information and discussion. For example, though I initially thought that over 50 years of reservations may be enough, the discussions at the otherindia site and particularly Weisskoff's artcle and the CASI study which came in the middle of the discussion, changed my view. More fine tuning is needed. I assume that this how some of the discussion groups operate. If one is not prepared to change or modify one's views, there is no point in participating in discussions.