When Science Is a Siren Song

When a group of British academic researchers reported last spring that women fond of eating breakfast cereal were more likely to give birth to boys, the story was lapped up by journalists the world over. "Skip breakfast for a daughter, eat up your cereals for a son," advised The Economist, just one of many publications to seize on the report.

The problem with this fascinating study? It appears to be wrong. An analysis led by Stan Young of the National Institute for Statistical Sciences found that the original conclusion was based on poor statistics and is probably the result of chance.

So far, Young's rebuttal, published in January, has received little notice. That it is ignored by many of the media outlets that lavished attention on the original report isn't surprising; in fact, the most remarkable thing is how ordinary that lack of attention may be. A lot of science, it turns out, can't withstand serious scrutiny.

Thoughtful analysis by John Ioannidis suggests that more than half of published scientific research findings can't be replicated by other researchers.

Part of the problem is that we've been conditioned to trust university research. It is based, after all, on the presumably lofty motives of its practitioners. What's not to like about science carried out by academics who have nobly dedicated their lives to understanding the unknown, furthering knowledge and serving humanity?

Within academia's ivied walls the view is a bit different. The university is not a peaceable kingdom, and life is far more Hobbesian. Henry Kissinger was on to something when he observed that "university politics are so vicious precisely because the stakes are so small." In contrast to the academia-vs.-industry trope, hubris, self-interest and ambition are not checked at the university door; arguably, they are essential for admission and required for professional success.

University researchers are in a constant battle for recognition and the rewards associated with success: research space, speaking engagements, funding and autonomy. Consequently, while academic research is often described as "curiosity-driven," the reality is messier, as (curiously) many researchers tend to pursue the trendiest technologies and explore topics that happen to be associated with the most generous levels of research support.

Moreover, since academic success is determined almost exclusively by the number and prestige of research publications, the incentives to generate results are exceedingly powerful and can encourage investigators to see patterns that may not exist, to disregard contradictory observations that might be important, to overvalue data that might be preliminary or unreliable, and to embrace conclusions that deserve to be viewed with far greater skepticism