Posts Tagged ‘Ioannidis’
“Nature is objective, and nature is knowable, but we can only view her through a glass darkly–and many clouds upon our vision are of our own making”*…
Sometimes it seems surprising that science functions at all. In 2005, medical science was shaken by a paper with the provocative title “Why most published research findings are false.” Written by John Ioannidis, a professor of medicine at Stanford University [see here and here], it didn’t actually show that any particular result was wrong. Instead, it showed that the statistics of reported positive findings was not consistent with how often one should expect to find them. As Ioannidis concluded more recently, “many published research findings are false or exaggerated, and an estimated 85 percent of research resources are wasted.”
It’s likely that some researchers are consciously cherry-picking data to get their work published. And some of the problems surely lie with journal publication policies. But the problems of false findings often begin with researchers unwittingly fooling themselves: they fall prey to cognitive biases, common modes of thinking that lure us toward wrong but convenient or attractive conclusions. “Seeing the reproducibility rates in psychology and other empirical science, we can safely say that something is not working out the way it should,” says Susann Fiedler, a behavioral economist at the Max Planck Institute for Research on Collective Goods in Bonn, Germany. “Cognitive biases might be one reason for that.”
Psychologist Brian Nosek of the University of Virginia says that the most common and problematic bias in science is “motivated reasoning”: We interpret observations to fit a particular idea…
How the co-founder of the Open Science Framework and the Center for Open Science is tackling human biases in science: “The Trouble With Scientists.”
* “Nature is objective, and nature is knowable, but we can only view her through a glass darkly–and many clouds upon our vision are of our own making: social and cultural biases, psychological preferences, and mental limitations (in universal modes of thought, not just individualized stupidity)” – Stephen Jay Gould
###
As we share and share alike, we might spare a thought for William “Willy” A. Higinbotham; he died on this date in 1994. A physicist who was a member of the team that developed the first atomic bomb, he later became a leader in the nuclear non-proliferation movement.
But Higinbotham may be better remembered as the creator of Tennis for Two, the first interactive analog computer game and one of the first electronic games to use a graphical display, which he built for the 1958 visitor day at Brookhaven National Laboratory. It used a small analogue computer with ten direct-connected operational amplifiers and output a side view of the curved flight of the tennis ball on an oscilloscope only five inches in diameter. Each player had a control knob and a button.
The 1958 Tennis for Two exhibit
“The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge”*…
We’ve looked before at the methodological problems that beset (too) much science, and at the work of John Ioannidis, who’s done more than anyone else to uncover them (see here and here). Ioannidis is back… and the news is troubling:
Over the past decade, scientists have increasingly become ashamed at the failings of their own profession: due to a lack of self-policing and quality control, a large proportion of studies have not been replicable, scientific frauds have flourished for years without being caught, and the pressure to publish novel findings—instead of simply good science—has become the commanding mantra. In what might be one of the worst such failings, a new study suggests that even systematic reviews and meta-analyses—typically considered the highest form of scientific evidence—are now in doubt.
The study comes from a single author: John Ioannidis, a highly respected researcher at Stanford University, who has built his reputation showing other scientists what they get wrong. In his latest work, Ioannidis contends that “the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, or conflicted.”
Systematic reviews and meta-analyses are statistically rigorous studies that synthesize the scientific literature on a given topic. If you aren’t a scientist or a policymaker, you may have never heard of them. But you have almost certainly been affected by them.If you’ve ever taken a medicine for any ailment, you’ve likely been given the prescription based on systematic reviews of evidence for that medication. If you’ve ever been advised to use a standing desk to improve your health, it’s because experts used meta-analyses of past studies to make that recommendation. And government policies increasingly rely on conclusions stemming from evidence found in such reviews. “We put a lot of weight and trust on them to understand what we know and how to make decisions,” Ioannidis says…
More at “The man who made scientists question themselves has just exposed huge flaws in evidence used to give drug prescriptions.” See also “The Inevitable Evolution of Bad Science” and “Trouble at the Lab.”
And lest we think “hard scientists” alone in their misery, consider the plight of economists: “The Emperor’s New Paunch.”
*Daniel J. Boorstin
###
As we check, check, and check again, we might send disingenuous birthday greetings to Trofim Denisovich Lysenko; he was born on this date in 1898. A Soviet biologist and agronomist, he believed the Mendelian theory of heredity to be wrong, and developed his own, allowing for “soft inheritance”– the heretability of learned behavior. (He believed that in one generation of a hybridized crop, the desired individual could be selected and mated again and continue to produce the same desired product, without worrying about separation/segregation in future breeds.–he assumed that after a lifetime of developing (acquiring) the best set of traits to survive, those must be passed down to the next generation.)
In many way Lysenko’s theories recall Lamarck’s “organic evolution” and its concept of “soft evolution” (the passage of learned traits), though Lysenko denied any connection. He followed I. V. Michurin’s fanciful idea that plants could be forced to adapt to any environmental conditions, for example converting summer wheat to winter wheat by storing the seeds in ice. With Stalin’s support for two decades, he actively obstructed the course of Soviet biology and caused the imprisonment and death of many of the country’s eminent biologists who disagreed with him.
Interestingly, some current research suggests that heretable learning– or a semblance of it– may in fact be happening, by virtue of epigenetics… though nothing vaguely resembling Lysenko’s theory.
You must be logged in to post a comment.