(Roughly) Daily

We’ve looked before at the methodological problems that beset (too) much science, and at the work of John Ioannidis, who’s done more than anyone else to uncover them (see here and here).  Ioannidis is back…  and the news is troubling:

Over the past decade, scientists have increasingly become ashamed at the failings of their own profession: due to a lack of self-policing and quality control, a large proportion of studies have not been replicable, scientific frauds have flourished for years without being caught, and the pressure to publish novel findings—instead of simply good science—has become the commanding mantra. In what might be one of the worst such failings, a new study suggests that even systematic reviews and meta-analyses—typically considered the highest form of scientific evidence—are now in doubt.

The study comes from a single author: John Ioannidis, a highly respected researcher at Stanford University, who has built his reputation showing other scientists what they get wrong. In his latest work, Ioannidis contends that “the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, or conflicted.”

 Systematic reviews and meta-analyses are statistically rigorous studies that synthesize the scientific literature on a given topic. If you aren’t a scientist or a policymaker, you may have never heard of them. But you have almost certainly been affected by them.

If you’ve ever taken a medicine for any ailment, you’ve likely been given the prescription based on systematic reviews of evidence for that medication. If you’ve ever been advised to use a standing desk to improve your health, it’s because experts used meta-analyses of past studies to make that recommendation. And government policies increasingly rely on conclusions stemming from evidence found in such reviews. “We put a lot of weight and trust on them to understand what we know and how to make decisions,” Ioannidis says…

More at “The man who made scientists question themselves has just exposed huge flaws in evidence used to give drug prescriptions.” See also “The Inevitable Evolution of Bad Science” and “Trouble at the Lab.”

And lest we think “hard scientists” alone in their misery, consider the plight of economists: “The Emperor’s New Paunch.”

*Daniel J. Boorstin

###

As we check, check, and check again, we might send disingenuous birthday greetings to Trofim Denisovich Lysenko; he was born on this date in 1898.  A Soviet biologist and agronomist, he believed the Mendelian theory of heredity to be wrong, and developed his own, allowing for “soft inheritance”– the heretability of learned behavior. (He believed that in one generation of a hybridized crop, the desired individual could be selected and mated again and continue to produce the same desired product, without worrying about separation/segregation in future breeds.–he assumed that after a lifetime of developing (acquiring) the best set of traits to survive, those must be passed down to the next generation.)

In many way Lysenko’s theories recall Lamarck’s “organic evolution” and its concept of “soft evolution” (the passage of learned traits), though Lysenko denied any connection. He followed I. V. Michurin’s fanciful idea that plants could be forced to adapt to any environmental conditions, for example converting summer wheat to winter wheat by storing the seeds in ice.  With Stalin’s support for two decades, he actively obstructed the course of Soviet biology and caused the imprisonment and death of many of the country’s eminent biologists who disagreed with him.

Interestingly, some current research suggests that heretable learning– or a semblance of it– may in fact be happening, by virtue of epigenetics… though nothing vaguely resembling Lysenko’s theory.

 source