Posts Tagged ‘experiments’
“The spirit of inquiry and the courage to challenge the status quo are at the heart of scientific progress”*…
Adam Mastroianni on the challenges– and opportunities– facing science…
Randomized-controlled trials only caught on about 80 years ago, and whenever I think about that, I have to sit down and catch my breath for a while. The thing everybody agrees is the “gold standard” of evidence, the thing the FDA requires before it will legally allow you to sell a drug—that thing is younger than my grandparents.
There are a few records of things that kind of look like randomized-controlled trials throughout history, but people didn’t really appreciate the importance of RCTs until 1948, when the British Medical Research Council published a trial on streptomycin for tuberculosis. Humans have possessed the methods of randomization for thousands of years—dice, coins, the casting of lots—and we’ve been trying to cure diseases for as long as we’ve been human. Why did it take us so long to put them together?
I think the answer is: first, we had to stop trusting Zeus.
To us, coin flips are random (“Heads: I go first. Tails: you go first.”). But to an ancient human, coin flips aren’t random at all—they reveal the will of the gods (“Heads: Zeus wants me to go first. Tails: Zeus wants you to go first”). In the Bible, for instance, people are always casting lots to figure out what God wants them to do: which goat to kill, who should get each tract of land, when to start a genocide, etc.
This is, of course, a big problem for running RCTs. If you think that the outcome of a coin flip is meaningful rather than meaningless, you can’t use it to produce two equivalent groups, and you can’t study the impact of doing something to one group and not the other. You can only run a ZCT—a Zeus controlled trial.
It’s easy to see how technology can lead to scientific discoveries. Make microscope -> discover mitochondria.
Clearly, though, sometimes those technologies get invented entirely inside our heads. Stop trusting Zeus -> develop RCTs.
Which raises the question: what mental technologies haven’t we invented yet? What brain switches are just waiting to be flipped?…
On reinvigorating science: “Declining trust in Zeus is a technology,” from @a_m_mastroianni.
Apposite to an issue he raises: “Citation cartels help some mathematicians—and their universities—climb the rankings,” from @ScienceMagazine.
[Image above: source]
###
As we deliberate on discovery, we might send micro-biological birthday greetings to a woman who modeled the attitude and behavior that Mastroianni suggests: Ruth Sager; she was born on this date in 1918. A pioneering geneticist, she had, in effect, two careers.
In the 1950s and 1960s, she pioneered the field of cytoplasmic genetics by discovering transmission of genetic traits through chloroplast DNA, the first known example of genetics not involving the cell nucleus. She identified a second set of genes were found outside of the cell’s nucleus, which, even though they were nonchrosomomal, also influenced inherited characteristics. The academic community did not acknowledge the significance of her contribution until after the second wave of feminism in the 1970s.
Then, in the early 1970s, she moved into cancer genetics (with a specific focus on breast cancer); she proposed and investigated the roles of tumor suppressor genes. She identified over 100 potential tumor suppressor genes, developed cell culture methods to study normal and cancerous human and other mammalian cells in the laboratory, and pioneered the research into “expression genetics,” the study of altered gene expression.
“Mistakes are, after all, the foundations of truth, and if a man does not know what a thing is, it is at least an increase in knowledge if he knows what it is not”*…
Controversy is essential to scientific progress. As Richard Feynman said, “science is the belief in the ignorance of experts.” Nothing is taken on faith, all assumptions are open to further scrutiny. It’s a healthy sign therefore that psychology studies continue to generate great controversy. Often the heat is created by arguments about the logic or ethics of the methods, other times it’s because of disagreements about the implications of the findings to our understanding of human nature. Here we digest ten of the most controversial studies in psychology’s history…
From “the Stanford Prison Experiment” and “the Milgram ‘Shock Experiments'” to “Voodoo correlations in social neuroscience” and “Libet’s Challenge to Free Will”– The British Psychological Society‘s “The 10 most controversial psychology studies ever published.”
(Lest one wonder whether all of this has any purchase in the real world, this review of Hooked…)
* Carl Jung
###
As we brace ourselves on our lab benches, we might spare a thought for Gustav Theodor Fechner; he died on this date in 1887. A philosopher and physicist with a keen interest in human behavior, Fechner is recognized (with Wilhelm Wundt and Hermann von Helmholtz) as the founder of experimental psychology. He formulated the rule known as Fechner’s law–that, within limits, the intensity of a sensation increases as the logarithm of the stimulus– a result indicative of his approach to studying the relationships between physical stimuli and the sensations and perceptions they cause. His approach, which came to be known as Psychophysics, has been influential ever since.
Amaze your friends!…
From the extraordinary resource that is The Public Domain Review, a compendium of do-it-yourself diversions from 1820– all “so clearly explained, as to be within the reach of the most limited capacity.”
Page through Endless Amusement for more things that it was apparently OK to try at home back then.
###
As we count our fingers to be sure that they’re all still there, we might recall that it was on this date in 1876 that Alexander Graham Bell first spoke through his experimental “telephone”– to his assistant, Thomas A. Watson, in the next room. Bell wrote in his notebook, “I then shouted into M [the mouthpiece] the following sentence: ‘Mr. Watson–come here–I want to see you.’ To my delight he came and declared that he had heard and understood what I said.”
Bell’s lab notebook, March 10, 1876
Playing the odds…
A P value is the probability of an observed (or more extreme) result arising only from chance.
It’s science’s dirtiest secret: The “scientific method” of testing hypotheses by statistical analysis stands on a flimsy foundation. Statistical tests are supposed to guide scientists in judging whether an experimental result reflects some real effect or is merely a random fluke, but the standard methods mix mutually inconsistent philosophies and offer no meaningful basis for making such decisions. Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous, and tests of medical dangers or treatments are often contradictory and confusing.
Replicating a result helps establish its validity more securely, but the common tactic of combining numerous studies into one analysis, while sound in principle, is seldom conducted properly in practice.
Experts in the math of probability and statistics are well aware of these problems and have for decades expressed concern about them in major journals. Over the years, hundreds of published papers have warned that science’s love affair with statistics has spawned countless illegitimate findings. In fact, if you believe what you read in the scientific literature, you shouldn’t believe what you read in the scientific literature.
“There is increasing concern,” declared epidemiologist John Ioannidis in a highly cited 2005 paper in PLoS Medicine, “that in modern research, false findings may be the majority or even the vast majority of published research claims.”
Ioannidis claimed to prove that more than half of published findings are false, but his analysis came under fire for statistical shortcomings of its own. “It may be true, but he didn’t prove it,” says biostatistician Steven Goodman of the Johns Hopkins University School of Public Health. On the other hand, says Goodman, the basic message stands. “There are more false claims made in the medical literature than anybody appreciates,” he says. “There’s no question about that.”
Nobody contends that all of science is wrong, or that it hasn’t compiled an impressive array of truths about the natural world. Still, any single scientific study alone is quite likely to be incorrect, thanks largely to the fact that the standard statistical system for drawing conclusions is, in essence, illogical. “A lot of scientists don’t understand statistics,” says Goodman. “And they don’t understand statistics because the statistics don’t make sense”…
What’s one to make of the stream of “eat this,” “avoid that” studies surfacing nearly daily? It’s an odds-on bet that readers will find out in the complete Science News story, “Odds Are, It’s Wrong.”
As we tell Monty that we’ll take what’s behind Door #2, we might recall that it was on this date in 1905 that Albert Einstein kicked off “Annus Mirabilis” with the publication of the first of his four epoch-making papers in Annalen der Physik— this one, proposing energy “quanta”– thus kicking off the year in which he reinvented physics and our understanding of reality.
The second of those papers, on Brownian motion, was the very first work of “statistical physics.”
Einstein, dressed for the patent office, 1905
Happy Náw-Rúz! This date in 1844 was the first day of the first year of the Bahai calendar.
You must be logged in to post a comment.