Posts Tagged ‘statistics’
“Gentlemen, you need to add armor-plate where the holes aren’t, because that’s where the holes were on the airplanes that didn’t return”*…
Allied bombers were key to Britain’s air offensive against Germany during the second world war. As such, the RAF wanted to armour their bombers to prevent them from being shot down. But armour is heavy – you cannot reinforce an entire bomber and still have it fly. So statistician Abraham Wald was asked to advise on where armour should be placed on a bomber.
After each wave of bombing, every returning aircraft was meticulously examined and a note was made of where each aircraft had sustained damage by the Germans. The image [above] conceptualises what Wald’s data might have looked like visually.
So what was Wald’s advice? Where should armour be added?
He essentially advised the RAF to add armour to places where you do not find bullet holes. Wait… what?!
Wald wisely understood that the data was based only on planes that survived. The planes that did not survive were likely to have sustained damage on the areas where we do not observe bullet holes – such as around the engine or cockpit…
Making better decisions: one of the most prevalent– and insidious– forms of selection bias, survivorship bias, illustrated: “How to armour a WWII bomber.”
See also: “How to avoid being duped by survivorship bias.”
###
As we think clearly, we might send productive birthday greetings to W. Edwards Deming; he was born on this date in 1900. An engineer, statistician, professor, author, lecturer, and management consultant, he helped develop the sampling techniques still used by the U.S. Department of the Census and the Bureau of Labor Statistics.
But he is better remembered as the champion of statistically-based production management techniques that first gained traction in post-WWII Japan, where many credit Deming as a key ingredient in what has become known as the Japanese post-war economic miracle of 1950 to 1960, when Japan rose from the ashes of war onto the its path to becoming the second-largest economy in the world– through processes shaped by the ideas Deming taught. In 1951, the Japanese government established the Deming Prize in his honor.
While his impact in Japan (finally) brought him to the attention of business leaders in the U.S., he was only just beginning to win widespread recognition in the U.S. at the time of his death in 1993.
“Oh, I am fortune’s fool!”*…
For many years, my life centered around studying the biases of human decision-making: I was a graduate student in psychology at Columbia, working with that marshmallow-tinted legend, Walter Mischel, to document the foibles of the human mind as people found themselves in situations where risk abounded and uncertainty ran high. Dissertation defended, I thought to myself, that’s that. I’ve got those sorted out. And in the years that followed, I would pride myself on knowing so much about the tools of self-control that would help me distinguish myself from my poor experimental subjects. Placed in a stochastic environment, faced with stress and pressure, I knew how I’d go wrong — and I knew precisely what to do when that happened.
Fast-forward to 2016. I have embarked on my latest book project, which has taken me into foreign territory: the world of No Limit Texas Hold ’em… The biases I know all about in theory, it turns out, are much tougher to fight in practice…
Maria Konnikova. a New York Times bestselling author and contributor to The New Yorker with a doctorate in psychology, decided to learn how to play poker to better understand the role of luck in our lives, examining the game through the lens of psychology and human behavior. An excerpt is adapted from her new book, The Biggest Bluff: How I Learned to Pay Attention, Master Myself, and Win: “The Hard Truth Of Poker — And Life: You’re Never ‘Due’ For Good Cards.”
* Shakespeare, Romeo and Juliet
###
As we ante up, we might spare a thought for Don Featherstone; he died on this date in 2015. An artist, he is surely best remembered for his creation of the plastic pink flamingo lawn ornament in 1957, while working for Union Products. It went on sale the following year– and now adorns lawns nationwide.
In 1996, Featherstone was awarded the 1996 Ig Nobel Art Prize for his creation; that same year, he began his tenure as president of Union Products, a position he held until he retired in 2000.
A Featherstone flock
“There are three types of lies — lies, damn lies, and statistics”*…
“Hiding in Plain Sight”
A chart’s purpose is usually to help you properly interpret data. But sometimes, it does just the opposite. In the right (or wrong) hands, bar graphs and pie charts can become powerful agents of deception, tricking you into inferring trends that don’t exist, mistaking less for more, and missing alarming facts. The best measure of a chart’s honesty is the amount of time it takes to interpret it, says Massachusetts Institute of Technology perceptual scientist Ruth Rosenholtz: “A bad chart requires more cognitive processes and more reasoning about what you’ve seen.”…
Five examples (like the one above) of the kinds of tricks that charts can try to pull, explained: “Five Ways to Lie with Charts.”
* Benjamin Disraeli
###
As we stack the deck, we might recall that it was on this date in 2010, at 2:32p EDT, that the U.S. stock markets suffered a “Flash Crash”– in a period of just 36 minutes, the S&P 500, Dow Jones Industrial Average, and Nasdaq Composite collapsed and rebounded (the Dow, e.g., lost 9% of its value, then recovered most of it).
Nearly five years later, the SEC charged a 36-year-old small-time trader who worked from his parents’ modest stucco house in suburban west London with having caused the collapse (using spoofing and layering, along with a form of front-running– all now explicitly outlawed). But many experts are not convinced; to this day, there are numerous theories– but no consensus– as to the cause(s) of the crash.
The DJIA on May 6, 2010 (11:00 AM – 4:00 PM EDT)
“Life is pleasant. Death is peaceful. It’s the transition that’s troublesome.”*…
Cause of death has changed over the years. In 1999, the suicide rate among 25- to 34-year-olds was 12.7 per 100,000 people. By 2016, that rate was almost 30 percent higher at 16.5.
These shifts over time are common and vary across sex and age groups.
With the release of the annual health report by the Centers for Disease Control and Prevention, I looked at the subcategories of mortality, as defined by the World Health Organization, focusing specifically on how the ten most common ways to die have changed over the years…
See (a full-sized and working version of) Nathan Yau’s animation of the changing causes of death, by sex and age group, in the U.S. from 1999 to 2016: “Shifting Causes of Death.”
* Isaac Asimov
###
As we memento mori, we might spare a thoughts for Gertrude Mary Cox; she died on this date in 1978. A pioneering statistician best known for her important work on experimental design, she founded the department of Experimental Statistics at North Carolina State University and later served as director of both the Institute of Statistics of the Consolidated University of North Carolina and the Statistics Research Division of North Carolina State University. In 1949 Cox became the first female elected into the International Statistical Institute and in 1956 was President of the American Statistical Association.
“Mathematics, rightly viewed, possesses not only truth, but supreme beauty”*…
Maryam Mirzakhani did not enjoy mathematics to begin with. She dreamed of being an author or politician, but as a top student at her all-girls school in Tehran she was still disappointed when her first-year maths exam went poorly. Her teacher believed her – wrongly – to have no particular affinity with the subject.
Soon that would all change. “My first memory of mathematics is probably the time [my brother] told me about the problem of adding numbers from 1 to 100,” she recalled later. This was the story of Carl Gauss, the 18th-century genius whose schoolteacher set him this problem as a timewasting exercise – only for his precocious pupil to calculate the answer in a matter of seconds.
The obvious solution is simple but slow: 1+2+3+4. Gauss’s solution is quicker to execute, and far more cunning. It goes like this: divide the numbers into two groups: from 1 to 50, and from 51 to 100. Then, add them together in pairs, starting with the lowest (1) and the highest (100), and working inwards (2+99, 3+98, and so on). There are 50 pairs; the sum of each pair is 101; the answer is 5050. “That was the first time I enjoyed a beautiful solution,” Mirzakhani told the Clay Mathematics Institute in 2008.
Since then, her appreciation for beautiful solutions has taken her a long way from Farzanegan middle school. At 17 she won her first gold medal at the International Mathematics Olympiad. At 27 she earned a doctorate from Harvard University. The Blumenthal Award and Satter Prize followed, and in 2014 she became the first woman to be awarded the Fields Medal, the highest honour a mathematician can obtain.
Before this particular brand of wonder became perceptible to Mirzakhani, she experienced feelings many of us can relate to: to the indifferent, her subject can seem “cold”, even “pointless”. Yet those who persist will be rewarded with glimpses of conceptual glory, as if gifted upon them by a capricious god: “The beauty of mathematics,” she warned, “only shows itself to more patient followers.”
This concept of “beauty” found in maths has been referred to over centuries by many others; though, like beauty itself, it is notoriously difficult to define…
For an experienced mathematician, the greatest equations are beautiful as well as useful. Can the rest of us see what they see? “What makes maths beautiful?”
[From The New Humanist, via the ever-illuminating 3 Quarks Daily]
Maryam Mirzakhani died last Friday, a victim of breast cancer; she was 40. As Peter Sarnak (a mathematician at Princeton University and the Institute for Advanced Study) said, her passing is “a big loss and shock to the mathematical community worldwide.” See also here.
* Bertrand Russell, A History of Western Philosophy
###
As we accede to awe, we might spare a thought for Andrey (Andrei) Andreyevich Markov; he died on this date in 1922. A Russian mathematician, he helped to develop the theory of stochastic processes, especially those now called Markov chains: sequences of random variables in which the future variable is determined by the present variable but is independent of the way in which the present state arose from its predecessors. (For example, the probability of winning at the game of Monopoly can be determined using Markov chains.) His work on the study of the probability of mutually-dependent events has been developed and widely applied to the biological and social sciences.
