Posts Tagged ‘statistics’
“There are three types of lies — lies, damn lies, and statistics”*…
“Hiding in Plain Sight”
A chart’s purpose is usually to help you properly interpret data. But sometimes, it does just the opposite. In the right (or wrong) hands, bar graphs and pie charts can become powerful agents of deception, tricking you into inferring trends that don’t exist, mistaking less for more, and missing alarming facts. The best measure of a chart’s honesty is the amount of time it takes to interpret it, says Massachusetts Institute of Technology perceptual scientist Ruth Rosenholtz: “A bad chart requires more cognitive processes and more reasoning about what you’ve seen.”…
Five examples (like the one above) of the kinds of tricks that charts can try to pull, explained: “Five Ways to Lie with Charts.”
* Benjamin Disraeli
###
As we stack the deck, we might recall that it was on this date in 2010, at 2:32p EDT, that the U.S. stock markets suffered a “Flash Crash”– in a period of just 36 minutes, the S&P 500, Dow Jones Industrial Average, and Nasdaq Composite collapsed and rebounded (the Dow, e.g., lost 9% of its value, then recovered most of it).
Nearly five years later, the SEC charged a 36-year-old small-time trader who worked from his parents’ modest stucco house in suburban west London with having caused the collapse (using spoofing and layering, along with a form of front-running– all now explicitly outlawed). But many experts are not convinced; to this day, there are numerous theories– but no consensus– as to the cause(s) of the crash.
The DJIA on May 6, 2010 (11:00 AM – 4:00 PM EDT)
“Life is pleasant. Death is peaceful. It’s the transition that’s troublesome.”*…
Cause of death has changed over the years. In 1999, the suicide rate among 25- to 34-year-olds was 12.7 per 100,000 people. By 2016, that rate was almost 30 percent higher at 16.5.
These shifts over time are common and vary across sex and age groups.
With the release of the annual health report by the Centers for Disease Control and Prevention, I looked at the subcategories of mortality, as defined by the World Health Organization, focusing specifically on how the ten most common ways to die have changed over the years…
See (a full-sized and working version of) Nathan Yau’s animation of the changing causes of death, by sex and age group, in the U.S. from 1999 to 2016: “Shifting Causes of Death.”
* Isaac Asimov
###
As we memento mori, we might spare a thoughts for Gertrude Mary Cox; she died on this date in 1978. A pioneering statistician best known for her important work on experimental design, she founded the department of Experimental Statistics at North Carolina State University and later served as director of both the Institute of Statistics of the Consolidated University of North Carolina and the Statistics Research Division of North Carolina State University. In 1949 Cox became the first female elected into the International Statistical Institute and in 1956 was President of the American Statistical Association.
“Mathematics, rightly viewed, possesses not only truth, but supreme beauty”*…
Maryam Mirzakhani did not enjoy mathematics to begin with. She dreamed of being an author or politician, but as a top student at her all-girls school in Tehran she was still disappointed when her first-year maths exam went poorly. Her teacher believed her – wrongly – to have no particular affinity with the subject.
Soon that would all change. “My first memory of mathematics is probably the time [my brother] told me about the problem of adding numbers from 1 to 100,” she recalled later. This was the story of Carl Gauss, the 18th-century genius whose schoolteacher set him this problem as a timewasting exercise – only for his precocious pupil to calculate the answer in a matter of seconds.
The obvious solution is simple but slow: 1+2+3+4. Gauss’s solution is quicker to execute, and far more cunning. It goes like this: divide the numbers into two groups: from 1 to 50, and from 51 to 100. Then, add them together in pairs, starting with the lowest (1) and the highest (100), and working inwards (2+99, 3+98, and so on). There are 50 pairs; the sum of each pair is 101; the answer is 5050. “That was the first time I enjoyed a beautiful solution,” Mirzakhani told the Clay Mathematics Institute in 2008.
Since then, her appreciation for beautiful solutions has taken her a long way from Farzanegan middle school. At 17 she won her first gold medal at the International Mathematics Olympiad. At 27 she earned a doctorate from Harvard University. The Blumenthal Award and Satter Prize followed, and in 2014 she became the first woman to be awarded the Fields Medal, the highest honour a mathematician can obtain.
Before this particular brand of wonder became perceptible to Mirzakhani, she experienced feelings many of us can relate to: to the indifferent, her subject can seem “cold”, even “pointless”. Yet those who persist will be rewarded with glimpses of conceptual glory, as if gifted upon them by a capricious god: “The beauty of mathematics,” she warned, “only shows itself to more patient followers.”
This concept of “beauty” found in maths has been referred to over centuries by many others; though, like beauty itself, it is notoriously difficult to define…
For an experienced mathematician, the greatest equations are beautiful as well as useful. Can the rest of us see what they see? “What makes maths beautiful?”
[From The New Humanist, via the ever-illuminating 3 Quarks Daily]
Maryam Mirzakhani died last Friday, a victim of breast cancer; she was 40. As Peter Sarnak (a mathematician at Princeton University and the Institute for Advanced Study) said, her passing is “a big loss and shock to the mathematical community worldwide.” See also here.
* Bertrand Russell, A History of Western Philosophy
###
As we accede to awe, we might spare a thought for Andrey (Andrei) Andreyevich Markov; he died on this date in 1922. A Russian mathematician, he helped to develop the theory of stochastic processes, especially those now called Markov chains: sequences of random variables in which the future variable is determined by the present variable but is independent of the way in which the present state arose from its predecessors. (For example, the probability of winning at the game of Monopoly can be determined using Markov chains.) His work on the study of the probability of mutually-dependent events has been developed and widely applied to the biological and social sciences.
“The only lasting truth is Change”*…
A running tally of world population, plus telling (and similarly constantly-updated) statistics on government and economics, society and media, the environment, food, water, energy, and health, all derived from sources including the United Nations Population Division, the World Health Organization (WHO), the Food and Agriculture Organization (FAO), the International Monetary Fund (IMF), and the World Bank: Worldometers.
* Octavia E. Butler
###
As we watch the world tick by, we might recall that it was on this date in 1923 that the Zero Milestone was dedicated just south of the White House at the north edge of the Ellipse, within President’s Park. Intended as the initial milestone from which all road distances in the United States should be reckoned, at present only roads in the Washington, D.C. area have distances measured from it.
“The generation of random numbers is too important to be left to chance”*…
Roman 12mm dice
Random numbers are central to more than we may realize. They have applications in gambling, statistical sampling, computer simulation and Monte Carlo modeling, cryptography (as applied in both communications and transactions), completely randomized design, even sooth-saying– in any area where producing an unpredictable result is desirable. So how they’re produced– the certainty that they are, in fact, random– matters enormously.
It’s no surprise, then, that random number generation has a long and fascinating history. Happily, Carl Tashian is here to explain.
“As an instrument for selecting at random, I have found nothing superior to dice,” wrote statistician Francis Galton in an 1890 issue of Nature. “When they are shaken and tossed in a basket, they hurtle so variously against one another and against the ribs of the basket-work that they tumble wildly about, and their positions at the outset afford no perceptible clue to what they will be even after a single good shake and toss.”…
From I Ching sticks and dice to the cryptographically-secure PRNG, “A Brief History of Random Numbers.”
[TotH to the eminently-numerate Reuben Steiger]
###
As we roll the bones, we might spare a thought for Samuel “Sam” Loyd; he died on this date in 1911. A chess player, chess composer, puzzle author, and recreational mathematician. A member of the Chess Hall of Fame (for both his play and for his exercises, or “problems”), he gained posthumous fame when his son published a collection of his mathematical and logic puzzles, Cyclopedia of 5000 Puzzles after his father’s death. As readers can see here and here, his puzzles still delight.
Loyd’s most famous puzzle was the 14-15 Puzzle, which he produced in 1878. His original authorship is debated; but in any case, his version created a craze that swept America to such an extent that employers put up notices prohibiting playing the puzzle during office hours.
“Without data, you’re just another person with an opinion”*…
It used to be that we’d see a poorly made graph or a data design goof, laugh it up a bit, and then carry on. At some point though — during this past year especially — it grew more difficult to distinguish a visualization snafu from bias and deliberate misinformation.
Of course, lying with statistics has been a thing for a long time, but charts tend to spread far and wide these days. There’s a lot of them. Some don’t tell the truth. Maybe you glance at it and that’s it, but a simple message sticks and builds. Before you know it, Leonardo DiCaprio spins a top on a table and no one cares if it falls or continues to rotate.
So it’s all the more important now to quickly decide if a graph is telling the truth…
Nathan Yau (Flowing Data) provides a very helpful (and very amusing) guide: “How to Spot Visualization Lies.”
* W. Edwards Deming
###
As we key our eyes open, we might send healthy birthday greetings to John Snow; he was born on this date in 1813. A physician and a leader in the adoption of anaesthesia and medical hygiene, he is considered the father of modern epidemiology, in large measure because of his work in tracing the source of a cholera outbreak in Soho, London, in 1854. His On the Mode of Communication of Cholera (1849) suggested that cholera was a contagious disease easily transmitted by contaminated water. But the widely-held theory was that diseases are caused by bad air led to his idea being ignored. Then, in London’s 1854 cholera emergency, he painstakingly correlated individual cholera casualties to the water supply they had used in each case. He then communicated his results with a map that underlined his point, and ended the deadly epidemic by removing the pump handle of the community water pump that he found to be the culprit.
Snow’s map of cholera cases
His findings inspired fundamental changes in the water and waste systems of London, which led to similar changes in other cities, and a significant improvement in general public health around the world. His mode of communicating them contributed to the rise of data visualization.
“When you have mastered numbers, you will in fact no longer be reading numbers, any more than you read words when reading books. You will be reading meanings.”*…
Errors of judgment about large numbers can have a big impact on the way you view policies and government decisions. The rationale goes like this: The National Science Foundation received $7.463 billion for fiscal year 2016 through the Consolidated Appropriations Act. The total United States budget outlay for 2016 was $3.54 trillion. If you’re someone who perceives the difference between a billion and a trillion as relatively small, you’d think the US is spending a lot of money on the National Science Foundation—in fact, depending on your politics, you might applaud the federal government’s investment or even think it wasteful. But, if you understand that a billion is a thousand times less than a trillion, you can calculate that the Foundation got a paltry 0.2 percent of the budget outlay last year. (It may be more straightforward to think of the budget as roughly one-half to one-third of reported costs for the proposed US-Mexico border wall, and let your values guide you from there.)…
On the significance of scale: “How to Understand Extreme Numbers.”
[The image above is, of course, from the ever-wonderful xkcd.]
* W.E.B. Du Bois
###
As we nudge ourselves toward numeracy, we might spare a thought for Sewall Wright; he died on this date in 1988. A geneticist, he was known for his influential work on evolutionary theory and also for his work on path analysis. He was a founder (with Ronald Fisher and J.B.S. Haldane) of population genetics– a major step in the development of the modern evolutionary synthesis combining genetics with evolution. He is perhaps best remembered for his concept of genetic drift (called the Sewall Wright effect): when small populations of a species are isolated, the few individuals who carry certain relatively rare genes may fail, out of pure chance, to transmit them. The genes may therefore disappear and their loss may lead to the emergence of new species– although natural selection has played no part in the process.
