(Roughly) Daily

Posts Tagged ‘Probability

“Everything we care about lies somewhere in the middle, where pattern and randomness interlace”*…

True randomness (it’s lumpy)

We tend dramatically to underestimate the role of randomness in the world…

Arkansas was one out away from the 2018 College World Series championship, leading Oregon State in the series and 3-2 in the ninth inning of the game when Cadyn Grenier lofted a foul pop down the right-field line. Three Razorbacks converged on the ball and were in position to make a routine play on it, only to watch it fall untouched to the ground in the midst of them. Had any one of them made the play, Arkansas would have been the national champion.

Nobody did.

Given “another lifeline,” Grenier hit an RBI single to tie the game before Trevor Larnach launched a two-run homer to give the Beavers a 5-3 lead and, ultimately, the game. “As soon as you see the ball drop, you know you have another life,” Grenier said. “That’s a gift.” The Beavers accepted the gift eagerly and went on win the championship the next day as Oregon State rode freshman pitcher Kevin Abel to a 5-0 win over Arkansas in the deciding game of the series. Abel threw a complete game shutout and retired the last 20 hitters he faced.

The highly unlikely happens pretty much all the time…

We readily – routinely – underestimate the power and impact of randomness in and on our lives. In his book, The Drunkard’s Walk, Caltech physicist Leonard Mlodinow employs the idea of the “drunkard’s [random] walk” to compare “the paths molecules follow as they fly through space, incessantly bumping, and being bumped by, their sister molecules,” with “our lives, our paths from college to career, from single life to family life, from first hole of golf to eighteenth.” 

Although countless random interactions seem to cancel each another out within large data sets, sometimes, “when pure luck occasionally leads to a lopsided preponderance of hits from some particular direction…a noticeable jiggle occurs.” When that happens, we notice the unlikely directional jiggle and build a carefully concocted story around it while ignoring the many, many random, counteracting collisions.

As Tversky and Kahneman have explained, “Chance is commonly viewed as a self-correcting process in which a deviation in one direction induces a deviation in the opposite direction to restore the equilibrium. In fact, deviations are not ‘corrected’ as a chance process unfolds, they are merely diluted.”

As Stephen Jay Gould famously argued, were we able to recreate the experiment of life on Earth a million different times, nothing would ever be the same, because evolution relies upon randomness. Indeed, the essence of history is contingency.

Randomness rules.

Luck matters. A lot. Yet, we tend dramatically to underestimate the role of randomness in the world.

The self-serving bias is our tendency to see the good stuff that happens as our doing (“we worked really hard and executed the game plan well”) while the bad stuff isn’t our fault (“It just wasn’t our night” or “we simply couldn’t catch a break” or “we would have won if the umpiring hadn’t been so awful”). Thus, desirable results are typically due to our skill and hard work — not luck — while lousy results are outside of our control and the offspring of being unlucky.

Two fine books undermine this outlook by (rightly) attributing a surprising amount of what happens to us — both good and bad – to luck. Michael Mauboussin’s The Success Equation seeks to untangle elements of luck and skill in sports, investing, and business. Ed Smith’s Luck considers a number of fields – international finance, war, sports, and even his own marriage – to examine how random chance influences the world around us. For example, Mauboussin describes the “paradox of skill” as follows: “As skill improves, performance becomes more consistent, and therefore luck becomes more important.” In investing, therefore (and for example), as the population of skilled investors has increased, the variation in skill has narrowed, making luck increasingly important to outcomes.

On account of the growth and development of the investment industry, John Bogle could quite consistently write his senior thesis at Princeton on the successes of active fund management and then go on to found Vanguard and become the primary developer and intellectual forefather of indexing. In other words, the ever-increasing aggregate skill (supplemented by massive computing power) of the investment world has come largely to cancel itself out.

After a big or revolutionary event, we tend to see it as having been inevitable. Such is the narrative fallacy. In this paper, ESSEC Business School’s Stoyan Sgourev notes that scholars of innovation typically focus upon the usual type of case, where incremental improvements rule the day. Sgourev moves past the typical to look at the unusual type of case, where there is a radical leap forward (equivalent to Thomas Kuhn’s paradigm shifts in science), as with Picasso and Les Demoiselles

As Sgourev carefully argued, the Paris art market of Picasso’s time had recently become receptive to the commercial possibilities of risk-taking. Thus, artistic innovation was becoming commercially viable. Breaking with the past was then being encouraged for the first time. It would soon be demanded.

Most significantly for our purposes, Sgourev’s analysis of Cubism suggests that having an exceptional idea isn’t enough. For radical innovation really to take hold, market conditions have to be right, making its success a function of luck and timing as much as genius. Note that Van Gogh — no less a genius than Picasso — never sold a painting in his lifetime.

As noted above, we all like to think that our successes are earned and that only our failures are due to luck – bad luck. But the old expression – it’s better to be lucky than good – is at least partly true. That said, it’s best to be lucky *and* good. As a consequence, in all probabilistic fields (which is nearly all of them), the best performers dwell on process and diversify their bets. You should do the same…

As [Nate] Silver emphasizes in The Signal and the Noise, we readily overestimate the degree of predictability in complex systems [and t]he experts we see in the media are much too sure of themselves (I wrote about this problem in our industry from a slightly different angle…). Much of what we attribute to skill is actually luck.

Plan accordingly.

Taking the unaccountable into account: “Randomness Rules,” from Bob Seawright (@RPSeawright), via @JVLast

[image above: source]

* James Gleick, The Information: A History, a Theory, a Flood

###

As we contemplate chance, we might spare a thought for Oskar Morgenstern; he died on this date in 1977. An economist who fled Nazi Germany for Princeton, he collaborated with the mathematician John von Neumann to write Theory of Games and Economic Behavior, published in 1944, which is recognized as the first book on game theory— thus co-founding the field.

Game theory was developed extensively in the 1950s, and has become widely recognized as an important tool in many fields– perhaps especially in the study of evolution. Eleven game theorists have won the economics Nobel Prize, and John Maynard Smith was awarded the Crafoord Prize for his application of evolutionary game theory.

Game theory’s roots date back (at least) to the 1654 letters between Pascal and Fermat, which (along with work by Cardano and Huygens) marked the beginning of probability theory. (See Peter Bernstein’s marvelous Against the Gods.) The application of probability (Bayes’ rule, discrete and continuous random variables, and the computation of expectations) accounts for the utility of game theory; the role of randomness (along with the behavioral psychology of a game’s participants) explain why it’s not a perfect predictor.

source

Written by (Roughly) Daily

July 26, 2021 at 1:00 am

“The urge to gamble is so universal and its practice so pleasurable that I assume it must be evil”*…

Gambling has existed since antiquity, but in the past 30 years it’s grown at a spectacular rate, turbocharged by the internet and globalisation. Problem gambling has grown accordingly, and become particularly prevalent in the teenage population. Even more troublingly, a study in 2013 reported that slightly over 90 per cent of problem gamblers don’t seek professional help. Gambling addiction is part of a suite of damaging and unhealthy behaviours that people do despite warnings, such as smoking, drinking or compulsive video gaming. It draws on a multitude of cognitive, social and psychobiological factors.

Psychological and medical studies have found that some people are more likely to develop a gambling disorder than others, depending on their social condition, age, education and experiences such as trauma, domestic violence and drug abuse. Problem gambling also involves complex brain chemistry, as gambling stimulates the release of multiple neurotransmitters including serotonin and dopamine, which in turn create feelings of pleasure and the attendant urge to maintain them. Serotonin is known as the happiness hormone, and typically follows a sense of release from stress or fear. Dopamine is associated with intense pleasure, released when we’re engaged in activities that deserve a reward, and precisely when that reward occurs – seeing the ball landing on the number we’ve bet on, or hearing the sound of the slot machine showing a winning payline.

For the most part, gambling addiction is viewed as a medical and psychological problem, though this hasn’t resulted in widely effective prevention and treatment programmes. That might be because the research has often focused on the origins and prevalence of addiction, and less on the cognitive premises and mechanisms that actually take place in the brain. It’s a controversial area, but this arguable lack of clinical effectiveness doesn’t appear to be specific to gambling; it applies to other addictions as well, and might even extend to some superstitions and irrational beliefs.

Can a proper presentation of the mathematical facts help gambling addiction? While most casino moguls simply trust the mathematics – the probability theory and applied statistics behind the games – gamblers exhibit a strange array of positions relative to the role of maths. While no study has offered an exhaustive taxonomy, what we know for sure is that some simply don’t care about it; others care about it, trust it, and try to use it in their favour by developing ‘winning strategies’; while others care about it and interpret it in making their gambling predictions.

Certain problem gambling programmes frame the distortions associated with gambling as an effect of a poor mathematical knowledge. Some clinicians argue that reducing gambling to mere mathematical models and bare numbers – without sparkling instances of success and the ‘adventurous’ atmosphere of a casino – can lead to a loss of interest in the games, a strategy known as ‘reduction’ or ‘deconstruction’. The warning messages involve statements along the lines of: ‘Be aware! There is a big problem with those irrational beliefs. Don’t think like that!’ But whether this kind of messaging really works is an open question. Beginning a couple of decades ago, several studies were conducted to test the hypothesis that teaching basic statistics and applied probability theory to problem gamblers would change their behaviour. Overall, these studies have yielded contradictory, non-conclusive results, and some found that mathematical education yielded no change in behaviour. So what’s missing?…

Catalin Barboianu, a gaming mathematician, philosopher of science, and problem-gambling researcher, asks if philosophers and mathematicians struggle with probability, can gamblers really hope to grasp their losing game? “Mathematics for Gamblers.”

For a deeper dive, see Alec Wilkinson’s fascinating New Yorker piece, “What Would Jesus Bet? A math whiz hones the optimal poker strategy.”

For cultural context (and an appreciation of the broader importance of the issue), see “How Gambling Mathematics Took Over The World.”

And for historical context, see (one of your correspondent’s all-time favorite books) Peter Bernstein’s Against the Gods: The Remarkable Story of Risk.

[image above: source]

* Heywood Hale Broun

###

As we roll the dice, we might spare a thought for Srinivasa Ramanujan; he died on this date in 1920. A largely self-taught mathematician from Madras, he initially developed his own mathematical research in isolation: according to Hans Eysenck: “He tried to interest the leading professional mathematicians in his work, but failed for the most part. What he had to show them was too novel, too unfamiliar, and additionally presented in unusual ways; they could not be bothered.” Seeking mathematicians who could better understand his work, in 1913 he began a postal partnership with the English mathematician G. H. Hardy at the University of Cambridge, England. Recognizing Ramanujan’s work as extraordinary, Hardy arranged for him to travel to Cambridge. In his notes, Hardy commented that Ramanujan had produced groundbreaking new theorems, including some that “defeated me completely; I had never seen anything in the least like them before.”

Ramanujan made substantial contributions to mathematical analysisnumber theoryinfinite series, and continued fractions, including solutions to mathematical problems then considered unsolvable. During his short life, he independently compiled nearly 3,900 results (mostly identities and equations).  Many were completely novel; his original and highly unconventional results, such as the Ramanujan prime, the Ramanujan theta functionpartition formulae, and mock theta functions, have opened entire new areas of work and inspired a vast amount of further research.  Nearly all his claims have now been proven correct.

See also: “Do not worry about your difficulties in Mathematics. I can assure you mine are still greater,” and enjoy the 2015 film on Ramanujan, “The Man Who Knew Infinity.”

source

“In the space between chaos and shape there was another chance”*…

Prince Hamlet spent a lot of time pondering the nature of chance and probability in William Shakespeare’s tragedy. In the famous “To be or not to be” speech, he notes that we helplessly face “the slings and arrows of outrageous fortune” — though a little earlier in the play he declares that “there’s a special providence in the fall of a sparrow,” suggesting that everything happens because God wills it to be so.

We can hardly fault the prince for holding two seemingly contradictory views about the nature of chance; after all, it is a puzzle that has vexed humankind through the ages. Why are we here? Or to give the question a slightly more modern spin, what sequence of events brought us here, and can we imagine a world in which we didn’t arrive on the scene at all?

It is to biologist Sean B. Carroll’s credit that he’s found a way of taking a puzzle that could easily fill volumes (and probably has filled volumes), and presenting it to us in a slim, non-technical, and fun little book, “A Series of Fortunate Events: Chance and the Making of the Planet, Life, and You.”

Carroll (not to be confused with physicist and writer Sean M. Carroll) gets the ball rolling with an introduction to the key concepts in probability and game theory, but quickly moves on to the issue at the heart of the book: the role of chance in evolution. Here we meet a key historical figure, the 20th-century French biochemist Jacques Monod, who won a Nobel Prize for his work on genetics. Monod understood that genetic mutations play a critical role in evolution, and he was struck by the random nature of those mutations…

Carroll quotes Monod: “Pure chance, absolutely free and blind, at the very root of the stupendous edifice of evolution: This central concept of modern biology is no longer one among other possible or even conceivable hypotheses. It is today the sole conceivable hypothesis, the only one that squares with observed and tested fact.”

“There is no scientific concept, in any of the sciences,” Monod concludes, “more destructive of anthropocentrism than this one.”

From there, it’s a short step to the realization that we humans might never have evolved in the first place…

Preview(opens in a new tab)

The profound impact of randomness in determining destiny: “The Power of Chance in Shaping Life and Evolution.”

See also: “Survival of the Luckiest.”

* Jeanette Winterson, The World and Other Places

###

As we blow on the dice, we might send carefully-calculated birthday greetings to Gabrielle-Émilie Le Tonnelier de Breteuil, Marquise du Châtelet, the French mathematician and physicist who is probably (if unfairly) better known as Voltaire’s mistress; she was born on this date in 1706.  Fascinated by the work of Newton and Leibniz, she dressed as a man to frequent the cafes where the scientific discussions of the time were held.  Her major work was a translation of Newton’s Principia, for which Voltaire wrote the preface; it was published a decade after her death, and was for many years the only translation of the Principia into French.

Judge me for my own merits, or lack of them, but do not look upon me as a mere appendage to this great general or that great scholar, this star that shines at the court of France or that famed author. I am in my own right a whole person, responsible to myself alone for all that I am, all that I say, all that I do. It may be that there are metaphysicians and philosophers whose learning is greater than mine, although I have not met them. Yet, they are but frail humans, too, and have their faults; so, when I add the sum total of my graces, I confess I am inferior to no one.
– Mme du Châtelet, to Frederick the Great of Prussia

source

“Science is a process”*…

 

klee

Paul Klee, “The Bounds of the Intellect,” 1927 (detail)

 

When my grandfather died last fall, it fell to my sisters and me to sort through the books and papers in his home in East Tennessee. My grandfather was a nuclear physicist, my grandmother a mathematician, and among their novels and magazines were reams of scientific publications. In the wood-paneled study, we passed around great sheaves of papers for sorting, filling the air with dust.

My youngest sister put a pile of yellowing papers in front of me, and I started to leaf through the typewritten letters and scholarly articles. Then my eyes fell on the words fundamental breakthroughspectacular, and revolutionary. Letters from some of the biggest names in physics fell out of the folders, in correspondence going back to 1979.

In this stack, I found, was evidence of a mystery. My grandfather had a theory, one that he believed to be among the most important work of his career. And it had never been published…

The remarkable– and illuminating– story of Veronique (Nikki) Greenwood’s quest to determine whether her grandfather was “a genius or a crackpot”: “My Grandfather Thought He Solved a Cosmic Mystery.”

* T.S. Kuhn, The Structure of Scientific Revolution

###

As we note that the history of science is, effectively, the history of the instruments developed to help us “see” things smaller, larger, smaller, farther, or outside our human sensory range, we might recall that it was on this date in 1664 that natural philosopher, architect and pioneer of the Scientific Revolution Robert Hooke showed an advance copy of his book Micrographia— a chronicle of Hooke’s observations through various lens– to members of the Royal Society.  The volume (which coined the word “cell” in a biological context) went on to become the first scientific best-seller, and inspired broad interest in the new science of microscopy.

source: Cal Tech

Note that the image above is of an edition of Micrographia dated 1665.  Indeed, while (per the above) the text was previewed to the Royal Society in 1664 (to wit the letter, verso), the book wasn’t published until September, 1665.  Note too that Micrographia is in English (while most scientific books of that time were still in Latin)– a fact that no doubt contributed to its best-seller status.

 

Written by (Roughly) Daily

November 4, 2018 at 1:01 am

“Mathematics, rightly viewed, possesses not only truth, but supreme beauty”*…

 

Maryam Mirzakhani did not enjoy mathematics to begin with. She dreamed of being an author or politician, but as a top student at her all-girls school in Tehran she was still disappointed when her first-year maths exam went poorly. Her teacher believed her – wrongly – to have no particular affinity with the subject.

Soon that would all change. “My first memory of mathematics is probably the time [my brother] told me about the problem of adding numbers from 1 to 100,” she recalled later. This was the story of Carl Gauss, the 18th-century genius whose schoolteacher set him this problem as a timewasting exercise – only for his precocious pupil to calculate the answer in a matter of seconds.

The obvious solution is simple but slow: 1+2+3+4. Gauss’s solution is quicker to execute, and far more cunning. It goes like this: divide the numbers into two groups: from 1 to 50, and from 51 to 100. Then, add them together in pairs, starting with the lowest (1) and the highest (100), and working inwards (2+99, 3+98, and so on). There are 50 pairs; the sum of each pair is 101; the answer is 5050. “That was the first time I enjoyed a beautiful solution,” Mirzakhani told the Clay Mathematics Institute in 2008.

Since then, her appreciation for beautiful solutions has taken her a long way from Farzanegan middle school. At 17 she won her first gold medal at the International Mathematics Olympiad. At 27 she earned a doctorate from Harvard University. The Blumenthal Award and Satter Prize followed, and in 2014 she became the first woman to be awarded the Fields Medal, the highest honour a mathematician can obtain.

Before this particular brand of wonder became perceptible to Mirzakhani, she experienced feelings many of us can relate to: to the indifferent, her subject can seem “cold”, even “pointless”. Yet those who persist will be rewarded with glimpses of conceptual glory, as if gifted upon them by a capricious god: “The beauty of mathematics,” she warned, “only shows itself to more patient followers.”

This concept of “beauty” found in maths has been referred to over centuries by many others; though, like beauty itself, it is notoriously difficult to define…

For an experienced mathematician, the greatest equations are beautiful as well as useful. Can the rest of us see what they see?  “What makes maths beautiful?

[From The New Humanist, via the ever-illuminating 3 Quarks Daily]

Maryam Mirzakhani died last Friday, a victim of breast cancer; she was 40.  As Peter Sarnak (a mathematician at Princeton University and the Institute for Advanced Study) said, her passing is “a big loss and shock to the mathematical community worldwide.”  See also here.

* Bertrand Russell, A History of Western Philosophy

###

As we accede to awe, we might spare a thought for Andrey (Andrei) Andreyevich Markov; he died on this date in 1922.  A Russian mathematician, he helped to develop the theory of stochastic processes, especially those now called Markov chains: sequences of random variables in which the future variable is determined by the present variable but is independent of the way in which the present state arose from its predecessors.  (For example, the probability of winning at the game of Monopoly can be determined using Markov chains.)  His work on the study of the probability of mutually-dependent events has been developed and widely applied to the biological and social sciences.

 source

 

Written by (Roughly) Daily

July 20, 2017 at 1:01 am

%d bloggers like this: