Most studies that involve statistical research remain largely inaccurate, and a large number of hypotheses use data samples which are inadequate.

Statistics Inaccuracies

<ul><li>Researchers extract a lot of false certainty from data which is used to study a particular hypothesis, due to their myopic thinking, confirmation bias, and other cognitive assumptions that they aren't even aware of.</li><li>Many researchers believe in small samples of data as if they stand for the whole of the population, and this can lead to unpredictable consequences.</li></ul>

Beliefs and Statistics

It points out our irrational bias and behavior towards our surroundings. How people succumb to irrational beliefs was demonstrated in an experiment at a University Kitchen.It found out that if there was a poster with eyes put up near the kitchen counter, then people were more likely to pay for the food, subconsciously taking those eyes in the poster as someone looking at them.

The Eye Poster Experiment

The Anchoring effect is a phenomenon studied in the art of persuasion and influence. It states that people tend to register the first piece of information that they get and 'anchor' it in their minds subconsciously. This makes them compare the second piece of information they get with the anchored information in their minds, automatically.

The Anchoring Effect

<div>
<figure> <img src="https://compote.slate.com/images/99ff8f76-d721-4ec7-871d-8ddfb5440c90.jpg"></figure>
Danny Kahneman's love affair with Amos Tversky began in the spring of 1969, when his dazzling and clever colleague, also a professor of psychology at Hebrew University in Jerusalem, came to give a talk to Kahneman's graduate seminar. Tversky told the students about a new study being done by researchers in Michigan on how regular people tend to think about statistics. The work suggested that we all have a natural grasp of probability, Tversky said. We're all inclined to be rational.
"Brilliant talk," said Kahneman when Tversky finished, "but I don't believe a word of it." He had strong reasons to be skeptical. For years, his own research had been derailed by faulty intuitions about how to use statistics. Even though Kahneman was highly trained in research methods, he'd kept on falling victim to the same mistake: His sample sizes were too small.
Kahneman also knew that he was not the only scholar screwing up. He'd seen a recent paper showing that the problem was ubiquitous: Even studies in the leading journals of psychology, performed by scientists who specialized in quantitative research, were <a href="https://replicationindex.wordpress.com/2015/09/22/the-statistical-power-of-abnormal-social-psychological-research-a-revew-by-jacob-cohen/">underpowered as a rule</a>. And now Tversky meant to argue that everyone's a natural statistician?
Most places that he went, Tversky was the smartest person in the room, and he wasn't often challenged. But Kahneman let him have it. ("You won't believe what happened to me," Tversky told a colleague later, titillated by the disagreement.) Their argument in class led to a conversation over lunch, and then a run of private meetings. What if they could show that Kahneman was right-that when people think about statistics they're inclined to make mistakes?
That summer, Kahneman and Tversky tested out their theory. They wrote a set of questions meant to test a person's statistical intuitions, and Tversky passed it out to researchers at a pair of scientific conferences, including one for the Mathematical Psychology Association. Back home in Jerusalem, they looked over the responses. Many of the scientists had made the same mistake that Kahneman had, placing too much trust in studies of small samples and tending to "extract more certainty from the data than the data, in fact, contain." This was not a form of wishful thinking, wrote Kahneman and Tversky in the first paper they put out together in 1971, called " <a href="http://pirate.shu.edu/~hovancjo/exp_read/tversky.htm">Belief in the Law of Small Numbers</a>." It was a bias of cognition, a hardware error of the mind, like one of those optical <a href="http://www.slate.com/articles/health_and_science/science/2016/03/the_science_of_the_black_and_blue_dress_one_year_later.html">illusions</a> that <a href="https://en.wikipedia.org/wiki/M%C3%BCller-Lyer_illusion">cannot be unseen</a>, no matter how many times it's pointed out. Neither knowledge nor expertise could save you: Even savvy professionals suffered from a "consistent misperception of the world."
This scientific study of scientific bias would ignite a romance of the mind, one that spanned several decades and ended up transforming both psychology and economics. Kahneman and Tversky went on to show that mistakes in human judgment are not exceptions but the rule, resulting from a host of mental shortcuts and distortions that cannot be avoided. We do not behave like " <a href="http://en.wikipedia.org/wiki/Rational_choice_theory">rational actors</a>," as economists once presumed; rather, we're predictably misguided-subject to a " <a href="https://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/2002/kahnemann-lecture.pdf">bounded rationality</a>." Tversky went on to win a MacArthur "genius" grant on the basis of their work. Kahneman would get a Nobel Prize.
The author Michael Lewis tells the tender, probing story of their lives and work together in his new book, The Undoing Project. It's a portrait of besotted opposites: Both Kahneman and Tversky were brilliant scientists, and atheist Israeli Jews descended from Eastern European rabbis, but in every other way they seemed to differ. Kahneman liked to smoke; Tversky hated cigarettes. Kahneman was a morning person; Tversky worked at night. Kahneman's office was a mess, with papers piled everywhere; Tversky preferred an empty room with nothing but a pencil and a desk. Kahneman could be shy, <a href="https://www.theguardian.com/science/2014/feb/16/daniel-kahneman-thinking-fast-and-slow-tributes">pessimistic</a>, and depressive. Tversky was pugnacious and outgoing. Kahneman's insight into bias was tied up with self-doubt, deriving from the careful excavation of weakness that he found within himself. Tversky seemed to study human folly through a telescope, as if he were peering at a far-off land.
Lewis says that colleagues never understood quite how these two managed to connect, let alone to spend so many hours every day, across so many years, inside each other's heads. "It was as if you had dropped a white mouse into a cage with a python," he writes, "and come back later and found the mouse talking and the python curled in the corner, rapt." In any case their peculiar synergy fueled an engine of disruption, laying groundwork for the field of behavioral economics and driving new approaches to the practice of <a href="http://www.slate.com/articles/arts/books/2016/12/how_daryl_morey_used_behavioral_economics_to_revolutionize_the_art_of_nba.html">sports management</a>, <a href="http://www.commonwealthfund.org/publications/newsletters/quality-matters/2013/june-july/in-focus">health care</a>, <a href="https://www.whitehouse.gov/the-press-office/2015/09/15/executive-order-using-behavioral-science-insights-better-serve-american">government</a>, <a href="http://www.forbes.com/sites/rogerdooley/2012/11/19/obama-behavioral/">presidential campaigns</a>, and <a href="http://www.nber.org/papers/w20609">education</a>. (That's just a random sample of domains in which their work has been influential; there are many, many more.)
Yet for all their fame and influence, Kahneman and Tversky's own discipline, psychology, remained somehow unaffected by the first and founding insight of their partnership. Other scholars would extend their work on judgment and heuristics while seeming to ignore the implications of their original study of statistics. If it were really true that some of the keenest minds in psychology were doing research wrong, then the aggregate effect would be enormous. The literature of published findings might be clogged with inconclusive data, and that would mean that each new research project in psychology was wobbling on hollow struts. In 1971, Kahneman and Tversky had cautioned that scientists' intuitions about randomness-their unwarranted belief that any small sample of the population could stand in for the whole-could lead to "unfortunate consequences in the course of scientific inquiry." It was as if the field forgot, or simply overlooked, what they'd demonstrated with such clarity and humor-that even virtuosic scientists, even Danny Kahneman and Amos Tversky types, might be prone to this mistake.
Neither were Kahneman and Tversky themselves inclined to dwell on what they'd found. Their survey of fellow researchers had served to illustrate the broader point, that everyone makes mistakes in a systematic way. That would be the target of their research as their partnership progressed-not the inefficiencies of scientific practice (small potatoes, those), but the very fundaments of human thought.
Even as their work produced what seemed to be an endless stream of insight, Kahneman and Tversky's personal relationship would be soured by mistakes that neither party could avoid. The Undoing Project chronicles their split in stinging detail. "It was worse than a divorce," Tversky's wife told Lewis. They seemed to suffer from the difference in their natures: one delicate, the other bruising. In 1983, Kahneman confessed that he'd been overcome by envy, since Tversky was getting most of the acclaim. "Tversky cannot control this," he told an interviewer at the time, "though I wonder whether he does as much to control it as he should." He began to feel rejected and ignored. "An episode such as the one we had yesterday wrecks my life for several days, (including anticipation as well as recovery), and I just don't want those anymore," he wrote to Tversky after a meeting in 1987.
"I realize my response style leaves a lot to be desired but you have also become much less interested in objections or criticism, mine or others'," Tversky answered. "One of the things I admired you for most in our joint work was your relentlessness as a critic." Tversky's letter castigated Kahneman for having lost his skepticism and willingness to change his mind. "I do not see any of this in your attitude to many of your ideas recently," the letter said.
After Tversky died from melanoma in 1996-the two remained in touch until the end-Kahneman gained more prominence himself. He would get his Nobel Prize in 2002 and the Presidential Medal of Freedom in 2013. But his reputation got its biggest boost five years ago, when Kahneman published <a href="http://www.slate.com/articles/life/science/2011/10/daniel_kahneman_s_thinking_fast_and_slow_reviewed_.html">Thinking, Fast and Slow</a>, a fascinating r&#233;sum&#233; of his work with Tversky and beyond, and also of the other research in psychology that has dug into our many hidden biases, illusions, fallacies, and neglects.
Kahneman had been terrified to write the book; according to Lewis, he worried that it would destroy his reputation. (Like a good behavioral economist, he even <a href="http://www.ttbook.org/book/transcript/sonic-sidebar-daniel-kahneman-thinking-fast-and-slow">paid his colleagues</a> to read the manuscript and provide anonymous advice about whether to abandon it.) To his surprise, Thinking, Fast and Slow became a huge success-a <a href="http://www.nytimes.com/2011/12/11/books/10-best-books-of-2011.html?_r=0">critically acclaimed</a>, <a href="http://www.nytimes.com/books/best-sellers/2012/01/15/hardcover-nonfiction/?module=DropDownNav&amp;action=click%C2%AEion=navbar&amp;contentCollection=Books&amp;version=Nonfiction&amp;referrer=http%3A%2F%2Fwww.nytimes.com%2Fbooks%2Fbest-sellers%2F2012%2F06%2F17%2Fcombined-print-and-e-book-nonfiction%2F&amp;pgtype=Reference">best-selling</a>, <a href="https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow#Awards_and_honors">award-winning</a> culmination of his long career.
Within a few months of its publication, though, the study of psychology went into a state of crisis. Many of the findings that Kahneman had cited in the book-and <a href="http://science.sciencemag.org/content/349/6251/aac4716">lots of others</a>, too-suddenly appeared to be quite fragile, maybe even spurious. In a section on " <a href="https://books.google.com/books?id=ZuKTvERuPG8C&amp;lpg=PP1&amp;pg=PT59">The Marvels of Priming</a>," for example, Kahneman had described the way that "primed" ideas spread across the mind like ripples on a pond, and that "the mapping of these ripples is now one of the most exciting pursuits in psychological research." But shortly after he finished the book, the enterprise of social-priming research fell into scandal and uncertainty. An influential scholar in the field was <a href="http://www.nature.com/news/2011/111101/full/479015a.html">outed as a fraud</a>. A bedrock finding in the field-drawn from a study that Kahneman called an " <a href="https://books.google.com/books?id=ZuKTvERuPG8C&amp;lpg=PP1&amp;pg=PT60">instant classic</a>"- <a href="http://blogs.discovermagazine.com/notrocketscience/2012/03/10/failed-replication-bargh-psychology-study-doyen/">crumbled under scrutiny</a>.
Other classic studies mentioned in the book started to dissolve as well, as scientists tried and failed to reproduce the original experiments. Kahneman had wowed his readers with the fact that a person is more easily amused when she <a href="http://www.slate.com/articles/health_and_science/cover_story/2016/08/can_smiling_make_you_happier_maybe_maybe_not_we_have_no_idea.html">holds a pen between her lips</a>, for example, and that <a href="http://uweb.cas.usf.edu/~drohrer/pdfs/Rohrer_et_al_2015JEPG.pdf">seeing images of money</a> will make her act more selfishly, and that <a href="http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html">a glass of lemonade can help restore her willpower</a>. But it wasn't long before each of these effects would be questioned or upended.
Indeed, the research that supported them ran afoul of the very problem that Kahneman and Tversky identified so many years ago: The researchers were fooled by faulty intuitions about randomness. Their studies had been underpowered, which meant that false-positive results may have crept into the literature.
Yet Kahneman himself left no room to doubt those suspect findings. " <a href="http://isites.harvard.edu/fs/docs/icb.topic1092050.files/Kahneman_Daniel._Thinking_Fast_and_Slow_pp._50-58.pdf">Disbelief is not an option</a>," he wrote in the book. "The results are not made up, nor are they statistical flukes. You have no choice but to accept that the major conclusions of these studies are true. More important, you must accept that they are true about you." He concluded the chapter on priming with what he called "a perfect demonstration" of its argument-a study from 2006 in which posters showing a pair of eyes or a bunch of flowers were put into a university kitchen. People were <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1686213/">more likely to pay for their tea and coffee</a> in the presence of the poster with the eyes, that study found. Something as subtle as this meager gesture at surveillance could prompt important changes in behavior.
But then, further study of the "watching-eyes effect"-with very <a href="http://lebs.hbesj.org/index.php/lebs/article/viewFile/lebs.2015.36/124">mixed results</a>-made even that perfect demonstration of priming seem <a href="http://stefanienorthover.com/docs/ehb16.pdf">a little fishy</a>.
The replication crisis in psychology does not extend to every line of inquiry, and just a portion of the work described in Thinking, Fast and Slow has been cast in shadows. Kahneman and Tversky's own research, for example, turns out to be resilient. Large-scale efforts to recreate their classic findings have so far <a href="https://osf.io/wx7ck/wiki/home/">been successful</a>. One bias they discovered-people's tendency to overvalue the first piece of information that they get, in what is known as the " <a href="https://en.wikipedia.org/wiki/Anchoring">anchoring effect</a> "-not only passed a replication test, but turned out to be much stronger than Kahneman and Tversky thought.
Still, entire chapters of Kahneman's book may need to be rewritten. The psychologist Uli Schimmack has devised a statistical measure called the <a href="https://replicationindex.wordpress.com/2016/01/31/a-revised-introduction-to-the-r-index/">R-index</a> to estimate the trustworthiness of a given body of research based on its reported sample sizes and effects. (It's like a "doping test for science," Schimmack says.) He recently applied this measure to the studies cited in each of 11 different chapters from Thinking, Fast and Slow, then assigned letter grades to each result. (The book has 38 chapters total.) A couple of the chapters came out looking very good, with R-index scores of 93 and 99-worthy of an A-plus grade from Schimmack for their rigor. But five other chapters, including the one on social priming, ended up with scores of less than 40-what Schimmack called an F. Taken all together, the chapters Schimmack looked at earned an average grade of C-minus.
How could Kahneman, of all people-a man so brilliant at describing weakness, so canny in his doubts, so quick to cut through bullshit-have signed his name to all this suspect science? Had his thinking gotten credulous, as Tversky had suggested in that letter written in the midst of their divorce? Or had he succumbed to normal human error, like the one that he discovered in the 1960s, right around the time he and Tversky met?
No one is a natural statistician, he'd argued at the time. Even scientists make mistakes. As usual, when it came to being wrong, Kahneman was right.
-
<a href="http://www.amazon.com/dp/0393254593/?tag=slatmaga-20">The Undoing Project</a>by Michael Lewis. W. W. Norton &amp; Company.
</div>

Danny Kahneman's love affair with Amos Tversky began in the spring of 1969, when his dazzling and clever colleague, also a professor of psychology at Hebrew University in Jerusalem, came to give a talk to Kahneman's graduate seminar. Tversky told the students about a new study being done by researchers in Michigan on how regular people tend to think about statistics.

How a Pioneer in the Science of Mistakes Ended Up Mistaken

Statistics Inaccuracies

Beliefs and Statistics

The Eye Poster Experiment

The Anchoring Effect

Avery B.'s ideas are part of this journey:

Related collections

Similar ideas