Many layers of uncertainty along with thinking errors of scientists (blind spots) make the research or evidence untrustworthy about 42 percent of the time, according to a study.
Many studies lack validity, but researchers already being involved in the studies develop ‘learned helplessness’ and start believing in evidence even though there is none. A lack of evidence to prove the contrary is not evidence to support the theory.

When Studies Are Untrustworthy

Problem Solving

“Absence of evidence is not evidence of absence.”

Quote

When we read scientific studies, it helps to keep in mind the following:
<ol>
<li>Scientists are prone to error just like everyone else.</li>
<li>Single source claims are dubious.</li>
<li>There is a lot we don’t know.</li>
<li>We should not be biased towards a particular outcome.</li>
<li>Independent tests of the findings can be done if possible.</li>
<li>Proof of something does not mean it is true, and a lack of proof does not mean it is false.</li>
</ol>

Advice For Reading Scientific Studies

The Zeigarnik Effect states that we remember unfinished and incomplete tasks much better than completed tasks. Things remain in our mind due to their being incomplete. 
There are several studies that have failed to replicate Zeigarnik’s experiment. But this does not mean the theory is false. It means the concept might be true for some people, but not all people.

Studies on The Zeigarnik Effect

It is a good idea to remain critical as most psychological experiments are situational and may not stand the test of time.
Certain ideas that claim scientific evidence like the 10,000 hour rule or Grit, study a specific group of people, and are not universal theories.

Accuracy of Psychological Experiments

There are very few absolutely certain, universal truths in life. We are quick to think a piece of information is 100 percent true if it's presented as a new, groundbreaking idea, making us have a lightbulb moment.
Apart from basic math, which is a universal truth, very few truths are found in our lifetime that resonate with us for decades.

Universal Truths

<h1> Why ‘Science-Backed’ Advice Is Not Always Better</h1><img width="665" height="435" src="https://i1.wp.com/dariusforoux.com/wp-content/uploads/2020/07/absence-of-evidence.png?resize=665%2C435&amp;ssl=1" alt="absence of evidence" srcset="https://i1.wp.com/dariusforoux.com/wp-content/uploads/2020/07/absence-of-evidence.png?resize=665%2C435&amp;ssl=1 665w, https://i1.wp.com/dariusforoux.com/wp-content/uploads/2020/07/absence-of-evidence.png?zoom=2&amp;resize=665%2C435&amp;ssl=1 1330w, https://i1.wp.com/dariusforoux.com/wp-content/uploads/2020/07/absence-of-evidence.png?zoom=3&amp;resize=665%2C435&amp;ssl=1 1995w"><div> 5 min read</div>There’s a lot of “science-backed” advice in books, articles, newspapers, and other places we get our advice from.&nbsp;Honestly, I&nbsp;also regularly refer to scientific studies.My goal for writing articles is to combine my personal experience with evidence. Sometimes I share mostly my&nbsp;experience;&nbsp;sometimes I share stories about other people’s experiences and research.&nbsp;But I always think, “How can we trust&nbsp;these&nbsp;sources?”&nbsp;Fortunately, I’m not the only person who thinks that.&nbsp;Even scientists say&nbsp;we should remain critical of the findings of scientific experiments. David Chester, Ph.D., a psychology professor at Virginia Commonwealth University, recently did a&nbsp;<a href="https://www.news.vcu.edu/article/An_untested_foundation_A_VCU_study_finds_that_many_published?fbclid=IwAR2c_IRpAk2ADBkJdwJEv_haAWSmIrYh2g2W7XNd271aUlAd_Xt1LVG4cv8" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener" dir="ltr">study of studies</a>. Together with his colleague, Emily&nbsp;Lasko, they wrote:&nbsp;<BLOCKQUOTE>“The findings of experimental psychology likely rest on an untested foundation.”&nbsp;</BLOCKQUOTE>They did a study of scientific experiments in the field of psychology. Chester and&nbsp;Lasko&nbsp;investigated 348 psychological experiments. Their study showed that roughly&nbsp;42%&nbsp;of the experiments lacked validity.&nbsp;Validity is an important concept in scientific research because it’s an indication of how&nbsp;<a href="https://linguistics.byu.edu/faculty/henrichsenl/ResearchMethods/RM%5C_2%5C_18.html" rel="noreferrer noopener" dir="ltr">sound research</a>&nbsp;is. In other words: How solid are the claims scientists make? Chester and&nbsp;Lasko&nbsp;are basically saying&nbsp;we don’t know.&nbsp;&nbsp;<h2>How to interpret psychology research&nbsp;</h2>One of the biggest critics of scientific experiments is&nbsp;Nassim&nbsp;Nicholas&nbsp;Taleb, author of <a href="https://www.amazon.com/gp/product/0812975219/ref=as_li_qf_asin_il_tl?ie=UTF8&amp;tag=dariusforoux-20&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0812975219&amp;linkId=5f8a0700fa0ca7fce5eda65062fca7ca" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener" data-amzn-asin="0812975219" dir="ltr">Fooled by Randomness</a>:&nbsp;<BLOCKQUOTE>“(M)any&nbsp;psychological “biases” are errors by researchers missing a layer of uncertainty in the model.”&nbsp;</BLOCKQUOTE>Taleb&nbsp;argues that scientists also make thinking errors. We shouldn’t take findings as the truth. We should remain critical at all times. But how do we know which experiments lack validity?&nbsp;Chester and&nbsp;Lasko&nbsp;didn’t disclose that information, which I understand.&nbsp;They didn’t want to point&nbsp;fingers at&nbsp;their colleagues. But we have&nbsp;to&nbsp;recognize that there are a lot of experiments.&nbsp;<a href="https://en.wikipedia.org/wiki/Category:Psychology_experiments" rel="noreferrer noopener" dir="ltr">This Wikipedia lists 76 popular ones</a>.&nbsp;You might have heard of&nbsp;“Learned Helplessness.”&nbsp;It’s an experiment I’ve&nbsp;referred to in the past.&nbsp;It’s also included in&nbsp;my productivity course. So&nbsp;does this mean we can’t trust&nbsp;science&nbsp;at all? No.&nbsp;&nbsp;If a study isn’t valid, it doesn’t automatically mean it’s invalid. David Chester explains this well: “Almost all of the manipulations we examined failed to provide the necessary evidence that they were valid, which does not mean they are invalid — their validity is just unknown.”&nbsp;A&nbsp;lack&nbsp;of proof is not proof. Pseudo intellectuals love to use the absence of evidence as evidence. The 17th century philosopher&nbsp;John Locke was probably the first who wrote about this thinking fallacy. The aphorism goes as follows:&nbsp;&nbsp;<BLOCKQUOTE>“Absence of evidence is not evidence of absence.”&nbsp;&nbsp;</BLOCKQUOTE>It’s ignorant to dismiss science. And yet, we see this all the time. That’s why we can all benefit from having basic knowledge about how to interpret scientific studies. Here are a few things to keep in mind when you read about scientific studies:&nbsp;<UL><LI>Scientists are humans, and humans make mistakes&nbsp;</LI><LI>We don’t know everything&nbsp;</LI><LI>Avoid claims that come from a single source&nbsp;</LI><LI>Don’t pick sides&nbsp;</LI><LI>If possible, test the ideas in your own life&nbsp;</LI><LI>Proof&nbsp;doesn’t&nbsp;mean it’s true&nbsp;</LI><LI>Lack of proof doesn’t mean it’s false&nbsp;</LI><LI>Understand that there’s no one size fits all&nbsp;</LI></UL><h2>Remain critical of theories&nbsp;&nbsp;</h2>One of my favorite parts in&nbsp;Berkshire Hathaway’s annual meeting&nbsp;includes&nbsp;Warren Buffett and Charlie&nbsp;Munger&nbsp;taking&nbsp;subtle digs at economists who believe in the Efficient Market Hypothesis. EMH states that share prices always reflect&nbsp;all available information and are efficiently priced. According to that theory, it’s impossible for investors to purchase undervalued stocks.&nbsp;&nbsp;That means it should be impossible to outperform the overall market. Tell that to Buffett and&nbsp;Munger, who consistently outperformed the market for decades. $1000 invested with Warren Buffett since 1965 would be worth more than $27 million last year. The comparable amount for the S&amp;P 500 was roughly $200,000 (source:&nbsp;<a href="https://www.barrons.com/articles/berkshire-hathaways-investing-returns-a-breakdown-by-numbers-51582633800" rel="noreferrer noopener" dir="ltr">Barron’s</a>):&nbsp;<figure><img src="https://i0.wp.com/dariusforoux.com/wp-content/uploads/2020/07/berkshire-returns.png?resize=665%2C325&amp;is-pending-load=1#038;ssl=1" alt="" srcset="data:image/gif;base64, https://dariusforoux.com/science-backed-advice/R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" width="1" height="1"></figure>Understanding that theories are not true for everyone is particularly important when it comes to psychology experiments. Let me give you&nbsp;an&nbsp;example. In the personal development world, many writers love to refer to the&nbsp;Zeigarnik&nbsp;effect.&nbsp;&nbsp;The idea is that people remember&nbsp;unfinished&nbsp;tasks better than completed&nbsp;tasks.&nbsp;Some experts claim that certain tasks remain in your mind because you haven’t finished them.&nbsp;But here’s the thing. Several other studies failed to replicate&nbsp;Zeigarnik’s&nbsp;experiment (<a href="https://en.wikipedia.org/wiki/Zeigarnik_effect" rel="noreferrer noopener" dir="ltr">Van Bergen, 1968</a>). Does that mean the theory is false? Not necessarily. It means the concept might be true for some people, but&nbsp;not all&nbsp;people.&nbsp;We must realize that experiments show results from a small group of participants. If a bunch of students kept on thinking about their unfinished tasks, it doesn’t mean we’re all built that way. Every person is different. Most psychology experiments are situational.&nbsp;&nbsp;Maybe an idea is true for a certain group of people or during a particular situation. For example, I have no issue with unfinished tasks myself. That stuff doesn’t linger on my mind because I practice mindfulness.&nbsp;&nbsp;In his book, <a href="https://www.amazon.com/gp/product/0735214484/ref=as_li_qf_asin_il_tl?ie=UTF8&amp;tag=dariusforoux-20&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0735214484&amp;linkId=22980bc91c39c52a16a334897c4ed602" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener" data-amzn-asin="0735214484" dir="ltr">Range</a>, David Epstein&nbsp;argues&nbsp;against several&nbsp;popular beliefs that claim&nbsp;scientific evidence.&nbsp;He&nbsp;breaks down the research behind popular ideas like the 10,000&nbsp;hours&nbsp;rule and Grit. He demonstrates we must remain critical because these studies only look at a very specific group of&nbsp;people. We can’t extrapolate these ideas to everything in life.&nbsp;<h2>Universal truths are sparse&nbsp;</h2>The problem with information is the way it’s presented. When we have an idea or learn something new, we’re quick to think&nbsp;it’s&nbsp;100% true. We love to present ideas as “this is the way it is!”&nbsp;&nbsp;The Coronavirus pandemic is a perfect example. I’ve read and watched interviews with dozens of virologists and epidemiologists. And nearly all of them spoke with full confidence. It seems like they’re describing hard science. But very few things in life are as straightforward as basic math.&nbsp;Ask 100 teachers to show a kid how to do basic math and you’ll see a unified approach. Basic math is a universal truth. We can’t argue with the fact that 5 x 5 is 25.&nbsp;&nbsp;But ask 100 virologists whether we should close schools during a pandemic and you get an intellectual war. Who knows it best? We don’t know.&nbsp;There are very few <a href="https://dariusforoux.com/17-hard-truths/" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener" dir="ltr">universal truths</a> in life;&nbsp;things we can say with absolute certainty. I personally live my life according&nbsp;to&nbsp;those truths.&nbsp;Most of&nbsp;them&nbsp;are common sense; things that have been passed&nbsp;from one generation to the next.&nbsp;&nbsp;One of those&nbsp;truths&nbsp;is something I learned from my&nbsp;mother, who in turn learned it&nbsp;from her father, and so forth. My grandfather firmly believes in the power of waking up every day at the same time. He’s a highly orderly person and that strategy works great for him. He’s still independent and lives on his own.&nbsp;<a href="https://dariusforoux.com/1-habit/" aria-label="undefined (opens in a new tab)" rel="noreferrer noopener" dir="ltr">And I follow his advice as well</a>. I simply tried it and it worked for me. The times I’m most consistent in life are&nbsp;when I wake up at the same time. You can always do your own experiments to see&nbsp;what works for you. We don’t have to be a scientist to try things.&nbsp;&nbsp;That’s the biggest lesson I’ve learned. What if Buffett and&nbsp;Munger&nbsp;simply accepted the Efficient Market Hypothesis? What if we all drove ourselves nuts if we didn’t finish a task? We must keep challenging ourselves and the status quo.&nbsp;&nbsp;But I’m glad&nbsp;that&nbsp;people like Chester and&nbsp;Lasko&nbsp;keep doing scientific research with an open mind. Findings like these show us that we never stop learning.&nbsp;<h1>Want To Get Tips For Personal Growth During A Crisis?</h1>I'm continually researching ways we can improve ourselves so we can get through this pandemic stronger. Join my newsletter to get my latest tips. I’ll also send you my Growth Kit, a bundle of 6 useful resources.<div data-element="fields" data-stacked="false"><button data-element="submit">JOIN NOW</button></div>No ads. No spam. Unsubscribe at any time.<h3>Spread The Word:</h3>

There’s a lot of “science-backed” advice in books, articles, newspapers, and other places we get our advice from. But's it's not always better.

Why ‘Science-Backed’ Advice Is Not Always Better

There’s a lot of “science-backed” advice in books, articles, newspapers, and other places we get our advice from. Honestly, I also regularly refer to scientific studies. My goal for writing articles is to combine my personal experience with evidence. Sometimes I share mostly my experience; sometimes I share stories about other people’s experiences and research. But I always think, “How can we trust these sources?”  Fortunately, I’m not the only person who thinks that. Even scientists say we should remain critical of the findings of scientific experiments. David Chester, Ph.D., a psychology professor at Virginia Commonwealth University, recently did a <a aria-label="undefined (opens in a new tab)" dir="ltr" href="https://www.news.vcu.edu/article/An_untested_foundation_A_VCU_study_finds_that_many_published?fbclid=IwAR2c_IRpAk2ADBkJdwJEv_haAWSmIrYh2g2W7XNd271aUlAd_Xt1LVG4cv8" rel="noreferrer noopener">study of studies </a>. Together with his colleague, Emily Lasko, they wrote:

They did a study of scientific experiments in the field of psychology. Chester and Lasko investigated 348 psychological experiments. Their study showed that roughly 42%  of the experiments lacked validity.  Validity is an important concept in scientific research because it’s an indication of how <a dir="ltr" href="https://linguistics.byu.edu/faculty/henrichsenl/ResearchMethods/RM%5C_2%5C_18.html" rel="noreferrer noopener">sound research </a> is. In other words: How solid are the claims scientists make? Chester and Lasko are basically saying we don’t know.   One of the biggest critics of scientific experiments is Nassim Nicholas Taleb, author of <a aria-label="undefined (opens in a new tab)" data-amzn-asin="0812975219" dir="ltr" href="https://www.amazon.com/gp/product/0812975219/ref=as_li_qf_asin_il_tl?ie=UTF8&amp;tag=dariusforoux-20&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0812975219&amp;linkId=5f8a0700fa0ca7fce5eda65062fca7ca" rel="noreferrer noopener">Fooled by Randomness </a>:

Taleb argues that scientists also make thinking errors. We shouldn’t take findings as the truth. We should remain critical at all times. But how do we know which experiments lack validity?  Chester and Lasko didn’t disclose that information, which I understand. They didn’t want to point fingers at their colleagues. But we have to recognize that there are a lot of experiments. <a dir="ltr" href="https://en.wikipedia.org/wiki/Category:Psychology_experiments" rel="noreferrer noopener">This Wikipedia lists 76 popular ones </a>.  You might have heard of “Learned Helplessness.” It’s an experiment I’ve referred to in the past. It’s also included in my productivity course. So does this mean we can’t trust science at all? No.

If a study isn’t valid, it doesn’t automatically mean it’s invalid. David Chester explains this well: “Almost all of the manipulations we examined failed to provide the necessary evidence that they were valid, which does not mean they are invalid — their validity is just unknown.”

A lack  of proof is not proof. Pseudo intellectuals love to use the absence of evidence as evidence. The 17th century philosopher John Locke was probably the first who wrote about this thinking fallacy. The aphorism goes as follows:   “Absence of evidence is not evidence of absence.”    It’s ignorant to dismiss science. And yet, we see this all the time. That’s why we can all benefit from having basic knowledge about how to interpret scientific studies. Here are a few things to keep in mind when you read about scientific studies:

<ul><li>Scientists are humans, and humans make mistakes  </li><li>We don’t know everything  </li><li>Avoid claims that come from a single source  </li><li>Don’t pick sides  </li><li>If possible, test the ideas in your own life  </li><li>Proof doesn’t mean it’s true  </li><li>Lack of proof doesn’t mean it’s false  </li><li>Understand that there’s no one size fits all  </li> </ul>

One of my favorite parts in Berkshire Hathaway’s annual meeting includes Warren Buffett and Charlie Munger taking subtle digs at economists who believe in the Efficient Market Hypothesis. EMH states that share prices always reflect all available information and are efficiently priced. According to that theory, it’s impossible for investors to purchase undervalued stocks.

That means it should be impossible to outperform the overall market. Tell that to Buffett and Munger, who consistently outperformed the market for decades. $1000 invested with Warren Buffett since 1965 would be worth more than $27 million last year. The comparable amount for the S&amp;P 500 was roughly $200,000 (source: <a dir="ltr" href="https://www.barrons.com/articles/berkshire-hathaways-investing-returns-a-breakdown-by-numbers-51582633800" rel="noreferrer noopener">Barron’s </a>):

Understanding that theories are not true for everyone is particularly important when it comes to psychology experiments. Let me give you an example. In the personal development world, many writers love to refer to the Zeigarnik effect.   The idea is that people remember unfinished  tasks better than completed tasks. Some experts claim that certain tasks remain in your mind because you haven’t finished them.  But here’s the thing. Several other studies failed to replicate Zeigarnik’s experiment (<a dir="ltr" href="https://en.wikipedia.org/wiki/Zeigarnik_effect" rel="noreferrer noopener">Van Bergen, 1968 </a>). Does that mean the theory is false? Not necessarily. It means the concept might be true for some people, but not all  people.

We must realize that experiments show results from a small group of participants. If a bunch of students kept on thinking about their unfinished tasks, it doesn’t mean we’re all built that way. Every person is different. Most psychology experiments are situational.   Maybe an idea is true for a certain group of people or during a particular situation. For example, I have no issue with unfinished tasks myself. That stuff doesn’t linger on my mind because I practice mindfulness.   In his book, <a aria-label="undefined (opens in a new tab)" data-amzn-asin="0735214484" dir="ltr" href="https://www.amazon.com/gp/product/0735214484/ref=as_li_qf_asin_il_tl?ie=UTF8&amp;tag=dariusforoux-20&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0735214484&amp;linkId=22980bc91c39c52a16a334897c4ed602" rel="noreferrer noopener">Range </a>, David Epstein argues against several popular beliefs that claim scientific evidence. He breaks down the research behind popular ideas like the 10,000 hours rule and Grit. He demonstrates we must remain critical because these studies only look at a very specific group of people. We can’t extrapolate these ideas to everything in life.

The problem with information is the way it’s presented. When we have an idea or learn something new, we’re quick to think it’s 100% true. We love to present ideas as “this is the way it is!”   The Coronavirus pandemic is a perfect example. I’ve read and watched interviews with dozens of virologists and epidemiologists. And nearly all of them spoke with full confidence. It seems like they’re describing hard science. But very few things in life are as straightforward as basic math.

Ask 100 teachers to show a kid how to do basic math and you’ll see a unified approach. Basic math is a universal truth. We can’t argue with the fact that 5 x 5 is 25.   But ask 100 virologists whether we should close schools during a pandemic and you get an intellectual war. Who knows it best? We don’t know.  There are very few <a aria-label="undefined (opens in a new tab)" dir="ltr" href="https://dariusforoux.com/17-hard-truths/" rel="noreferrer noopener">universal truths </a>in life; things we can say with absolute certainty. I personally live my life according to those truths. Most of them are common sense; things that have been passed from one generation to the next.

One of those truths is something I learned from my mother, who in turn learned it from her father, and so forth. My grandfather firmly believes in the power of waking up every day at the same time. He’s a highly orderly person and that strategy works great for him. He’s still independent and lives on his own.

<a aria-label="undefined (opens in a new tab)" dir="ltr" href="https://dariusforoux.com/1-habit/" rel="noreferrer noopener">And I follow his advice as well </a>. I simply tried it and it worked for me. The times I’m most consistent in life are when I wake up at the same time. You can always do your own experiments to see what works for you. We don’t have to be a scientist to try things.   That’s the biggest lesson I’ve learned. What if Buffett and Munger simply accepted the Efficient Market Hypothesis? What if we all drove ourselves nuts if we didn’t finish a task? We must keep challenging ourselves and the status quo.   But I’m glad that people like Chester and Lasko keep doing scientific research with an open mind. Findings like these show us that we never stop learning.

Want To Get Tips For Personal Growth During A Crisis?

Bite-sized knowledge
to upgrade
your career

Why 'Science-Backed' Advice Is Not Always Better

When Studies Are Untrustworthy

TRADITIONAL APHORISM

Advice For Reading Scientific Studies

Studies on The Zeigarnik Effect

Accuracy of Psychological Experiments

Universal Truths

IDEAS ABOUT

MORE LIKE THIS

Got clutter you can't seem to shift? 5 smart tricks to make it move

The Sagan Standard: Extraordinary Claims Require Extraordinary Evidence - Effectiviology

Bias from Liking/Loving: Why We Comply With Those We Love - Farnam Street

It's time to

READ

LIKE

A PRO!

Bite-sized knowledgeto upgradeyour career

Why 'Science-Backed' Advice Is Not Always Better

When Studies Are Untrustworthy

TRADITIONAL APHORISM

Advice For Reading Scientific Studies

Studies on The Zeigarnik Effect

Accuracy of Psychological Experiments

Universal Truths

IDEAS ABOUT

MORE LIKE THIS

Got clutter you can't seem to shift? 5 smart tricks to make it move

The Sagan Standard: Extraordinary Claims Require Extraordinary Evidence - Effectiviology

Bias from Liking/Loving: Why We Comply With Those We Love - Farnam Street

It's time to

READ

LIKE

A PRO!

Bite-sized knowledge
to upgrade
your career