The world is full of evidence and studies, some good and some poor.
<ul>
<li>One major problem is that scientific lingo often means something different from everyday language. Words like theory, significant, and control have entirely different meanings in the realm of science.</li>
<li>Another problem is that experiments can suffer from problems in how they're designed, how they're analysed, and how scientific journals review them.</li>
</ul>

Problems with how we view scientific studies

One type of conflict of interest is financial. It could be someone who received funding from a company with a vested interest in the study's outcome. Or that person has a relationship with the company that could lead to benefits in the future.
A recent analysis found that from 7 to 32 percent of randomized trials in top medical journals were funded by medical industry sources.

Be aware of conflicts of interest

<ul>
<li>A hypothesis is often the first step of the scientific method. It is a proposed and still-unproven explanation that can be tested through experiments and observations.
</li>
<li>In science, a theory is a widely accepted idea backed by data, observations, and experiments. Of course, established scientific theories can later be changed or rejected if there's enough data to support it.
</li>
</ul>

The difference between a hypothesis and a theory

If you are looking at a clinical trial, a psychology study, or an animal study, it must be a randomized, placebo-controlled, double-blind study.
<ul>
<li>Randomized: The participants in the study are randomly placed into the experimental group and the comparison group.</li>
<li>Placebo-controlled: In medical studies, one comparison group gets a placebo, such as a sugar pill. This is to see the effects of the actual drug. (However, placebo effects can be so strong that they often relieve pain, among other health problems.)</li>
<li>Double-blind: A study is "blind" if the participants don't know whether they are in the experiment or the control group. A study is "double-blind" if the researchers in personal contact with participants also don't know which treatment they are administering.</li>
</ul>

The gold standard: double-blind, placebo-controlled, randomized tests

In science, "statistically significant" is the effect that can be picked up with a particular statistical tool called a p-value. A good p-value (or calculated probability) is arbitrary and can vary between scientific fields. The cut-off for statistically significant is a p-value of 0.05.
One problem is that if you run a study multiple times or do a whole bunch of different statistical analyses on the same data, your results could look meaningful purely by chance.

Understand "significance"

<ul>
<li>The Impact Factor is the most commonly used metric to assess a scientific journal's influence. It counts the number of times a journal's papers have been mentioned in other papers, relative to their own output.</li>
<li>You can find a journal's Impact Factor in the Journal Citation Reports analysis that appears annually. Otherwise, search for "impact factor." The most prestigious journals have Impact Factors in the high-20s to mid-30s.</li>
<li>The Impact Factor is controversial. Some science generates more citations, but it doesn't mean they're better. For-profit journals will often publish anything (even without peer review)</li>
</ul>

Not all journals are created equal

<ul>
<li>Correlation: Scientists may find that two variables are correlated. They may be related, but it doesn't mean that one is causing the other. It could be a coincidence, or perhaps a third variable is causing both of the other two.</li>
<li>Causation: Lots of correlative evidence can lead to a stronger case that something is causing something else. It is often combined with systematically ruling out other possibles causes. However, the best way to show causation is to perform a controlled experiment.</li>
</ul>

Correlation is not causation

Studies can suffer from selection bias when people are recruited from a specific group that are not representative of the whole. Scientists select a smaller group to study, but the chosen set isn't random enough and is therefore somehow biased in favor of a specific outcome of the study. 
Selection bias can also occur when certain types of people are more likely to want to be involved or are more committed to staying in a longer experiment.

Selection bias

In a peer review system, independent, anonymous experts read over a paper submitted to a journal. They can recommend revisions to the text, new experiments that should be added, or even that the journal shouldn't publish the paper.
But reviewers aren't asked to ensure that the results are absolutely correct. It would be too time-consuming and impractical. That means that a peer review is mostly beneficial, but not perfect.

Correlation is not causation

Similar ideas to Correlation is not causation

Receiving feedback is a skill