How to be a more savvy science reader

Studies can suffer from selection bias when people are recruited from a specific group that are not representative of the whole. Scientists select a smaller group to study, but the chosen set isn't random enough and is therefore somehow biased in favor of a specific outcome of the study.

Selection bias can also occur when certain types of people are more likely to want to be involved or are more committed to staying in a longer experiment.

• Correlation: Scientists may find that two variables are correlated. They may be related, but it doesn't mean that one is causing the other. It could be a coincidence, or perhaps a third variable is causing both of the other two.
• Causation: Lots of correlative evidence can lead to a stronger case that something is causing something else. It is often combined with systematically ruling out other possibles causes. However, the best way to show causation is to perform a controlled experiment.

If you are looking at a clinical trial, a psychology study, or an animal study, it must be a randomized, placebo-controlled, double-blind study.

• Randomized: The participants in the study are randomly placed into the experimental group and the comparison group.
• Placebo-controlled: In medical studies, one comparison group gets a placebo, such as a sugar pill. This is to see the effects of the actual drug. (However, placebo effects can be so strong that they often relieve pain, among other health problems.)
• Double-blind: A study is "blind" if the participants don't know whether they are in the experiment or the control group. A study is "double-blind" if the researchers in personal contact with participants also don't know which treatment they are administering.

In science, "statistically significant" is the effect that can be picked up with a particular statistical tool called a p-value. A good p-value (or calculated probability) is arbitrary and can vary between scientific fields. The cut-off for statistically significant is a p-value of 0.05.

One problem is that if you run a study multiple times or do a whole bunch of different statistical analyses on the same data, your results could look meaningful purely by chance.

One type of conflict of interest is financial. It could be someone who received funding from a company with a vested interest in the study's outcome. Or that person has a relationship with the company that could lead to benefits in the future.

A recent analysis found that from 7 to 32 percent of randomized trials in top medical journals were funded by medical industry sources.

In a peer review system, independent, anonymous experts read over a paper submitted to a journal. They can recommend revisions to the text, new experiments that should be added, or even that the journal shouldn't publish the paper.

But reviewers aren't asked to ensure that the results are absolutely correct. It would be too time-consuming and impractical. That means that a peer review is mostly beneficial, but not perfect.

• The Impact Factor is the most commonly used metric to assess a scientific journal's influence. It counts the number of times a journal's papers have been mentioned in other papers, relative to their own output.
• You can find a journal's Impact Factor in the Journal Citation Reports analysis that appears annually. Otherwise, search for "impact factor." The most prestigious journals have Impact Factors in the high-20s to mid-30s.
• The Impact Factor is controversial. Some science generates more citations, but it doesn't mean they're better. For-profit journals will often publish anything (even without peer review)