How Do You Know If You Have a Losing or Inconclusive A/B Test? - Deepstash
Onboarding Matters

Learn more about marketingandsales with this collection

How to create a successful onboarding process

Why onboarding is crucial for customer retention

How to measure the success of onboarding

Onboarding Matters

Discover 54 similar ideas in

It takes just

8 mins to read

How Do You Know If You Have a Losing or Inconclusive A/B Test?

How Do You Know If You Have a Losing or Inconclusive A/B Test?

An inconclusive test might mean the numbers are less than a percent off, or neither variation got any traffic at all. When your tests don’t have enough data or if the numbers are too close, they are considered inconclusive or statistically insignificant.

7

63 reads

MORE IDEAS ON THIS

#3-#4

Look Beyond Your Core Metrics

You might have hidden data in your losing test results.

E.g. Look at your “losing” ad to see if it drove less traffic but had higher conversions. If you’d only been looking at traffic & revenue, you might not have noticed this ad works be...

7

9 reads

6 Ways to Leverage Data From Losing or Inconclusive A/B Testing #1-#2

Try Something Really Different

Inconclusive test results could mean your variations are too close. A/B testing can help you see if a small change impacts conversions, but sometimes those tiny tweaks don’t have much impact at all.

Analyze Different Traffic Segm...

7

16 reads

#5-#6

Look for Biases & Get Rid of Them

Look for factors that could have impacted your test.

  • Did you run a promotion?
  • Was it during a traditionally busy/slow season in your industry?
  • Did a competitor’s launch impact your tests?

Try to separa...

7

15 reads

CURATED FROM

IDEAS CURATED BY

phaedrus

Digital Marketer Welcome to my Deepstash where I share ideas I find most valuable for those who wish learn more about digital marketing!

Related collections

Other curated ideas on this topic:

How Do You Plan an A/B Test?

Decide what to test, create two versions, decide on how long to run the test, collect enough data, analyze.

If you’re running an on-site test, you’ll want to think of all the sales-related pieces of your website, and then figure out which elements to split test:

  • headlines

7 Things You Need To Understand About Anxiety And Depression

No, we are not over yet. After discussing the topic with my friend’s Dad (he is a doctor), I’ve made a list of things you need to understand and avoid anxiety attacks (hard to breathe moments).

  1. Think of your anxiety as a biological problem, rather than thinking there is something wron...

Read & Learn

20x Faster

without
deepstash

with
deepstash

with

deepstash

Personalized microlearning

100+ Learning Journeys

Access to 200,000+ ideas

Access to the mobile app

Unlimited idea saving

Unlimited history

Unlimited listening to ideas

Downloading & offline access

Supercharge your mind with one idea per day

Enter your email and spend 1 minute every day to learn something new.

Email

I agree to receive email updates