How to Get Useful Data From Losing and Inconclusive A/B Tests - Deepstash
Onboarding Matters

Learn more about marketingandsales with this collection

How to create a successful onboarding process

Why onboarding is crucial for customer retention

How to measure the success of onboarding

Onboarding Matters

Discover 54 similar ideas in

It takes just

8 mins to read

How Do You Know If You Have a Losing or Inconclusive A/B Test?

How Do You Know If You Have a Losing or Inconclusive A/B Test?

An inconclusive test might mean the numbers are less than a percent off, or neither variation got any traffic at all. When your tests don’t have enough data or if the numbers are too close, they are considered inconclusive or statistically insignificant.

7

62 reads

6 Ways to Leverage Data From Losing or Inconclusive A/B Testing #1-#2

Try Something Really Different

Inconclusive test results could mean your variations are too close. A/B testing can help you see if a small change impacts conversions, but sometimes those tiny tweaks don’t have much impact at all.

Analyze Different Traffic Segments

  • new versus returning customers
  • buyers versus prospects
  • specific pages visited
  • devices used
  • demographic variations
  • locations or languages

7

15 reads

#3-#4

Look Beyond Your Core Metrics

You might have hidden data in your losing test results.

E.g. Look at your “losing” ad to see if it drove less traffic but had higher conversions. If you’d only been looking at traffic & revenue, you might not have noticed this ad works better statistically, if not in rough numbers.

Dig into that data to find out why it drove less traffic & use that to improve your next set.

Remove Junk Data

Test results can be skewed by junk data. Remove that to see trends more clearly & drill down to find crucial trends. Remove:

  • bot traffic.
  • your traffic.
  • competitor traffic.

7

9 reads

#5-#6

Look for Biases & Get Rid of Them

Look for factors that could have impacted your test.

  • Did you run a promotion?
  • Was it during a traditionally busy/slow season in your industry?
  • Did a competitor’s launch impact your tests?

Try to separate your results from those impacts. If you can’t, try rerunning the test.

Look at how your test was run. Did you randomize who saw which versions? Were both versions mobile-optimized?

Run Your A/B Tests Again

The goal is to continuously improve performance. Once you’ve completed a test & determined a winner (or determined there was none), test again.

7

15 reads

CURATED BY

phaedrus

Digital Marketer Welcome to my Deepstash where I share ideas I find most valuable for those who wish learn more about digital marketing!

stash-superman-illustration

Explore the World’s

Best Ideas

200,000+ ideas on pretty much any topic. Created by the smartest people around & well-organized so you can explore at will.

An Idea for Everything

Explore the biggest library of insights. And we've infused it with powerful filtering tools so you can easily find what you need.

Knowledge Library

Powerful Saving & Organizational Tools

Save ideas for later reading, for personalized stashes, or for remembering it later.

# Personal Growth

Take Your Ideas

Anywhere

Organize your ideas & listen on the go. And with Pro, there are no limits.

Listen on the go

Just press play and we take care of the words.

Never worry about spotty connections

No Internet access? No problem. Within the mobile app, all your ideas are available, even when offline.

Get Organized with Stashes

Ideas for your next work project? Quotes that inspire you? Put them in the right place so you never lose them.

Join

2 Million Stashers

4.8

5,740 Reviews

App Store

4.7

72,690 Reviews

Google Play

Sean Green

Great interesting short snippets of informative articles. Highly recommended to anyone who loves information and lacks patience.

samz905

Don’t look further if you love learning new things. A refreshing concept that provides quick ideas for busy thought leaders.

Shankul Varada

Best app ever! You heard it right. This app has helped me get back on my quest to get things done while equipping myself with knowledge everyday.

Ashley Anthony

This app is LOADED with RELEVANT, HELPFUL, AND EDUCATIONAL material. It is creatively intellectual, yet minimal enough to not overstimulate and create a learning block. I am exceptionally impressed with this app!

Laetitia Berton

I have only been using it for a few days now, but I have found answers to questions I had never consciously formulated, or to problems I face everyday at work or at home. I wish I had found this earlier, highly recommended!

Jamyson Haug

Great for quick bits of information and interesting ideas around whatever topics you are interested in. Visually, it looks great as well.

Giovanna Scalzone

Brilliant. It feels fresh and encouraging. So many interesting pieces of information that are just enough to absorb and apply. So happy I found this.

Ghazala Begum

Even five minutes a day will improve your thinking. I've come across new ideas and learnt to improve existing ways to become more motivated, confident and happier.

Read & Learn

20x Faster

without
deepstash

with
deepstash

with

deepstash

Access to 200,000+ ideas

Access to the mobile app

Unlimited idea saving & library

Unlimited history

Unlimited listening to ideas

Downloading & offline access

Personalized recommendations

Supercharge your mind with one idea per day

Enter your email and spend 1 minute every day to learn something new.

Email

I agree to receive email updates