Explore the World's Best Ideas
Join today and uncover 100+ curated journeys from 50+ topics. Unlock access to our mobile app with extensive features.
A/B Testing
A/B testing involves splitting your audience/user base into two groups. The two groups receive two versions of a feature. After a specified time, the product manager will look at the data gathered to know which version should be used for the entire user base.
The point of running a test is to use data to best improve your product.
16
155 reads
Develop a hypothesis
- The first step is to decide what you're trying to find out. Base your hypothesis on real pain points within your product. Find something that you need to fix and consider ways to fix it.
- Decide how you'll be measuring the results.
17
153 reads
Set the parameters for the test
Before you deploy the test, you need to make sure it meets the minimum acceptance criteria.
Every test in your backlog needs to have a detailed:
- Hypothesis
- Definition
- Implementation plan
- Expected outcome
Once you've set the minimum criteria, you can deploy.
17
140 reads
A/B Testing vs Multivariate Testing
These two tests are not opposites, but two ways of approaching the same problem - the need to test your ideas.
Multivariate testing is used if you have many variables to test at once and if your userbase is big enough to create multiple segments.
17
136 reads
What to Do With The Data
The biggest mistake you can make after running an A/B test is to just pick the winner and forget about the test.
Testing is about understanding the reason why. Your users have given you more information about themselves, and to serve them properly you need to take advantage of that. Also make the data available to other teams as they may gain unique insights from it.
16
128 reads
A/B Testing Tools
If you’re a larger organization, Optimizely may be suitable for deploying powerful tests and managing the insight you glean from them.
If you’re on a tight budget, many developers like to keep it simple with Google experiments.
17
134 reads
IDEAS CURATED BY
Tiffany Miller's ideas are part of this journey:
Learn more about product with this collection
Essential product management skills
How to work effectively with cross-functional teams
How to identify and prioritize customer needs
Related collections
Similar ideas
Read & Learn
20x Faster
without
deepstash
with
deepstash
with
deepstash
Personalized microlearning
—
100+ Learning Journeys
—
Access to 200,000+ ideas
—
Access to the mobile app
—
Unlimited idea saving
—
—
Unlimited history
—
—
Unlimited listening to ideas
—
—
Downloading & offline access
—
—
Supercharge your mind with one idea per day
Enter your email and spend 1 minute every day to learn something new.
I agree to receive email updates