A Beginner's Guide To A/B Testing: An Introduction

A/B testing works by randomly showing two versions of the same asset (ad, website, pop-up, offer, etc.) to different users. The random part is important because this provides more accurate information without skewing the results. 

One version is the “control” group, or the version already in use. The second version changes a single element. You can change multiple elements, but it does make it harder to tell what change made the difference. This is called multivariate testing (more on this later).

• When you see what types of content your audience responds to, you gain insight into who your audience is & what they want.
• A/B tests help to increase conversion rates. Knowing what works gives you data that helps you streamline the conversion process.
• It’s hard to predict what type of content people will respond to. Testing regularly helps you stay ahead of changing consumer behavior.
• When visitors see content they like, they stay on your site longer. Testing to find the type of content your users like will help you create a better site — and one that users want to stay on.

Decide what to test, create two versions, decide on how long to run the test, collect enough data, analyze.

If you’re running an on-site test, you’ll want to think of all the sales-related pieces of your website, and then figure out which elements to split test:

• headlines
• CTA text
• CTA location
• CTA button color or surrounding space
• pop-ups
• featured images
• copy
• the number of fields in a form

With off-site tests, you’re probably testing either an ad or a sales email. Send out two versions and then track which one converts better. Then you can adapt your structure, copy, images, or offers.

1. Decide what feature you want to test.
2. Create two versions of the same ad, landing page, app, etc.
3. Decide how long your test will run. I suggest at least two weeks, but it may be longer or slightly shorter depending on your traffic and industry.
4. Choose a testing tool to help you run your test (more on that later).
5. Launch!
6. After a couple of weeks, take a look at the results. Which version won?
7. Rinse and repeat. A/B testing is most effective when done continually.

Testing more than one thing at a time, such as headlines and calls to action, is called a multi-variate test, and is more complicated to run. 

You’ll also need to consider how your systems can handle split tests as well as have staff on hand able to analyze multiple results and compile the data into digestible amounts. 

Multivariate testing puts a lot more on your plate at once: but it shouldn’t necessarily be avoided. If you have the right procedures in place to handle the extra workload, then go ahead – but if you want a more simplistic approach: one A/B test at a time is just fine.