First of all, what is A/B split testing? In basic terms A/B split testing is a variety of marketing testing where customers randomly receive almost identical, yet slightly different, test samples.

This method can efficiently be used in various niches such as brochures, email campaigns and landing pages. A/B split testing can help you to improve conversion rates, better understand visitor behavior, test out new products and much more. Email newsletter services like Mailchimp have already implemented A/B split testing in their feature range.

There are plenty of useful tools already made like Google Website Optimizer which cost you nothing. Continue reading to find comprehensive guides on A/B split testing, exciting case studies, useful tools and resources.

Know What to Test

The goal of A/B split testing is to improve your website’s performance, not to build a whole website. Find out what exactly you want to test before testing it. Help out with case studies and have a closer look at your own website.

If you’re not sure start with a bit. Some of the most popular things to test are call to action buttons, headlines, forms, text length and layout.

Give it Time

Don’t jump to any conclusions after you’ve run the test for a couple of days. A solid A/B split test might even take a month. Giving up too early on a test will only cost you time and the results will be marginal.

Once you start seeing a solid path where the results are going you can close the test or start a new one. There are also calculators available to test results confidence if you’re unsure.

Analyze Results

You’ve run your first A/B test, great, now what? Don’t take it for granted that the test version is always a better solution. Examine your results carefully and see where the flaws were and what you could improve even more. One test often won’t be enough.

You’ll have to do several or even a multivariate one. Once you’ve done a decent amount of tests you’ll see where the actual results are going.

Mix it up

Like you just found out – one test won’t be enough. Be creative with your tests. Don’t stick to the ordinary A/B option, 50/50 audience. Create various templates and vary the proportions. Change something else in each test. At the end you’ll have rich and valuable material to analyze.

However don’t get too fancy. Don’t torture your regular visitors with too many testing. Address tests to your new visitors then see how they adapt. To sum up, follow the guidelines but don’t take them as a must. Be creative with some limitations though.

Articles & Tutorials

  1. The Ultimate Guide To A/B Testing: This article by Paras Chopra, the founder of Visual Website Optimizer, is meant to be the best guide you will ever need for A/B testing.
  2. Split Testing Landing Pages to Boost Conversions: Running split tests on campaign landing pages is an important way to test and optimise conversions. Learn how to use split tests and optimize landing pages.
  3. Using Google Analytics to get insights from A/B testing: In this article Felipe Wesbonk from Traffic Builders explains the integration from VWO (Visual Website Optimizer) in Google Analytics and the way you can use this plugin in making custom reports.
  4. How to Do A/B Split Testing on Your Mailing List: Learn how to perform split testing in your mailing campaigns, check out case study and improve your results.
  5. A Crash Course on A-B Split Testing: Another article giving you an insight in AB testing advantages and disadvantages.
  6. The Pitfalls Of A/B Ad Split Testing, Part 2: Article digging into ad testing and examining some pitfalls of AB ad split testing.
  7. A/B Split Testing vs. Multivariate: Pros & Cons: Which of the two primary testing methods should you employ? Find out pros and cons of each test, when to use them together and why.
  8. Gathering Variables for A/B Split Testing: This article will take a look at elements of a website that should affect its users, teach you what to test and where to gain ideas.
  9. Split Testing: Very detailed and comprehensive article about split testing basics, showing how these tests work, case study and tools to do the job.

Case Studies

  1. Ecommerce website? Bring price and add-to-cart button closer to increase sales: Insight into Taylor Gifts A/B test they recently did to improve add-to-cart clickthrough.
  2. Split testing works. Case study from AgileTask: Short case study showing how call to action buttons can significally improve user reactions.
  3. Writing Decisions: Headline tests on the Highrise signup page: 37signals was experimenting and rotated some headlines and subheads on the Highrise signup page to see if headlines have an effect on signups. Want to know the answer?
  4. Split Testing: How To Increase Your Adsense Earnings 94% Overnight: In this post Brian Armstrong from StartBreakingFree.com shares some tips on using Split Testing to increase his AdSense earnings.
  5. Skype Boosts Homepage Conversion Rate with Radical Simplicity: Skype designed two new homepages to draw user focus to the call to action. See which was the most succesful option.
  6. Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement: This post will show exactly how one start-up improved their homepage conversion rate (visitor to sign-up flow) more than 20%, then 16% again, with a few simple changes and Google Website Optimizer.
  7. A/B Test Case Study: Can Split Test Results Be Trusted?: Little A/B experiment done for the Official Vancouver 2010 Olympic Store to optimize conversion.
  8. How to Double Your Subscriber Growth With 10 Minutes Work: See some split testing examples, learn how to do one yourself and find out which web elements to test and why.
  9. A/B testing browser extensions: do annoying notifications cause uninstallations?: Most people refer to web pages when they talk about A/B testing. However, A/B testing can be useful in other domains too. At InvisibleHand, Visual Website Optimiser was used to test the performance of different versions of a browser extension.

Tools & Resources

  1. Google Website Optimizer: Google Website Optimizer is the easiest and cheapest tool to do A/B and multivariate tests.
  2. Visual Website Optimizer: Visual Website Optimizer is an easy to use A/B testing tool featuring point-and-click test designer and WYSIWYG editor for creating variations. If you aren’t sure if Google Website Optimizer will work for you this is definitely the second best choice.
  3. VWO Free Testing Tools: Visual Website Optimizer also offers four great free tools split testing. These include landing page analyzer, test significance calculator, test duration calculator and A/B Ideafox where you can gain ideas for various A/B tests.
  4. genitify: genitfy enables webpage authors to accelerate their development cycles. It is like A/B testing but it goes beyond it.
  5. A/B Theme Testing Plugin (premium): This WordPress plugin rotates themes to assign them evenly between visitors.
  6. Vanity: Vanity is an experiment driven development framework for Rails. It will help you to create and run A/B tests though it requires some heavy knowledge.
  7. phpScenario: phpScenario is an object-oriented split testing library written in PHP. It’s designed to do simple A/B testing with as little code as possible, but should also be as powerful as you need it to be.
  8. ShrimpTest: ShrimpTest is an A/B testing solution for WordPress under development at Automattic.

This post may contain affiliate links. See our disclosure about affiliate links here.

Written by
Editorial Team
We are 1stWebDesigner and we’re on a mission to help you build a better web. Our team produces content created by web design professionals, for web design professionals.
All Posts Written by Editorial Team