First of all, what is A/B split testing? In basic terms A/B split testing is a variety of marketing testing where customers randomly receive almost identical, yet slightly different, test samples. This method can efficiently be used in various niches such as brochures, email campaigns and landing pages. A/B split testing can help you to improve conversion rates, better understand visitor behavior, test out new products and much more. Email newsletter services like Mailchimp have already implemented A/B split testing in their feature range.
More and more people are becoming aware of A/B split testing and not without reason. Case studies and examples show how greatly testing can improve a site’s performance, usability and conversions. What’s more, it costs you comparatively nothing. There are plenty of useful tools already made like Google Website Optimizer which cost you nothing. Continue reading to find comprehensive guides on A/B split testing, exciting case studies, useful tools and resources.
A Few Quick Tips To Get You Started
Know what to test
The goal of A/B split testing is to improve your website’s performance, not to build a whole website. Find out what exactly you want to test before testing it. Help out with case studies and have a closer look at your own website. If you’re not sure start with a bit. Some of the most popular things to test are call to action buttons, headlines, forms, text length and layout.
Give it time
Don’t jump to any conclusions after you’ve run the test for a couple of days. A solid A/B split test might even take a month. Giving up too early on a test will only cost you time and the results will be marginal. Once you start seeing a solid path where the results are going you can close the test or start a new one. There are also calculators available to test results confidence if you’re unsure.
You’ve run your first A/B test, great, now what? Don’t take it for granted that the test version is always a better solution. Examine your results carefully and see where the flaws were and what you could improve even more. One test often won’t be enough. You’ll have to do several or even a multivariate one. Once you’ve done a decent amount of tests you’ll see where the actual results are going.
Mix it up
Like you just found out – one test won’t be enough. Be creative with your tests. Don’t stick to the ordinary A/B option, 50/50 audience. Create various templates and vary the proportions. Change something else in each test. At the end you’ll have rich and valuable material to analyze. However don’t get too fancy. Don’t torture your regular visitors with too many testing. Address tests to your new visitors then see how they adapt. To sum up, follow the guidelines but don’t take them as a must. Be creative with some limitations though.
In the screenshot above you can see Basekit’s original pricing page (on the left) and the redesign (on the right) which increased conversations by 25%. The goal was to increase the number of people who visit the Buy Now page after visiting their Plans and Pricing page.
As you can see the redesigned page is more organized, smoother, cleaner, brighter and more user friendly. This design showed a consistent 25% improvement in conversations for the entire duration of the test.
Even though they accomplished their goal with this redesign, they’re already working on a new redesign to do another split test. As we already discussed – the key to success lots of tests.
This A/B test showed the importance of pricing table design. Remember that pricing table article we did recently? It’s a great example of efficient A/B testing in action.
Articles & Tutorials
This article by Paras Chopra, the founder of Visual Website Optimizer, is meant to be the best guide you will ever need for A/B testing.
Running split tests on campaign landing pages is an important way to test and optimise conversions. Learn how to use split tests and optimize landing pages.
In this article Felipe Wesbonk from Traffic Builders explains the integration from VWO (Visual Website Optimizer) in Google Analytics and the way you can use this plugin in making custom reports.
Learn how to perform split testing in your mailing campaigns, check out case study and improve your results.
Another article giving you an insight in AB testing advantages and disadvantages.
Article digging into ad testing and examining some pitfalls of AB ad split testing
Which of the two primary testing methods should you employ? Find out pros and cons of each test, when to use them together and why.
This article will take a look at elements of a website that should affect its users, teach you what to test and where to gain ideas.
Very detailed and comprehensive article about split testing basics, showing how these tests work, case study and tools to do the job.
10. Fundamentals of A/B and Multivariate Testing
Look at the fundamentals of both A/B and multivariate testing and learn how to create a split test step by step.
Insight into Taylor Gifts A/B test they recently did to improve add-to-cart clickthrough.
Dustin Curtis outlines his experiment with Twitter links and shows the power language has on clickthrough rates.
Short case study showing how call to action buttons can significally improve user reactions.
37signals was experimenting and rotated some headlines and subheads on the Highrise signup page to see if headlines have an effect on signups. Want to know the answer?
In this post Brian Armstrong from StartBreakingFree.com shares some tips on using Split Testing to increase his AdSense earnings.
Skype designed two new homepages to draw user focus to the call to action. See which was the most succesful option.
This post will show exactly how one start-up improved their homepage conversion rate (visitor to sign-up flow) more than 20%, then 16% again, with a few simple changes and Google Website Optimizer.
Little A/B experiment done for the Official Vancouver 2010 Olympic Store to optimize conversion.
See some split testing examples, learn how to do one yourself and find out which web elements to test and why.
Most people refer to web pages when they talk about A/B testing. However, A/B testing can be useful in other domains too. At InvisibleHand, Visual Website Optimiser was used to test the performance of different versions of a browser extension.
Tools & Resources
Google Website Optimizer is the easiest and cheapest tool to do A/B and multivariate tests.
Visual Website Optimizer is an easy to use A/B testing tool featuring point-and-click test designer and WYSIWYG editor for creating variations. If you aren’t sure if Google Website Optimizer will work for you this is definitely the second best choice.
Visual Website Optimizer also offers four great free tools split testing. These include landing page analyzer, test significance calculator, test duration calculator and A/B Ideafox where you can gain ideas for various A/B tests.
genitfy enables webpage authors to accelerate their development cycles. It is like A/B testing but it goes beyond it.
This WordPress plugin rotates themes to assign them evenly between visitors.
Vanity is an experiment driven development framework for Rails. It will help you to create and run A/B tests though it requires some heavy knowledge.
phpScenario is an object-oriented split testing library written in PHP. It’s designed to do simple A/B testing with as little code as possible, but should also be as powerful as you need it to be.
ShrimpTest is an A/B testing solution for WordPress under development at Automattic.
A/B split testing showcase with nearly 150 case studies. Yet you can only view one per week for free.
Showcase of various split tests with screenshots, results and comments.