What is A/B Testing?
A/B testing is the method of comparing two designs to see which one works better. The projects can range from webpage design, mobile interface, marketing ads, and much more.
“AB testing is essentially an experiment where two or more variants of a page are shown to users at random, and statistical analysis is used to determine which variation performs better for a given conversion goal.”
— Optimiely
History
According to LinkedIn, A/B testing can be traced back to the 1700s (Kramer). In its current form, Harvard Business Review states the method started in the 1990s (Gallo, 2017). This shows the success of the method and how it can be used in multiple disciplines.
Tools for A/B Testing
From Google Analytics to other software, there are multiple tools online to help with A/B testing. Some resources include:
Steps of A/B Testing
There are multiple steps to using A/B testing. From getting started to completing the test, here is what you need to know.
Getting Started
- Establish your research question.
- What are you trying to accomplish?
- Study your current analytics.
- Assess the page that needs an update.
- Set a goal.
- What elements are you trying to improve?
Create Variations & Choose a Tool
- Create a new design for the page.
- Apply the change to target what you want to improve.
- Choose the tool or software you want to use.
Run Experiment
- Launch both options.
- Wait for results.
- The test should last several weeks until you have enough data for a clear winner (Patel, 2020).
Analyze Results
- Determine the winner.
- Assess what elements work.
- Analyze the successful elements of the winner.
Do It Again
Using one variant won’t give you the most accurate results. There is always room for improvement when it comes to web design (Kolowich). After you have completed this test, run another one with another slight variant. By doing this over again helps develop the best design that will help keep your users coming back.
Studies with A/B Testing
Here are a few examples of A/B testing in practice.
Performable Button Test
Back in 2011, Joshua Porter and his team conducted an A/B test to determine the right color for the “Get Started Now!” button on their homepage. The two pages tested one homepage with a green button, and the other one with a red button (Porter, 2011). After 2,000 website visits, the red button outperformed the green button by 21% (Porter, 2011). The main goal of the test was to improve the efficiency of the homepage through A/B testing.
Adding FAQ
The design team working for Kiva.org conducted an A/B test to see if adding an FAQ section to their homepage would increase donations. At the end of the test, the homepage with the FAQ section received an 11% increase in contributions than the one without the FAQ section (AB Tasty, 2016). An A/B test was an easy and effective method to increase donations to the site.
Conclusion
A/B testing is straightforward to understand and easy to practice. The data collected from each study will help your website create a better user experience for your readers.
References:
AB Tasty. (2016, February 26). A/B Testing Examples ” 5 A/B Test Case Studies You Can Learn From. Retrieved from https://www.abtasty.com/blog/learn-from-5-ab-test-case-studies/
Gallo, A. (2017, November 27). A Refresher on A/B Testing. Retrieved from https://hbr.org/2017/06/a-refresher-on-ab-testing
Kolowich, L. (n.d.). How to Do A/B Testing: A Checklist You’ll Want to Bookmark. Retrieved from https://blog.hubspot.com/marketing/how-to-do-a-b-testing
Kramer, N. (n.d.). The Origin of A/B Testing. Retrieved from https://www.linkedin.com/pulse/origin-ab-testing-nicolai-kramer-jakobsen
Patel, N. P. N. (2020, March 24). What is A/B Testing in Digital Marketing, How it Works, Tools (Guide). Retrieved from https://www.crazyegg.com/blog/ab-testing/
Porter, J. (2011, August 2). The Button Color A/B Test: Red Beats Green. Retrieved from https://blog.hubspot.com/blog/tabid/6307/bid/20566/the-button-color-a-b-test-red-beats-green.aspx
Jennifer Coffey 
Leave a Reply