A/B testing is an optimization technique that creates a copy of an existing webpage, allowing you to change a specific feature (like a button or sign-up form) on the copied version and test which page performs better. The better-performing version is kept, which should result in higher clicks, conversions, or traffic.
How A/B Testing Works
To begin A/B testing, use tracking tools to observe how visitors interact with your website and note which features cause them to stop engaging. Then, pick one of those features, create a copy of the webpage, and tweak the element to see which performs better. For example, if visitors aren’t clicking on a button, it might be because of the color. To test this, create two versions of the button by changing the color of one while keeping the other the same. Run both versions simultaneously and measure engagement results over 30 days.
A common testing misstep is testing random web page elements. To ensure a test provides meaningful results, use tracking tools like heatmaps (which show how users engage with specific parts of a webpage), recordings of visitor browsing behavior on your site, and polls to learn how users interact with your site and where problem points are. These help pinpoint the exact web page elements that need testing as well as potential changes that can help them perform better.
Once you know what’s wrong, use a webpage editor—often included with A/B testing services—to easily change the feature causing engagement problems. By clicking on a button within an editor, for example, you should be able to easily change color, size, and font type. You can also drag the button and place it elsewhere. Once you’ve made your adjustments, save the new version. Finally, publish both versions (the new and original) while pushing an equal amount of traffic to both, and measure which performs best.
Most A/B testing tools offer reports that tell you how well your test version performed against the original. For example, it might tell you the new version of your button received a 5% boost in clicks. If it performed better than the original and you’re satisfied with how well it performed, you can publish it as the new version of your webpage and stop the A/B test. If it didn’t perform better or performed the same as the original, you can make different changes and retest them.
“A/B testing is not something that you do just once, have a mini happy dance party, and you move on with life. A/B testing, or testing in general, should be a part of your strategy and your culture. You can choose one A/B test per month. Understand that these changes you are making are incremental and not all tests will make a vast improvement, but if you can run one A/B test per month, you will see the rewards.”
— Bethanie Nonami, Chief Innovation Officer, Marley Nonami
How to Conduct A/B Testing in 7 Steps
To conduct an efficient A/B testing process, begin by learning which website elements (forms, buttons, or pricing tiers, for example) stop customers from taking important steps toward a purchase or similar action. Then, use testing tools like Chrome extensions; webpage editors; and specific, measurable, achievable, relevant, and time-bound (S.M.A.R.T.) goals to get your conversion rates back on track.
Here’s how to do A/B testing in 7 steps:
1. Choose Conversion Rate Optimization & A/B Testing Tools
A conversion rate optimization tool is a way to collect data on how a user engages or browses your site. We recommend starting with a conversion rate optimization (CRO) toolkit because they include tools that help you learn which website elements (buttons, or forms, for example) cause people to pause or drop off your site. Then, an A/B testing tool allows you to change those elements and test new variations.
Here are the CRO tools that save resources in the testing process:
- Session Recordings – Session recordings allow you to watch people engage with your website from the moment they first click on your site to when they exit it. In doing so, you can pinpoint spots that trip them up and keep them from buying or engaging.
- Funnel Analysis – Funnel analysis reports show you in what stage of the buying journey people tend to drop off. This helps you hone messaging and engagement tactics for webpage elements that are designed specifically for that stage.
- Heatmaps – Heatmaps give you a visual representation of where people click, scroll, and move their mouse on your website so you can better understand what people respond to.
- Polls or Surveys – Sometimes it’s easiest to understand why users don’t engage or buy simply by asking. Polls and surveys aim to do just that, reaching out to customers and prospective customers directly about their experience with your site.
- Segmentation Tools – Often, it’s helpful to know what type of visitor is engaging on your site and what type isn’t. Segmentation tools show you demographic data (age, gender, buying behaviors) on website visitors so you know what type of customer you’re capturing.
- A/B Testing – These capabilities allow you to test new versions of website elements against your original elements to learn what changes will boost conversions.
- A WYSIWYG Editor – A what-you-see-is-what-you-get (WYSIWYG) editor allows you to change the design and placement of website elements (like buttons, text, or layout, for example) for testing purposes.
When you use these tools together, you can plan an efficient testing strategy, launch a test on elements that really need testing, understand what changes you need to make from clear reports, and then implement those changes easily to make the journey from first website visit to purchase as easy and smooth as possible.
2. Determine Where & What to Test
Begin by exploring how people interact with your website and which elements (buttons, text, or forms, for example) they use to engage with you or complete a purchase. Then, use funnel analysis tools, heatmaps, session recordings, and polls to narrow in on which steps and elements cause them to drop off your website why.
Watch Users as They Navigate Your Website via Session Recordings
Session recording and heatmap tools allow you to watch your visitors as they navigate your website. A good session recording tool will even allow you to pick a starting point, like a landing page, to get an understanding of what path customers take from that landing page to final purchase.
Once you pick a starting point—a specific landing page, for example—make note of what users interact with as soon as the page loads and the steps they take to learn more information, engage with webpage elements, or buy a product/service. These notes might highlight pieces of content, forms, or CTA buttons.
The problem may be that you have other buttons or calls-to-action that are neglected and you want these CTAs to get more attention. Or, you may find that while most people make it to the shipping and handling form, more than half drop off at this point and don’t complete a purchase.
Narrow in on Problem Conversion Steps Using Funnel Analysis
While watching a few users engage with your site gives you a sense of what’s working and what isn’t, it’s unlikely that you’ll be able to draw major conclusions about problem areas without using a tool that can track all of your site visitor behavior. To do so, set up a funnel analysis tool like Freshmarketer to gather data on all site visitor behavior over a certain period of time.
Each Freshmarketer funnel analysis report gives you an exact percentage of your current drop-off rate at each of your specified steps in the conversion funnel. It may be that when visitors get to your email request page, for example, they only click on the field and input their email 60% of the time. That’s 40% in drop-offs at that stage. Note your current drop-off rate and where in the funnel it significantly drops so you can target that with A/B testing.
Use Heatmaps to Pinpoint Problem Elements in Conversion Steps
Now that you know what conversion steps are problematic, you want to narrow in further on the exact elements in that conversion step that cause people to leave. Heatmaps are the best way to do this; they offer a visual view of where website visitors click, scroll, or move their mouse. The more people click, the darker the coloring. Lighter coloring means fewer people clicked on that spot on your website. This is helpful for pinpointing elements that underperform and so hinder conversions.
For example, if your funnel analysis shows your contact form is where people drop off, use heatmaps to understand where. You may notice that many people click on the first and last name fields, as well as home addresses and phone numbers fields, because the color density is dark. But, on the email address field, the color density lightens, meaning fewer people click on it to fill it out. This may indicate a field malfunction or that this field frustrates people.
Gather User Feedback via Polls
At this point, you have a pretty good idea where people drop off your conversion funnel and what elements are the root cause, but you may not know why. Gathering information about why people behave the way they do is often only doable by asking. But, you don’t just want to ask one person. You want to ask enough people why they’re dropping off or failing to click on a field or button so you can detect a trend. Polls help you do this.
For example, Freshmarketer’s poll and survey feature allows you to trigger polls based on visitor behavior. If you’ve noticed that people leave your website after they click on the shipping and handling page, for example, you can trigger a poll asking them how you can improve your form. Then, you can see all responses in one place to detect trends. If a large percentage of people complained the form was too long, you consider testing shorter versions.
3. Create Conversion-boosting Goals
Now that you know where in the conversion path people most commonly leave, which elements could be the root of the problem, and where your performance currently stands, you can create achievable goals to boost conversions. For the highest likelihood of achieving them, your goals should be specific, measurable, achievable, relevant, and time-bound (also known as S.M.A.R.T. goals). Once you’ve outlined your goals, you can set up A/B testing to achieve them.
Learn the S.M.A.R.T. Goal Method to Goal Setting
S.M.A.R.T. goals are written to be very specific and, as such, help ensure you can accomplish real business initiatives through your available resources, circumstances, and expertise. A S.M.A.R.T goal must state what you will achieve and how you will achieve it using action words, be within your scope to accomplish, and come with a specific target end date.
Write Your S.M.A.R.T. Goal
Writing S.M.A.R.T goals primarily means being specific and ensuring your goals are achievable. For example, you may have found people fail to fill in a certain form that requests email addresses. You would like to boost the number of email addresses you receive via this form so you can add customers to your newsletter contact list and, thereby, grow your customer base.
Here are the S.M.A.R.T. elements that would correspond with your email collection goal:
- Specific & Measurable – Write out how many emails you need to collect to reach, say, a 10% boost in your newsletter contact list.
- Achievable – Make a note of the tools and in-house resources (like staff and budget) you will use to achieve your goal.
- Relevant – Write out why this goal is important to your business. For example, if you know a certain percentage of your newsletter contacts will make a purchase based on what customers have done in the past, you may say it is relevant because you can achieve a revenue boost from newsletter sign-ups.
- Time-Bound – Write when you want to achieve this goal using specific time periods like weeks or months.
As you write, use bold action words and phrases like “I will…,” “Use A/B testing tools to …,” and “Create a 10% boost by…” For examples to model your goal after, read our article on S.M.A.R.T. goal examples. Don’t be afraid to tweak it until you have an action plan for what you want to achieve and how you plan to achieve it. Once you have your S.M.A.R.T goal written, you can use it to guide you as you set up your A/B test.
4. Create A/B Test Versions
A/B testing tools are used to create two variations of a web page element (like a headline), then direct traffic to each to measure which best performs. Use a WYSIWYG editor to change elements you’d like to test. Then, set the metric you want to measure (clicks or signups, for example) and the audience you want to see the test (the visitors of a specific landing page, for example). Finally, launch your test.
Download Free A/B Testing Tools
Freshmarketer has a unique Chrome extension called Freshmarketer-A/B Testing & Heatmap Software, which gives marketers easy access to testing tools right from their Chrome browsers. To download the Freshmarketer Chrome Extension, search for “Freshmarketer” in the Chrome Web Store. Then, download the app, click on it, and sign in using the pop-up window.
Create and Edit a New Variation of Your Webpage
Once you’ve downloaded Freshmarketer’s Chrome extension, you can enter the URL of the webpage you’d like to test in Chrome. Then, click the Freshmarketer Chrome extension button and click “A/B Testing.”
Now, you can begin editing your web page to create a new variation. Simply hover over the different elements on the page. They will highlight as you hover. Click on what you’d like to edit and begin editing by changing text or using the tabs to change the layout, colors, and more. Your editing options will automatically appear when you click on the element you’d like to edit.
By default, this page will be named “Variation 1.” You can create new variations by selecting “Add New Variation” on the “Variations” tab.
Set Your Tracking Goals Based on S.M.A.R.T. Objectives
Before launching your test, you will need to decide what metrics, like clicks or signups, you want to track. Example metrics you can track include clicks, page visits, revenue, form submissions, and more.
Clicks are the easiest goal to track, but other goals require a different process. To track clicks, select the element you’d like to begin tracking clicks for (a button, for example) and select “Track clicks” from the menu that pops up. A text box will appear. Enter your goal name and save it. From there, you will see an arrow above that element to indicate clicks are being tracked.
From there, click “Next | Goals” to set other tracking metrics. Then, select “Add another goal” to add new goals you’d like to track.
Once you’ve selected the goals you’d like to track on this variation, be sure to select one as the primary by clicking the button next to the goal. Only one goal can be set as the primary for a more focused final report, but you can add as many secondary goals as you’d like.
Set Your Audience
To select the audience for your A/B test—which can be a subset of your typical website visitors or a segment defined by demographic information and user behavior—click on “Targeting.” You can choose from a standard audience or a custom audience.
Under the standard audience, you can track referral traffic, social traffic, new visitors, returning visitors, and more. If a standard audience doesn’t list the audience you want to track, click “Custom audience” and combine criteria to create a new audience. Once you’ve chosen an audience, click “Continue.”
Set Your Traffic Allocation
Now it’s time to decide how much of your target audience you want to participate in your A/B test. To do so, select the “Preflight” tab and scroll down to “Set Traffic.” For the most part, you want to leave this at 100% (all of your current website traffic). If, however, you are unsure of your test and want to start with a small sample size to reduce risk, choose a smaller percentage.
If you want to use less than 100% of your audience, be sure to use Freshmarketer’s sample size calculator to verify you audience size will be enough to collect meaningful data; tracking too small an audience will mean you can’t reasonably draw conclusions about buyer behavior.
From there, split your audience equally among all versions of your page, including your original, by setting the percentage you want sent to each. It is recommended that you split your audience equally among all versions to ensure your data isn’t lopsided.
Verify Your Tracking Code & Launch Experiment
On this page, click “Verify” (located next to the tracking code). If the extension tells you that your Freshmarketer isn’t snippet properly integrated with your website, this means you will not be able to track your goals. To fix this, follow the steps in this guide from Freshmarketer. Then, head back to your Freshmarketer tab and select the “Launch Experiment” button on the top right-hand corner of the screen.
5. Calculate Duration of A/B Test
To calculate how long you should run a test, use a duration calculator like the one offered by Freshmarketer. This is important because Google gets upset if it detects traffic is allocated to a webpage other than what appears on search engine results pages for an extended period of time. Running long tests also introduces new variables that skew results, like different shopping seasons. The calculator tells you the minimum run time you need to gather reliable insights.
The beauty of Freshmarketer’s duration calculator is that it’s free and explains exactly how to use it as you input numbers. For example, in the section for inputting your “Baseline conversion rate,” it explains that this is the existing conversion rate of the webpage you’re testing. Simply input your numbers, and watch as the calculator adjusts to reveal the number of days you should run your test.
Freshmarketer’s calculator offers an explanation for what numbers should be input into each calculator field, but if you would like more information, view a more thorough explanation here.
6. Collect Qualitative Data from A/B Tests
While you’re running your A/B test, you want to be sure you supplement quantitative data (such as clicks or signups) with more qualitative data (survey and polls). These help you learn not just what users like about a particular version of your webpage, but why. Be sure to create these to address the specific webpage features you’ve changed, and position them on your site so visitors see them alongside the elements you’re testing.
Create Your Poll Questions
To create your poll questions, open your Freshmarketer web app and click on “Polls and Feedback” from the left-hand menu. From there, choose the type of question you’d like to ask from the “Question Type” drop-down menu. This allows you to choose a long or short-answer question.
Next, begin typing the questions you would like visitors to answer in the text box under “Question Type.” Craft your questions around your observed problem elements. For example, you may select a long-answer question to ask visitors what they liked or didn’t like about a form that’s not performing well. Alternatively, if you’re planning on triggering it when people click out of the page, you can ask why they’re leaving. Click “Add another question” to add questions.
Customize Your Thank-You Page
Once you’ve added all your questions, click the “Thank you for your feedback” link. From there, you can customize the thank you page that appears after a visitor has complete your poll. Add a message you want your participants to see in the “Thank you message” text box. Also, add a URL to direct them to a particular page after they’ve completed the poll. Then, underneath the URL input box, add the text you’d like visitors to see on the CTA button they click to go to the destination URL. Lastly, click “Next.”
Decide Your Poll Design
As you type your poll questions, you can see how your poll will appear via the preview located at the bottom right-hand corner of your screen. Once you’ve added your questions, you can also adjust the theme and button positions of your poll in the “Configuration” tab. Choose from dark, lite, or neutral themes for your poll, as well as right and left button positions.
Set Your Poll to Launch on A/B Test Pages
While polls can be set to appear on specific pages, you can also set poll trigger behavior. This feature allows you to pick what user behavior causes the poll to appear. For example, you may want your poll to appear to visitors as they leave your page or when they scroll to a particular point on a page. Use the drop-down “Trigger Behavior” menu to select from the options available. Then, click “Next.”
Now, it’s time to specify that you want your polls to launch on your test web pages. Type or paste the URL of your test versions in the text box next to “Simple Match.” This will ensure that whenever that your test pages are being browsed and visitors engage in the trigger behavior you set on the previous page, the poll will appear.
Launch Your Poll & Review Results
Now, click “Launch Poll” to make your poll go live on your test webpages. As data comes in, you can view the response reports under the “Polls Report” right-hand menu option in the Freshmarketer dashboard.
7. Assess Your Results & Enact an Action Plan
In this final step, access your A/B testing, polls and feedback, and heatmap reports to learn which of the features you’re testing is working better and why. Freshmarketer offers intuitive and clear reports so you’re not left guessing how variations performed. Read your reports and develop next steps from them. Next steps might be launching a winning version, or deciding to test again for more significant boosts in conversions, and even gathering improvement ideas you can implement site-wide.
Determine Which Version Won (If Any)
When you’ve run your test for the time required to produce significant insights, take a look at Freshmarketer’s available reports. If they tell you that a version converted at a higher percentage than your original version (by earning more clicks, for example), you can either send all your traffic to that version, or create a variation of that version to conduct further testing for even better results. If you choose to do the latter, go back to step two of this guide.
However, if your results show your variations performed worse than the original, you can stick with your original version or test a new version. Given that you haven’t accomplished your S.M.A.R.T goals, we recommend going back to step two and continuing testing new versions until you do.
Lastly, you may find your new version(s) didn’t show a significant boost or drop in conversions. At this point, we recommend going back to step two to continue testing until you inch closer to your S.M.A.R.T goal. As you do, remember that test results often don’t show significant data because the test wasn’t bold enough. So, consider making larger changes, like testing a green button against an orange one instead of testing two buttons of different shades of blue.
Gather Poll Response Insights
Interpreting poll responses means even your losing versions can offer valuable insights, sometimes even sitewide. Look at your poll responses to see how you can improve the elements you’re testing. For example, responders may report your offers aren’t as competitive as you thought they were. If so, your next A/B test might focus on pricing. In this case, begin the process again from step three.
Frequently Asked Questions (FAQs)
What is A/B Testing?
A/B testing is the process of creating different variations of a web page element (like a CTA, title, or form), then sending an equal amount of traffic to each variant simultaneously to see which version performs best. Once you see which one performs better, you can make webpage adjustments to improve visitor engagement.
What Tools Do I Use for A/B Testing?
The tools needed for A/B testing can be broken out into three categories: tracking and observation tools that clearly demonstrate what features are not working on your site, testing tools that allow you to edit page elements and launch different versions to track performance, and analysis tools that show you reports on your testing.
The following are tools that businesses use for A/B testing:
- Freshmarketer: Freshmarketer is a full conversion-rate optimization suite that offers all the tools you need to pinpoint underperforming elements (like CTA buttons or headlines)—including heatmaps, funnel analysis, polls, and session recordings—then test variations of these elements and learn insights to boost performance.
- Unbounce: Unbounce is an A/B testing tool for website landing pages. It offers expertly pre-optimized templates with basic editing and A/B testing capabilities.
- Google Analytics & Google Optimize: These are bare-minimum A/B testing tools that solopreneurs can use for free to create element variations, then test them to see which performs best.
While there are a plethora of testing tools on the market, they all have their own strengths, weaknesses, and specializations. For more information on the top tools available to you, read our A/B testing tools buyer’s guide.
What Is the Difference Between A/B Testing and Multivariate Testing?
A/B testing is creating different versions of a web-page element (like a form). Multivariate testing involves changing many variables and testing to see which overall webpage version performs better. Understanding why one multivariate test version performs better requires much more data than doing the same with A/B test versions. Therefore, multivariate testing is often not suitable for small businesses that lack the traffic and data to run them correctly.
What Is the Difference Between A/B Testing and Split Testing?
A/B testing and split testing are synonymous. Both begin by creating two variations of a web page element, then running them against each other to see which one performs best. This requires splitting website traffic between both pages to see which reaches goals like increased visitors clicking on a CTA or purchasing a product. Once you determine which element performs better you can publish the winning version.
How Much Does A/B Testing Cost?
A/B testing can be conducted for free using Google Analytics and Google Optimize. However, this process can be tedious as the process requires a learning curve. Other A/B and CRO testing tools (like Freshmarketer’s $99/month plan) offer a full suite of tools to help you easily narrow in on what needs testing, then conduct that testing to see what improvements you need to make for maximum website engagement.
The Bottom Line
A/B testing is the process of assessing different variations of an element on a webpage to determine which performs best. Testing helps companies reach their business goals by first learning what website elements (like CTA buttons) users find problematic, then testing new versions of those elements to determine what changes will improve site engagement and sales.
Unfortunately, none of this can be done manually. The good news is, Freshmarketer offers a full conversion-rate-optimization suite that gives users all the tools they need to complete effective testing. At $99 a month, it’s an affordable option for small businesses that rely heavily on website traffic for brand awareness and sales. Plus, sign-up is easy; click below to get started.