The Definitive Primer on A/B Testing Emails in HubSpot

Increasing revenue involves more than just pushing out promotional content and hoping that it piques your audience’s interest. Truly effective sales and marketing teams rely on A/B testing to take the guesswork out of boosting conversion rates and build campaigns that drive results.

With A/B email testing (also known as split testing), two versions of a campaign are created and distributed to two sample groups. In most cases, version A and version B are identical except for one variable, such as a different subject line, call to action placement, or background color. By analyzing how each version performs, sales and marketing teams can determine which approach is more effective and adjust their strategies accordingly.

According to email marketing provider Litmus, brands that A/B test their emails generally see 20% more revenue than those that don’t. Despite this clear upside to A/B testing, Litmus estimates that nearly 40% of brands never or rarely A/B test their broadcast and segmented emails.

So, what are you waiting for? Set up your first HubSpot A/B test and get ready for reduced bounce rates, improved content engagement, and an increase in conversion rates. Read on to learn more about:

1. What to consider when setting up a HubSpot A/B test
2. How to set up HubSpot A/B testing
3. How to analyze your results using a third-party tool like Actiondesk

HubSpot A/B testing 101

When creating a HubSpot A/B testing email, remember what you learned in your middle school science experiments: work with a large sample size and run tests long enough to get statistically significant results. But how large is large enough, and how long is long enough?

• Determining your sample size: To see statistically significant results from an A/B test, you need to have a substantial number of HubSpot contacts in your subscriber list — ideally at least 1,000. You can then create smaller (yet still substantial) test groups from which to secure meaningful feedback before sending the campaign to your entire prospect list. Many sales professionals rely on online sample size calculators to determine how many subscribers to include in their A/B test.
• Determining your testing time frame: How long you run your A/B test depends on your sample size and existing traffic. According to the email experts at Mailchimp, it takes about one hour to determine the winning metric for clicks and two hours to determine the winning metric for opens. You can also count on tools like Actiondesk to determine a statistically significant winner.
• Keeping it consistent: While many elements of an email can be tested (different headers, colors, etc.), focus on testing one variable at a time. Make sure that both versions are sent out simultaneously and run for the same length of time. Try to not change any experiment settings as the test is running, as doing so can significantly skew your test results.

How to conduct HubSpot A/B testing

Ready to run your own A/B test? To get started with HubSpot A/B testing, follow these steps:

1. Access the email tool in your HubSpot account by navigating to “Marketing” and then “Email.”

2. Click on “Create email,” select a template, and design version A of your A/B test.

3. Click “Run a test” in the top-left corner.

4. Name your A and B emails so that you can easily differentiate them. For example, names like “Color version” and “Black and white version” may be helpful. You’ll design version B shortly — for now you’re just naming it.

5. Set the size of your A/B distribution group, select your winning metric (open rate, click rate, or click-through rate), set your test distribution, and hit “Create test.”

6. Toggle between your version A and B emails using the dropdown in the top-left corner. Your version B email will be a duplicate of your version A email until you edit it. Elements you may consider experimenting with include subject lines, key phrasing, and header images. Just remember: only change one variable per A/B test to avoid inconclusive results.

7. Click on the “Settings” tab and fill out the subject line and other necessary information.

8. Click on the “Send or schedule” tab and select your recipients. Hit “Review and send” or schedule your test for a later date.

9. Sit back and wait for the results to roll in!

The A/B testing results are in. Now what?

When your test period concludes, HubSpot will automatically send the winning email to the rest of your recipients. But that doesn’t mean the A/B testing process is over. Now it’s time to dig into the results and unearth key insights that will inform your future campaign strategies. A thorough review of your HubSpot A/B test usually involves statistical methods and analysis. This process can be extremely time-consuming, but not with Actiondesk.

Actiondesk is a spreadsheet interface that can pull email performance data into easy-to-understand reports, helping you quickly analyze your HubSpot A/B tests and reveal key marketing gems that will make your campaigns shine. You can also use the tool to compare multiple A/B tests at once, saving you from having to spend hours jumping between screens to analyze all the components of a successful email.

With Actiondesk, determining the subject lines, design elements, and calls to action that will resonate the most with your prospects is easier than ever. Try Actiondesk for free today and start building campaigns that work.