Improve the Accuracy of Your A/B Tests With These Email Marketing Best Practices

We push testing in this email marketing blog. A lot. For good reason. Testing is how you learn what works and what doesn’t. Testing is how you continually improve your email marketing.

However, there’s an aspect I hadn’t considered before with the seemingly simple A/B split test: knowing you can trust your results! Is a 2% difference really a 2% difference or only an anomaly? Did you test a large enough sample size to get an accurate comparison? How confident can you be?

ExactTarget’s Jay Miller has written up some wonderful advice to help you make fact-based decisions using the data you glean.

First, he says, you must figure out how big the effect you’re trying to measure really is. Are you striving for a 10% improvement or a 100% improvement? You have to know beforehand because that determines the sample size you’ll need. You’ll need a larger one to measure a smaller change and vice versa.

He also addresses the issue of confidence, and the margin for uncertainty you can allow for when testing email marketing results.

It’s easy to read even with the talk of statistics. Plus, he has included three online calculator tools you can use to help you set up your tests, determine your sample size and evaluate the results.

Read more of Jay’s recommendations on email marketing best practices for A/B testing here.

Email Deliverability Best Practices: Can We Say Enough About Testing? No!

Testing is a topic that has come up a lot lately at our email marketing vendor offices. We encourage constant testing all the time. Not to say you should test every single factor every single time. But always be testing by choosing something to test every time you do an email. It could be anything from your From name to the color of your call-to-action button.

Constant testing falls into the category of email deliverability best practices for several reasons. One, anything you can do to improve your email deliverability rate even a fraction is a good thing (more emails delivered, more possible sales!). And once you’ve gotten into the inbox, anything you can do to improve interaction with your recipient is going to improve your email marketing ROI.

Given that we’re focused on testing as an email deliverability best practice right now, MarketingSherpa’s article on email testing mistakes to avoid is quite timely! If you’re setting out to do more testing, good for you! And definitely read the article before you start. It will help you avoid mistakes and learn more (to improve more!) from the tests you run.

Among the mistakes covered are:

Mistake #1. Not having the right analytics behind your tests—As the article says, sometimes you’re not yet ready to start testing! Get your analytics lined up first.

Mistake #2. Not segmenting email lists for tests—“Without segmented lists you don’t get good test results.”

Mistake #3. Stopping tests after one big win—This dovetails perfectly with the message we at ClickMail are focused on right now: You must always be testing! It doesn’t matter if you have a win. What matters is you keep testing.

Mistake #4. Testing too often—OK, you must keep testing, but keep it in check too…something you’ll figure out as you get into it.

Mistake #5. Overlooking email copy tests—Don’t get all caught up in your colors and photos and what’s above the fold or below. The message itself is a crucial component of your email marketing. Test it too.

Mistake #6. Always testing additions, rather than subtractions—What would happen if you took elements away from your email instead of adding? Test and find out!

This is just the briefest of summaries of this informative article. So make it one of your email deliverability best practices to read the full article…right now.

Unless you test.

It’s only by testing that you can continue refining your email marketing campaigns, improving open rates, conversion rates…and ultimately email marketing ROI, right?

The easiest way to test email marketing campaigns is with an A/B split test. An A/B split test means splitting your list into two, then trying two different things with it (like two different subject lines). Whichever one performs better tells you something. Maybe a shorter, more promotional subject line performs better than a longer, dryer one. That tells you your audience might prefer the shorter, more sales-y subject lines, and you can use that knowledge going forward for your next email marketing campaign.

You can test pretty much any email marketing and design component in your email campaigns:

• From names
• Subject lines
• Personalization
• Offers
• Calls to action
• Images
• Delivery days or times
• Click throughs
• Amount of copy
• Type of content
• Html vs. text
• Frequency
• Layout

To do an A/B split test, divide your list into two groups: one is A, one is B. Now change one, just one, thing about the emails between them. (If you have more than one variable, you can’t accurately measure your results.) Then send out your emails to both groups at the same time (to avoid the time variable). Finally, compare your results and learn. That’s all there is to it.

You can either do an A/B test to your entire list (if it’s small), or you can do an A/B to a statistically significant size portion of your list. Your email test group should be at least 10,000 email addresses to be statistically significant.

Wait, there is one more step: Keep on testing. Make it a habit. If you do, you’ll keep on learning, improving, and increasing that email marketing ROI.