The Whitelist

Improve the Accuracy of Your A/B Tests With These Email Marketing Best Practices

We push testing in this email marketing blog. A lot. For good reason. Testing is how you learn what works and what doesn’t. Testing is how you continually improve your email marketing.

However, there’s an aspect I hadn’t considered before with the seemingly simple A/B split test: knowing you can trust your results! Is a 2% difference really a 2% difference or only an anomaly? Did you test a large enough sample size to get an accurate comparison? How confident can you be?

ExactTarget’s Jay Miller has written up some wonderful advice to help you make fact-based decisions using the data you glean.

First, he says, you must figure out how big the effect you’re trying to measure really is. Are you striving for a 10% improvement or a 100% improvement? You have to know beforehand because that determines the sample size you’ll need. You’ll need a larger one to measure a smaller change and vice versa.

He also addresses the issue of confidence, and the margin for uncertainty you can allow for when testing email marketing results.

It’s easy to read even with the talk of statistics. Plus, he has included three online calculator tools you can use to help you set up your tests, determine your sample size and evaluate the results.

Read more of Jay’s recommendations on email marketing best practices for A/B testing here.

Email Deliverability Best Practices: Can We Say Enough About Testing? No!

Testing is a topic that has come up a lot lately at our email marketing vendor offices. We encourage constant testing all the time. Not to say you should test every single factor every single time. But always be testing by choosing something to test every time you do an email. It could be anything from your From name to the color of your call-to-action button.

Constant testing falls into the category of email deliverability best practices for several reasons. One, anything you can do to improve your email deliverability rate even a fraction is a good thing (more emails delivered, more possible sales!). And once you’ve gotten into the inbox, anything you can do to improve interaction with your recipient is going to improve your email marketing ROI.
(more…)

Yesterday we talked about how to do an A/B split test. Today in MediaPost’s EmailInsider blog, Chad White talks about the importance of A/B tests…timely! But he also goes further in suggesting some unusual email marketing elements to test.

Savvy email marketers know there’s a wealth of information to be gleaned from their email service provider’s (ESP’s) reporting tools. But despite the depth of that data, you’re not learning anything new. If your open rate is 32%, for example, then that’s your open rate. It might be higher or lower than your last campaign, but you won’t know why.

Unless you test.

(more…)