blog

When you’ve worked hard on an email campaign, you want to know how well it did. Depending on your CRM/email service, you probably have a plethora of statistics to consider: send rate, open rate, click-through rate, hard fails, soft fails… the list is either fantastic or overwhelming, depending on your point of view. While it’s great to have lots of data, it’s only useful if you know what to look at; you typically want to pull some meaningful, simplified picture out of this vast amount of information. Many people like to treat one metric as the end-all, be-all of email success: the open rate.

The open rate is a simple concept with a convoluted behind-the-scenes life. The idea is to count how many people open your email once they receive it. Simple enough. After all, they can’t read your content and take action for your organization if they don’t open the email. So open rate should be pretty important, right?

Not so fast. For starters, everyone has different email habits, so recording a particular action (like opening an email) might not mean what you think. For example, I feel like my inbox is cluttered if I have too many unread messages, so I make a practice of opening everything, just so they get marked as read. I might read the content, I might not. On the other hand, I know people who only open emails when they really want to read them, and keep tons of unread emails sitting around. Just because someone opened your email doesn’t mean that they read it.

Worse, the usefulness is seriously limited by the way open rates are measured. Here’s how it works: when you send an email through a program that tracks open rates, the program embeds a tiny image in the email, usually one transparent pixel. That image lives on your web server; when the email is opened and images are displayed, the server records that someone has asked for that image. That’s an “open”: when that invisible tracking image is displayed.

Let’s think about what could go wrong here:

• Images only work in HTML (rich) emails, not plain text emails. If your recipient only gets text emails (which is more common than you’d think), there’s no image to track. It’s impossible to track open rates on text-only emails.
• Many email clients don’t display images by default. Often, your recipient has to actively choose to display them. If they don’t display the images, you have no way to tell if they opened the email.
• What if your recipient views the email multiple times? They could check it briefly on their iPhone, read it carefully in Outlook at work, and open it again when they get home to finish taking action. This could be interpreted differently by different services.

In the end, it sounds like “open rate” actually means “people who received the email in HTML and opened it with images enabled.” That’s not quite the same. I wouldn’t rely on it as a measure of how many people actually read your email. What is it useful for, then?

• It provides a rough measure of the effectiveness of your subject lines. Remember: when someone chooses to open an email, all they can see is the subject line and the sender.
• It can be a useful point of comparison. Are your supporters more energized about saving the rainforest, or saving the whales? Would they rather donate money or sign a petition? Comparing across multiple emails, you can see which emails your subscribers are more likely (or unlikely) to open. For example, a steady decline in open rate could indicate email fatigue.
• It gauges the approximate number of potential action takers. After all, no one will take action on your campaign if they don’t even open your emails. But, given the caveats above, it is an approximate measure only.

There is no single, magical measure of email success. However, some measures are more useful than others. Your best data will always come from a full analysis of multiple measures, but if you need one at-a-glance metric, consider the click-through rate as a more reliable signal of success. A click-through measures people who click on a link in the email and arrive at your site. (Loren McDonald suggests comparing the number of clicks to opens; you might also compare the number of clicks to the total messages sent successfully, since you can receive clicks from people who opened the email without your system recording it as an open.) The click-through rate tells you how many people who saw your content were engaged enough to follow through and learn more. While not a perfect count of users-who-clicked, it comes closer to its intended purpose than the open rate.

Since your end goal is usually to have the recipient take a specific action (donate, sign a petition, etc), what you really want to know is how many people made it to that goal, and where others dropped off along the way. Different metrics will tell you different pieces of the story. Open rate suggests the successfulness of your subject line; click-throughs suggest whether recipients were influenced by your content; if you can pull in data on how many of your click-throughs actually completed your action, you can gauge the success of your landing pages. Each statistic gives you an idea of where to improve your campaigns in the future. Only by interpreting a range of statistics do you get the full picture what’s going on.  But if you’re looking for one quick measure of success, just take your open rates with a grain of salt.

This entry was posted on Monday, June 30th, 2008 at 2:01 pm and is filed under Marketing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.