185: Email open rates are less reliable but still valuable

Open rates have been a traditional way to measure email campaigns. Whether a recipient opens an email is taken as a measure of the relevance of things the recipient can see in the preview pane. That’s mostly the subject line, but it also includes the from address, and in some cases the preview text, or a portion of the first line of the email.

That metric is becoming less valuable because of changes in the way emails are handled.

Let’s start with a quick discussion of how open rates work in the first instance. How does the sender know that the recipient has opened the email?

The internet is based on a request and response system. Your web browser, or your email client, makes a request to a server to get some resource, like html code or an image. The server can log that request.

When you send an email, your email service provider includes a tracking pixel, which is a tiny, invisible image embedded in the email. When the email is opened – provided you have images turned on – the email client requests this image from the server. The tracking pixel has code that’s unique to the recipient and to the email campaign, so the ESP can identify which recipients have opened which emails.

Anything that disrupts that process makes open rates less reliable.

There are two fairly large things that disrupt that process.

The first is Apple’s Mail Privacy Protection. Apple decided that open information should be private – that the email sender has no business knowing whether or not you opened their email. So – for the Apple users who opt into this program – Apple opens every email and caches all the images before the email even gets to the recipient. That inflates the reported open rate for emails sent to Apple devices, making overall rates less valuable.

You could filter out known Apple users and rely on the open rate of the remaining recipients, but we’re not done with the problems yet.

It’s not just Apple. A lot of corporations do something similar to what Apple does. They’re concerned about malware, viruses, and other security threats to their networks, so they create a firewall that emails have to get through before they’re delivered to the recipient. These systems often open every email and click on every link before the email is delivered. Once again, that messes with your open rates.

So what do you do about this?

First, don’t abandon open rates altogether. The absolute value of the open rate has declined because many of the opens are coming from these automated systems and not from human users. But the open rate still has a relative value. You can still test two subject lines and choose the one that gets the higher open rate, because even if 30 percent of the opens are phony, that will affect both panels equally, so the winner is still the winner.

Second, since some percent of your opens are phony, you should make your test panels larger. In other words, if you need two panels of at least a thousand names to get a statistically significant result, you might need to bump that up 30 percent or so to make sure you’re still hitting that target with actual users and not with automated systems.

Third, make sure you’re not misrepresenting things – for example, with your advertisers. Don’t give them open rates as if they mean what they used to mean.

Fourth, talk to your systems people. If you can get timestamps on opens, a smart programmer might be able to tease out the real opens from the programmatic opens.