The 5 Times When You Absolutely Must do A/B Testing

The world needs more A/B testing.

Why? Because A/B testing is the most effective way to raise conversion rates. Tragically, according to Econsultancy, a paltry 38% of companies are doing A/B testing.

There’s obviously a problem here.

What I’m proposing in this article are five times when you absolutely must do A/B testing. The five A/B testing times that I’ve listed below are not optional. They are essential.

You’re going to face one or more of these times within the next 6 months. When you do, you’ll be glad you read this article. (Thank me later.)

Source: Placeit.net

1. Do A/B testing when you redesign your website.

Redesign a website is part of life in the online world. Unfortunately, website designs can ruin traffic and conversion rates.

For the designer and developer, redesigning a website is just another day on the job. For the SEO and CRO, it’s apocalyptic. SEOs cringe because a redesign causes an eruption of 404s, lost linkbacks, broken robots.txts, plummeting rank, and a lot of other very bad things that I can’t write about or you will have nightmares.

For example, when Digg redesigned their website, look what happened:

Image from ReadWrite.com

That’s bad.

CROs fear redesigns because conversions start to tank, and revenue drops like an ice cream cone in the hands of a toddler. You feel like face palming your entire life.

Before the redesign, users knew exactly where to go, what to click and how to convert. Suddenly, they show up, and it’s like somebody switched all the furniture around. They’re banging their shins on the coffee table and tripping over the sofa instead of easily converting. It’s like my mom when Facebook “changed things around AGAIN!”

Why are you redesigning your site?

There are plenty of bad reasons to redesign a website, and very few good ones. I’m here to tell you that most reasons for redesigning a website are bad ones. Let me be specific.

Bad Reasons to Redesign Your Site

Here’s a typical list of reasons why a company redesigns their website.

It’s been two years since they redesigned it. In Internet years that’s like forty.
It looks outdated.
There’s a “redesign” line item in the budget.
The designers and developers are bored.
Someone thought it would be a good idea.
The VP of Marketing got persuaded by the attractive design intern.
The CEO didn’t like the color scheme of the old website.
Some over-zealous usability designer drank nine cups of coffee before 10am.
Etc., etc., ad nauseum.

Good Reasons to Redesign Your Site

There are only three good reasons to redesign your site.

You plan to improve traffic.
You plan to improve conversions.
You plan to improve usability.

These three reasons are a cycle of awesomeness. Higher traffic helps to improve conversions. Improved usability also raises conversions. Better usability also improves traffic due to SERP CTRs and lower bounce rate, which, in turn, raise conversion rates, which also lead to higher usability ratings, which improve traffic, etc., etc., ad awesomeum.

What kind of website “redesigns” are we talking about?

A “redesign” runs the gamut from a color scheme tweak to a total CMS-and-up revamp. When should you launch A/B testing with vigor? Here are the types of redesign that require A/B testing.

When you change your website organization or architecture.
When you rebrand.
When you add or subtract substantial amounts of copy.
When you add or subtract significant numbers of pages.
When you make any change to a page in the conversion funnel.
When you change anything on any landing page.

Parting Advice

I’ll share three valuable tidbits of advice to close out this section.

If you must redesign, do A/B and usability testing before you plan the redesign. Use this data as the basis for redesigning your website. The three-pronged goal of a website redesign should be to improve usability, traffic, and conversions. If your redesign efforts don’t intend to accomplish any one of the three goals in the cycle of awesomeness, don’t do it. If you do intend to accomplish any one of these three goals, then you need hard data to make definitive plans on how it’s going to happen.
After redesign, compare your traffic numbers with your conversion rates. Bill Ross of LinchpinSEO.com writes this: “Traffic numbers might be the easiest to analyze at first, but I’d also look at conversion data, as it might be that you are getting slightly less traffic, but it’s converting at a much higher level, thus offsetting the traffic loss.” He brings up a good point — the variation between traffic and conversion. You can lose a little bit in one area as long as you gain equally in another area. Remember my circle of awesomeness?
After you redesign, unleash a flurry of A/B testing. Now that your shiny new website is up and running, it’s time to test the heck out of it. Restrain your analytical passions, and only run one test at a time. Also, exercise patience and keep each test running for at least two weeks.

2. Do A/B testing when you change a service, plugin, or feature.

Sometimes, even oftentimes, you’ll need to change a feature on your website. When you do, it’s time to test

I want to stress specifically that you do A/B testing when the change affects customer data or the purchase process. Here are a few possibilities.

Shopping cart service or plugin.

Your shopping cart is an extremely sensitive part of your website. It’s where money changes hands, where emotions are high, and where you can potentially lose a torrent of traffic. Every tweak to the shopping cart will create a compensatory shift in conversions. Be sure you test these conversions to find out specifically where, how, and why your conversions may be decreasing or improving.

You want a shopping cart that will best advance your conversion goals. If you try one shopping cart service that has 50% abandonment, and another with 60% abandonment, at least you’ll know about this and can take action.

Email service providers or forms.

One of my most important conversions is email signups. For other consultants, content marketers, or SaaS businesses, email signup conversions are clutch. If you change your email provider, you’ll need to do some A/B testing on the signup landing page or whatever capture form you use.

Most quality email services have their own custom signup pages. Some are crap. Some are awesome. If you make a switch and find that you’re converting at 4% as opposed to your previous 8%, you know that something has gone very wrong.

I encourage you to test a variety of services. Robert Tyson of The Tyson Report made this remark: “I use dozens of different forms and they vary at converting traffic into email subscriptions from 1% to 50%.”

3. Do A/B testing when you change prices.

Conversion optimization is one of the quickest ways to increase revenue. Another quick way to raising revenue is to change your price. But it comes with risks. You can either improve your revenue or wreck it with a price change.

How to A/B Test a Price Change

If you are considering a price change, test it first. Simply A/B test the two different price points.

Obviously, there will always be customers who will spring for low prices. Other customers, however, may be more likely to convert if the price is above a certain point. You won’t know unless you test.

From the folks at TipsAndTricks-hq.com, here’s how to A/B test a price change:

The best way to find out how your conversion constant reacts to the price change is to setup a split test (A/B Testing) with your sales page. So you are going to have two copies of your sales page with two different prices for the product (for example: sales page A will offer the product for $47 and sales page B will offer the product for $97). You then serve “sales page A” to 50% of your customers and serve “sales page B” to the other 50% of your customers (preferably randomly). Once you have enough data, you can then setup another split testing scenario with another two price points and gather more data.

This will help you measure the conversion constant for different price points. When you plug these numbers in the formula mentioned above it will help you find out the sweet spot.

A/B Testing Price Changes…And Beyond

This type of test may produce some corollary observations. For example, let’s pretend you’re selling a low-priced software widget. It costs $2. Tons of people are buying it, because it’s so cheap.

But the huge increase in sales has created a crushing amount of customer service. You don’t have time to deal with it. Unfortunately, hiring a customer service agent will cost you more than the number of conversions can sustain. What do you do?

After A/B testing, you decide to raise your price. Now, each widget costs $40. You get fewer sales, but you get way more money per sale. Plus, you don’t have nearly as many customer service inquires. You’re able to devote your time to product development, customer service, and customer retention. Plus, you’re making more money. (You’ve already called the local Lamborghini dealership.)

This is what A/B testing can do. Not only does it give you actionable data on a constrained dataset, but it also provides additional observations that will affect your conversion rates and revenue.

4. Do A/B testing when you think your conversion rates might be screwed.

I have an insatiable thirst for more data, more information, and more statistics on conversion optimization.

I see this kind of thing every day:

My word. 3,000% huh? Let me do that too!

Or this one….

I’m a sucker for stats like that. “Oooh, what kind of trick did this guy pull off to increase his conversions?!”

The fact is, you come across a lot of gems, but you have to rake through a bunch of crap. (The articles above, by the way, are actually pretty good. I don’t place them in the crap category at all.)

The only sure data is going to come from A/B tests. If you read information regarding what your average conversion rate is supposed to be, or you hear about miraculous conversion increases, or just have a sneaking suspicion that your rates are off, it’s time to test.

Don’t make any changes on your site based on someone else’s A/B test. That’s like taking someone else’s eyeglass prescription and having glasses made for yourself. It doesn’t work that way.

You do your own testing to get your own results. Then you take action.

5. Do A/B testing when you just want to raise revenue.

So maybe none of the above points apply to you.

You’re not getting off that easy. I opened this article up with a line that I endorse: The world needs more A/B testing.

If you want to raise your revenue, then you need to A/B test. It’s just that simple. A/B testing is at the core of conversion optimization. You may raise your conversion rates by sheer luck. But that’s about as likely as winning the lottery. (Odds: 1 in 176 million.)

Don’t you want something more likely? Okay, then A/B test. Odds of success: 1 in 1.

The only way to make powerful, effective, and informed decisions is to test. Target an area, make a hypothesis, and test it.

In one year alone, Google conducted 7,000 A/B tests. A/B testing is the practice responsible for Obama’s massive fundraising of $60,000,000. A/B testing is the path to success.