Conversion

What is Hypothesis Testing and How It’s Done – A Complete Guide with Examples!

Disclosure: Our content is reader-supported, which means we earn commissions from links on Crazy Egg. Commissions do not affect our editorial evaluations or opinions.

hypotesis-testing-introduction

When attempting to optimize your web presence for maximum leads and conversions, you may come across terms like hypothesis testing.

While the term sounds like something from a science test, marketers intent on boosting their digital results are increasingly turning to scientific methods to squeeze a little more juice from their online campaigns.

If you want the most leads, revenue, and rate of return, here are the steps to follow. The scientific method just met the digital marketing world, and you’re about to reap the rewards.

Here’s how to use hypothesis testing to boost the results of all of your B2B marketing campaigns.

What is Hypothesis Testing?

Just like you learned in science class, hypothesis testing is the process of making an observation, forming a question based on the information that you’ve gleaned, and then attempting to solve that problem using the scientific method.

That’s the simplified version. The beauty of the scientific method is that it’s gradual and deliberate and each phase is recorded – and your hypothesis continually altered – until you’ve come to a solid enough conclusion.

While using hypothesis testing and the scientific method to get more results from marketing is not a new phenomenon, you are at a distinct advantage over marketers of the past.

That’s because you have a myriad of tools you can use to test and collect relevant data in order to prove (or disprove) the hypotheses you construct.

In this way, marketing can be akin to a science. Your job is to ask the right questions and then devise the proper tests to prove your theories.

The end result, of course, will be a web presence that your audience expects. It attracts leads en masse and churns out conversions while ramping up revenue and ROI.

Sounds amazing, doesn’t it?

Here’s how to start forming hypotheses to make all that happen for your B2B campaigns.

What a Hypothesis Is – And What it Isn’t

A hypothesis is nothing more than a question based on a particular observation that you will then set out to prove.

For a question to be a hypothesis, it must be provable using actual data.

For instance, you can prove if altering a headline will increase conversions by up to 20%. You shouldn’t form a hypothesis that states, “Will changing the title boost conversions?”

In other words, your hypotheses should be concrete, not vague.

However, as you continually test and prove or disprove your hypothesis, you are encouraged to continually alter them and manipulate them until you come to valid and provable conclusions.

To form a proper hypothesis, you must develop an inherent curiosity about the minutiae of your web campaigns. You must be willing to dig into the details.

Here’s a great example.

Let’s say that you have ten blog posts. Two are highly successful. They receive a ton of views, experience extended times on site, and they’re shared on various social channels.

By all accounts, those blogs are certainly ones to emulate.

You observe that the successful blogs have larger images and shorter paragraphs than the other, less successful posts.

Your hypothesis might then be to ask, “Will the other blogs receive just as much attention with larger images and better formatting?”

You can then run tests to gauge the accuracy of your hypothesis. By making the other blogs’ images larger and paragraphs shorter, did more traffic arrive? Did more shares occur?

That’s hypothesis testing in a nutshell.

Why A/B Testing and Multivariate Testing Qualify as Hypothesis Testing

hypotesis-testing-ab-multivariate
Image source: Optimizely

You can see, then, how popular methods like A/B testing and multivariate testing count as hypothesis testing.

Using various tools, you can run countless tests on a variety of web elements to test your questions and prove your theories.

For instance, if you’re running a retargeting ad, you might ask if your image, copy, and demographics are most appropriate. You can then run A/B and multivariate tests to see if there’s any increase in traffic and sales.  

hypotesis-testing-retargeting-ad

Other Hypothesis Testing Examples

Hypothesis: The dud has a headline that is formed like a question. Could altering the headline to a more powerful announcement make all the difference?

You can then A/B test headlines to see if the slower-to-perform landing page does indeed catch up with the others.

  • Your emails are experiencing a low open rate as compared to the past.

Hypothesis: You observe that you started using a new email template around the time the drop occurred. Could changing the template or reverting to plain text help with conversions?

You can then A/B test various email templates and even plain text emails to see if you can bring your traffic back up to par.

  • Your website is experiencing a higher-than-average bounce rate compared to times in the past.

Hypothesis: Would changing the color of your website to a more soothing shade help to keep people on-site longer?

Your images may be too small, as well, and your headlines could use some tightening up.

With multivariate testing, you can test all those elements at once until you determine if the changes perform positively, negatively, or if your results stay about the same.

Notice how all of these hypotheses can be tested with data. That’s the crucial aspect of hypothesis testing. Without data, your questions can never be answered, and you’ll never truly understand the validity of your conclusions.

The Null Hypothesis and the Alternate Hypothesis

Null hypothesis is a term from inferential statistics, of which hypothesis testing is also derived. The null hypothesis is generally a statement that relates to the hypothesis you’re seeking to prove.

For instance, if you are enlarging the images of three blogs to match the high traffic rate of your other two blogs, your null hypothesis might be that no change will occur between the two samples.

This isn’t the result you hope for, but it gives you a frame of reference for that which you are attempting to disprove.

Your hypothesis is that traffic will rise by enlarging images and your null hypothesis is that image size doesn’t really matter at all.

Otherwise referred to as a research hypothesis, an alternate hypothesis is generally the opposite of the null hypothesis.

It seeks to disprove your null hypothesis. For instance, in the example we’ve used, your alternative hypothesis is that merely enlarging the images on three blogs will help it match the attention of the other two.

hypotesis-testing-null-alternate

How to Use Hypothesis Testing for Your Own Benefit

Step 1: Decide What to Test

Here is where you’ll select the web elements you wish to improve. Do you want to improve your lower performing blogs? Get the read and open rates bumped up on your email newsletters? Maybe you simply want more conversions on your website.

Once you have your test subject, your experiments are almost ready to begin.

Step 2: Define Your Hypothesis

This is the crux of hypothesis testing. It’s the question you’ll ask that you will then seek to prove using relevant data.

If your website isn’t converting, you may ask if your copy is persuasive enough. If your emails aren’t being opened, you may wonder if your template or lack thereof is to blame.

The questions you ask will form the basis for the experiments you’ll subsequently engage in.

Step 3: Define Your Variables

Whether you are using A/B testing or multivariate testing, or some other type of hypothesis testing, figure out what changes you want to make and then test with your audience.

You may decide to change the title, record the data, and then make your observations. Or, you might want to change the title, image, and even button color as a multivariate test.

Step 4: Test Your Hypotheses

With your original hypothesis, null hypothesis, and alternative hypothesis fresh on your mind, start the testing process.

There are many tools you can use, such as Crazy Egg, and a host of others. Each offers the ability to live test and even record visitor data in real time.

Step 5: Calculate Your Results, Analyze, & Act

With your experiment ready to go, start testing. Give each test some time to generate adequate data so that you’re not making alterations based on whims or emotion.

Instead, go by the facts. If your hypotheses are indeed proven, you’ll have clear evidence that your alterations work.

If your null or alternate hypotheses are proven instead, go back to the drawing board and find other variables. More importantly, you’ll want to run more tests until you find the ideal combination of elements that appeal to your audience the most.

Q&A About Hypothesis Testing

Q. How long should you allow before calibrating hypothesis testing results?

Ideally, you should allow A/B or multivariate tests to run for at least seven days. For example, you might test Monday through Sunday, then calibrate the results. You can of course go longer than seven, but a week’s worth of data analysis will give you a good foundation to work with.

Q. Should you run tests more than once to really prove your hypotheses?

This is a great question. If you feel that your results may have been a fluke, feel free to run them again, perhaps with different variables.

Hypothesis testing isn’t a practice to be conducted a single time. It’s an ongoing process that can be duplicated and replicated across various channels and over time.

You are encouraged to test as often as possible to give your hypotheses at least 95% certainty.

Q. What variables should I test?

If it can be tested, it should be tested. Visitors to your website may stick around because of your headlines, persuasive copy, image sizes, ease of navigation, or any other reason.

Your job is to be as inquisitive as possible and ask plenty of questions to give your tests real substance.

Once you know what to test and how, you’ll find yourself coming to evidence-based conclusions that boost your results; quite possibly beyond your expectations.

Q. What tools should I use to perform hypothesis testing?

The aforementioned tools like Crazy Egg can help you run live testing at any scale. Run tests for as long as you want and replicate your tests across every variable.

Start Testing with Crazy Egg Now

Speaking of early innovators of web hypothesis testing tools, Crazy Egg offers a free trial.

Use that trial to begin A/B and multivariate testing your own hypotheses. You can then explore heatmaps and scrollmaps of your test variants to see where visitors are pausing, clicking, and taking certain actions.

hypotesis-testing-crazyegg

You can even record visitor sessions in order to witness how people use your site and to identify where any problems or roadblocks may exist.

Conclusion

If your web presence could use any improvement, hypothesis testing is the scientific way to boost your results.

Figure out what you want to improve upon and get creative with your observations. Ask plenty of questions and form hypotheses you can then test.

Gather up and use the right tools, like Crazy Egg, and then gauge your test results. Are your hypotheses proven or did your null hypotheses win out, thus proving your alternate hypotheses?

This makes web testing systematic and, well, scientific, for more accurate digital marketing improvements.

Whether you’re after more traffic, conversions, or revenue and ROI, hypothesis testing should definitely be one of the tools in your marketing toolbox.


Mary is a self-acknowledged numbers nerd who loves using analytics to drive business decisions. She currently consults for a handful of B2B/SaaS startups out of her home office in Austin.

Make your website better. Instantly.

Over 300,000 websites use Crazy Egg to improve what's working, fix what isn't and test new ideas.

Free 30-day Trial