Compared to something like redesigning your entire website from scratch, headline testing is one of the most pragmatic ways to increase audience engagement. Simply put, it can give you quality data about what resonates with your customers and what makes them click away from your site. Knowing these things can lead you to make improvements to your customer experience and get more traffic in the future.
Naturally, managing a site with a lot of traffic opens the doors to A/B testing opportunities, but what if your property isn’t getting enough hits to begin with? In this case, there are several alternative techniques you can apply without having a single visitor land on your web pages.
5 Simple Headline Testing Methods That Work On Any Site
Getting to the top of the first page of Google Search results depends on many factors, including the perceived quality of your posts, the amount of resources pointing to your pages, and the catchiness of your headlines. All of these factors can be improved by implementing a great headline formula, or by using the right approaches at the right time and under the right circumstances.
Keep in mind that any headline testing methodology will ultimately require some amount of traffic—or an equivalent type of data—to test their real potency. If you’re just getting started and don’t have many visitors yet, you may want to try any or all of the following five headline testing methods.
1. Five-second testing
Five-second testing aims to determine how real audiences will respond to your website design, including the headlines of your posts and/or articles. The approach works by showing your headlines to someone for five seconds and then asking them about the most memorable parts they’re able to recall.
As you can probably tell, this method isn’t based on any observable statistics or verifiable experiments like a properly implemented A/B test. However, it’s still a good starting point for gathering helpful evidence from prospective visitors, and it teaches you about people’s first impressions of your headlines.
That said, five-second testing can also give you information that you may never have gotten otherwise, since it’s fairly rare for a dissatisfied visitor to send you a message about how your headline didn’t grab their attention enough.
In practice, a great way to run this experiment en masse is with an online service like Lyssna’s five-second testing tool. Lyssna’s tool makes it easy thanks to its intuitive testing methodologies and large pool of participants—which you can target by industry, company size, and job function.
During testing, try to determine whether or not your headlines are conveying the right message for your intended audience. For example, if a participant can’t identify the purpose of your pages after viewing it for five seconds, then it’s probably time to rework your copy. You may need to distill your call-to-action (CTA) so that it’s punchier and more clear.
Lastly, it’s also a good idea to ask the participants the same questions for consistency and record their answers for future reference—with permission, of course.
2. Headline analyzer
Unless you’re a deeply gifted freelance writer from Macedonia or something, the chances of writing the perfect headline on your first try are likely very slim. To speed up the revision process and avoid wasting precious time, however, you can use what’s called a headline analyzer.
A headline analyzer is an online tool that grades your headlines based on a variety of different parameters, including sentiment, word and character count, and emotional charge. One such online tool is the headline analyzer from MonsterInsights.
Typically, a tool like this will assign your headline a numerical score between zero and 100. A score of zero probably means your headline was either complete gibberish or nonexistent in the first place, whereas 100 means your headline can be expected to convince users to click on your page when they see it in search results.
To create a headline that will score very poorly, for example, you could make it extremely long (so that it gets truncated in search results) with no emotionally charged words like “shameless” or “silly” (so that it will be less enticing to readers). Instead, you should aim to include several common words, one or two “power” words, and a positive sentiment rather than a neutral one.
Note: the headline of this post, “How To Run Headline Testing Without Any Traffic,” scored 75 out of 100, which is above the tool in question’s threshold for good results.
Continuing with our example headline, both its word and character counts have an appropriate length, meaning it will be displayed fully on Google and other search engines. This gives it a solid chance to attract an above-average number of clicks.
3. Customer surveys
Customer surveys are another great way to test your headlines without relying on existing web traffic. Generally speaking, there are two relevant types of headline customer surveys: those limited to two options, and those with more than two options.
Surveys featuring two options
Two-option surveys allow you to compare two distinct headlines in a one-on-one matchup against each other. You start by coming up with two headlines, then you plug them into a tool like PickFu and wait for the results. Depending on the given tool you use, you should start to see results after the first hour of posting your survey, if not sooner.
Keep in mind that head-to-head surveying and A/B testing are very similar concepts, albeit with one key difference. Namely, while A/B testing typically requires a sizable amount of live traffic to get viable results, two-option surveys are submitted to a wider spectrum of internet users who aren’t necessarily aware of your brand’s existence. This makes A/B testing a more reliable approach in many cases, but it’s also a more difficult experiment to pull off given the prerequisites.
Surveys with multiple options
Surveys that feature multiple options—also known as ranked surveys—are a type of online questionnaire that helps you identify the best headline among closely similar or vastly different options.
These days, the majority of examples reveal that survey respondents tend to prefer descriptive headlines over those that are witty or show signs of clickbait. This is likely because they do a good job of informing the user about the incoming major points and then following up by delivering. Therefore, it’s safe to assume that the average internet audience doesn’t like to be cheated into reading something that was misrepresented by the headline.
4. Crowdsourcing marketplaces
Crowdsourcing marketplaces such as Amazon Mechanical Turk and Microworkers allow you to outsource a small task to thousands of people from all over the world who can complete it remotely.
This kind of practice will vary from platform to platform, but the underlying concepts are mostly the same.
To start testing your headlines with Amazon Mechanical Turk, for example, you first need to create a new project, name the project, and choose your preferred project keywords. Next, you need to set how much you’re willing to pay for someone to complete each task. For reference, the recommended rates for something as simple as picking a favorite headline vary between $0.01 and $0.02, which includes any additional fees by the provider of the crowdsourcing service.
Naturally, you also need to decide how many participants you’d like to include in your project. A good suggestion is to start with 1,000 participants and experiment with that number until you’re satisfied with the results or the project is over.
Keep in mind that some platforms offer a simple user interface (UI) for creating your projects, but others may require editing source code to personalize your survey tasks. Either way, all platforms should have clear FAQ sections to guide you through the entire task-creation process from beginning to end.
Finally, once you’re satisfied with how a task looks, click finish to provide your payment and publish your project. In most cases, the results will encourage you to pick the headline with the most votes by the global audience in your respective survey. That said, you may still want to account for nuance at times.
5. Social media polls
Some of today’s most prominent social media platforms offer marketers a unique chance to test their ideas by leveraging the power of polls. In essence, social media polls are one of the quickest ways to gain insight into your brand’s positioning in the market—including seeing how your new headlines would perform in the real world.
Before going any further, it’s important to remember that polls and surveys do not refer to the same thing. Typically, polls are shorter voting cards that social media users can create to ask one simple question. In most cases, they can be created and completed on-page, meaning the users don’t have to click on a link to take them to a third-party service or a different page on the same site to vote in a poll.
Comparatively, surveys can have multiple questions and usually take longer to complete. This distinction is important because, on average, surveys are abandoned more often than polls by the participating audience. This is especially true if the users aren’t offered any compensation for casting their votes.
In any case, social media polls can be a valuable tool in optimizing your headlines before they go live, and here are some tips on how to do just that:
Set clear objectives
Start by asking questions. Is your goal to optimize the headline of an existing article or to test three new headlines for an upcoming one that has yet to be published? How much time and energy are you willing to devote to finding the so-called perfect headline, and at what point should you stop and move on to other tasks?
Answering these questions will give you a clearer picture of what you’re trying to accomplish with your polls. For example, some common poll objectives include identifying your target audience, testing the number of people who are interested in your new article, and comparing the number of impressions you’d potentially get on your social media polls against a competitor’s account.
Make it concise
Your headline polls should be simple, concise, and should contain no more than two promising headlines to choose from. If you’re working with multiple headlines, you should break them up into multiple polls and make separate posts.
Analyze the polls
Once the votes are in, analyze the results and record them in a spreadsheet so that you can track progress. Did the votes surprise you? In what way were they different than you were expecting them to be? In this step, you’re presented with an excellent opportunity to check your initial goals, compare them with your results, and tweak your future polls to get more precise participant feedback.
Post the results
Lastly, don’t leave the participants wondering about the poll’s final outcome—especially if it was popular. Besides, sharing the results could be as simple as creating a simple follow-up post or enabling the auto-response feature on Facebook or X/Twitter to send the results to every participant once the poll has run its course.
How Much Traffic Is Required For A/B Headline Testing?
It’s no surprise that conducting highly effective conversion optimization tactics requires at least some significant web traffic. That said, the more traffic your site gets, the more A/B experiments you can perform to optimize your headlines.
For startups, niche sites, and smaller ecommerce shops, you can expect to receive fewer than one hundred thousand unique visitors—which roughly translates to 500 conversions each month. Anything below this number is considered low web traffic.
Once again, conducting experiments on a site with low traffic presents its own sets of challenges, including:
- The experiment’s sample size or the amount of traffic
- The uplift, or the difference between the conversion rates in variant A and variant B
For example, if you want to test two CTA-related headlines, you can run an A/B test for several weeks and, say, receive 10,000 hits per headline. In this hypothetical, you’ll get the following results:
- Variant A—200 conversions achieved from a total of 10,000 visitors, equalling a 2% conversion rate
- Variant B—500 conversions achieved from a total of 10,000 visitors, equalling a 5% conversion
In this scenario, variant B is the clear winner, featuring a 150% improvement over variant A.
Naturally, this leads us to believe that having more traffic equals faster results. If your site receives more than 100,000 unique visitors per month, you can expect to get a reliable set of data within one or two weeks at most. Conversely, if you get less than 100,000 visitors, obtaining a statistically reliable sample size would take a lot longer.
Nevertheless, it’s still possible to conduct viable A/B headline testing with low traffic. In this case, you’ll need to have a statistically significant difference in the uplift between variants A and B to draw viable conclusions from the tests.
How do you determine if your A/B test is statistically significant?
Simply put, there’s no ideal amount of traffic that you need on your site before you can carry out an A/B headline test. If you’re unsure, you can always use an A/B test calculator to check the statistical validity of your project before performing the test itself. At a minimum, you’d need at least 1,000 visitors per week on the page you’re testing, coupled with roughly 50 weekly conversions.
A/B testing tips for sites with low traffic
Sites that don’t get very many hits can greatly benefit from assessing their headline performance in Crazy Egg’s A/B testing tool.
The tool is very simple to use and doesn’t require complicated set-ups to test the effectiveness of various page elements—including product descriptions, sales copy, and article headlines.
It works in three steps:
First, specify the type of test you’d like to run.
Second, choose if you want to run automated split testing or perform a manual traffic split if it suits your needs better.
Third, set a specific goal to determine which headline performs the best in the test environment, and deploy it to drive more traffic.
Keep in mind that our A/B testing solution integrates with most major apps and services. It also supports Google Tag Manager and features an intuitive UI to set up your first testing project in minutes. For owners of smaller sites that don’t have the budget, traffic, and expertise needed to carry out complex A/B testing campaigns on their own, this can be a true difference-maker.
Recap
Website traffic is an important factor in A/B headline testing, but it doesn’t have to be the only one. If your site is relatively new or doesn’t get enough hits to conduct a proper headline test, you can opt for some of the following alternative headline testing methods:
- Five-second testing
- Headline analyzer
- Customer surveys
- Crowdsourcing marketplaces
- Social media polls
In the end, the key is to avoid falling into the trap of feeling married to any one particular headline you personally like. Instead, it behooves you to trust the insight you can gain from tools and large quantities of real people.
If you’re interested in learning more, you may also want to check out our guides on FOMO marketing and web hosting services.
