A sitemap is (usually) an XML document, containing a list of pages on your website that you have chosen to tell Google and other search engines to index.
Google often uses the sitemap file as a guide to the pages available on your website — even though it may decide not to index every page you list on your sitemap.
The sitemap also carries information about each page, including when it was created and last modified, and its importance relative to other pages on your site. This speeds up the process of indexing pages.
A sitemap is one of those rare things in SEO that Google has given the big thumbs up to. Normally, Google likes to talk to website owners in cryptic riddles.
However, for sitemaps, Google makes it pretty clear:
“If your site’s pages are properly linked, our web crawlers can usually discover most of your site. Even so, a sitemap can improve the crawling of your site, particularly if your site meets one of the following criteria …”
Which websites need a sitemap?
In “the following criteria”, Google claims that sitemaps are helpful for…
- Large websites
- Websites with lots of content that’s not well linked
- New websites
- Websites with rich media content
It’s clear from the above that the majority of websites should have a sitemap.
Are there any risks to having a sitemap?
Sitemaps are a rare, beautiful, wonderful part of SEO. Having a sitemap poses no risks to website owners — only benefits.
While Google does make a point of saying it cannot guarantee that its bots will even read a sitemap, it also points out:
“In most cases, your site will benefit from having a sitemap, and you’ll never be penalized for having one.”
To sum up, Google has confirmed that you won’t get punished for a sitemap and that it might help your website. With that information, you can see precisely why every website that wants to rank on Google needs one. Whenever I start working with a client on an SEO campaign, I always recommend a sitemap as part of the process.
Should I have an XML sitemap or an HTML sitemap?
The difference is that XML sitemaps are written for search engines and HTML sitemaps are written for humans.
An XML sitemap is specifically written for search engine spiders. A search engine spider can quickly and easily extract all the important pieces of information about your site by looking at the XML file.
An HTML sitemap enables lost human users to find a page on your site that they are looking for. Remember that search engines reward sites that are easy to use and that, in many cases, you can improve your site’s ranking by improving your user experience. It may be the case that you provide a better user experience by providing an HTML sitemap.
The Crazy Egg blog has an HTML sitemap. Look for it in the footer 😉
Additionally, a well-designed HTML sitemap should have an anchor link pointing to every webpage on your website. This ensures that your entire website is search engine crawlable. In theory, a well-designed HTML sitemap would relieve the need of an XML sitemap.
So which sitemap should you use? The easy answer to this question is both. Cater to the search engine spiders with the XML sitemap and to your fleshy human friends with the HTML sitemap. Everyone wins!
Content syndication, duplication and sitemaps
Another reason to have an XML sitemap is to tackle content duplication issues.
It’s not uncommon for websites who publish lots of content to have their content syndicated on other websites.
It’s also not uncommon for website or blog content to be republished without the author’s permission.
Both are examples of content duplication.
Google does not like content duplication. Google’s Panda update penalized websites with duplicate content in a bid to eliminate spammy sites from the search results pages. The update eliminated duplicate content from the search results, only showing the originally published piece of content.
An XML sitemap enables you to show Google that you are the original content creator. The sitemap will show the date the content was published, even down to the hour.
High-quality content
In 2017, we now know that Google greatly values and rewards quality content.
Back in 2015, Google released an update often referred to as “the Quality Update”. Google updated the signals of quality content in its core ranking algorithm. The result was that high-quality content was rewarded, while low-quality and farmed content were more likely to be penalized.
As I mentioned earlier, XML sitemaps are a fast way for you to tell Google when you’ve added new content to your website or when you’ve updated old content. If you’re regularly publishing quality content, you’ll want to let Google know as quickly as possible each time you publish or update, in order to see ranking improvements.
Remember that when we talk about quality content, this doesn’t just refer to the length of an article (though research has shown that long-form content performs better). Although there are no official guidelines, we know that content quality may be judged according to the relevance of content, natural usage of keywords and keyword variations, readability and sentence structure, relevant internal and external links, and content formatting, including heading tags and visual media.
What goes in a sitemap and how do I create one?
Let’s get down to the nitty-gritty.
While you might agree that sitemaps are brilliant, you might be a little intimidated by them because of just how — well — “geeky” they are.
You’re not wrong. I mean, just look at that thing. To some, it’s an object of beauty. To others, it’s more complicated than 360-degree live streaming is to your grandma.
For the lucky ones amongst us, some content management systems (CMS) can auto-generate XML sitemaps. I hear your sigh of relief a mile off. If you’re a WordPress user, I’d recommend the Yoast SEO Plugin, which can generate an XML sitemap.
If you’re not using WordPress and your CMS doesn’t auto-generate sitemaps, there are a number of free tools that can create a sitemap for you. I’d personally recommend Screaming Frog and XML-sitemaps.com, though there are many, many others.
Creating a sitemap XML file with XML-sitemaps.com
First, go to XML-sitemaps.com and paste in your website URL:
Next, let their crawler navigate through your website. If your website has over 500 URLs, you’ll need to setup a paid account. When the crawler is finished, download the “.XML” file.
Creating a sitemap XML file with Screaming Frog
For Screaming Frog to work, you will need the latest Java update for your computer, but once you load it, simply follow the instructions below to produce a sitemap:
- Get it to crawl the website.
- When the crawl is completely finished, click “Create XML Sitemap”.
- Choose which pages you want to include and exclude. You might be thinking that you’d want to include every page, but it’s worth excluding any duplicate content you might have on your website and any thank you pages you have set up for tracking goal completion.
- Enter the last modified date, list the priority of your URLs, input the frequency that your pages are likely to change, and decide whether you want to include your website’s images in your sitemap. All of this is optional and it’s impossible to say whether or not Google or Bing really requires this information. But they might, so you might as well do it.
- Select a folder to save your sitemap to. We recommend calling it “sitemap.xml”.
There are plenty of other sitemap creating tools out there and most of them work in pretty much the same way. As such, this advice should work for most sitemap creating services.
After you have created a sitemap, it’s important that you submit it to both Google Search Console and Bing Webmaster Tools, in order to speed up the process of indexing your site.
Submitting your sitemap to Google Search Console:
To register your sitemap with Google Search Console, follow these steps:
- Check your XML Sitemap is on your server and the URL loads (or use a sitemap testing tool, of which there are many). Usually, you upload your sitemap.xml file to the root directory of your website. It’s quite normal to have it located at: https://mywebsite.com/sitemap.xml
- Log in to Search Console
- Under “Crawl,” choose “Sitemaps”
- Click “Add/Test Sitemap”
- Enter the URL of the sitemap and click “Submit Sitemap”
Submitting your sitemap to Bing Webmaster Tools:
BTW – Bing is giving away $100 of advertising credit for signing up!
To register a sitemap with Bing:
- Check your XML Sitemap is on your server and the URL loads
- Log in to Bing Webmaster Tools
- Click on “Configure My Site” then “Sitemaps”
- Enter the sitemap URL in the “Submit a Sitemap” text box
- Hit “Submit”
As well as registering changes to your content quickly, Google Webmaster Tools has the added benefit of informing you if there are any errors within your sitemap that need fixing.
Wrap-up
Creating an XML sitemap and an HTML sitemap is worthwhile for website owners who want to make sure they’re doing everything they can to please the SEO gods. Though there’s no concrete evidence that a sitemap will benefit your website, there’s a good chance that it will — and with no risk of penalization, you should definitely set one up.
About the Author: Charlie Marchant is head of Digital PR at Exposure Ninja. Her team of content marketers work closely with SEO to ensure their content is generating maximum results. Charlie is the co-author of The Ultimate Guide to Digital PR and Content Marketing.
