The sense and nonsense of XML sitemaps

Fact: if your website is set up the right way, you shouldn’t need an XML sitemap at all. You shouldn’t need to think about your category’s XML sitemaps or about including images in your post’s XML sitemap. But why do we keep talking about them like it’s the most important thing ever for SEO? It’s an almost daily subject in our support. That might be, because it’s a convenient list of all the pages on your website. It makes sense that Google is able to crawl all pages of your website if you list them on a page, right?

Google is almost human

Over the last years, we have been talking a lot about Google becoming more ‘human’, so to say. Google is quite good at mimicking the user’s behavior on a website and uses this knowledge in their ranking methods. If your website is user-friendly and gives users the answers they were looking for in Google, chances are your website will do well in the search result pages.

Structure is a sitemap within your website

In the process of setting up your website, you should look at the keywords you’d like to address and translate that to a proper site structure. Using, for instance, the internal linking tool in our Yoast SEO plugin, you are able to create structured links to all the pages of your website. That simply means that Google is able to follow all links and find all pages. That means you have set up a great infrastructure within your website for search engines.

But why should I use XML sitemaps in that case?

Sitemaps, both HTML and XML, come in handy when your site structure and internal linking structure really aren’t that good, to be honest. When you’re dealing with a huge, messy inheritance of the previous owner, years of writing (more or less unstructured) content, or if you simply haven’t thought about internal links that much, your XML sitemap is probably a life saver.

In addition to pointing Google to all your content, XML sitemaps can also optimize crawling of your website by a search engine bot. XML sitemaps should include the last modified date. This date will immediately tell a search engine which pages should be crawled and which haven’t changed since the last crawl and can be excluded from this crawl. This is a huge benefit of using XML sitemaps.

Analyzing your XML sitemaps

In Google Search Console’s Sitemap section, you can find errors in your sitemaps. Pages that are linked there, but don’t work. There’s a huge list of possible errors in the Google Search Console Help documentation.

Apart from that, an important thing to analyze is the types of XML sitemaps you have. You can find these in Google Search Console and in the SEO › XML Sitemaps section in our plugin. In WordPress, sitemaps are created for post types and taxonomies, where sometimes you just don’t need an XML sitemap for all of these. Our SEOs come across websites that have XML sitemaps for filter types (in eCommerce shops for instance), or for dimensions and things like that. If these ‘pages’ don’t make sense to the user, by all means, exclude that content type for the search results in our Yoast SEO plugin. Only serve sitemaps that matter.

There is a reason Google included an XML sitemap section in Google Search Console. Google likes to know every page of your website. They want to see everything, to see if it contains interesting information to answer their user’s search queries. Your XML sitemap is like a roadmap to all the different POI’s on your site, to all the tourist attractions. And yes, some are more interesting than others. Last year, the XML sitemaps served by our plugin contained a priority percentage. Heavy users of our plugin sometimes requested an option to alter that percentage and we never got to that. We decided to remove the percentage altogether as it just did not work as intended – on Google’s side. That emphasizes even more, that it’s just a list of pages. A convenient list, nevertheless.

Should every website have an XML sitemap?

Perhaps I have already answered this question. Yes, I think every website should have an XML sitemap. Or multiple XML sitemaps to provide a lot of links in a better format. It’s a way to make sure search engines find every page on your website, no matter how much of a mess you make of your website. But you should really put your best effort in making that XML sitemap an extra and not a necessity.

If the crawlability of your website depends on your XML sitemap, you have a much larger problem on your hands. I really do think so. Hopefully, you can still go back to the drawing board, invest a bit in a good keyword research training. Restructure the site. Use our internal linking tool when going over your most visited pages again and insert the right links. And then, when most of your pages can be reached via your website itself, rely on that nice, comforting XML sitemap to serve Google any forgotten leftovers and help you to further optimize the crawling of your website.

Read more: WordPress SEO tutorial: definite guide to higher ranking »