What is a Sitemap (And Do I Need One for SEO)? A Simple Guide to HTML and XML Sitemaps
SEO is a broad term for a long list of best practices and technical recommendations that can improve your site’s rankings in search engines. Creating a sitemap is one of many technical SEO best practices that can help boost your organic traffic.
In this post, we are going to cover what sitemaps are and why they matter, the difference between HTML and XML sitemaps, how to create a sitemap, and how to submit a sitemap to Google.
What is a Sitemap?
Much like a geographical map helps you find places in the real world, a sitemap is a file that lists all the URLs of your website to tell search engines about the organization of your site content.
You can think of a sitemap as a table of contents for both users and search engines that helps them differentiate between your pages, better understand your site hierarchy, and easily navigate through your website. However, there are two different types of sitemaps, and we need to be specific about which we are talking about: the sitemaps that are specifically built for search engines spiders (XML sitemaps), and the visual sitemaps which are generally created for users (HTML sitemaps).
What is an HTML Sitemap?
An HTML sitemap is designed with the intent of guiding visitors and allowing them to easily navigate the pages on your website. HTML sitemaps are typically designed to look like a simple, bulleted outline of the site navigation, with links to each page that use page titles as anchor text. By following the links included in the HTML sitemap, users should be able to find all the information that they are looking for on your website. HTML sitemaps are usually linked to from the website’s footer, at the bottom of the page.
In addition to giving visitors a simple way to access to your website content, they can also provide a secondary crawl path for search engine bots to get to your webpages and read and index your content.
When asked about the function of HTML sitemaps and how much SEO value they add to a website, Google Webmaster Trends Analyst John Mueller said:
“Is there any SEO value in HTML sitemaps? Sometimes. It can definitely make sense to have these kinds of sitemaps, which are essentially a mapping of your category and your detail pages – especially if we can’t crawl a website normally otherwise. If you have a really complicated navigation structure, maybe if you have pages that are almost connected through search forms rather than a logical structure, then having at least one place where we can understand the structure of the site, based on links, that can really help us.”
An HTML sitemap can simply be a page that replicates links that are already available in the top navigation and footer. You don’t have to link to every single webpage from your HTML sitemap. Pages that drive revenue but aren’t accessible via the header/footer navigation should be included to help Googlebot find and crawl them, and to inject a small amount of organic search visibility into those pages.
Here’s a well-organized HTML sitemap example taken from Apple.com that includes a link to all key category and subcategory landing pages:
While HTML sitemaps have limited SEO value, they are still recommended. However, in this post we’re going to focus on the kind of sitemap that is most important for SEO: the XML Sitemap.
What is an XML Sitemap?
Much like an HTML sitemap, an XML sitemap is also a list of the URLs on your site, but this version is built exclusively for search engines bots as a way to guide them to all the pages you want to be indexed. An XML sitemap is a list of webpages on a site written in a specific format (XML or text files marked up with tags) that makes the site information more readable for Googlebot.
In general, Googlebot has two basic jobs:
- Explore the web from page to page
- Record information about the links it visits along the way, and how they’re related
Google uses this information to create search results and determine what searches a particular piece of content is most relevant for. This is where the XML sitemaps come in handy, as creating a Bot-focused sitemap can help them thoroughly crawl your site and gather the information they need to form the SERP (Search Engine Results Page) for each search query.
According to Google Search Console Help Center:
“A sitemap is a file where you provide information about the pages, videos, and other files on your site, and the relationships between them. Search engines like Google read this file to more intelligently crawl your site. A sitemap tells the crawler which files you think are important in your site, and also provides valuable information about these files: for example, for pages, when the page was last updated, how often the page is changed, and any alternate language versions of a page.”
XML sitemaps can include valuable metadata associated with your site’s pages. This metadata allows Googlebot to quickly access information such as when a page was last updated, how often each page gets updated, and the importance of the page relative to other pages on your website. Metadata can also be used to give additional information about the specific types of content you offer on your webpages, such as images and videos.
Here’s an example of an XML sitemap taken from the Costco website:
Does My Website Need a Sitemap?
The short answer is YES. Whether you are planning on building a website, or if you already have a website and are trying to rank higher in the search results, then implementing an HTML and an XML sitemap is essential.
While they are not required, sitemaps are highly recommend as they allow users and spiders to easily find all the content on your website.
Another advantage of creating an XML sitemap is from the standpoint of “crawl budget.” Google uses its resources wisely by allocating a certain budget of Googlebot’s time to websites. Crawl budget represents the number of simultaneous parallel connections Googlebot may use to crawl the site, as well as the time it has to wait between fetches.
Having said that, if you have a large website with hundreds of URLs, and you recently updated just a dozen of your pages, you want Google to focus on the updated pages rather than crawling the entire site and killing your crawl budget. You can update the modification dates for the updated pages in your XML sitemap, signaling Google to focus its efforts only on the new pages.
According to Google, you need to have an XML sitemap if your website meets any of the following criteria:
- Your site is really large. As a result, it’s more likely Google’s web crawlers might overlook crawling some of your new or recently updated pages.
- Your site has a large archive of content pages that are isolated or not well linked to each other. If your site pages do not naturally reference each other, you can list them in a sitemap to ensure that Google does not overlook those pages.
- Your site is new and has few external links to it. Googlebot and other web crawlers crawl the web by following links from one page to another. As a result, Google might not discover your pages if no other sites link to them.
- Your site uses rich media content, is shown in Google News, or uses other sitemap-compatible annotations. Google can take additional information from sitemaps into account for search, where appropriate.
How Do I Create a Sitemap?
There are different ways of creating an XML sitemap. Here are some of the most commonly used options:
- Build an XML map manually. This method allows you customize your sitemap, but it is only recommended for small websites with fewer pages, as it would be very time-consuming and labor-intensive process for larger websites. This method may also require a web developer to implement.
- XML sitemap add-ons provided by your website’s platform. There are many website builders and Content Management System (CMS) platforms out there that offer tools and plugins to automate the process of building an XML sitemap and can help create it quickly and easily. This is typically the best solution, especially for larger sites.
- WordPress: https://wordpress.org/plugins/google-sitemap-generator/
- Magento: https://marketplace.magento.com/weltpixel-module-sitemap.html
- Use an online sitemap generator. This is generally a good option when you need to quickly create a sitemap. You won’t have to install anything extra on your site, and the process is typically quick and simple. For example, xml-sitemaps.com is a free online sitemap generator service that creates an XML sitemap that can be submitted to Google, Bing and other search engines. This tool can also generate an HTML sitemap to allow your website visitors to navigate your site more easily.
How Do I Submit an XML Sitemap to Google?
After creating an XML sitemap, the last step is to submit it to Google Search Console to let Google know about it, so Googlebot can immediately begin crawling your sitemap and indexing your webpages. Here are the step-by-step instructions for how to submit your sitemap to Google Search Console:
- Sign in to your Google Search Console
- In the left rail navigation, select “Sitemaps”:
- On the Google Search Console sitemap page, you will see a list of XML sitemaps that were previously uploaded (if there are any), with some information about submission date, last read date, status (success/errors), and discovered URLs. At the top of this page, you will see an option to add a new sitemap:
- Next, enter your XML sitemap URL (normally something like example.com/sitemap.xml) in the “Add a new sitemap” box and hit SUBMIT.
- Upon completion, check this section of Google Search Console occasionally to make sure there are no errors/warnings detected by Google.
As a side note, it is worth the time and effort to periodically audit your XML sitemap to make sure it includes only URLs that return a 200 OK status code, with self-referencing canonical tags, rather than pages that have a 3xx redirect header response code or pages that are 404ing and no longer exist.
There are two different kinds of sitemaps: HTML (visual) sitemaps, which are intended to be seen by users as well as search engine spiders, and XML sitemaps, which are typically only accessed by search engines. Both kinds of sitemaps are beneficial for SEO, albeit in slightly different ways:
- HTML sitemaps help provide a crawl path to pages that may otherwise be difficult for search engines to find, allowing link value to reach these pages, which may improve their rankings.
- XML sitemaps help ensure that search engines can find and index all pages on your site. They do not pass link value, but they can provide additional information (such as last-modification dates) that may be beneficial in optimizing your use of crawl budget.
The latter is more important for SEO, particularly for large websites. There are several ways to implement XML sitemaps, whether it is manually, though a CMS, or via a third-party tool.
Submitting XML sitemaps to Google is as easy as entering their URLs into Google Search Console. This allows for regular monitoring as well.
Do you need help with creating a new sitemap or auditing an existing sitemap? Our technical SEO team is ready to help – reach out for a free consultation today.
You May Find These Interesting
What Will Future Data Strategies Look Like? Top 10 Takeaways from our “Managing Complex Change” Series Panel
On August 6th, the agency hosted a panel discussion to dive into what the future data ecosystem will look like, given the intense focus on first party data and how the landscape is changing due to the death of the third party cookie, global privacy regulations and...
This article originally appeared on The Stagwell Group's website. A few weeks back, experts from global brand performance agency ForwardPMX and digital-first creatives Code and Theory led a virtual discussion about key strategies and tactics for Conversion Rate...
Taking place in late July 2020, The Stagwell Group’s Transformation Summit offered a moment in time to look back and reflect on the changes the ad and marketing industry has seen since the start of the COVID-19 pandemic. For brands, the most significant of these...