Sitemaps are a must if you plan to start a website or already have started a website. Sitemaps are the maps that Google uses to navigate your website, understand the structure of your website, check how frequently different pages of the site update, and most importantly, take the sitemap and stores it in its index called Caffeine.
The sitemap is crucial for indexing your website, and without indexing, no matter how much content you post or how good the content is, it will never be shown to the users.
With establishing the importance of the XML sitemap, let’s move to the core of this article: creating an XML sitemap and some of the best practices to ensure a smooth, error-free indexing process. Along with that, some things to avoid while creating and submitting your sitemap to Google.
First., let’s create the sitemap.
Creating The Sitemap
A sitemap can be created either automatically or manually. The manual process includes a little bit of coding, very basic but very time-consuming. If there were no other option of creating a sitemap automatically, we would have elaborated on this topic, but since there is a better way, we’ll skip it. But for those who want to know, here’s how to do it. For all those who want to save time, follow these steps:
1. Sitemap Using Yoast
If you use WordPress as your CMS, then Yoast is the fastest and best way to create a sitemap. Don’t worry, and we’ll also include the sitemap generation method for other CMS. To create a sitemap using Yoast, you must have a website with different pages, posts, and the Yoast plugin installed. You can install it from the WordPress plugins pages. It is free of cost.
After the installation, in the WordPress dashboard, you’ll see an “SEO” option at the bottom-left corner. Under that option, click on general. On the General page, select the “Features” option, which will bring a list of different Yoast features. In addition, an image is provided for better understanding.
The seventh option is the XM sitemaps option. All you have to do is click on the question mark just beside it and select the see the sitemap option. This will bring you to a page that contains a directory of your website. A sitemap is just a directory of all the links to the various pages of your website.
2. Sitemap using Screaming Frog
Screaming Frog offers a much more complicated but powerful sitemap creation tool. The downside here is that there is no online version, and the free version is limited to only 500 pages of the website. So why are we recommending it? Because of the functionality. The application is free to download, and here are all the features and what they do.
Screaming Frog features
After you have installed the application:
- Choose the “Crawl” option and then enter the URL of your website (root domain). This will begin the scan of your website.
- Choose the ” Create Sitemap” option after the crawl is 100% complete, choose the “Create Sitemap” option. This will bring up a dialogue box with all the features that you can add to the sitemap.
- Take a look at the image below for a better understanding.
- Pages:
This is the tab that allows you to tweak the sitemap setting, the pages you want to include, and what to exclude. Noniondexed pages are usually the backend of your website, the admin panel, and a few other RSS feed URLs, which are completely unnecessary and should not be included in the sitemap. But for some reason, if you want to include them, tick the box. But we will not recommend it as it increases the crawl load on Googlebots, slowing the process down.
The canonicalized option lets you add the canonicalized versions of the pages. This means that there is already a URL pointing to a particular page of your website, but there is also another different version of the URL that links to the same page. Including canonicalized links increases the sitemap size and reduces crawl execution speed. Paginated URLs and PDFs are other sections of your website that are not needed to be sent to Google for crawling. But it is good to have the control to add them to the sitemap.
The Response code section allows you to add pages or links with the particular response codes. By default, only 2xx codes are included in the sitemap, which means only the reachable, error-free pages are included. But if you want, you can add “No response” and “Blocked by Robots.txt” pages. But, again, we do not recommend it.
3xx codes refer to the redirected URLs, leading to another page. 4xx means that the page does not exist and the link is broken, while 5xx codes refer to the pages that are not available on the server. We’d recommend not ticking any of these boxes.
- Last Modified:
This section allows you to set a custom date in the sitemap that tells the crawlers (googlebots) when the page was updated. You can either let it be “server response” or use a custom date. The sitemap will suggest the date you choose to the crawlers as the date of update. This will cause them to crawl your site on that particular date.
Remember, this is not commanding the crawler but a suggestion. The crawlers will consider it. We recommend keeping this setting as “server response.”
- Priority:
The priority tab tells the search engine crawlers the priority of different pages of your website. Not all pages are equally important to Google. For example, pages with posts that are frequently updated and need to be shown in the search engine are of higher priority. This is also true for news websites, with the latest news being the most important.
So you can set the priority of different pages, with 1 being the most important page and 0 being the least important one. Keeping your posts or products pages as a high priority is recommended, while other pages, such as privacy policy, contact, etc., must be of least priority.
Again, the sitemap will have the “priority” tag, which will suggest Google’s crawlers, not command them.
- Change frequency:
This option allows you to suggest to Google crawlers how often the content of the pages of your website is updated. It is recommended to leave it to default, but you can follow the page priority if you want.
- Images:
You can also suggest Google to crawl the images of your website. But this does not happen natively. It can only be done if you are using an image CDN to store images in the form of caches. The image CDN provider gives a link to the image depository of your website. Add the link in this tab if you want it to be indexed in the sitemap, although it is not necessary.
- Hreflang tag:
The Hreflang tag tells Google that there is an identical copy of the main website, but it is in a different language. This allows Google to apply the same crawl settings. Use this if you have a website version in a different language.
That’s all! Your sitemap is ready. This is why we called it the easiest and fastest method. But what’s the fastest and easiest method of making a sitemap for non-WordPress users? Let’s check that.
3. The Sitemap For Non-WordPress Users
There is no Yoast plugin if you are not using WordPress. But don’t worry, you won’t have to go for the manual method to create the sitemap. There is a very simple and easy online sitemap creation method, but it involves a few extra steps later in sitemap submission.
To create your website’s sitemap, head over to any online sitemap creation website. We prefer XML Sitemap Creator because of the clean interface and quick creation process, but if you want, you can also go for Screaming Frog if there are less than 300 pages on your site.
On the website, enter your root domain, which is the main URL. For example, if your domain is yourwebsite.com, so enter this exact URL and nothing else. Then click on start. Depending on the size of your website, website will generate the sitemap either within a minute, or it might take over an hour.
After the sitemap is created, you’ll have the option to download it. The file will be in .xml format. Store it and save it. It will come in handy when the submission starts.
4. For Wix Users
Many people use Wix as their CMS, mostly eCommerce website owners. The sitemap creation process for Wix is also very simple because Wix automatically creates a sitemap for the site. You can reach the sitemap with the URL yourwebsite.com/sitemap.xml. This link is what you’ll have to use to submit it to Google.
Unfortunately, there is no control that you get for the sitemap editing. Wix is quite restricted in every feature, so this is the best you can do. The process is the same for other closed-structure website managers such as Square Space and Shopify. Just add “sitemap.xml” after your website’s root domain, and you’ll get to the sitemap.
Submitting The Sitemap
Submitting the sitemap is the next step to getting your website indexed. Again, we’ll explain the indexing process is very brief. If you want to read a detailed version of this process, you can read our article on How to Index Website on Google. But here’s the brief version of it:
Log in to Google Search Console, and this is where you can access all the details about your website. If you do not have an account, create one and use your primary email address. Once the account is created, you’ll have to add your website as the property.
Adding the property is easy, but it consumes a lot of time. We have explained the entire process in detail in the linked article.
In Google Search Console, go to Sitemaps, which will be on the left-hand side of the page. Here, you can add the sitemap of your website.
Mostly, for WordPress, Wix, Squarespace, and Shopify users, you can add the link to the sitemap page, and Google will confirm that its crawlers can access the page. Once you submit the link, please wait for Google to index it for a few hours. Google will show any indexing issues in this section of Google Search Console.
But for websites that are not using WordPress or other popular CMS, adding a sitemap is a little more complicated. Remember that .xml file that you downloaded from the sitemap creation website? This file needs to be uploaded to the root directory of your website. So how to do it?
Go to cPanel or any other software that lets you access your website’s files. In that section, select the public.html folder and add the sitemap.xml file in your domain’s public folder. This will create a sitemap on your website, which now can be accessed via the link; yourwebiste.com/sitemap.xml.
After the file has been uploaded, head over to Google Search Console and add the link in the sitemap section. After a few hours, your website’s sitemap will be successfully submitted and read. Congratulations, you created a sitemap and indexed your website on Google.
The same process is applicable for Bing and most of the other search engines. As mentioned above, creating a sitemap for your website and getting it indexed was the entire process. While for some websites, it can be complicated, for most websites using WordPress or other CMS, sitemap generation is very easy.
There are a few things to consider, such as the size of the sitemap. There is a limit of 50 MB for sitemap size, meaning that there should not be more than 50,000 URLs in one sitemap. This is the extreme limit, and no website lets their single sitemap file reach that high, as this increases the crawl load, resulting in very poor crawl frequency.
The best method to avoid this is by creating several fragments of the sitemap, each fragment having not more than 2,000 URLs. This concludes the article.
