Halo !!! Saya Kang Ismet, ini adalah blog tentang AMP HTML dan cara penerapannya

How to add Robots.txt file on Blogger ? Step by Step - Andro Lift


If you do not know What is a robots.txt file? And how to add a custom robots.txt file to Blogger, you can read this article thoroughly.

It is very important for SEO of your blog, it may be completely removed from your blog search engine due to its misuse. You can follow this guide to know the benefits and use of Robots.txt File.

The direct contact of Robots.txt File is linked to your blog's indexing and crawl rate. Generally all search engines have their own robots or bots which crawl your blog and the links in it and index it in the search engine. Because of which, whatever you update in the blog, it also shows in the whole update search engine and controle what it will be crawl in your blog robot.txt.

Add Custom Robots.txt File Blogger

What is robots.txt?

Robots.txt is a text file with some simple code, which gives instruction to web crawlers, indexing and indexing which page to crawl your blog.

This file is saved in any website or blog server so that search engine bots can be managed. It is used primarily to prevent crawling any non-importance webpages (Demo page, Archive, Labels etc).

That is, in simple language, through Robots.txt file, you have the authority of your blog, which page you want to show in search engine and what page you want to hide

Note- Search engine scans the robots.txt file before crawling any web page.

By default Blogger's robots.txt file looks like this, as shown in the screenshot.

To see the Custom Robots file, you can type slash (/) after the homepage URL of your blog and type in robots.txt and press Enter.

robots.txt file in blogger
Now we see what this Robots.txt code means?

User-agent: Mediapartners-Google

This code is for Google AdSense robots, which governs what kind of adsense adsense ads you receive on your blog and website.

User-agent: *

This code means all search engine bots can crawl our blog. Asterisk (*) means all robot (googlebot, yahoo etc).

Disallow: / search

This means that search engine crawl will not occur after your domain name, even before the search keyword will appear.

Ex- https://www.androlift.ooo/search/label/Blogging
This link is the label name (Menu) of our blog. Similarly, the number of labels in our blog will not be indexed.

Allow: /

This code helps you index by crawling the homepage of your blog.

Sitemap

Sitemap gives the search engine information about all the existing and new posts in our blog. This means that whenever the web crawler scans our robots.txt file, the sitemap will get information from all of our published posts.

Robots.txt File Blogger How do I add

To add a custom robots.txt file to Blogger I first of all will be able to login to blogger and get into the Dashboard.

- Now you can go to Settings and click on the Search Preferences option.

- In this page, you will find a section called Crawlers and indexing, where there will be an option of Custom robots.txt [Disabled]. To enable this, you have to click on Edit and tick on Yes.

After this, copy the code below to paste in the box of Custom robots.txt and click on Save Changes

Copy Robots.txt code

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: / search

Allow: /

Sitemap: http://example.com/sitemap.xml

Note- http://example.com/sitemap.xml Change the Sitemap inside your Sitemap.

Add robots.txt file in blogger

I have explained the complete process of adding custom Robots.txt file to your Blogger, by following it, you can also add robots.txt file to your blog.

If you want to hide a post or page from here, then you should have a proper knowledge of robots.txt because the misuse of it can seo your blog completely. Do not tamper with its settings without any knowledge at all.

If you have any questions related to this topic then you can ask in the comment and you can like our Facebook page (Andro Lift) for blogging tips of this method.