How to add a robots.txt file to Blogger blog

Google now lets you edit your robot.txt file to blogger. What are Robots.txt files, robots files are plain text files which instruct the search engines what to not index. For example, you can restrict the indexing of some pages of your  blog that you wouldn't like  to appear in the search. that is if bots respect and adhere to the robots Exclusion Standard, big players like Google is likely to follow this but bad  robots that scan the Internet looking for information, the vulnerabilities of web sites, e-mail addresses to spammers will heed the robots.txt file.

Your Robots.txt files are visible and available to anyone not use  them to hide information. How to set robot.txt in blogger ,to do this you must first  move to the new Blogger interface, as they will remove old interface  in the coming months there is no obstacle that you do not move  immediately to new interface . In the new interface, click on the Search Settings and Preferences, click the Custom robots.txt in empty text field, enter the robot.txt 

When you enter text, click the Enable custom content robots.txt? Yes, and Save changes and that's it.
By default robots.txt  for Blogger  looks like this one below
  User-agent:Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: http://www.name of your blog /feeds/posts/default?orderby=updated

23 comments:

  1. Robot.txt is the file that contains the pages which should not be indexed or crawled. But this pages can still be give rank for the keywords. It will have the restrictions for the crawlers. It is usually used to controlled the behaviour of automated robots.

    ReplyDelete
    Replies
    1. you mean we can add this file for our privacy policy page, or disclaimer pages only? if we post robot.txt with every page then google crawl not indexed our pages? please clear me. thank you

      Delete
  2. thanks

    skyneel.blogspot.com

    ReplyDelete
  3. Amazing Blog robot.txt file....

    Thanks for Sharing !!

    ReplyDelete
  4. thanks for sharing.

    ReplyDelete
  5. I like your post and the way you are explaining about robots.txt. It is nice post and contain genuine data for learning how we create robots.txt file for a site.

    ReplyDelete
  6. Thanks !!
    Was wondering how to change it.
    by the way what does the default tag indicate?

    ReplyDelete
  7. thanx by http://myearningmethods.com

    ReplyDelete
  8. can you tell me how to send multiple sitemaps?

    ReplyDelete
  9. hello guys
    this is my blog http://freepillsreviews.blogspot.com/ and I want to increase targeted traffic , and I don't know what is robot txt and what is use of this and and how I increase traffic via this ?

    ReplyDelete
  10. Robot.txt is the file that contains the pages which should not be indexed or crawled. It is basically used to controlled the behavior of automated robots.

    ReplyDelete
  11. Thanks for this article. Traffic on my blog www.funWithPuzzles.com was going down and wrong values in robots.txt was the reason for it.

    I have updated now robots.txt file now.

    ReplyDelete
  12. Very informative post. please carry on.....

    ReplyDelete
  13. Thank you very much www. giasuchuyen. com

    ReplyDelete
  14. very useful article.keep blogging. for some more similar articles visit http://webcodeexpert.blogspot.in/

    ReplyDelete
  15. Thank you for sharing this information regarding blog robots.txt, was very useful

    ReplyDelete
  16. great information

    viewmovienow.blogspot.in

    ReplyDelete
  17. Robot.txt is the file that contains the pages which should not be indexed or crawled. It is basically used to controlled the behavior of automated robots.

    ReplyDelete

Pages