How to Add your Dynamic Sitemap to your site’s robots.txt

icon1 Posted by Rochelle in How To..., Site Building Strategies on 01 31st, 2008 | 6 responses

You can add your dynamic sitemap’s location to your robots.txt. The reason to do this is so you can tell the search engine/spider where your sitemap.xml file is. This may help index your site quicker.

This idea comes from forum member WebMogul. You can see his instructions here.

 

My instructions to add your dynamic sitemap to your robots.txt:

Step One
 Go to your robots.txt file (click here if you don’t know how to do this or see How To Add a robots.txt if you don’t yet have a robots.txt).
Step Two
 

Copy contents of add-dynamic-sitemap-to-robots-txt.txt and paste contents ABOVE all other code in your robots.txt file. (Click here to see an example of what a robots.txt will look like with the dynamic sitemap added.)

  
 Change the following bits of code before you save your changes:
 
  • Change "sitemap: http://YourSiteURL.com/sitemap.xml" to the URL of the your site’s sitemap.xml.
  
 Save changes.
done

Simple as that!

Rochelle

6 Responses to “How to Add your Dynamic Sitemap to your site’s robots.txt”

  1. Webmogul says:
    February 3, 2008 at 2:28 pm

    Thanks for the pingback Rochelle!

  2. Rochelle says:
    February 4, 2008 at 12:03 pm

    You are very welcome, WebMogul : )

  3. How to Add a Dynamic Sitemap to your Site | Niche Store Strategies says:
    February 4, 2008 at 12:04 pm

    [...] Instructions to add your dynamic sitemap to your robots.txt are located here. [...]

  4. How to Add a Dynamic Sitemap to your Site | Niche Store Strategies says:
    February 24, 2008 at 5:35 pm

    [...] Add the dynamic sitemap to your robots.txt (click here if you don’t yet have a robots.txt). Instructions to add your dynamic sitemap to your robots.txt are located here. [...]

  5. nadiya says:
    April 23, 2008 at 12:51 am

    Hi Rochelle – I am thankful that you have these instructions as they help out immensely. I’m confused about this one ..I assumed all this time that this was being done for the previous version (pre v3) as V3 has a sitemap. Is this dynamic sitemap / robot.txt different then the one we have on the BANS sitemap . .being a newbie I want to make sure I’m not confusing the terminology. Thank you.

  6. Rochelle says:
    April 23, 2008 at 8:37 am

    Nadiya,

    You are correct that BANS v1 and v2 do not automatically create sitemaps and v3 does. So you don’t need to add either Mark’s sitemap or WebMogul’s dynamic sitemap if your site is v3. But you will still want to follow these instructions to add the dynamic sitemap to your robots.txt.

    If that doesn’t answer your question please let me know.

    Rochelle

Leave a Reply