Ministry of Web Design
Contact Us
CrossRoad Web Studio
At CrossRoad web design studio, we create beautiful, custom websites for churches, ministries and Christian organizations. We believe that, "Whatever you do, work heartily, as for the Lord and not for men". Col 3:23
Sign up for our Newsletter: 

Effective use of robot.txt

You can use robots.txt to provide an autodiscovery mechanism for the spider to find the XML Sitemap file. The search engines can be told to find the file with one simple line in the robots.txt file: Sitemap: sitemap_location The sitemap_location should be the complete URL to the Sitemap, such as http://

You can place this anywhere in your file. For full instructions on how to apply robots.txt, see You may also find it valuable to use Dave Naylor’s robots.txt generation tool to save time and heartache (http://www

You should use great care when making changes to robots.txt. A simple typing error can, for example, suddenly tell the search engines to no longer crawl any part of your site. After updating your robots.txt file it is always a good idea to check it with the Google Webmaster Tools Test Robots.txt tool.

CrossRoad offer church website design.

More web design posts

Back to recent posts

Get in touch The best way to get in touch with us is via email You can also call us from 9:00 to 18:00 Monday through Friday. We will be glad to hear from you.
About Us CrossRoad is established and run by Christian web designers. We offer ministries, Christian organization and churches website design services.