CMSBuffet.com
Helping Our Clients Achieve Their Goals

Robots.txt

Robots.txt file does not improve your search engine positioning.
It provides Search Engines robots information concerning which files you will not allow to be crawled and indexed in the search engines.
When the search engine robot crawls your site it looks for the robots.txt file.
If it doesn't find one it assumes automatically that it may crawl and index the entire site.

This allows all robots to crawl all files.
User-agent: *
Disallow:

This Disallows all robots to crawl a folder called /cmsbuffet/ .
User-agent: *
Disallow: /cmsbuffet/

This web site validates your Robots.txt
http://tool.motoricerca.info/robots-checker.phtml

 

 

Here is the latest explanation from Matt Cutts:

Choose Font Size:

Recently Launched Sites

NyGates.com

NY Gates

New York is the greatest city in the world, but we know all too well that you can never be too careful when it comes to security. Business owners throughout the New York City rely on Rolling Gates, Roll Up Gates and many different types of Security Gates to keep their businesses safe and protected 24/7.

Contact Us:

Phone: 647-477-2992
Email: info @ CmsBuffet.Com
Helping You Achieve Web Visibility.
Follow Us On Twitter