Typical Robots.txt Mistakes
Marketing | Programming and Tec

Typical Robots.txt Mistakes

The File Name Contains Upper Case The only possible file name is robots.txt, nor Robots.txt, or ROBOTS. TXT. 2. Using Robot.Txt Instead of Robots.txt Once again, the File must be called robots.txt. 3. Incorrectly Formatted Instructions: Disallow: Googlebot The only correct option is: User-agent: Googlebot Disallow: / 4. Mentioning Several Catalogs in Single ‘Disallow’ Instructions…