Web Gurus

[skip]

I have a question about google robot.txt file. The following is the question from googles FAQ.


How does the robots.txt file analysis tool work if I have a site located in a subdirectory?

A robots.txt file is only valid when it's located in the root of a site. So, if you are looking at a site in your account that is located in a subdirectory (such as http://www.example.com/mysite/), we show you information on the robots.txt file at the root (http://www.example.com/robots.txt). You may not have access to this file, but we show it to you because the robots.txt file can impact crawling of your subdirectory site and you may want to make sure it's allowing URLs as you expect.


SO I put the txt file in the subcategory lets call it mysite.com/shop then i moved it to the highest folder but my site was reporting errors. Is it ok in the "/shop file or not? haha I am lost.

[themoch]

skip,

you want it in http://www.yoursite.com/robots.txt

if it is in http://www.yoursite.com/shop/robots.txt
it is no good.

basically, you want robots.txt to be in what ever folder contains your "index" file (otherwise known as the homepage file - the page that loads when people enter your web address in) for the main site...

i hope that helps a little....

[skip]

That helps a lot!!! HAHA

I need this shit put into to plain english sometimes. After a 8 hour day on the torch and 3 hours getting lossed in web development it becomes confusing and tiring.

NOW for the second part of my question.

A couple days ago I moved the file from Mysite.com/shop to mysite.com/... That move caused errors in the coding (I think it would be called the PHP)
So I moved the file back and the errors went away. My question now is.

What if I duplicated the file and had it in both locatons so it will be in the right place for the bots and it wont cause errors on my mysite.com/shop/index files

[themoch]

i'm not sure.

most likely if you are using the google tools page then what has happened is the expected file has moved (ie google doesn't know you moved it to the other folder so it says there's an error when it tries to find it on your site), and you need to request a re-scan of your page in order for them to re-find the robots.txt file.

simple explanations of robots.txt files can be found here:

http://www.robotstxt.org/wc/faq.html

and that's really all i can do to help at the moment without seeing your setup.

good luck!

[skip]

You rock!

I will return to the madness after dinner.