Block search engines google, yahoo indexing specific folder using robot.txt

by rajesh 2012-08-16 13:03:35

To block the search engines including google and yahoo can be done using a single file named robot.txt

create a filenamed robot.txt in the websites home folder. The file should be accessible like http://hiox.org/robot.txt


Inside the file add the following code to block the folder

User-agent: *
Disallow: /folder-name/


you can replace the folder-name with your folder name that should not be indexed by the search engines.

we can have multiple such blocks as

User-agent: *
Disallow: /folder-name1/
Disallow: /folder-name2/



931
like
0
dislike
0
mail
flag

You must LOGIN to add comments