|
| Author |
Message |
rich
Joined: 13 Dec 2005
Posts: 34
Location: U.K
|
 Posted: 1/13/2006, 4:41 am Post subject: Your Robots.txt |
 |
|
Again, seem to have read a plethora of posts relating to this subject as far as what ppl say you should have in the robots.txt.
I'd just be interested to know what the PRO's here thought about this. |
|
| Back to top |
|
 |
Thoul
Joined: 14 Sep 2005
Posts: 169
|
| Posted: 2/2/2006, 3:50 pm Post subject: |
|
|
As far as my forums go, I block things that should be private, like admin panels, moderator, and PM functions. I also block subdirectories that shouldn't be accessed directly, like language/ or includes/. Guest posting isn't allowed, so I usually block the posting form, too.
_________________
Phantasy Star: The Fringes of Algo | phpBB Smith |
|
| Back to top |
|
|
BiggDawgg
Joined: 18 Apr 2006
Posts: 12
|
|
| Back to top |
|
|
dingointeractive
Joined: 18 Apr 2006
Posts: 10
Location: Presque Isle, maine
|
| Posted: 5/6/2006, 6:44 am Post subject: |
|
|
If you want indepth info on this check this link
http://www.robotstxt.org/
_________________
Looking for mods in all my forums PM me for details
Hosting Forum |
|
| Back to top |
|
|
|
