Snitz Forums 2000
Snitz Forums 2000
Home | Profile | Register | Active Topics | Members | Search | FAQ
Username:
 Password:
Save Password
Forgot your Password?

 All Forums
 Community Forums
 Community Discussions (All other subjects)
 INKTOMI spider/bot		  New Topic  Topic Locked
 Printer Friendly
Author Previous Topic Topic Next Topic  

weeweeslap
Senior Member

USA
1077 Posts
Posted - 19 December 2003 :  02:47:48  Show Profile  Visit weeweeslap's Homepage  Send weeweeslap an AOL message  Send weeweeslap a Yahoo! Message
Is this bot useful to crawl your site? Does it serve a purpose in search engine rankings? What do I benefit from if it lives on my site, literally? If it's useless or near useless, can someone make me a robots.txt file to make it go away from my site, it really hogs my site.
coaster crazy

OneWayMule
Dev. Team Member & Support Moderator

Austria
4969 Posts
Posted - 19 December 2003 :  03:11:32  Show Profile  Visit OneWayMule's Homepage  Send OneWayMule an ICQ Message
The robots.txt to disallow the Inktomi robot to crawl your site would look like:
User-agent: Slurp
Disallow: /


More info about the Inktomi Robot (Slurp):
http://www.robotstxt.org/wc/active/html/slurp.html
http://www.inktomi.com/slurp.html
My MODs:
Birthdays - Custom Policy - F.A.Q. Administration - Forum Rules - Guestbook
Links Manager - MyOwnGoogle - Profile Views - Search Log - WebSearch

Useful stuff:
Forum and MOD Installation - MOD Installation Guide - Snitz v3.4.05 Readme - Free ASP Hosts - Support Snitz
Go to Top of Page

weeweeslap
Senior Member

USA
1077 Posts
Posted - 19 December 2003 :  19:08:08  Show Profile  Visit weeweeslap's Homepage  Send weeweeslap an AOL message  Send weeweeslap a Yahoo! Message
thanks
coaster crazy
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Topic Locked
 Printer Friendly
Jump To:
Snitz Forums 2000 © 2000-2021 Snitz™ Communications Go To Top Of Page
This page was generated in 0.27 seconds. Powered By: Snitz Forums 2000 Version 3.4.07