Snitz Forums 2000
Snitz Forums 2000
Home | Profile | Register | Active Topics | Members | Search | FAQ
Username:
 Password:
Save Password
Forgot your Password?

 All Forums
 Help Groups for Snitz Forums 2000 Users
 Help: MOD Implementation
 Slurp online		  New Topic  Topic Locked
 Printer Friendly
Author Previous Topic Topic Next Topic  

Bassman
Junior Member

Netherlands
256 Posts
Posted - 07 October 2003 :  14:32:56  Show Profile  Visit Bassman's Homepage
Slurp? Its a search bot.

http://www.inktomi.com/slurp.html

Here the problem, the slurp bot is stuck on my site, it takes up to 5 slurp bots on cal.asp (event calendar) and I have know idea how to stop it.

If slup bots can't get what it what's it starts over searching for something. Here is a screenshot from active the users:



my active users page:

http://www.scoutingforum.com/active_users.asp

The code 
<meta name="robots" content="noindex,nofollow">
is not helping, as you can see the bot keeps coming back...like a fly to sh**

5 Emails to the slurp people did no help, got no response back. Can sombody help, ore tel me how to get slurp from de cal.asp page. All other pages are no problem.

work mule
Senior Member

USA
1358 Posts
Posted - 07 October 2003 :  15:08:26  Show Profile
http://www.inktomi.com/slurp.html

quote:

noindex meta-tag: Slurp obeys the noindex meta-tag. If you place

<META NAME="robots" CONTENT="noindex">

in the head of your web document, Slurp will retrieve the document, but it will not index the document or place it in the search engine's database.


It sounds like they will still visit the page, just not index it. I mean logically, they have to retrieve the page before they see that meta tag.

My guess is you'll need to use the robots.txt file to instruct them to ignore that page. The slurp page has a section for describing that.
Edited by - work mule on 07 October 2003 15:09:05
Go to Top of Page

MarcelG
Retired Support Moderator

Netherlands
2625 Posts
Posted - 04 November 2003 :  09:34:38  Show Profile  Visit MarcelG's Homepage
I have the same issue...I don't mind that slurp indexes my site, but I do have a problem with it trying to index the calendar, as the calendar creates virtually inlimited links...
So when I use the robots.txt file or use the metatag NOINDEX, it will be used for ALL pages...wouldn't it ?
portfolio - linkshrinker - oxle - twitter
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Topic Locked
 Printer Friendly
Jump To:
Snitz Forums 2000 © 2000-2021 Snitz™ Communications Go To Top Of Page
This page was generated in 0.42 seconds. Powered By: Snitz Forums 2000 Version 3.4.07