Snitz Forums 2000
Snitz Forums 2000
Home | Profile | Register | Active Topics | Members | Search | FAQ
Username:
 Password:
Save Password
Forgot your Password?

 All Forums
 Snitz Forums 2000 MOD-Group
 MOD Add-On Forum (W/Code)
 Auto Update Spider ID MOD : Active Users		  New Topic  Topic Locked
 Printer Friendly
Previous Page | Next Page
Author Previous Topic Topic Next Topic
Page: of 19

acoustika
Junior Member

Denmark
311 Posts
Posted - 14 December 2003 :  19:30:08  Show Profile  Visit acoustika's Homepage  Send acoustika an ICQ Message
I realized another change I made that could have something to do with it maybe. I have this in Forum.asp after all the include files :-)

'Redirect forum 17 til classifieds
if FORUM_ID = 17 then
response.Redirect ("classifieds.asp")
end if
'redirect ****
May The Code Be With You
www.UniqueDk.dk *** The Open Forums of Denmark ***
Edited by - acoustika on 14 December 2003 19:30:33
Go to Top of Page

masterao
Senior Member

Sweden
1678 Posts
Posted - 15 December 2003 :  06:25:05  Show Profile  Visit masterao's Homepage
Search spiders can handle dynamic pages a lot better today than ever, and the topics on my forum can be found by doing a search on Google, for example. I don't use the topicmaker mod on my forum, and Im not familiar with how it works, so I cannot answer anything about it.

How does that redirect code work? Is it only for particular forums, or?
Jan
===========
FR Portal Forums | Active Users 4.0.20 Mod
Go to Top of Page

acoustika
Junior Member

Denmark
311 Posts
Posted - 15 December 2003 :  12:47:01  Show Profile  Visit acoustika's Homepage  Send acoustika an ICQ Message
It's only for forum 17
It's because I first have made a normal forum for "buy,sale,trade" then I found the Classifieds and wanted to make the Normal forum link to the classifieds, I had my problems with this, but did it by throwing in his code, that means my forum 17 does exist, but there aren't any topics, and when my users hit the forum 17 it actually goes to Forum.asp, but then these lines check if it's forum 17 and if it is they will be redirected to the classifieds, that's the only forum it's concerning :-)
But I think that removing the javascript might have fixed it. :-)
I just have to watch out for spiders now and see what there are doing.
May The Code Be With You
www.UniqueDk.dk *** The Open Forums of Denmark ***
Edited by - acoustika on 15 December 2003 12:48:14
Go to Top of Page

masterao
Senior Member

Sweden
1678 Posts
Posted - 15 December 2003 :  19:15:01  Show Profile  Visit masterao's Homepage
Ok. As said earlier, Im not good with javascript, so I hope that it was the cause for your problem .
Jan
===========
FR Portal Forums | Active Users 4.0.20 Mod
Go to Top of Page

Jag24
Junior Member

182 Posts
Posted - 02 April 2004 :  10:11:10  Show Profile  Visit Jag24's Homepage  Send Jag24 an ICQ Message
Any more updates on the spiders ?
Go to Top of Page

aspwiz
Junior Member

250 Posts
Posted - 08 April 2004 :  07:21:06  Show Profile  Visit aspwiz's Homepage
Hi.... Javascript (especially related to navigation, and frames) often poses a problem for search engine spiders.

I am happy to offer any SEO (Search Engine Optimisation) advice to members of this forum. Just pop along to my site and post in the SEO forum.

Sorry I've been absent from this mod for a while... I've just been very hecticcally busy!
Go to Top of Page

nickw
Junior Member

Ireland
193 Posts
Posted - 03 June 2004 :  17:42:16  Show Profile
So, here I am with this MOD, and I am getting notification on a new BOT.

Agent="msnbot/0.11 (+http://search.msn.com/msnbot.htm)"

How do I add this to my list?

Thanks,
Nick
Go to Top of Page

masterao
Senior Member

Sweden
1678 Posts
Posted - 03 June 2004 :  19:26:38  Show Profile  Visit masterao's Homepage
Add it to the current list of agents so that it looks like this (addition marked red):

"ZyBorg/||ZYBORG||DEAD LINK CHECKER**" & _
"Mediapartners-Google||GOOGLE ADSENSE||CONTEXTUAL ADVERT BOT" & _
"msnbot||MSNBot||Prototype Web-crawling robot"

Thanks for the tip, nickw.
Jan
===========
FR Portal Forums | Active Users 4.0.20 Mod
Go to Top of Page

nickw
Junior Member

Ireland
193 Posts
Posted - 04 June 2004 :  12:12:12  Show Profile
Excellent Jan ;-) Now we can build this list up :-)

Now it is: Edited 9th June 2004
AgentCsvStr = 	"Mediapartners-Google||GOOGLE ADSENSE||ADSENSE CRAWLER**" & _
			"Googlebot||GOOGLE||SEARCH ENGINE SPIDER**" & _
			"NPBot||Name Protect||Domain Research Bot**" & _
			"Scooter||ALTAVISTA||SEARCH ENGINE SPIDER**" & _
			"ia_archiver||ALEXA||SEARCH ENGINE SPIDER**" & _
			"FAST-WebCrawler||ALL THE WEB||SEARCH ENGINE SPIDER**" & _
			"Speedy Spider||ENTIRE WEB||SEARCH ENGINE SPIDER**" & _
			"ArchitextSpider||EXCITE||SEARCH ENGINE SPIDER**" & _
			"ArchitectSpider||EXCITE||SEARCH ENGINE SPIDER**" & _
			"Ask Jeeves/Teoma||ASK JEEVES / TEOMA||SEARCH ENGINE SPIDER**" & _
			"Slurp/||INKTOMI||SEARCH ENGINE SPIDER**" & _
			"Slurp.so/1.0||INKTOMI||SEARCH ENGINE SPIDER**" & _
			"UltraSeek||INFOSEEK||SEARCH ENGINE SPIDER**" & _
			"InfoSeek Sidewinder||INFOSEEK||SEARCH ENGINE SPIDER**" & _
			"MantraAgent||LOOKSMART||SEARCH ENGINE SPIDER**" & _
			"Lycos_Spider_(T-Rex)||LYCOS||SEARCH ENGINE SPIDER**" & _
			"HenryTheMiragoRobot||MIRAGO||SEARCH ENGINE SPIDER**" & _
			"MSNBOT/0.1||MSN SEARCH||SEARCH ENGINE SPIDER**" & _
			"Gulliver||NORTHERN LIGHT||SEARCH ENGINE SPIDER**" & _
			"Scrubby||SCRUB THE WEB||SEARCH ENGINE SPIDER**" & _
			"teoma_agent1||TEOMA||SEARCH ENGINE SPIDER**" & _
			"marvin/infoseek||WEBSEEK||SEARCH ENGINE SPIDER**" & _
			"SlySearch/1.3||SLYSEARCH||SEARCH ENGINE SPIDER**" & _
			"Szukacz||SZUKACZ.PL||SEARCH ENGINE SPIDER**" & _
			"IE 5.5 Compatible Browser||IE 5.5 Compatible Browser||Unknown**" & _
			"almaden||IBM||RESEARCH BOT**" & _
			"Google CHTML Proxy/1.0||GOOGLE PROXY SERVER||PROXY SERVER**" & _
			"http://grub.org||GRUB||RESEARCH BOT**" & _
			"NutchOrg||NUTCH||OPEN SOURCE SPIDER**" & _
			"InternetSeer.com||INTERNET SEER||WEBSITE MONITORING SERVICE**" & _
			"Baiduspider+||BAIDU||SEARCH ENGINE SPIDER**" & _
			"Xenu Link Sleuth||XENU LINK CHECKER||DEAD LINK CHECKER**" & _
			"Mozilla/5.||Mozilla 5.x||**" & _
			"Mozilla/4.||Mozilla 4.x||**" & _
			"Mozilla/3.||Mozilla 3.x||**" & _
			"NetMonitor/||NetMonitor||WEBSITE MONITORING SERVICE**" & _
			"TurnitinBot/1.5||TurnItInBot||PLAGIARISM RESEARCH BOT**" & _
			"dloader(NaverRobot)||NAVER ROBOT||KOREA TELECOM**" & _
			"ZyBorg/||ZYBORG||DEAD LINK CHECKER**" & _
			"QuepasaCreep||QUEPASA.COM||SEARCH ENGINE SPIDER**" & _
			"Microsoft URL Control||POSSIBLE EMAIL COLLECTOR||POSSIBLE VUNERABILITY SCANNER**" & _
			"Avant Browser||Avant Browser||Avant Browser**" & _
			"msnbot/0.11||MSNBot||Prototype Web-crawling robot**" & _
			"Openbot/3.0||Openfind.com||Prototype Web-crawling robot**"
Nick
Edited by - nickw on 09 June 2004 14:29:54
Go to Top of Page

nickw
Junior Member

Ireland
193 Posts
Posted - 04 June 2004 :  16:58:59  Show Profile
Mmmm... Hi Again Jan...

So today, I get another email... even thought the above is added... ideas?

Email:
Agent="msnbot/0.11 (+http://search.msn.com/msnbot.htm)"
IP: 213.94.253.31
Nick
Go to Top of Page

OneWayMule
Dev. Team Member & Support Moderator

Austria
4969 Posts
Posted - 05 June 2004 :  04:53:46  Show Profile  Visit OneWayMule's Homepage  Send OneWayMule an ICQ Message
You probably have to change this line:
			"msnbot||MSNBot||Prototype Web-crawling robot"

to
			"msnbot/0.11||MSNBot||Prototype Web-crawling robot"
My MODs:
Birthdays - Custom Policy - F.A.Q. Administration - Forum Rules - Guestbook
Links Manager - MyOwnGoogle - Profile Views - Search Log - WebSearch

Useful stuff:
Forum and MOD Installation - MOD Installation Guide - Snitz v3.4.05 Readme - Free ASP Hosts - Support Snitz
Go to Top of Page

masterao
Senior Member

Sweden
1678 Posts
Posted - 05 June 2004 :  07:18:16  Show Profile  Visit masterao's Homepage
I second OWMs suggestion.
Jan
===========
FR Portal Forums | Active Users 4.0.20 Mod
Go to Top of Page

nickw
Junior Member

Ireland
193 Posts
Posted - 05 June 2004 :  09:27:30  Show Profile
Already tried that guys... same problem :(
Nick
Go to Top of Page

masterao
Senior Member

Sweden
1678 Posts
Posted - 05 June 2004 :  16:10:47  Show Profile  Visit masterao's Homepage
I checked the code a little and I missed something. You need to add ** after the entry. ** is used to split the crawler-list into the array.

It should look like this:

"msnbot||MSNBot||Prototype Web-crawling robot**"
Jan
===========
FR Portal Forums | Active Users 4.0.20 Mod
Go to Top of Page

nickw
Junior Member

Ireland
193 Posts
Posted - 06 June 2004 :  09:39:07  Show Profile
Thanks Jan, I'll give that a go, and edit the list above
Nick
Go to Top of Page
Page: of 19 Previous Topic Topic Next Topic  
Previous Page | Next Page
 New Topic  Topic Locked
 Printer Friendly
Jump To:
Snitz Forums 2000 © 2000-2021 Snitz™ Communications Go To Top Of Page
This page was generated in 0.15 seconds. Powered By: Snitz Forums 2000 Version 3.4.07