I was reading back over this thread, and saw something I missed....
Someone wanted to know why i decided not to list the adsense crawler as a search engine spider....
well, I did that because although it is a spider, it has no relevance to search engine results
.... more in fact to do with what adverts show up for google adsense.
It seems my first code I posted was not 100% complete, so let me point out how it was supposed to go...
The first line in the agents block should read as follows:-AgentCsvStr = "Mediapartners-Google||GOOGLE ADSENSE||ADSENSE CRAWLER**" & _
Then, round about line 289, there should be this code:-response.write "<tr>" & _
"<td class=""nw ffs"" valign=""middle"" width=""5%""> </td>" & _
"<td class=""nw ffs"" valign=middle>"
If instr(AGENTSTR,"SEARCH ENGINE SPIDER") or instr(AGENTSTR,"ADSENSE CRAWLER") then
response.write "CRAWLER"
c=c-1
else
response.write fLang(strLangMOD_Ls3kAU_01130) & c
end if
response.write "</td>"
...and finally at the end of the whatpage function, i have this code:if strOnlineLocation <> "" then strPage = strOnlineLocation
If inStr(Agentstr,"SEARCH ENGINE SPIDER") then strpage=replace(strpage,"Viewing","Indexing")
If inStr(Agentstr,"SEARCH ENGINE SPIDER") then strpage=replace(strpage,"Browsing","Indexing")
If inStr(Agentstr,"SEARCH ENGINE SPIDER") then strpage=replace(strpage,"Reading","Indexing")
If inStr(Agentstr,"SEARCH ENGINE SPIDER") then strpage=replace(strpage,"Posting new topic in","Indexing Topic")
If inStr(Agentstr,"SEARCH ENGINE SPIDER") then strpage=replace(strpage,"Replying to","Indexing")
If inStr(agentstr,"SEARCH ENGINE SPIDER") then strpage=replace(strpage,">Home page<",">Indexing home page<")
If inStr(Agentstr,"ADSENSE CRAWLER") then strpage=replace(strpage,"Viewing","Deciding advert context for")
If inStr(Agentstr,"ADSENSE CRAWLER") then strpage=replace(strpage,"Browsing","Deciding advert context for")
If inStr(Agentstr,"ADSENSE CRAWLER") then strpage=replace(strpage,"Reading","Deciding advert context for")
If inStr(Agentstr,"ADSENSE CRAWLER") then strpage=replace(strpage,"Posting new topic in","Deciding advert context for topic")
If inStr(Agentstr,"ADSENSE CRAWLER") then strpage=replace(strpage,"Replying to","Deciding advert context for")
If inStr(agentstr,"ADSENSE CRAWLER") then strpage=replace(strpage,">Home page<",">Deciding advert context for homepage<")
WhatPage = strPage
I have the portal mod installed, so some of that last chuck will differ for this mod....
I just thought I would give reason for why I initially made the decision to treat adsense crawler not as a search engine spider.
I think this change should be reflected in the mod chuck... what does everyone else prefer?