| Author |
 Topic  |
|
|
Webbo
Average Member
United Kingdom
982 Posts |
 Posted - 14 March 2014 : 16:59:05
|
We have a hidden private forum for the moderators use and we had a 'guest' visitor this evening which turned out to be a Googlebot (IP 66.249.75.211)
Whilst I appreciate that even if the bot crawled the private forum and the results showed up in a search the searcher couldn't access the forum through a Google link, could it still take a cache of the posts in the hidden forum and list them as a preview?
If this is the case is there anythng that can be done to prevent bots from crawling parts of the forum, say for instance by forum ID without it affecting the public sections of the site? |
|
|
HuwR
Forum Admin
United Kingdom
20595 Posts |
Posted - 15 March 2014 : 02:25:54
|
| Are you sure they are actually accessing the posts rather than just the url, I would be extremely surprised and somewhat horrified if they were able to index data that required a login to view it. |
MVC .net dev/test site | MVC .net running on Raspberry Pi |
 |
|
|
Webbo
Average Member
United Kingdom
982 Posts |
Posted - 15 March 2014 : 03:44:49
|
I've got the 'active users' mod installed and alongside the 'guest' user with the above IP (Googlebot) it stated: Viewing Topic: topic name
What prompted me to look at the active users was when directly viewing the topic there was one guest displayed as viewing the topic.
I do know it is possible to give Google bots and analytics permissions to view content that requires a log in by creating a specific log in for them within the admin settings for Google accounts, but I have not done so.
Unless it is the mod just showing that the url is being hit on by the IP address but they cannot gain access to the content, however I would have thought that the mod would have listed the forum.asp page instead
|
|
|
|
HuwR
Forum Admin
United Kingdom
20595 Posts |
|
|
ruirib
Snitz Forums Admin
Portugal
26364 Posts |
Posted - 15 March 2014 : 07:50:55
|
I am sorry, but this has been discussed tens of times. The Active Users mod is deceiving you. It doesn't know whether you have access to the full page or not. It checks the page any visitor tries to access, but due to the moment when that is done, it has no way to know whether the page displayed in full or whether the message denying access is being shown. The same mod also shows even anonymous users accessing the admin pages!
There is nothing to worry about. |
Snitz 3.4 Readme | Like the support? Support Snitz too |
|
|
|
Webbo
Average Member
United Kingdom
982 Posts |
Posted - 16 March 2014 : 03:34:16
|
| Thankyou Rui |
|
|
|
Comforse
Starting Member
Canada
1 Posts |
Posted - 21 March 2014 : 12:17:34
|
| Thanks from me too,ruirib, for clarifying things! |
|
|
| |
Topic |
|
