| Author |
 Topic  |
|
bobby131313
Senior Member
USA
1163 Posts |
 Posted - 21 January 2009 : 20:42:00
|
Mike I think the problem Phorum had was not your first post, I think it was the reference to them not being popular that set him off. Ever tell a die hard Mac user that they've never gotten a virus not because they are so secure, but because no one cares enough to write one for them? 
OK, back to topic... Is there any data on false positives yet? For example, what if an AOL proxy IP gets in your DB mistakenly? Could block 100K people accidentally.< |
Switch the order of your title tags |
 |
|
|
MrMike2000
Starting Member
16 Posts |
Posted - 21 January 2009 : 23:46:17
|
quote:
Originally posted by bobby131313
Mike I think the problem Phorum had was not your first post, I think it was the reference to them not being popular that set him off.
Probably so. He can get as cranky as he wants, but it's not my fault that Phorum is much less popular than other message boards. I'm sorry if hearing it was painful or embarrassing for him.
quote:
Originally posted by bobby131313Ever tell a die hard Mac user that they've never gotten a virus not because they are so secure, but because no one cares enough to write one for them?
I'd only do that if I was behind bullet-proof glass. :)
quote:
Originally posted by bobby131313OK, back to topic... Is there any data on false positives yet? For example, what if an AOL proxy IP gets in your DB mistakenly? Could block 100K people accidentally.
Yup, it's possible. We recommend checking the email first and foremost- that's the most reliable indicator. We don't have any false positive data, and to be honest, I'm not sure how we'd collect or verify it. If you have any ideas we'd be interested in hearing them.
Here's how it breaks out:
Email - the single most reliable indicator of a bot. 99.9% or better.
IP - the second most reliable indicator of a bot. 95% or better.
Name - the least reliable indicator of a bot. Varies too much to be assigned a real percentage. Maybe 30%.
If both the email and IP match, that's pretty conclusive. You can actually check all three parameters (IP, email, and name) by using another query type that we just added to the API. The new query request type is 'MULTI', and a correctly formatted URL will appear like this (this wouldn't format correctly here so I had to leave off the "http://):
botscout.com/test/?multi&name=krasnhello&mail=krasnhello@mail.ru&ip=84.16.230.111&format=xml
1) The 'multi' parameter must be present to trigger the 'MULTI' type test
2) Notice that there is no value after the 'multi' parameter in the GET string. Any value assigned to the 'multi' parameter will be ignored.
3) Each field item (name, ip, email) must be explicitly assigned as a "name=value" pair.
4) The order of the field items doesn't matter, but they must all be present or the system will kick out the following error: Missing or Malformed Data for MULTI test.
The MULTI test will return the following XML structure (if 'format=xml' is specified):
<response>
<type test="IP">
<matched>Y</matched>
<count>3</count>
</type>
<type test="MAIL">
<matched>Y</matched>
<count>0</count>
</type>
<type test="NAME">
<matched>Y</matched>
<count>2</count>
</type>
</response>
The field order (IP, MAIL, NAME) will not vary and all of the the top-level elements will be returned each time with a secondary 'match' and 'count' element .
By omitting the 'format=xml' option the data can also be returned as a standard format of a pipe-delimited string, which would look similar to this:
Y|MULTI|IP|3|MAIL|1|NAME|2
In the pipe-delimited format, if any of the elements match then the first field will be a 'Y'. If none of them match then it's returned as 'N' (but with the rest of the fields still present).
Please note that the MULTI test is most useful if you intend to implement some sort of threshold-based scoring. You could, however, use it to get the email and IP 'match' values and decide based on those two alone. Otherwise, we recommend just checking the email address.< |
|
|
|
Shaggy
Support Moderator
Ireland
6780 Posts |
Posted - 22 January 2009 : 04:21:25
|
quote:
Originally posted by MrMike2000
Name - the least reliable indicator of a bot. Varies too much to be assigned a real percentage. Maybe 30%.
Also, a legit member could also be using one of the blacklisted usernames.
< |
 Search is your friend
“I was having a mildly paranoid day, mostly due to the
fact that the mad priest lady from over the river had
taken to nailing weasels to my front door again.” |
|
|
|
Podge
Support Moderator
Ireland
3775 Posts |
|
|
Shaggy
Support Moderator
Ireland
6780 Posts |
Posted - 22 January 2009 : 05:25:36
|
Personally, I'd say that's pretty conclusive, Podge; I'd actually stop the registration on the e-mail alone.
< |
Search is your friend
“I was having a mildly paranoid day, mostly due to the
fact that the mad priest lady from over the river had
taken to nailing weasels to my front door again.” |
|
|
|
MrMike2000
Starting Member
16 Posts |
Posted - 22 January 2009 : 06:19:12
|
quote:
Originally posted by Podge
If the ip and email return a count of 1 and 1 respectively does that mean its a bot ?
If both the IP and the email match, chances are overwhelming that it's a bot.
As Shaggy said, personally I'd stop the registration on the e-mail alone. The email address is dead giveaway since real users will never have the same email as a bot.< |
|
|
|
Podge
Support Moderator
Ireland
3775 Posts |
Posted - 22 January 2009 : 06:44:18
|
Suppose a legit (but stupid) user registers on one of your honeypot forums. Does this automatically mean that they are a spammer ? What if someone deliberatley sets out to tarnish someone's email address and manages to spoof their i.p.? From your records do you know if there are many spams reported with very small counts (i.e. 2 or less)?
How about these features for the Spam Control Admin Area
1. Auto lock members posting spam (for bot that registered before bot prevention mod was added to a forum
2. Set spam threshold
3. Toggle Register botcheck
4. Toggle Post botcheck
5. Create member and lock at registration if botcheck fails
6. Email Administrator notifications of botcheck failures
7. Set BotScout api key
Any others ?
< |
Podge.
The Hunger Site - Click to donate free food | My Blog | Snitz 3.4.05 AutoInstall (Beta!)
My Mods: CAPTCHA Mod | GateKeeper Mod
Tutorial: Enable subscriptions on your board
Warning: The post above or below may contain nuts. |
|
|
|
JJenson
Advanced Member
USA
2121 Posts |
Posted - 22 January 2009 : 07:33:41
|
| Podge personally I would like a list from the admin area to see which accounts were caught by this script in a list in the admin area. I personally don't have emails sent to me from the forum on anything.< |
|
|
|
Podge
Support Moderator
Ireland
3775 Posts |
Posted - 22 January 2009 : 09:00:14
|
I'm not that much in favour of adding too much db stuff other than in FORUM_CONFIG_NEW. What I could do is give the option to create a normal member account and lock it straight away but give them a member title to read "SPAM Bot" or something similar. That way they could never register again with the same email address if the forum admin removes the mod or the botscout service is down/withdrawn, etc.
Anyone test the code for me ? I'm mainly interested to see if the xml components work for everyone. Some people may need to change these lines of code.Set xml = Server.CreateObject("Microsoft.XMLHTTP")
' Set xml = Server.CreateObject("MSXML2.ServerXMLHTTP")
Set xmlDoc = Server.CreateObject("Microsoft.XMLDOM")
< |
Podge.
The Hunger Site - Click to donate free food | My Blog | Snitz 3.4.05 AutoInstall (Beta!)
My Mods: CAPTCHA Mod | GateKeeper Mod
Tutorial: Enable subscriptions on your board
Warning: The post above or below may contain nuts. |
|
|
|
MrMike2000
Starting Member
16 Posts |
Posted - 22 January 2009 : 12:53:54
|
quote:
Originally posted by Podge
Suppose a legit (but stupid) user registers on one of your honeypot forums.
Humans can't register on our honeypot network- the bot trapping network isn't visible to humans (real users), and we also take some tricky steps to make certain they aren't real users. I don't want to go into details publicly, but essentially humans can't register on the bot traps. They'd have to really try to register themselves and even then most of wouldn't be able to do it.
quote:
Originally posted by Podge
What if someone deliberatley sets out to tarnish someone's email address and manages to spoof their i.p.?
As far as I'm aware, you can't spoof a specific IP address directly. The best you can do is go through a proxy server, which will report the proxy server's IP address. There's no way I can pick a specific IP to appear to come from unless I've already managed to bot or infect a specific user's PC, then take direct control of it and deliberately hit the bot trapping network.
quote:
Originally posted by Podge
From your records do you know if there are many spams reported with very small counts (i.e. 2 or less)?
Yes, there are loads and loads of them. This is because email addresses are cheap and easy to manufacture. By using a catch-all address on a single domain you can effectively have an unlimited number of email addresses, millions and millions and millions of them. Spammers often use this tactic to evade spam filters, but it won't evade us. If they use an email address just once and then brush up against us, we have that address stored forever. 
quote:
Originally posted by Podge
How about these features for the Spam Control Admin Area
1. Auto lock members posting spam (for bot that registered before bot prevention mod was added to a forum
2. Set spam threshold
3. Toggle Register botcheck
4. Toggle Post botcheck
5. Create member and lock at registration if botcheck fails
6. Email Administrator notifications of botcheck failures
7. Set BotScout api key
Any others ?
Wow, these would be great options. The only one I'm not sure of is the threshold setting- there would need to be some sub-options under that I think, so you could set individual options for the IP, email, and name matches.< |
|
|
|
Podge
Support Moderator
Ireland
3775 Posts |
|
|
MrMike2000
Starting Member
16 Posts |
Posted - 22 January 2009 : 18:40:31
|
quote:
Originally posted by Podge
Ok. I'm sold on the i.p. and email address matching = spam i.e. a count of 2 is enough in this case.
If anyone is interested, we're looking for 1 or 2 folks who'd like to beta test the bot trapping system. All that's needed is for you to place a single small file on your system somewhere, and then add a simple call to the form server on BotScout. The rub is that we only have the code written in PHP, using cURL to make the call to the form server. It would need to be translated into ASP to work with the Snitz Forum code. 
A brief explanation of what it does:
The form-server calling code is placed on a page. When the page loads the call is made and the form server prints an invisible form in the page. When a bot sucks up the page it sees the form, fills out the fields, and submits it. The form is submitted to the small file placed on your system.
The reason behind the need for it to submit to the file on your system is that the bots (the bot owners, really) will eventually "learn" to avoid a form that submits directly to BotScout. But submitting the form to the same server that the form is on looks perfectly natural.
The small file on your system simple takes the submitted data and resubmits it to BotScout on the fly, so the bot never knows that it just added itself to the BotScout database. :)
< |
|
|
|
Podge
Support Moderator
Ireland
3775 Posts |
|
|
MrMike2000
Starting Member
16 Posts |
Posted - 22 January 2009 : 21:26:58
|
quote:
Originally posted by Podge
Can you email me the php code and I'll look at it ?
I'll email you a copy as soon as I get it into working form. It's close, hopefully tomorrow.< |
|
|
|
MrMike2000
Starting Member
16 Posts |
Posted - 22 January 2009 : 23:39:23
|
quote:
Originally posted by MrMike2000
quote:
Originally posted by Podge
Can you email me the php code and I'll look at it ?
I'll email you a copy as soon as I get it into working form. It's close, hopefully tomorrow.
UPDATE: Errrr. I just realized that it's in PHP. I'll probably need to find someone that's familiar with both languages. < |
|
|
|
Topic |
|
Anti-Bot Modification for Snitz Forum
