tags:Spam, WordPress
Akismet Worst Offenders Extension
June 4th, 2006, by Rich.
This last few weeks the site has been very heavily hit by comment spammers hawking their usual reprobate websites and wasting internet bandwidth. Akismet has been doing a sterling job of catching this spam and not one message has made it onto the site (I wrote about Akismet’s effectiveness in the pre-launch testing previously). In the bad old days before Akismet I’d have to go through the “unmoderated comments list” in order to find the occasional real comment amidst all the spam, this is no longer necessary, which is wonderful. Comment-Spam-Nirvana has not been reached yet, however.
In order to help keep Akismet working well, and also, to ensure that “false positives” do not go unnoticed it is still necessary to trawl through the “spam list” and look for real comments. So although the problem has been turned on it’s head, the requirement on the responsible user is still the same.
The latest version of the Akismet plugin (v1.15) makes this “de-spamming” process easier, but it still leaves the poor website owner with the responsibility of looking at every single spam message in case there are any real comments that have been mistakenly marked as spam.
Informing the Akismet server about these false positives is important because it helps improve Akismet’s accuracy, which benefits everyone by ensuring fewer false positives – one hand washes the other, so to speak.
So I wrote a small addition to Akismet 1.15 (pictured above) that tries to help. It pre-processes the spam comments and identifies the worst offenders in terms of the domain that’s being advertised, or (perhaps more usefully) the IP Address of the spamming computer.
It’s not uncommon for me to get several hundred spam comments each day, so certain machines and websites are hitting my site many times. What the plugin does is make those worst offenders really obvious, so they can be removed en masse, reducing the ham-hunting to a smaller and more managable task.
Download it here: Akismet 1.15 plus Worst Offenders Extension. A stand alone version is available which works with newer versions of Akismet, see the discussion on the forum for more details.
hi, I had to close off comments altogether on my blogs because I was getting an unmanageable amount of spam. The good folks on the Pivot project implemented a technique in their latest version called Hashcash. It’s a really simple technical countermeasure and it’s completely eliminated spam on my blogs without the need for and bayes analysis or blacklists or other difficult-to-manage technique, or the user-unfriendliness of captchas, quizes, etc. The downside is that those commenting have to have javascript enabled on their browser, but I feel that’s a small price to pay. I still get valid comments and the spam log indicates that I’m still getting hit thousands of times a day by spam attempts. I probably won’t last forever, but in the meantime it’s very effective and unlike email-style filtering techniques, it doesn’t generate false-positives.
Methods like Hashcash only work for regular comments, it’s impossible for protocols designed for blog-to-blog communication like Trackback or Pingback.
Rich, plugin looks pretty cool.
There was also an interesting chat regarding hashcash on zedrdave’s blog recently. To paraphrase Marco, if the bot groks javascript, hashcash is toast and you’re back to square one.
Pivot also has “hardened trackback” to eliminate trackback spam. It also requires javascript on the browser and has also been 100% effective so far.
Sure it might not work forever. Maybe someone will bother to write a smart enough spambot that gets it.
But in the meantime, I find it odd, although not completely out of character for geeks, to dismiss a solution that’s simple but might not work forever in favour of a suite of complex techniques that aren’t nearly as effective.
Dismiss hashcash, no, but, I’ll go into more detail (because deep down I’m a scientist first; geek comes a distant second).
I’m not (and I dont think Matt was) dismissing hashcash, just discussing a few of it’s most obvious weaknesses. It’s a technique, it has flaws, so does everything else.
The fundamental difference between static spam catching mechanisms and any dynamic service (not just Akismet) is that the dynamic service recognizes the spam rather than obfuscating the interaction.
It is a weakness of recognition mechanisms that they can result in false positives. This extension shows how the volume of spam received can be used to massively simplify the process of spotting such (rare) errors.
What you gain on the swings you lose on the roundabouts.
Akismet is arguably more effective because the techniques it uses for recognition can adapt over time. If a side by side test of dynamic -vs- static system were conducted over 10 years, it’s possible (maybe even likely) that static could beat dynamic initially, but once enough bots can circumvent the static system, it’s game over.
That’s not to say that using a static system in front of a dynamic one is not a good idea; many people use other anti-spam plugins together with Akismet, and report that it works very well.
What I do think is a fundamental flaw of hashcash is the reliance on JavaScript. I’m a strong advocate of web accessibility and try to ensure every page I make is valid. The result of a website that requires javascript for interaction, is facilities that are not equally available for all people – i.e. – someone accessing the site through a screen reader may not be able to leave a comment. That’s a bad thing.
First of all it’s well worth trying out the Bad Behaviour plugin – I use it and as a result I’ve had to wait 3 days to build up 11 spam messages in Akismet for it to spot a grouping – meanwhile it’s rejected 222 connections from things claiming to be browsers but aren’t (or break the HTTP specs in some way) in the last 7 days.
cheers!
Chris
See also: Akismet htaccess extension
The extension (of this extension) enables automated htaccess modification, so known spam machines are banned before they even get as far as WordPress and Akismet.
[...] Several of Nafcom’s comments are eaten up by the hungry Akismet plugin in WordPress. Now, I am trying Akismet Worst Offenders Extension to see if it works better. [...]
[...] Rich Boakes written an extension to Akismet that makes deleting all the spam comments much easier. His extension adds a “Worst Offenders” list at the top of the Akismet page. The list is ordered by hits of IP address and helped me delete 2,407 comments with 3 or 4 clicks and saved me scrolling through an endless list of spams. [...]
I get the following error when I try to use your extension on http://www.alabamawatch.com
Parse error: syntax error, unexpected $end in /home/alabhcom/public_html/wp-content/plugins/akismet/akismet.php on line 428
What should I do?
[...] Check out the Worst Offenders Extension and the htaccess extension. [...]
[...] Akismet Worst Offenders plugin is just for you. It groups your suspected spam comments by IP address or web site. Then a delete button allows you to easily delete those bulk spam comments right away. [...]
WP Plugin : Akismet Worst Offenders Extension…
Spam対ç–ã¨ã—ã¦ã‚ˆã使ã‚れるプラグインã¨ã—ã¦æœ‰å㪠Akismet ã§ã™ãŒã€ã“ã®ãƒ—ラグインを利用ã—ã‚„ã™ãæ”¹é€ ã—ãŸãƒ—ラグインã®ç´¹ä»‹ã§ã™ã€‚
……
If anyone’s got an inkling of what might be breaking on Clay’s installation, RSVP! Looking around the net it seems that the error he describes can be seen in cases where there are unmatched brackets, so I might need to do some charcter escaping somewhere. If anyone else is seeing the issue, please be sure to mention it.
[...] update #3: akismet worst offenders plugin egy hasznos kiegészÃtés a sokak által használt akismet pluginhez. Ezekre klikk, ha menteni akarod a posztot. [...]
[...] Akismet Worst Offenders Extension是Akismet的一个扩展，在原Akismetçš„åŸºç¡€ä¸Šæ·»åŠ ä¸€ä¸ªå¯¹æˆªåˆ°çš„spamçš„ip和网å€è¿›è¡Œç»Ÿè®¡ã€‚效果è§ä¸‹å›¾ï¼š [...]
[...] Richard has added a small plugin called Worst Offenders Extension to Akismet 1.15 that helps you view and delete mass grouping of spam easily. It checks against the spam comments and identifies the worst offenders and isolates spammers by domain or IP Address. [...]
[...] Get it for yourself and see what I mean [...]
[...] Akismet Worst Offenders Extension – boakes.org (tags: akismet blogs download extension plugin spam wordpress) [...]
[...] 　　Akismet Worst Offenders Extension，Akismet的一个扩展，在原Akismetçš„åŸºç¡€ä¸Šæ·»åŠ ä¸€ä¸ªå¯¹æˆªåˆ°çš„spamçš„ip和网å€è¿›è¡Œç»Ÿè®¡ã€‚ [...]
[...] So a very neat plug-in for WordPress called the Akismet Worst Offenders Extension just made life a lot easier. [...]
[...] I’ve started to use a great extension to Akismet which tallies the spam caught by IP and web site. The Akismet Worst Offenders Extension by Rich Boakes shows that around a third of the spam in the last 15 days related to an IP and web site hosted by Inhoster. I’ve tried complaining to Inhoster before without success but then found an e-mail to its upstream provider, Wvfiber, produced good results. So, I’ve just fired off another e-mail to Wvfiber. Akismet has caught 2,027 spam in the last fifteen days so it’s time for me to fight back. [...]
[...] I didn’t understand the point of this plugin at all when it was announced a few days ago, but Neville Hobson just posted something that made it make sense. All it does is give you a list of all the IP addresses with multiple spams, so when you’re checking Akismet hasn’t caught anything legitimate you can delete everything at once. [...]
[...] Mehr Grund zum Jubeln für Wordpress-Blogger: Akismet an sich ist ja schon eine feine Sache um Spam Comments den Garaus zu machen, aber etwas nervig (und anstrengend!) ist es schon die Liste der Spam Comments einzeln auf false negatives durchgehen zu müssen. Ich habe in letzter Zeit einige Kommentare und auch Trackbacks aus der Akismet bedingten Verbannung retten müssen… Nun gibt es jedenfalls ein Plugin, Akismet Worst Offenders, das einfach alle Kommentare die vom selben Host stammen gruppiert und man sie so mit einem Mausklick löschen kann. Einfach runterladen, vorhandene akismet.php mit der neuen Version überschreiben, fertig. Awesome! Manchmal lohnt es sich eben doch die Links im Dashboard durchzugucken
[...]
[...] I, like most other bloggers, am fighting the good fight against spam. A few weeks back I tried Bad Behavior on recommendation – it made no impact, if anything the amount of spam I had to wade through went up, so I’m back to working with just Akismet. Happily, there’s a new extension on the block that snuggles up cosily to Akismet. The “Worst Offenders” extension lists the most common offenders in your Akismet queue, batching them up so you can delete them in one fell swoop. It needs the latest version of Akismet, something I haven’t got around to installing yet, so I can’t tell you how well it works, but it seems like a good idea [...]
[...] Good news this morning from WV Fiber, the upstream provider to Inhoster. My complaints about two Inhoster IPs – 85.255.117.18 and 85.255.117.250 – were finally listened to after hundreds of comment spam were sent to my blog. I used Akismet to trap the deluge with the Akismet worst offenders extension providing running totals. [...]
great plugin.
it seems like i have to click the “delete worst offenders” button 2 or three times, each time some unknown subset of the worst offenders gets deleted. does this make any sense? i’m running 2.0.2
Hi John, um, sorry it doesn’t make a whole pile of sense, yet. What do you mean you “have” to click the button several times? What are you trying to achieve that makes this multiple clicking a requirement? I’m certainly baffled by the “unknown subset”, so if you can elaborate a little that would be cool.
Rich-
I will wait for some spam to build up and them make some screenshots.
John
[...] You also have an Akismet Worst Offenders Extension that pre-processes the spam comments and identifies the worst offenders in terms of the domain that’s being advertised, or (perhaps more usefully) the IP Address of the spamming computer. [...]
[...] Well after the upgrade akismet had gotten ovre written, and along with it was the worse offenders version.  Well While I was looking at it to verify he was using the latest Akismet code in his version I noticed he had his htaccess one there… and so far he hadn’t any issues. When I looked at it before i saw experimental and stayed away. But I’m tired of spam… even if it isn’t as much as it used to be, because of Bad Behavior. The htaccess version also includes the worse offenders, making it very easy to decide who to add to the list. But what that means is if you spam too much from an IP, it’ll ban you from the site altogether. And looking through the list of IPs it added and why, I didn’t see any problems with the bans. But if you find yourself banned… email me, or comment here from an un-banned ip. [...]
[...] Check out the Worst Offenders Extension and the htaccess extension. [...]
[...] You see when a spammer hits this site for example, they’ll normally submit up to 20 or so comments all with the same domain. So instead of using Akismet’s ‘Delete all’, Rich’s Akismet Worst Offenders steps in. It allows you to do delete comments based on a common domain so it might say ‘26 from blogspot.com’ and I can just delete those. It cuts down the overall list and reduces the odds of me getting rid of a genuine comment. [...]
[...] Una solución simple y efectiva, sin el uso de javascript, preguntas extrañas o inserción de códigos especiales para validación. Inclusive hay una extensión para este plugin llamada Akismet Worst Offenders que te permite ver en grupos, los comentarios spam mas recurrentes y eliminarnos, permitiéndote realizar una moderación de los comentarios mas efectiva. [...]
[...] About the same time we passed 5000 comments*, so 20 out of 21 comments are spam**. Nice. Many many thanks to akismet.com and the “Worst Offenders” plugin. [...]
[...] Akismet – (Automatic-Kismet) (Comment Spam Blocker)This plugin needs a wordpress.com api key to work (free and commercial versions available.) Have you ever heard of comment spam? Start a blog and you’ll find out what it is – fast. It is a sleazy technique that uses software to take advantage of the open dialog nature of blogs. The software auto-posts irrelevant text comments, stuffed with links, in the comment sections of posts. This plugin will save you hours of clean-up work removing comment spam. I use a modified version of the plugin created by Rich Boakes. This version will automatically create a list of the IP addresses used by the spammers and write them in a ‘deny access list’ in your .htacess file. Any attempts by those computers to access your server get denied. This addition really cuts down on the spam and also the bandwidth used up by spammers searching your site. [...]
[...] Worst Offenders Bad Behavior Spam Karma [...]
[...] First off, I installed the Worst Offenders plugin done by Rich Boakes. You can read about it here and see a screen shot of what it does (see the release history and download here). This is absolutely one of my favorite plugins now. I absolutely love the fact that I can just quickly see obvious spam and delete it without having to scroll through pages of spam to make sure I’m not losing comments. Cheers to you for making this one! [...]
[...] First off, I installed the Akismet Worst Offenders plugin, which sorts through your spam, aggregates the common offenses into bunches, and then allows you to delete entire classes of spam all at once. This makes it dead easy to throw out the vast majority of your spam without actually looking at it. You can then sort through the leftovers looking for false-positives. The other nice thing about this plugin is that it automatically bans the offending IP addresses via .htaccess when you delete the associated spams. This is a great tool for spam management, but it doesn’t do that much to reduce your spam load, despite the IP blocking. [...]
[...] the AkismetWorst Offenders Plugin at http://boakes.org/akismet-worst-offenders Akismet, Block IP, plugin, spamIf you enjoy this post, please subscribe to our RSS Feed Written by [...]
[...] (16-06-07) There is a PHP powered solution at: Akismet Worst Offenders Extension with more "bells & whistles" added… but currently seems [...]
[...] you hours of clean-up work removing comment spam. I use a modified version of the plugin created by Rich Boakes. This version will automatically create a list of the IP addresses used by the spammers and write [...]
[...] the file to akismet.php and drop into your plugins/akismet/ folder. It’s the same idea as the Akismet Worst Offenders I blogged last year but rewritten from scratch. Looks dog ugly but maybe someone will carry the [...]