Jeff Croft

I’m a product designer in Seattle, WA. I recently worked at Simply Measured, and previously co-founded Lendle.

Some of my past clients include Facebook, Microsoft, Yahoo, and the University of Washington.

I’ve authored two books on web and interactive design and spoken at dozens of conferences around the world.

I’m currently accepting contract work and considering full-time opportunities.

Blog entry // 01.31.2012 // 9:11 AM // 15 Comments

How I shut down comment spam on this site

In the past, I’ve only allowed comments on recent blog posts, not older ones. Because I didn’t blog for the longest time, I had no recent blogs posts, which meant I got no comments (or comment spam). When I re-launched JeffCroft.com recently and started writing again, I actually started getting comments again (thanks, guys!). Of course, I also started getting comment spam again. Like, a ton of it. On average, I was getting about 10 comment spam posts per hour.

Frustrated, I went to my standby from the good ‘ol days, Akismet. I quickly found that it wasn’t working for me. It was letting through most of the spam and also preventing a lot of ham from being posted. After a tweet on the matter, Mike Davidson pointed me down a path that has worked beautifully for two weeks now — not a single comment spam post.

Of course, I’m risking that by sharing the technique with you guys, but what the hell. It’s exceptionally simple. Here’s what I do, in a nut: pick a string to be your “secret”. Any string. Set that string somewhere in your backend that you can access it easily (for me, that’s my Django settings.py file). Then, add a hidden field for the comment secret to your comment form. Give it no value. Then, using Javascript, wait a few seconds after someone begins typing a comment and then set the value of the hidden field to your secret string. Finally, when the comment is posted, check on the backend to make sure the posted secret string matches the one you’ve got set. That’s it.

Here are the relevant bits of code from my app:

I hope this proves helpful!

Comments

  1. 001 // Jacob Kaplan-Moss // 01.31.2012 // 9:28 AM

    Nice trick!

    Another clever trick is to try to outwit spambots with a honeypot: create an input called something common like “name” or “url”, but set it to hidden with CSS or JavaScript. Then, if this field contains anything, reject the comment. Works pretty well for me.

    I learned that one from Ned Batchelder: http://nedbatchelder.com/text/st…. He’s got a couple of other related tricks there, too.

  2. 002 // Mike Papageorge // 01.31.2012 // 9:35 AM

    +1 for the honey pot and Ned Batchelder. Has worked great for years on many projects..

    Suprised am I though, to hear that about Akismet. Never had issues with that on WP sites, but hey, they weren’t jeffcroft.com ;)

  3. 003 // Matt Wilcox // 01.31.2012 // 9:35 AM

    Problem here being you can’t comment if JS is off or broken.

    I require my commentors to approve their comment by clicking on a validation link sent to the email they used to submit a comment with.

    I don’t think I’ve ever had spam get through to the front end, though I’ve had one or two on the back-end. Those just get deleted after 30 days.

  4. 004 // Matt Wilcox // 01.31.2012 // 9:38 AM

    I am thinking of a much more rigorous thing for my redesign. Not just to avoid spam, but to avoid pointless reactionary comments like “that sux” or “awesome” etc.

    I’m going to ask you a question about the article I wrote. If you didn’t read it properly, or simply can’t be arsed to answer - consider yourself filtered out of the conversation due to lack of decent appropriate things to say.

  5. 005 // Jeff Croft // 01.31.2012 // 10:05 AM

    Problem here being you can’t comment if js is off or broken.

    That’s a feature, not a bug. If you’re not using a JS-enabled browser, there’s virtually no chance I want your comment on my site.

    I’m going to ask you a question about the article I wrote. If you didn’t read it properly, or simply can’t be arsed to answer - consider yourself filtered out of the conversation due to lack of decent appropriate things to say.

    That’s a good idea. I like it.

  6. 006 // Matt Wilcox // 01.31.2012 // 10:14 AM

    It’s your site and your rules: but browser environment has little relation to worth of comment i think. What if im commenting from my phone (as now) but my mobile provider has mangled the JS (as happens, O2 insert html and js into pages served over their network)? Or i’m temporarily behind some firewall or network filter, like at a college or on holiday somewhere?

    Open question btw, not a challenge.

  7. 007 // Jeff Croft // 01.31.2012 // 10:30 AM

    Matt: Honest (if not PC) answer: if you can’t use Javascript for one of those reasons, you’ve got bigger problems than commenting on my site. Half the internet isn’t working for you, and you’re used to it. You’re either resigned to the fact that a lot of sites don’t work for you, or you’re trying like hell to change your situation. Either way, I wish you luck.

  8. 008 // Matt Wilcox // 01.31.2012 // 11:35 AM

    That’s a fair enough assessment. I don’t agree with leaving it in the hands of other people to sort out when the reason for the problem is a lack of effort on the developer’s side. It’s not technically impossible, or even hard. It’s just time consuming to accommodate. But we’re talking edge cases and personal priorities.

    To be clear: I’m not having a go at your position, just saying I personally don’t sit on the side of the argument that allows me to walk away from people in those situations. I want to engage one and all, not just the privileged majority.

  9. 009 // alex pilon // 01.31.2012 // 11:39 AM

    Problem here being you can’t comment if js is off or broken.”

    Is this really the blogs problem?

    Another variation on this technique (which by the way is a clever alternative to the honey pot method) would be not only to use a single secret, but perhaps generate a secret key on the fly so that it is always different. This way it is less susceptible to a bot figuring out what the key is and always submitting it with the form.

    The symfony php framework employs this technique to counter CSRF. It would be a great to apply to comments via the javascript method and block spam.

    I think having to verify a comment via email is also in the same “sucks to use” / “what if my phone sucks” boat. If you phone is so bad that it does not properly run javascript, who is to say that an email verification method isn’t going to be even worse.

  10. 010 // Tobias Sjösten // 01.31.2012 // 11:57 AM

    Oh man, guerrilla anti-spam. This subject is very dear to me. I have tried a lot of techniques but they always find a way through after a while.

    What I did similar to this was to time the comment attempt and not allow submissions faster than seven seconds.

    I also tried setting up a honeypot form element, asking for a phone number, that I hid with CSS. Whenever it’s filled in I know the comment can safely be rejected. Didn’t work as good for me as for Jacob though.

    One technique that did work very good for a long time was to resolve the commenter’s email domain. If there’s no MX pointers then it’s an invalid email address. Very efficient until they figured out they can use random hotmail.com or gmail.com addresses.

    However the single most efficient technique I’ve applied is to follow their spammy links and contact the site’s administrator. Many times it’s a small business with no insight in the work ethics of the shady SEO consultant they’ve hired. One hour of work for me resulted in four (!) webmasters thanking me for bringing it to their attention and saying they would fire their SEO partner.

    It would be interesting to know if you’ve measured any effects of this approach you descirbe here?

  11. 011 // Jeff Croft // 01.31.2012 // 12:11 PM

    Another variation on this technique (which by the way is a clever alternative to the honey pot method) would be not only to use a single secret, but perhaps generate a secret key on the fly so that it is always different. This way it is less susceptible to a bot figuring out what the key is and always submitting it with

    Yeah, I definitely thought about that. Didn’t feel necessary to me at this time for this site, but it’s a good idea.

    The symfony php framework employs this technique to counter csrf. It would be a great to apply to comments via the javascript method and block spam.

    Yeah, so does Django. Definitely a good parallel to make.

  12. 012 // Joe Casper // 01.31.2012 // 12:16 PM

    +1

    I usually use the honeypot trick for all forms I create and I would say it works 99% of the time but this seems like the next layer to add to my honeypot (That sounded kinda bad).

    Thanks for sharing Jeff!

  13. 013 // sun // 02.01.2012 // 3:05 PM

    You’re definitely going to be happy for a couple of weeks with this approach.

    But protecting against comment spam via JS is not a solution. Most comment spam is caused by replay attacks of spambots controlled in a bot network. Modern spambots learned to correctly interpret CSS already. On top of that, the cutting-edge implementations have learned to load, interpret, and execute JS like regular user agents.

    Executing JS in a pure processing/scripting layer without UI is not only a goal of spammers; it’s also the goal of a range of open-source projects in order to automate JS unit testing. The “smart” spambots just simply copy and integrate that code into their application stack.

    Thus, it’s merely a matter of time until the right spambot hits your site. If one does, and if it’s backed by a bot network, you’ll witness hundreds of spam posts within a single minute. I took the time to separately benchmark all existing practices on the net/market in parallel and ad infinitum, from which a few have been mentioned above.

    Lastly, none of the techniques protects against human comment spam, which slowly but certainly becomes the most problematic technique for site owners; primarily originating from countries having an extremely low labor cost. Benchmarks clearly showed that only the dedicated external spam fighting web services are able to cope with that, since all of the other techniques rely on the poor and dated assumption that user agents of spammers don’t support certain technologies. That’s utterly wrong regarding human spammers, and even more so with regard to current spambot trends.

  14. 014 // Jeff Croft // 02.01.2012 // 3:37 PM

    Hi Sun-

    Thanks for the insight. To be clear, the assumption here is not that spambots can’t post my form because they don’t have Javascript, but rather that they won’t post my form because they don’t want to hang around waiting for the timer to run. As you noted, spam bot are typically capable of slamming someone with hundreds of messages per minute. If that’s their M.O., and I can force them to only be able to post a few in minute, the hope is that they’ll consider my site not efficient enough and move on to the next one.

    We’ll see if it hold out or not. For now, it’s working. If it stops, I’ll move on to the next thing.

  15. 015 // Dave A. // 02.06.2012 // 8:05 AM

    Hi Jeff, thanks for this blog post, as it gave me something to think about while learning Asp.Net MVC. I wrote an article over at CodeProject based on the concept you applied here, but implementing it within MVC.

    http://www.codeproject.com/Artic…

    Thanks, Dave

Tags for this entry
Links in this entry