Formmail tarpitting

29 Aug 2003

I've been trying to come up with a way to tarpit formmail spammer probes/attacks, and I haven't had much luck yet. This is an outline of what I've done and what I plan on trying next. If anyone has any thoughts on this, please let me know. In particular, I'm looking for any approaches I've overlooked; I'm sure there's a lot.

Background: Matt's old version of Formmail, up to at least version 1.9, had serious terrible bad vulnerabilities that would let a spammer use it to send any email anywhere, no matter how much you tried to secure it.

At my last job (ISP helpdesk), I get complaints every now and then of spam coming from our mail server; it was almost always spammers using Formmail to do their dirty work. I'd have to track down websites where an old copy of Formmail was lurking, shut it down, and try to clear the mail queue of as much crap as possible. (This got a lot easier once I discovered [ngrep|http://www.packetfactory.net/projects/ngrep/] and had the root password ~SlashdotJournal_29August2003). Eventually I went and replaced all the copies w/the NMS version of Formmail, which did the trick wonderfully. I could drop it in to a website under attack and it would work right away: spam would stop, legitimate requests would still work.

I still get Formmail probes on my website all the time. A while back, I decided to send the spammers something more than just a [404|http://saintaardvarkthecarpeted.com/wheredidthispagego?] page. Using Apache's ~ScriptAliasMatch directive, any request with that matches "/cgi-bin/formmail" (case-insensitive) in the URL gets redirected to my copy of (ta-da!) Formmail Weasel.

Formmail Weasel is a boringly simple Perl script that parses the request made to it, logs everything to a database, and displays an innocuous "thanks for the submission" page (not that the robot ever read it). There's another script that displays the last ten requests in horrible tables. That's it so far.

Once, I got curious and sent off a fake reply to an address mentioned in one of the probes, making it look like a vulnerable Formmail script had been found. (Future plans for Formmail Weasel include the ability to send off these fake replies automatically, and x-ray vision.) Within a week, there were all kinds of attempts to send spam going on -- maybe one a minute or so. After a few weeks of this, the spammer figured out that it wasn't working, and stopped.

That was interesting, and moderately gratifying, but I wanted to cause pain. I want to imagine spammer wails of dismay. Tarpitting immediately leaped to mind. But I can't simply tarpit port 80 and be done with that: I'm still running Apache to serve a few websites, and I don't want to interfere with that. Besides, Formmail probes go by website, not IP addresses, so I need to have www.somethingorother resolve to my server in order to attract scans.

First I decided to try directing Formmail requests to a separate port. Using Apache's ~RedirectMatch directive and a separate ~VirtualHost thingy, I sent all requests for formmail to port 2348 (aka port random) where Formmail Weasel would be listening and Apache would be logging. For good measure, I set up tcpdump too.

My first hope was that the probe robots looking for Formmail scripts would follow the redirect, and I'd be able to capture the traffic on port 2348 w/tcpdump for analysis. ("Lookee here: spammers use SYN packets! Guess we know what to look for now, professor!")

My second hope was that I could provoke an attack by sending off a fake reply, and see whether the attack robots would follow the redirect. Maybe, if I was extraordinarily lucky, I could just tarpit port 2348 and be done with it.

I forgot about it for a week after sending off the decoy email. Today I checked the Apache logs and the tcpdump file: nothing on either one. But when I checked the main logs for my website, there had been half a dozen requests for formmail; the robots simply didn't follow the redirect. I made sure that the redirect was still working, then cried for a bit.

As I see it, this leaves me with a couple options that don't involve deep heavy network hacking:

• Leave Formmail Weasel the way it is: an essentially passive annoyance to spammers.
• Be a little more crafty.

From the requests I've seen, Formmail probes will look for a few common variations on the extension (.pl, .cgi) with some capitalization variations thrown in for good measure (formmail, Formmail, ~FormMail). This gives me a way of distinguishing an attack (an attempt to send spam) from a probe (seeing whether or not there's a script that can be exploited).

Formmail Weasel could designate one of these (let's say ~FormMail.cgi) as one that is the signal of an attack. Probes that came in for other variations would result in an email being sent off to the spammer, but with the attack address in it. In other words, any probes for FormMail.pl, formmail.cgi, or Formmail.cgi would result in an email back to the spammer indicating that ~FormMail.cgi was successful. At that point, the spammer (hopefully) takes the bait and begins the attack.

At this point we can use the [Linux iptables string matching kernel module|http://www.netfilter.org/documentation/pomlist/pom-extra.html#string] to look for packets that have the request in them, and tarpit them. You'd have to be specific about what exactly to look for: something like "GET ~or /cgi-bin/~FormMail.cgi", plus the host/site/whatever directive. But this is a small enough part of the request, and close enough to the beginning, that it should serve as a way of flagging that address as one that should be tarpitted.

Another option, and probably an easier and more effective one, would be to have Formmail Weasel set up separate iptables rules to tarpit the addresses that are part of the attack. You could age them and phase them out after a short/long while. One possible problem with this is that I've seen Formmail attacks that have come from many different IP addresses simultanteously; these usually end up being open proxies. You'd have to take care not to flood your firewall with tarpitting rules.

Original entry