Any help for an odd qmail spam exploit?

I’ve been using qmail on the Xenite server for several years now. The current server, I believe, was set up in 2004. In 2005 email spammers began using Xenite as a relay and we were told to change our /var/qmail/control/helohost to explicitly reference mail.xenite.org so we did that, naively thinking the evil email spammers could not exploit us again.

Well, somewhere along the way a spammer dissected our custom contact form and used that Perl script to do his little relaying, so we made changes there, too.

Over the past couple of weeks I’ve noticed I had problems sending email from Xenite. People were not receiving my messages. I checked Spamhaus and their database doesn’t record our IP address. I figured we weren’t being exploited by spam.

In 2005 we were told to use the “tail -f maillog” command to check and see how much outbound email we had. At the time it showed a continuous flow of email and when that stopped we knew we had closed the exploit.

Sadly, that command (along with Spamhaus) failed to show me what was going on this time. I finally called our hosting ISP tech support and they told us we had about 110,000 spam emails in our outbound queue awaiting processing. No wonder no one was receiving my messages.

Despite the fact we pay for 24/7 support the tech only suggested we upgrade both the server and qmail (and they will be glad to charge us $75 per hour to assist). Qmail hasn’t had an official upgrade in 10 years, so all you’re left with are 3rd party patches and after-market upgrades.

For a software application that is supposedly secure, qmail certainly seems to be full of holes when it comes out of the box because, so far as I know, we have not installed any patches.

In searching the Web for help on what may be happening I’ve come across references to a tcp-something-or-other patch that people have been advised to install through the years. Based on file names mentioned on those sites I conclude that we don’t have it installed. However, those Web sites don’t describe the problems we’re having so I’m not sure of whether that fix will fix our problem.

Whomever exploited qmail on Xenite was able to invoke the qmail-remote program from other hosts. We had specified only Xenite network hosts in our virtualdomains and rcpthosts files (both also under the qmail/control directory). To date I’ve only found one old forum discussion from about 2003 where this problem was described but the people in the forum offered nothing to resolve the issue.

Along the way I found that you can use /etc/hosts.deny and /etc/hosts.allow to keep other hosts from using your email services. This weekend I set up a line in hosts.deny that reads “ALL: ALL” and then I added our hosts to hosts.allow (e.g., “ALL: .xenite.org”). As I understand it, this supposedly prevents everyone else from using our mail service.

Alas! Sadly, it appears that it also prevents everyone else’s email from reaching me, so unless I whitelist every domain I could possibly expect email from I’ll have to undo my “fix”. Ick.

My partner pointed out that it will probably cost us less to just set up a new server and migrate everything over to that than to pay our hosting ISP $75 an hour to help us upgrade our O/S and email. Since I’ve wiped out more than one server in the past she normally does the migrations and upgrades and she doesn’t have time for it right now.

So I’m kind of stuck. Do I need to install the tcp-something-or-other patch that has been sporadically referred to? I know that qmail’s tcp-env program is used to set environment variables for the qmail-remote program. I know that qmail-remote is used to send email to other hosts so I need that (I was able to temporarily disable the spam by renaming the file to something else until I could figure out a different solution).

qmail-qstat is not reporting any unusual amounts of outbound messages right now (just a couple dozen or so, mostly bounces for spam from other domains and Vbulletin forum emails)and “ps -ef” doesn’t show all those spammy invocations of qmail-remote,so I’m pretty sure I’ve stopped the most recent exploit.

But can anyone suggest a way to resolve this WITHOUT having to install that tcp-something-or-other package? Why don’t I want to install it? I have no idea. I just DON’T trust stuff written by other people, I guess. Besides which, we’re going to do something about the server by June (I hope).

Any help will be appreciated. Oh, and if you believe qmail cannot be compromised, trust me, qmail is vulnerable to spam injection. Maybe I’ve been better off with qmail than sendmail but it would be nice if there were a more effective solution.

Oh yeah: I’ve checked out LifeWithQmail and QmailRocks and neither site seemed to be helpful in this matter. I’m a qmail-moron so I may have missed something that was covered in typical UNIX-hack fashion (all UNIX documentation is written in human-unintelligible cryptic nonsense).