Plugging QPSMTPD Service With QMAIL

QPSMTPD is a flexible smtp daemon written in Perl. Apart from the core SMTP features, all functionality is implemented in small extension plugins using the easy to use object oriented plugin API. Basically I uses Qmail Based mail server’s with a custom qmailqueue. We uses the tcpsvd (TCP/IP Service Daemon) for the smtp service and the mails are passed to a custom qmailqueue which is basically a perl script with custom filters for filtering out the mails. QPSMTPD has a verity of custom plugins which includes SPF check, DKIM check and even for DMARC also. If you are a Perl guy, then you can build custom plugins. The main reason why i got attracted to QPSMTPD was because of its plugin nature.

In this blog i will explain on how to setup QPSMTPD along with QMAIL MTA. As i mentioned earlier, i’m using a custom qmailqueue, where i have some custom filtering which varies from client to client. So i will be using QPSMTPD to do initail filtering like checking DNSBL/RBL, SPF check, DKIM, DMARC etc. A breif info about various qpsmtpd plugins are available here

Ths qpsmtpd source is available in Github. The soucre comes with default run scripts which can be used with runit/daemontools. Clone the qpsmtpd source and install the dependency Perl modules.

$ git clone https://github.com/smtpd/qpsmtpd/

$ cpan -i Net::DNS

$ cpan -i MIME::Base64

$ cpan -i Mail::Header

Now create a user for the qpsmtp service, say ”smtp” with home folder as the location of the qpsmtpd folder and chown the qpsmtpd folder using the smptp user. Add sticky bit to the qpsmtpd folder by running chmod o+t qpsmtpd, in order to make supervise start the log process. By deafult inside the source folder there will be a sample config folder called “config.sample”. Copy the entire folder and create a new config folder.

$ cp config.sample config

In the config folder, edit the ”IP” in order to mention which ip the qpsmtpd daemon should bind. Putting “0” will bind to all the interfaces. Now if we go through the qpsmtpd’s run script in the source folder, it depends on two binaries softlimit and tcpserver. The softlimit binary comes with the daemontools debian package and the tcpserver binary comes with the ucspi-tcp debian package. so let’s install those two packages.

$ apt-get install ucspi-tcp daemontools runit

Now start the qpsmtpd server. I’m using runit for service supervision.

$ update-service --add /usr/local/src/qpsmtpd qpsmtpd

The above command will add the service. We can check the service status using sv s qpsmtpd command. This will show us whether the serivce is running or not. Now go inside the “config” folder and open the “plugin” file. This is where we enable the plugins, by addin the plugin names with corresponding options. By default the ”rcpt_ok” plugin must be enabled. This plugin handles the qmail’s rcpthosts feature. It accepts the emails for the domains mentioned in the ”rcpthosts” file present in the config folder. If this is not enabled it will not accept any mails. So the best way to understand how each plugin works is comment out all the plugins except “rcpt_ok” and then add the plugins one by one. The plugins are available in the “plugin” folder in the qpsmtpd source folder. All the basic info about the plugins are mentioned in the plugin files itself.

Now most commonly used plugins are auth for SMTP AUTH, DNSBL/RBL, spamassassin, etc. We can enable these plugins by adding the names in the config/plugin files. For example, since i’m using a custom qmailqueue, once the qpsmtpd has accepted the mail, it has to be queued to my custom QMAILQUEUE. So i need to enable the “queue plugin”. I can enable the plugin by adding the below line to the plugin file inside the config folder.

queue/qmail-queue /var/qmail/bin/qmail-scanner-queue

If you are using any other MTA, you can provide the corresponding MTA’s queue. For example for postfix ”postfix-queue”, and for exim use ”exim-bsmtp”, or if you want to use QPSMTPD as a relaying server, you can use ”smtp-forward” plugin for relaying mails to another SMTP server. So once the mail has been accepted by qpsmtpd, it will queue the mail to my custom qmail queue, and then it will start the mail delivery. Similarly i use ldap backend for smtp authentication. So i need to enable ”auth/auth_ldap_bind” plugin for this. Like that we can add other plugins too. By default DMARC plugin is not added, but we can get it from here.

Use tools like swaks for sending test mails, because plugins like check_basicheaders will not accept mails without proper headers, so using telnet to send mails wont work some times. Swaks is a good tool for sending test mail. We can increase the loglevel, by editing config/loglevel file. It’s better to increase the log level to debug so that we will get more details of errors. Some plugins needs certain Perl modules, if it’s missing the error will popup in the qpsmtpd logs, so use cpan and install those perl modules.

By default the run script uses tcpserver to start the service. There many other ways of deployments like forkserver,pre-fork daemon,Apache::Qpsmtpd etc. To use the default TLS plugin, we need to use the ”forkserver model”. The forke server model script is availbale in the same run script, but it is commented by default. The default spool directory will be a ”tmp” folder inside the QPUSER’s ie, the user “smtp” home folder. In my case i’m using a separate folder for spool, /var/spool/qpsmtp, for such cases, edit lib/Qpsmtpd.pm and go to ”spool_dir” subroutine and add ”$Spool_dir = “/var/spool/qpsmtpd/”;”. Now create the spool directory with owner as ”smtp” user and folder permission ”0700” and then restart the qpsmtpd service.

Now to enable TLS, enable the tls plugin in the config/plugin file like this ”tls cert_path priv_key_path ca_path”. If there is no TLS certificate available ,then we can generate using a perl script ”tls_cert”, which is available at the plugins folder. Now we need to edit the config/tls_before_auth file and put the value “0”, otherwise AUTH will not be offered unless TLS/SSL are in place. Now we can try sending a test mail using swaks with TLS enabled. Below is my swaks output.

=== Trying 192.168.42.189:587...
=== Connected to 192.168.42.189.
<-  220 beingasysadmin.com ESMTP  send us your mail, but not your spam.
 -> EHLO deeptest.beingasysadmin.com
<-  250-beingasysadmin.com Hi deeptest [192.168.42.184]
<-  250-PIPELINING
<-  250-8BITMIME
<-  250-SIZE 5242880
<-  250-STARTTLS
<-  250 AUTH PLAIN LOGIN CRAM-MD5
 -> STARTTLS
<-  220 Go ahead with TLS
=== TLS started w/ cipher xxxxxx-xxx-xxxxxx
=== TLS peer subject DN="/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=Server/CN=debwheez.beingasysadmin.com/emailAddress=postmaster@debwheez.beingasysadmin.com"
 ~> EHLO deeptest.beingasysadmin.com
<~  250-beingasysadmin.com Hi deeptest [192.168.42.184]
<~  250-PIPELINING
<~  250-8BITMIME
<~  250-SIZE 5242880
<~  250 AUTH PLAIN LOGIN CRAM-MD5
 ~> AUTH PLAIN AGRlZXBhawBteWRlZXByb290
<~  235 PLAIN authentication successful for deepak - authldap/plain
 ~> MAIL FROM:<deepak@beingasysadmin.com>
<~  250 <deepak@beingasysadmin.com>, sender OK - how exciting to get mail from you!
 ~> RCPT TO:<deepakmdass88@gmail.com>
<~  250 <deepakmdass88@gmail.com>, recipient ok
 ~> DATA
<~  354 go ahead
 ~> Date: Wed, 01 May 2013 23:19:54 +0530
 ~> To: deepakmdass88@gmail.com
 ~> From: deepak@beingasysadmin.com
 ~> Subject: testing TLS + Auth in qpsmtpd
 ~> X-Mailer: swaks v20120320.0 jetmore.org/john/code/swaks/
 ~>
 ~> This is a test mailing
 ~>
 ~> .
<~  250 Queued! 1367430597 qp 9222 <>
 ~> QUIT
<~  221 beingasysadmin.com closing connection. Have a wonderful day.
=== Connection closed with remote host. 

HARAKA – a NodeJS Based SMTP Server

Today i came across a very interesting project in GITHUB. HARAKA is an SMTP server written completely in NodeJS. Like the qpsmtpd, apart from the core SMTP features we can improve the functionality using small plugins. There are very good pluginsi for HARAKA, basically in javascripts. Like Postfix,Qmail, we can easily implements all sorts of checks and features with the help of these plugins.

Setting up HARAKA is very simple. In my setup, i will be using HARAKA as my primary smtp server, where i will implement all my filterings and then i will relay to a qmail server for local delivery. There is plugin written by @madeingnecca in github, for directly delivering the mails to user’s INBOX (mail box should be in MAILDIR format). In the real server’s we use LDAP backend for storing all the USER databases. So before putting HARAKA into production, i need a to rebuild the auth plugin so that HARAKA can talk to LDAP for user authentication in SMTP.

So first we need to install NodeJS and NPM (Node Package Manager). There are several ways for installing NodeJS. We can compile it from the source, or we can use NVM (Node Version Manager), or we can install the packages from APT in Debian machines. But i prefer source code, because official APT repo has older versions of NodeJS, which will create compatibility issue. Current version is “v0.10.4”. Building NodeJS from source is pretty simple.

Just Download the latest source code from “http://nodejs.org/download/”

$ wget http://nodejs.org/dist/v0.10.4/node-v0.10.4.tar.gz

$ tar xvzf node-v0.10.4.tar.gz && cd node-v0.10.4

$  ./compile 

$ make && make install

Once NodeJS is installed, we can go ahead with HARAKA.

$ git clone https://github.com/baudehlo/Haraka.git

Now go inside to the Haraka folder and run the below command. All the dependency packages are mentioned in the package.json file.

$ npm install

The above command will install all the necessary modules mentioned in the package.json file and will setup HARAKA. Now we can setup a separate service folder for HARAKA.

$ haraka -i /etc/haraka     

The above command will create the haraka folder in /etc/ and it will create creates config and plugin directories in there, and automatically sets the host name used by Haraka to the output of the hostname command. Now we need to setup up the port number and ip which HARAKA SMTP service should listen. Go to config folder in the newly created haraka service folder and open the “smtp.ini” file, and mention the port number and ip.

Now before starting the smtp service, first let’s disable all the plugins, so that we can go in steps. In the config folder, open the “plugin” file, and comment out all the plugins, because by default haraka will not create any plugin scripts, so most of them mentioned in that will not work. So we will start using the plugins, once we have copied the corresponding plugin’s js files to the plugin directory inside our service directory.

Let’s try running the HARAKA foreground and see if it starts and listens on the port we mentioned.

$ haraka -c /etc/haraka

Once HARAKA SMTP service starts, we can see the line ”[NOTICE] [-] [core] Listening on :::25” in the STDOUT, which means HARAKA is listening on port 25. We can just Telnet to port 25 and see if we are getting SMTP banner.

Now we can try out a plugin. Haraka has a spamassassin plugin. So will try it out. So first install spamassassin and start the spam filtering.

$ apt-get install spamassassin spamc

Now from the plugin folder inside the git source folder of HARAKA, copy the spamassassin.js and copy it to the plugin folder of our service directory. By default plugin folder is not created inside the service directory, so create it. Now we need to enable the service. Inside the config folder of our service directory, create a config file “spamassassin.ini”, and inside the file fill in the necessary details like, “reject_thresold”, “subject_prefix”, “spamd_socket”. Now before starting the plugin, we need to add it in the plugin, inside the config folder. Once spamassassin plugin is added, we can start the HARAKA smtp service. If the plugin is added properly, then we can see the below lines in the stdout,

[INFO] [-] [core] Loading plugin: spamassassin
[DEBUG] [-] [core] registered hook data_post to spamassassin.hook_data_post

Now using swaks, we can send a test mail see, if spam assassin is putting scores for the emails. Like this we can enable all other plugins, based on our needs.

Since i’m going to relay the mails, i need to make HARAKA to accept mails for all my domains. For that i need to define all my domains on HARAKA. In the config folder, open the file “host_list”, and add all the domains for which HARAKA should accept mails. There also a regular expression option available for, which can be done in “host_list_regex” file.

Now we need to add, smtp relay, for that edit the “smtp_forward.ini” file and mention the relay host ip, port number and auth details(if required). Now we can restart the HARAKA service and we can check SMTP relay by sending test mails using swaks.

I haven’t tried the Auth plugin yet, but soon i will be trying it. If possible, i will try to use LDAP backend for authentication, so that HARAKA can be used a full fledged SMTP service. More developments are happening in this, hope it wil become a good competitor …