Debian, mcollective

MCoMaster – an HTML5 Based GUI for Mcollective With Redis Discovery Method

Mcollective is a framework to build server orchestration or parallel job execution systems. It’s completely built on RUBY, and offcourse Open Sourced. There are a quite good plugins available for mcollective. In my previous blog’s i’ve explained on how to setup Mcollective with ActiveMQ and also how to use Mongo Discovery method with Mcollective. In this blog i will explain on how to use Mcollective with Rabbitmq connector and a new project called mcomaster, an HTML5 web interface to mcollective, which uses REDIS Discovery method in the backend.

Setting up RabbitMQ

First add the RaabitMQ APT repo,

$ echo "deb testing main" >> /etc/apt/sources.list

$ curl -L -o ~/rabbitmq-signing-key-public.asc 

$ apt-key add ~/rabbitmq-signing-key-public.asc

$ apt-get update

And install the RabbitMQ server

$ apt-get install rabbitmq-server

Now using rabbitmqctl create a vhost ”/mcollective”. Also create a user called ”mcollective” in Rabbitmq using rabbitmqctl and give full permission for this user to the new mcollective vhost.

$ rabbitmqctl add_vhost /mcollective

$ rabbitmqctl add_user mcollective mcollective

$ rabbitmqctl set_permissions -p /mcollective mcollective ".*" ".*" ".*"

There is also managemnet gui for rabbitmq, we can perform all the above tasks from that also. For that we need enable the management plugin in rabbitmq.

$ rabbitmq-plugins enable rabbitmq_management

And restart rabbitmq server. After restarting, we can access the gui using ”http://ip:55672”. Login user is ”guest”, password is ”guest”. We need to create two exchanges that the mcollective plugin needs. We can use the rabbitmqadmin command as mentioned in the Mcollective Documentation or we can use the Management GUI for that.

$ rabbitmqadmin declare exchange vhost=/mcollective name=mcollective_broadcast type=topic

$ rabbitmqadmin declare exchange vhost=/mcollective name=mcollective_directed type=direct 

Configuring Mcollective

Now install the mcollective from APT repository. We can use the puppetlabs APT repo for the latest versions.

$ echo "deb precise main" >> /etc/apt/sources.list

$ wget -o /tmp/pubkey.gpg && apt-key add /tmp/pubkey.gpg

$ apt-get intstall mcollective

Now go to ”/etc/mcollective/” folder and open the server.cfg file and add the below lines to enable the rabbitmq connector.

direct_addressing = 1

connector = rabbitmq
plugin.rabbitmq.vhost = /mcollective
plugin.rabbitmq.pool.size = 1       => increase this value, if you are going to use rabbitmq cluster =
plugin.rabbitmq.pool.1.port = 61613
plugin.rabbitmq.pool.1.user = mcollective
plugin.rabbitmq.pool.1.password = marionette

And restart the mcollective service, to make the changes to come effect.

Setting up McoMaster

Mcomaster is an HTML5 web interface to mcollective, which uses REDIS discovery method. Thanks to @ajf8 for such a wonderful project. Clone the repository from github.

$ git clone https//

Now copy the meta.rb registration file to the mcollective extension directory of all the Mcollective servers.

$ cp mcollective/registration/meta.rb /usr/share/mcollective/plugins/mcollective/registration/meta.rb

We need to enable the registration agent on ALL nodes with the following server.cfg settings

registerinterval = 300
registration = Meta

Now enable direct addressing by adding the below line in the server.cfg of all the server’s where we have copied the meta registration plugin.


We need one mcollective node which receives the registrations and saves them in redis. Ensure that Redis server is installed on that server. We can use the same server where we are going to install mcomaster.

$ cp mcollective/agent/registration.rb /usr/share/mcollective/plugins/mcollective/agent/

Add the redis server details to the server.cfg on the server which receives the registrations. = localhost
plugin.redis.port = 6379
plugin.redis.db = 0

Install mcollective clinet on the server where we are installing the mcomaster. Then configure the discovery agent, which will query the redis database for discovery data.

$ cp mcollective/discovery/redisdiscovery.* /usr/share/mcollective/plugins/mcollective/discovery/

And add the following settings to the client.cfg = localhost
plugin.redis.port = 6379
plugin.redis.db = 0
default_discovery_method = redisdiscovery
direct_addressing = yes

Using mco command we can check the node discovery using redisdiscovery method, which is the default mwthod now.. The old mc method will also works.

root@testcloud:~# mco rpc rpcutil ping --dm redisdiscovery -v
info 2013/05/29 10:08:45: rabbitmq.rb:10:in `on_connecting' TCP Connection attempt 0 to stomp://mcollective@localhost:61613
info 2013/05/29 10:08:45: rabbitmq.rb:15:in `on_connected' Conncted to stomp://mcollective@localhost:61613
Discovering hosts using the redisdiscovery method .... 4

 * [ ============================================================> ] 4 / 4

deeptest                                : OK

ubuntults                               : OK

debwheez                                : OK

testcloud                               : OK

---- rpcutil#ping call stats ----
           Nodes: 4 / 4
     Pass / Fail: 4 / 0
      Start Time: 2013-05-29 10:08:46 +0530
  Discovery Time: 30.95ms
      Agent Time: 85.24ms
      Total Time: 116.19ms
info 2013/05/29 10:08:46: rabbitmq.rb:20:in `on_disconnect' Disconnected from stomp://mcollective@localhost:61613

Go to McoMaster repo folder, and run bundle install to install all the dependency gems. Copy th existing application.example.yml and create a new application.yml file. Edit this file and fill in the Username, Password, Redis server details etc. Create a new database.yml and fill in the database details. I’m using MYSQL as the database backend. Below is my config.

  adapter: mysql
  database: mcollective
  username: mcollective
  password: mcollective
  host: localhost
  socket: "/var/run/mysqld/mysqld.sock"

  adapter: mysql
  database: mcollective
  username: mcollective
  password: mcollective
  host: localhost
  socket: "/var/run/mysqld/mysqld.sock"

We need to migrate the database,

$ RAILS_ENV=production rake db:reset

After that, we need to compile the assets,

$ rake assets:precompile

Let’s start the server,

$ rails server -e production

The above command will make the application to listen to default port ”3000”. Access the McoMaster GUI from the browwser using the username and password added in the ”application.yml” file. Once we login, we will be able to see the nodes discovered through the redisdiscovery method. If we add the plugins like service and process, we will be able to see the plugins showing up in the mcomaster, and we can also run these plugins with in the McoMaster.

This is a screenshot of the MCoMaster dashboard.

Debian, mcollective

Runing MCollective in a Multi-Ruby Environment

Mcollective one of the best open sourced orchestration tools. It’s build on Ruby and depends on Ruby1.8. But now a days most of the Ruby apps are build on Ruby1.9+, so the default Ruby versions will be 1.9+. This ususally causes problem for Mcollective to run. By default when we install Mcollective from APT, it will install Ruby1.8 also. So the best hack to make Mcollective run under such Multi-Ruby setup is edit the /usr/sbin/mcollectived file,

From #!/usr/bin/env ruby*, Change it to #!/usr/bin/env ruby1.8.

This will help the mcollective to work normally. That’s it, a small hack, but will helps us in Multi Ruby environment

Debian, Qmail, SMTP

Plugging QPSMTPD Service With QMAIL

QPSMTPD is a flexible smtp daemon written in Perl. Apart from the core SMTP features, all functionality is implemented in small extension plugins using the easy to use object oriented plugin API. Basically I uses Qmail Based mail server’s with a custom qmailqueue. We uses the tcpsvd (TCP/IP Service Daemon) for the smtp service and the mails are passed to a custom qmailqueue which is basically a perl script with custom filters for filtering out the mails. QPSMTPD has a verity of custom plugins which includes SPF check, DKIM check and even for DMARC also. If you are a Perl guy, then you can build custom plugins. The main reason why i got attracted to QPSMTPD was because of its plugin nature.

In this blog i will explain on how to setup QPSMTPD along with QMAIL MTA. As i mentioned earlier, i’m using a custom qmailqueue, where i have some custom filtering which varies from client to client. So i will be using QPSMTPD to do initail filtering like checking DNSBL/RBL, SPF check, DKIM, DMARC etc. A breif info about various qpsmtpd plugins are available here

Ths qpsmtpd source is available in Github. The soucre comes with default run scripts which can be used with runit/daemontools. Clone the qpsmtpd source and install the dependency Perl modules.

$ git clone

$ cpan -i Net::DNS

$ cpan -i MIME::Base64

$ cpan -i Mail::Header

Now create a user for the qpsmtp service, say ”smtp” with home folder as the location of the qpsmtpd folder and chown the qpsmtpd folder using the smptp user. Add sticky bit to the qpsmtpd folder by running chmod o+t qpsmtpd, in order to make supervise start the log process. By deafult inside the source folder there will be a sample config folder called “config.sample”. Copy the entire folder and create a new config folder.

$ cp config.sample config

In the config folder, edit the ”IP” in order to mention which ip the qpsmtpd daemon should bind. Putting “0” will bind to all the interfaces. Now if we go through the qpsmtpd’s run script in the source folder, it depends on two binaries softlimit and tcpserver. The softlimit binary comes with the daemontools debian package and the tcpserver binary comes with the ucspi-tcp debian package. so let’s install those two packages.

$ apt-get install ucspi-tcp daemontools runit

Now start the qpsmtpd server. I’m using runit for service supervision.

$ update-service --add /usr/local/src/qpsmtpd qpsmtpd

The above command will add the service. We can check the service status using sv s qpsmtpd command. This will show us whether the serivce is running or not. Now go inside the “config” folder and open the “plugin” file. This is where we enable the plugins, by addin the plugin names with corresponding options. By default the ”rcpt_ok” plugin must be enabled. This plugin handles the qmail’s rcpthosts feature. It accepts the emails for the domains mentioned in the ”rcpthosts” file present in the config folder. If this is not enabled it will not accept any mails. So the best way to understand how each plugin works is comment out all the plugins except “rcpt_ok” and then add the plugins one by one. The plugins are available in the “plugin” folder in the qpsmtpd source folder. All the basic info about the plugins are mentioned in the plugin files itself.

Now most commonly used plugins are auth for SMTP AUTH, DNSBL/RBL, spamassassin, etc. We can enable these plugins by adding the names in the config/plugin files. For example, since i’m using a custom qmailqueue, once the qpsmtpd has accepted the mail, it has to be queued to my custom QMAILQUEUE. So i need to enable the “queue plugin”. I can enable the plugin by adding the below line to the plugin file inside the config folder.

queue/qmail-queue /var/qmail/bin/qmail-scanner-queue

If you are using any other MTA, you can provide the corresponding MTA’s queue. For example for postfix ”postfix-queue”, and for exim use ”exim-bsmtp”, or if you want to use QPSMTPD as a relaying server, you can use ”smtp-forward” plugin for relaying mails to another SMTP server. So once the mail has been accepted by qpsmtpd, it will queue the mail to my custom qmail queue, and then it will start the mail delivery. Similarly i use ldap backend for smtp authentication. So i need to enable ”auth/auth_ldap_bind” plugin for this. Like that we can add other plugins too. By default DMARC plugin is not added, but we can get it from here.

Use tools like swaks for sending test mails, because plugins like check_basicheaders will not accept mails without proper headers, so using telnet to send mails wont work some times. Swaks is a good tool for sending test mail. We can increase the loglevel, by editing config/loglevel file. It’s better to increase the log level to debug so that we will get more details of errors. Some plugins needs certain Perl modules, if it’s missing the error will popup in the qpsmtpd logs, so use cpan and install those perl modules.

By default the run script uses tcpserver to start the service. There many other ways of deployments like forkserver,pre-fork daemon,Apache::Qpsmtpd etc. To use the default TLS plugin, we need to use the ”forkserver model”. The forke server model script is availbale in the same run script, but it is commented by default. The default spool directory will be a ”tmp” folder inside the QPUSER’s ie, the user “smtp” home folder. In my case i’m using a separate folder for spool, /var/spool/qpsmtp, for such cases, edit lib/ and go to ”spool_dir” subroutine and add ”$Spool_dir = “/var/spool/qpsmtpd/”;”. Now create the spool directory with owner as ”smtp” user and folder permission ”0700” and then restart the qpsmtpd service.

Now to enable TLS, enable the tls plugin in the config/plugin file like this ”tls cert_path priv_key_path ca_path”. If there is no TLS certificate available ,then we can generate using a perl script ”tls_cert”, which is available at the plugins folder. Now we need to edit the config/tls_before_auth file and put the value “0”, otherwise AUTH will not be offered unless TLS/SSL are in place. Now we can try sending a test mail using swaks with TLS enabled. Below is my swaks output.

=== Trying
=== Connected to
<-  220 ESMTP  send us your mail, but not your spam.
 -> EHLO
<- Hi deeptest []
<-  250-8BITMIME
<-  250-SIZE 5242880
<-  250-STARTTLS
<-  220 Go ahead with TLS
=== TLS started w/ cipher xxxxxx-xxx-xxxxxx
=== TLS peer subject DN="/C=XY/ST=unknown/L=unknown/O=QSMTPD/OU=Server/"
 ~> EHLO
<~ Hi deeptest []
<~  250-8BITMIME
<~  250-SIZE 5242880
<~  235 PLAIN authentication successful for deepak - authldap/plain
 ~> MAIL FROM:<>
<~  250 <>, sender OK - how exciting to get mail from you!
 ~> RCPT TO:<>
<~  250 <>, recipient ok
 ~> DATA
<~  354 go ahead
 ~> Date: Wed, 01 May 2013 23:19:54 +0530
 ~> To:
 ~> From:
 ~> Subject: testing TLS + Auth in qpsmtpd
 ~> X-Mailer: swaks v20120320.0
 ~> This is a test mailing
 ~> .
<~  250 Queued! 1367430597 qp 9222 <>
 ~> QUIT
<~  221 closing connection. Have a wonderful day.
=== Connection closed with remote host.