Monitoring, NodeJS, Sensu

UCHIWA – Awesome Dashboard for Sensu

It’s been more than a year since i’ve started playing with Sensu. It was one of coolest monitoring projects that i’ve ever worked with. Perfect for Cloud infrastructure and backed by a cool community. Though Sensu is still in Juvenile (v 0.12) state, its mature enough to tackle majority of the monitoring issues. Recently i started migrating all our monitoring from traditional Nagios to Sensu and the pretty cool thing with sensu is, we can directly use the Nagios plugins with Sensu. That’s an easy task for migration. We don’t have to rebuild all the check’s to make with Sensu. But most of the people outside was pointing to the sensu’s default dashboard. Though the dashboard doesn’t looks pretty fancy, it can do all the functions. But having a good dashboard which can display the current status is always a time saver.

So i was being searching for a good dashboard and the first choice on the google search was Sensu-Admin. A Rails project, which needs a backend DB. But still i was not satisfied with it, and i started looking out for something different. The second choice was sabisu. Sabisu uses Cloudant’s hosted Couchdb with Lucene. We just need to store all the events in a Redis List and a custom script which reads the data from the Redis List and pushes it to the Cloudant’s CouchDB. So we basically need a Cloudant account and the Webapp makes Lucene queries to the Cloudant DB and displays the results on the Sabisu dashboard. Though i tried to rebuild the same setup locally, like running a CouchDB+Lucene locally and sending the same data to the local couchdb. With some codehack’s i was able to make the webapp talk to my local CouchDB and display the results on the dashboard.

But then, I found a super cool dashboard project called UCHIWA which was started in Github a few days back by Simon Plourde. Uchiwa is simple dashboard built with NodeJS and uses SocketIO for real time updates. The screenshot’s looks super cool and i decided to give it a try. It has only one dependency, NodeJS, no backend DB’s required as it talks to the Sensu’s API in realtime.

Setting Up NodeJS

For Ubuntu, we can use the chris-lea’s PPA.

apt-get install python-software-properties        # required for "apt-add-repository" binary

apt-add-repository ppa:chris-lea/node.js

apt-get update

apt-get install nodejs

now we have the latest NodeJS on our system, we can start setting up Uchiwa.

Setting Up Uchiwa Dashboard

Uchiwa’s source is available in Github.

git clone https://github.com/palourde/uchiwa.git

Once cloned, the repository contains the “package.json” file which contains the list of necessary dependencies. We can use “npm” (node package manager) to install all these.

cd uchiwa

npm install

Now we need to create a config file for the app. There is a sample config file available in the repo. So we need to mention the Sensu’s API IP and Port number and also the auth credentials if any. Plus auth credentials for accessing Uchiwa Dashboard page.

once all these are set, we are done. We just need to start the service.

node app.js 

We can access the page via http://localhost:3000/, or we can proxy pass from the webserver. Instructions for Nginx is available on the Readme of the project. Now we need to keep this app running all the time. So it’s better to create a init/upstart process for the same, so that the process will start automatically when the system reboots. There is a cool Node project called forever which is a simple CLI tool for ensuring that a given script runs continuously.

I’ve created an upstart script for Uchiwa, bu putting a conf file “uchiwa.conf” in the “/etc/init” directory. Below is the content for the conf file. Once the file is in place, we have to do a reload of the upstart configuration. initctl reload-configuration will do the trick.

description "uchiwa - dashboard for sensu"
env APP_PATH = "/usr/local/uchiwa/"

start on startup
stop on shutdown

script
  cd $APP_PATH
  exec forever start app.js
end script

Uchiwa looks pretty cool and neat and it has the stash support also. There are couple of addons required like “Downtime”, but Uchiwa is a pretty new project and i’m sure that this project is gonna grow soon. It has already received 99 stars on the Github. Kudos to Simon Plourde for Open Sourcing this awesome project.

Advertisements
Standard
Debian, NodeJS, SMTP

HARAKA – a NodeJS Based SMTP Server

Today i came across a very interesting project in GITHUB. HARAKA is an SMTP server written completely in NodeJS. Like the qpsmtpd, apart from the core SMTP features we can improve the functionality using small plugins. There are very good pluginsi for HARAKA, basically in javascripts. Like Postfix,Qmail, we can easily implements all sorts of checks and features with the help of these plugins.

Setting up HARAKA is very simple. In my setup, i will be using HARAKA as my primary smtp server, where i will implement all my filterings and then i will relay to a qmail server for local delivery. There is plugin written by @madeingnecca in github, for directly delivering the mails to user’s INBOX (mail box should be in MAILDIR format). In the real server’s we use LDAP backend for storing all the USER databases. So before putting HARAKA into production, i need a to rebuild the auth plugin so that HARAKA can talk to LDAP for user authentication in SMTP.

So first we need to install NodeJS and NPM (Node Package Manager). There are several ways for installing NodeJS. We can compile it from the source, or we can use NVM (Node Version Manager), or we can install the packages from APT in Debian machines. But i prefer source code, because official APT repo has older versions of NodeJS, which will create compatibility issue. Current version is “v0.10.4”. Building NodeJS from source is pretty simple.

Just Download the latest source code from http://nodejs.org/download/”

$ wget http://nodejs.org/dist/v0.10.4/node-v0.10.4.tar.gz

$ tar xvzf node-v0.10.4.tar.gz && cd node-v0.10.4

$  ./compile 

$ make && make install

Once NodeJS is installed, we can go ahead with HARAKA.

$ git clone https://github.com/baudehlo/Haraka.git

Now go inside to the Haraka folder and run the below command. All the dependency packages are mentioned in the package.json file.

$ npm install

The above command will install all the necessary modules mentioned in the package.json file and will setup HARAKA. Now we can setup a separate service folder for HARAKA.

$ haraka -i /etc/haraka     

The above command will create the haraka folder in /etc/ and it will create creates config and plugin directories in there, and automatically sets the host name used by Haraka to the output of the hostname command. Now we need to setup up the port number and ip which HARAKA SMTP service should listen. Go to config folder in the newly created haraka service folder and open the “smtp.ini” file, and mention the port number and ip.

Now before starting the smtp service, first let’s disable all the plugins, so that we can go in steps. In the config folder, open the “plugin” file, and comment out all the plugins, because by default haraka will not create any plugin scripts, so most of them mentioned in that will not work. So we will start using the plugins, once we have copied the corresponding plugin’s js files to the plugin directory inside our service directory.

Let’s try running the HARAKA foreground and see if it starts and listens on the port we mentioned.

$ haraka -c /etc/haraka

Once HARAKA SMTP service starts, we can see the line ”[NOTICE] [-] [core] Listening on :::25” in the STDOUT, which means HARAKA is listening on port 25. We can just Telnet to port 25 and see if we are getting SMTP banner.

Now we can try out a plugin. Haraka has a spamassassin plugin. So will try it out. So first install spamassassin and start the spam filtering.

$ apt-get install spamassassin spamc

Now from the plugin folder inside the git source folder of HARAKA, copy the spamassassin.js and copy it to the plugin folder of our service directory. By default plugin folder is not created inside the service directory, so create it. Now we need to enable the service. Inside the config folder of our service directory, create a config file “spamassassin.ini”, and inside the file fill in the necessary details like, “reject_thresold”, “subject_prefix”, “spamd_socket”. Now before starting the plugin, we need to add it in the plugin, inside the config folder. Once spamassassin plugin is added, we can start the HARAKA smtp service. If the plugin is added properly, then we can see the below lines in the stdout,

[INFO] [-] [core] Loading plugin: spamassassin
[DEBUG] [-] [core] registered hook data_post to spamassassin.hook_data_post

Now using swaks, we can send a test mail see, if spam assassin is putting scores for the emails. Like this we can enable all other plugins, based on our needs.

Since i’m going to relay the mails, i need to make HARAKA to accept mails for all my domains. For that i need to define all my domains on HARAKA. In the config folder, open the file “host_list”, and add all the domains for which HARAKA should accept mails. There also a regular expression option available for, which can be done in “host_list_regex” file.

Now we need to add, smtp relay, for that edit the “smtp_forward.ini” file and mention the relay host ip, port number and auth details(if required). Now we can restart the HARAKA service and we can check SMTP relay by sending test mails using swaks.

I haven’t tried the Auth plugin yet, but soon i will be trying it. If possible, i will try to use LDAP backend for authentication, so that HARAKA can be used a full fledged SMTP service. More developments are happening in this, hope it wil become a good competitor …

Standard