Debian, mcollective

MCoMaster – an HTML5 Based GUI for Mcollective With Redis Discovery Method

Mcollective is a framework to build server orchestration or parallel job execution systems. It’s completely built on RUBY, and offcourse Open Sourced. There are a quite good plugins available for mcollective. In my previous blog’s i’ve explained on how to setup Mcollective with ActiveMQ and also how to use Mongo Discovery method with Mcollective. In this blog i will explain on how to use Mcollective with Rabbitmq connector and a new project called mcomaster, an HTML5 web interface to mcollective, which uses REDIS Discovery method in the backend.

Setting up RabbitMQ

First add the RaabitMQ APT repo,

$ echo "deb http://www.rabbitmq.com/debian/ testing main" >> /etc/apt/sources.list

$ curl -L -o ~/rabbitmq-signing-key-public.asc http://www.rabbitmq.com/rabbitmq-signing-key-public.asc 

$ apt-key add ~/rabbitmq-signing-key-public.asc

$ apt-get update

And install the RabbitMQ server

$ apt-get install rabbitmq-server

Now using rabbitmqctl create a vhost ”/mcollective”. Also create a user called ”mcollective” in Rabbitmq using rabbitmqctl and give full permission for this user to the new mcollective vhost.

$ rabbitmqctl add_vhost /mcollective

$ rabbitmqctl add_user mcollective mcollective

$ rabbitmqctl set_permissions -p /mcollective mcollective ".*" ".*" ".*"

There is also managemnet gui for rabbitmq, we can perform all the above tasks from that also. For that we need enable the management plugin in rabbitmq.

$ rabbitmq-plugins enable rabbitmq_management

And restart rabbitmq server. After restarting, we can access the gui using ”http://ip:55672”. Login user is ”guest”, password is ”guest”. We need to create two exchanges that the mcollective plugin needs. We can use the rabbitmqadmin command as mentioned in the Mcollective Documentation or we can use the Management GUI for that.

$ rabbitmqadmin declare exchange vhost=/mcollective name=mcollective_broadcast type=topic

$ rabbitmqadmin declare exchange vhost=/mcollective name=mcollective_directed type=direct 

Configuring Mcollective

Now install the mcollective from APT repository. We can use the puppetlabs APT repo for the latest versions.

$ echo "deb http://apt.puppetlabs.com precise main" >> /etc/apt/sources.list

$ wget -o /tmp/pubkey.gpg apt.puppetlabs.com/pubkey.gpg && apt-key add /tmp/pubkey.gpg

$ apt-get intstall mcollective

Now go to ”/etc/mcollective/” folder and open the server.cfg file and add the below lines to enable the rabbitmq connector.

direct_addressing = 1

connector = rabbitmq
plugin.rabbitmq.vhost = /mcollective
plugin.rabbitmq.pool.size = 1       => increase this value, if you are going to use rabbitmq cluster
plugin.rabbitmq.pool.1.host = stomp.example.com
plugin.rabbitmq.pool.1.port = 61613
plugin.rabbitmq.pool.1.user = mcollective
plugin.rabbitmq.pool.1.password = marionette

And restart the mcollective service, to make the changes to come effect.

Setting up McoMaster

Mcomaster is an HTML5 web interface to mcollective, which uses REDIS discovery method. Thanks to @ajf8 for such a wonderful project. Clone the repository from github.

$ git clone https//github.com/ajf8/mcomaster.git

Now copy the meta.rb registration file to the mcollective extension directory of all the Mcollective servers.

$ cp mcollective/registration/meta.rb /usr/share/mcollective/plugins/mcollective/registration/meta.rb

We need to enable the registration agent on ALL nodes with the following server.cfg settings

registerinterval = 300
registration = Meta

Now enable direct addressing by adding the below line in the server.cfg of all the server’s where we have copied the meta registration plugin.

direct_addressing=1

We need one mcollective node which receives the registrations and saves them in redis. Ensure that Redis server is installed on that server. We can use the same server where we are going to install mcomaster.

$ cp mcollective/agent/registration.rb /usr/share/mcollective/plugins/mcollective/agent/

Add the redis server details to the server.cfg on the server which receives the registrations.

plugin.redis.host = localhost
plugin.redis.port = 6379
plugin.redis.db = 0

Install mcollective clinet on the server where we are installing the mcomaster. Then configure the discovery agent, which will query the redis database for discovery data.

$ cp mcollective/discovery/redisdiscovery.* /usr/share/mcollective/plugins/mcollective/discovery/

And add the following settings to the client.cfg

plugin.redis.host = localhost
plugin.redis.port = 6379
plugin.redis.db = 0
default_discovery_method = redisdiscovery
direct_addressing = yes

Using mco command we can check the node discovery using redisdiscovery method, which is the default mwthod now.. The old mc method will also works.

root@testcloud:~# mco rpc rpcutil ping --dm redisdiscovery -v
info 2013/05/29 10:08:45: rabbitmq.rb:10:in `on_connecting' TCP Connection attempt 0 to stomp://mcollective@localhost:61613
info 2013/05/29 10:08:45: rabbitmq.rb:15:in `on_connected' Conncted to stomp://mcollective@localhost:61613
Discovering hosts using the redisdiscovery method .... 4

 * [ ============================================================> ] 4 / 4


deeptest                                : OK
    {:pong=>1369802330}

ubuntults                               : OK
    {:pong=>1369802330}

debwheez                                : OK
    {:pong=>1369802330}

testcloud                               : OK
    {:pong=>1369802326}



---- rpcutil#ping call stats ----
           Nodes: 4 / 4
     Pass / Fail: 4 / 0
      Start Time: 2013-05-29 10:08:46 +0530
  Discovery Time: 30.95ms
      Agent Time: 85.24ms
      Total Time: 116.19ms
info 2013/05/29 10:08:46: rabbitmq.rb:20:in `on_disconnect' Disconnected from stomp://mcollective@localhost:61613

Go to McoMaster repo folder, and run bundle install to install all the dependency gems. Copy th existing application.example.yml and create a new application.yml file. Edit this file and fill in the Username, Password, Redis server details etc. Create a new database.yml and fill in the database details. I’m using MYSQL as the database backend. Below is my config.

production:
  adapter: mysql
  database: mcollective
  username: mcollective
  password: mcollective
  host: localhost
  socket: "/var/run/mysqld/mysqld.sock"

development:
  adapter: mysql
  database: mcollective
  username: mcollective
  password: mcollective
  host: localhost
  socket: "/var/run/mysqld/mysqld.sock"

We need to migrate the database,

$ RAILS_ENV=production rake db:reset

After that, we need to compile the assets,

$ rake assets:precompile

Let’s start the server,

$ rails server -e production

The above command will make the application to listen to default port ”3000”. Access the McoMaster GUI from the browwser using the username and password added in the ”application.yml” file. Once we login, we will be able to see the nodes discovered through the redisdiscovery method. If we add the plugins like service and process, we will be able to see the plugins showing up in the mcomaster, and we can also run these plugins with in the McoMaster.

This is a screenshot of the MCoMaster dashboard.

Advertisements
Standard

2 thoughts on “MCoMaster – an HTML5 Based GUI for Mcollective With Redis Discovery Method

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s