Using Mongo Discovery Method in MCollective

I’ve been playing around with MCollective for the past few months. But this time i wanted to try out theMongo discovery method. The response time was quite faster for the Mongo discovery method. So i really wanted to try it out. Setting out MCollective Server/Client is prettyl simple. You can go through my previous blog. Now we need to install the Meta registration plugin on all the MCollective Servers. Just Download and Copy meta.rb in the MCollective registration plugin folder. In my case, i’ve Debian based machine’s, so the location is, /usr/share/mcollective/plugins/mcollective/registration/. This will make the metadata available to other nodes.

Now add the below three lines into the server.cfg of all the MCollective server’s.

registration = Meta
registerinterval = 300
factsource = facter

Now install the mongodb registration agent on one of the nodes, which will be our slave node.. Do not install this on all the nodes. There is a small bug in this agent. So follow the steps mentioned here and modify the registration.rb file. Now install mongoDB server on the slave node. Also add the below lines to the server.cfg in the slave machine.

plugin.registration.mongohost = localhost
plugin.registration.mongodb = puppet
plugin.registration.collection = nodes

Now restart the mcollective service. If we increase the log level to debug, then we can see the below lines in the mcollective.log. This indicates that the plugin is getting activated and it is receiving request from the machines, whose fqdn is shown in the below line.

D, [2012-11-29T15:51:34.391762 #12731] DEBUG -- : registration.rb:97:in `handlemsg' Updated data for host vagrant-debian-squeeze.vagrantup.com with id 50b650d4454bc346e4000002 in 0.0027310848236084s
D, [2012-11-29T15:50:05.810180 #12731] DEBUG -- : registration.rb:97:in `handlemsg' Updated data for host ubuntults.vargrantup.com with id 50b650c0454bc346e4000001 in 0.00200200080871582s

Initially, i used the default registration.rb file which i downloaded from the github. But it was giving me an error handlemsg Got stats without a FQDN in facts. So don’t forget to modify theregistration.rb

Now go connect to mongoDB and verify that the nodes are getting registered in it.

$ mongo
 MongoDB shell version: 2.0.4
 connecting to: test
 > use puppet
 switched to db puppet
 > db.nodes.find().count()
 2

So, now both my master and slave have been registered into the mongoDB. Now in order to use theMongo Discovery Method, we need to install the mongodb discovery plugin and also we need to enable the direct addressing mode. so we need to add direct_addressing = 1 in the server.cfg file.

Now we can use the –dm option to specify the discovery method.

$ mco rpc rpcutil ping --dm=mongo -v
  Discovering hosts using the mongo method .... 2

  * [ ========================================================> ] 2 / 2


  vagrant-debian-squeeze                  : OK
    {:pong=>1354187911}

  ubuntults                               : OK
	    {:pong=>1354187880}



  ---- rpcutil#ping call stats ----
            Nodes: 2 / 2
      Pass / Fail: 2 / 0
      Start Time: Thu Nov 29 16:48:00 +0530 2012
  Discovery Time: 68.48ms
  	  Agent Time: 108.35ms
  	  Total Time: 176.83ms

$ mco rpc rpcutil ping --dm=mc -v
  Discovering hosts using the mc method for 2 second(s) .... 2

  * [ ========================================================> ] 2 / 2


  vagrant-debian-squeeze                  : OK
 	    {:pong=>1354188083}

  ubuntults                               : OK
	    {:pong=>1354188053}



  ---- rpcutil#ping call stats ----
            Nodes: 2 / 2
   	  Pass / Fail: 2 / 0
   	  Start Time: Thu Nov 29 16:50:52 +0530 2012
  Discovery Time: 2004.24ms
  	  Agent Time: 104.28ms
  	  Total Time: 2108.51ms

From the above commands, we can see the difference in the Discovery Time.

Now for those who want GUI, R.I.Pienaar has develeoped a web gui called Mco-Rpc-Web. He has uploaded a few screencasts, which will give us a short demo on all of these.

Hiera-HTTP and RIAK

For the past few days, i was playing with hiera-puppet. I strongly wanted to use hiera in my puppet modules. Yesterday i was going through craigdunn’s blog, there i saw his new blog post about thehiera-http,a Hiera back end to connect to any HTTP RESTful API and return data based on a lookup.hiera-http is available as a rubygem or from craigdunn’s Github page. In his blog he has clearly explained how he uses hiera-http to query data from couchdb. Since i’m a fan of Riak, i wanted to try the same from Riak.

I used the riak-ruby-client to pass my json data to Riak. Below is a simple ruby script which i used,

#! /usr/local/bin/ruby
require 'rubygems'
require 'riak'

client = Riak::Client.new(:host => '192.168.1.120', :http_port => 8098)
bucket = client.bucket("hiera_bucket")
deb = Riak::RObject.new(bucket, "Debian")
deb.content_type = "application/json"
deb.data = { "db_name" => "testdb", "db_pass" => "P@ssw0rd" }
deb.store

where hiera_bucket is the name of the Bucket and Debian is the key.

Now we need to create a hiera.yaml,

:backends:

   – http

:http:
   :host: 192.168.1.120
   :port: 8098
   :output: json
   :failure: graceful
   :paths: /riak/hiera_bucket/Debian

We can test the working using hiera cli hiera-c hiera.yaml db_name
As per the hiera-http dcumentation, we can use the facter variables, so the path can be :paths: /riak/hiera_bucket/%{operatingsystem}. For those who wants a gui for Riak, i prefer rekon. It’s very simple to install and configure, also we can edit our values directly using this.

Vagrant: Make Virtualization Easier

Vagrant is a light weight virtualization tool, build over Oracle’s VirtualBox. It’s completely written in ruby and it’s very easy to install and configure. The only dependency is VirtualBox. Once VirtualBox is installed, we can either use RubyGems to install Vagrant or we can get the installer from Vagrant.

I prefer ruby gems, So

$ gem install vagrant

That’s it, it will install vagrant. Now we need to get the base box. Vagrant uses the base boxes to build the VM’s. We can download the base boxes from Vagrant Boxes. we can directly use the url to create the boxes, but it’s alwasy good if we have a downloaded copy of the base boxes.

Basically we need to follow just 3 steps. add,initialize,up. So once we have the base box just add the base box.

$ vagrant box add new_box_name our_downloaded_basebox_file

example,
$ vagrant box add mybox precise32.box

Now just do a vagrant init to create the Vagrantfile. This is the main configuration file. If we go through the Vagrantfile, we can see a bunch of options like port forward, provisioning, setting network and so on. If only one vm is required, then we just have to add the new_box name which we created at the config.vm.box= option int the Vagrantfile. And vagrant up will start the VM.

But one of the most important feature of Vagrant is it suports multiple VM’s over one single box. But we have to define those VM’s in the Vagrantfile. Below i’ve defined two VM’s in my Vagrantfile, also comment out config.vm.box=base

config.vm.define :ubuntu do |ubuntu_config|
	 ubuntu_config.vm.box = "precise32"
end
config.vm.define :puppet do |puppet_config|
	 puppet_config.vm.box = "precise32"
end

Where ubuntu and puppet are the name of the VM’s. And precise32 is the name of the the box which i’ve created. Now, vagrant up will start all the VM’s. But we can mention the name to start a specific VM. Like vagrant up ubuntu. It will start only the ubuntu VM.

Provisioning is another important feature of Vagrant. We can use puppet,chef and shell scripts to bootstrap the vm’s. I saw @mitchellh’s talk at PuppetConf 2012, in which he mentioned about how to create a Fully Automated Puppet Master using the shell provisioner. I’ve tried this out using a shell script that will install puppet master and genrate the ssl certificates. It’s working fine, soon i will post it here.