Debian, puppet, virtualization

Using Foreman With Puppet and Libvirt

TheForeman is one of the best provisioning tools available. It’s purely open-sourced. And it natively supports puppet for provisioning the nodes. Foreman can talk to libvirt also, which makes us easy to create a VM and provision it on the way. In this blog i will be explaining on how to install Foreman from the source, how to integrate it with puppet to receive the logs and facts and make Foreman to use Libvirt for building VM’s.

Setting up Foreman

First will install the basic depenencies. Since i’m using the git repository of Foreman for installation, git package has to be installed. Moreover we also need a database for Foreman. I’m going to use Mysql for that.

$ apt-get install git mysql-server ruby-mysql libmysql-ruby1.9.1 libmysqlclient-dev libvirt-dev 

Now clone the repository from github. The newer build’s works with Puppet 3.0.

$ git clone https://github.com/theforeman/foreman.git -b develop

Ensure that ”ruby and bundler” is installed in the machine.

$ bundle install --without postgresql sqlite

Now we can start configuring Foreman. Copy the sample config files.

$ cp config/settings.yaml.example config/settings.yaml
$ cp config/database.yml.example config/database.yml

Now create a database for FOreman and add the database details in the database.yml. Now add the puppet master details in the settings.yaml. Since i’m going to use the Foreman in production mode, i’ve commented out the Development and test environment setting in database.yml. Once the config files are set, we can now go ahead with db migration.

$ RAILS_ENV=production bundle exec rake db:migrate

Now we can check whether the server is fine or not by using the following command. The below command will start the Foreman with the builtin web server, and we can access the webui from http://foreman_ip:3000 in the browser. By default there is no authentication set for the WebUI. But LDAP Authentication can be set for the WebUI. Details are availabe in the foreman’s documentation.

$ RAILS_ENV=production rails server

Once the Foreman server is working fine, we can configure puppet to send its logs and facts to foreman. In the puppet clients, add report = true in the puppet.conf file. Now in the puppet master, we need to do a few stuffs.

Copy this foreman report file to puppet’s report library.

In my case it is /usr/lib/ruby/vendor_ruby/puppet/reports/ and rename it to foreman.rb. Now add reports=log, foreman in the puppet.conf file. Also add the foreman url in the foreman.rb file.

foreman_url='http://foreman:3000    # or use ip instead of foreman, if DNS/Host entry is not there for Foreman

Now for sending facts to puppet, we can put a cron job to execute the below command

$ rake puppet:import:hosts_and_facts RAILS_ENV=production

Now once the puppet clients starts running, they will send the logs to Foreman, and can be viewed in the WebUI.

Foreman and Libvirt

Now in the same machine i’ve installed libvirt and libvirt-ruby. Now create a user “foreman” and generate ssh-key for the user. Now copy the public key to the “authorized_keys” file of the root user. This is actually needed if your libvirt host is different.

Now go to the Foreman WebUI, Go to More —–> provisioning —–> Compute Resources. Now click on “New Compute Resource”, Add a name for the Resource, Select the provider as Libvirt, and URL is qemu:///system, since libvirt and foreman resides on the same system. We can also test the connection to libvirt. IF the parameters we entered are fine, Foreman can talk to libvirt directly.

Standard
Debian, puppet

Setting Up a PuppetMaster in Debian Based Machine

I still do not know why i like Puppet so much, But i always loves to play around with it. It’s a very powerful tool for the system admins. I’ve never tried chef before, but i’m very happy with puppet. Thanks to Luke and PuppetLabs for designing such a good tool. My colleague sarguru is working on a newer release of our deepOfix Mail Server, which will be using puppet for config management. In this blog i will explain hot setup Puppet in “StandAlone”  as well as in ” Server-Client” mode.

Puppet Master

We can install puppet from the APT. For testing i’m not going to daemonize my puppet master, i will be installing the two basic packages.

“apt-get install puppet puppet-common” 

Now, if you have already installed the puppet and if the old ssl certificates are still existing, it can be removed using the below command.

“puppet cert clean  –all”

If you are not running a dns server, enusre you have proper FQDN entries in the “/etc/hosts” for server and clients,if any. Now we can start the puppet master. It’s always better to use the debug mode during testing.

“puppet masterc  –debug  –no-daemonize”

This will start the puppet daemon, and will automatically create a self signed certificate for the puppet master.

StandAloneMode

Now we have puppet master running on the machine. In the StandAloneMode, we can create a “*.pp” file with all the resources and we can invoke puppet to apply the resources locally.

“puppet apply  *.pp  –debug”

This will the the resource which we mentioned in our puppet policy file.

If a module has to be applied, we have to mention the module path as well as the module that we are gonna apply.

“puppet apply –debug –modulepath=/etc/puppet/modules -e “include modulename””

Note:- Puppet has a very good feature, where we can simulate the changes without actually applying the resources. For that we have to use one option “–noop” (no operation) while executing the puppet apply. This is very helpful for us to simulate the results and see if the resources are being properly.

“”puppet apply –debug –modulepath=/etc/puppet/modules -e “include modulename” –noop”

Client-Server Model

So now we have puppet master running on one machine. On the client machine, similarly install “puppet and puppet-common” packages from APT. Ensure that that the machine can resolve the FQDN of the Puppet Master.

Now run the puppet agent.

“puppet agent –debug –no-daemonize”

If it throws any Name server error, then we can mention the server manually,

“puppet agent –server fqdnofmaster –debug –no-daemonize”

So now on the terminal where we are running the puppet master, we can see a certificate request from the client.

We can also see the csr by running the following command on the puppet master.

“puppet cert list”

We can certify the csr by running,

“puppet cert sign fqdnofclient”

Now we can run the puppet agent again to check whether the agent successfully talk with the the master.

In the client-server model, we need to specify the modules which are going to apply on specific nodes and for default nodes if any. These things are specified in “/etc/puppet/manifests/sites.pp”. Here we will mention the node and the modules to be applied.

Syntax is ,

node ‘node name in fqdn’ {

include modulename

. . . . . . 

}

Now when we run the puppet agent,appropriate modules will be installed to the clients as per the site.pp file. We can always verify the syntax of the puppet policy file using “puppet parser validate *.pp”.

Standard