Debian, virtualization

Setting up Apache Cloudstack

Today i was completely playing around with virtualization. I was playing around with Foreman and KVM, then i got WebVirtmanager to play around, which is working perfectly with LVM storage pool. It’s almost a week since i saw a few videos related to Apache Cloudstack, so today i decided to give it a try. In this blog i will explain on how to set up Apache CloudStack on an ubuntu 12.10 Machine. Apache Cloudstack is one of the coolest cloud platform’s available. It supports hypervisors like KVM, XEN, vSphere. The latest version is 4.0.1-incubating. The source can be downloaded from here. There is a very good documentation available from Cloudstack.

Building Debian Packages from the Source

First we need to install the below dependency packages.

1.  Apache Ant
2.  JDepend
3.  Apache Maven (version 3)
4.  Java (Java 6/OpenJDK 1.6)
5.  Apache Web Services Common Utilities (ws-commons-util)
6.  MySQL
7.  MySQLdb (provides Python database API)
8.  Tomcat 6 (not 6.0.35)
9.  genisoimage
10. dpkg-dev and their dependencies

Maven 3, which is not currently available in 12.10. So, we’ll need to add a PPA repository that includes Maven 3

$ add-apt-repository ppa:natecarlson/maven3

The current ppa supports only ubuntu 12.04 aka Precise, so edit /etc/apt/sources.list.d/natecarlson-maven3-quantal.list and replace “quantal” with “precise”. So now the content of the file looks like below one

deb precise main
deb-src precise main

Now we can start installing the dependencies,

$ apt-get install ant debhelper openjdk-6-jdk tomcat6 libws-commons-util-java genisoimage python-mysqldb libcommons-codec-java libcommons-httpclient-java liblog4j1.2-java python-software-properties maven3

Now we can resolve the buildtime depdencies for CloudStack by running the below command.

$ mvn3 -P deps.

Now there is a small bug, which add the dependency of “chkconfig” package to a few of the cloudstack packages. But “chkconfig” is required for Redhat based machines, not for debian based machines. So edit “Debian/control” file inside the apache cloudstack source folder and remove “chkconfig” from the dependency list. After that we can start building the debian packages.

$ dpkg-buildpackage -uc -us

The above command will build 16 debian packages.

Setting up a Local APT repo

Now we can set up a local apt repo so that we can install all these 16 packages along with their corresponding dependencies. First ensure that “dpkg-dev” is installed. After that copy all the packages to a specific location in order to create the local repo.

$ mkdir -p /var/www/cloudstack/repo/binary
$ cp *.deb /var/www/cloudstack/repo/binary
$ cd /var/www/cloudstack/repo/binary
$ dpkg-scanpackages . /dev/null | tee Packages | gzip -9 > Packages.gz

We need to configure the local machine to use this local repo. Add the local repository in echo "deb http://server_url/cloudstack/repo/binary ./" > /etc/apt/sources.list.d/cloudstack.list and run “apt-get update”. Now we can install the cloudstack packages.

$ apt-get install cloud-agent cloud-agent-deps cloud-agent-libs cloud-awsapi cloud-cli cloud-client cloud-client-ui cloud-core cloud-deps cloud-python cloud-scripts cloud-server cloud-setup cloud-system-iso cloud-usage cloud-utils

Now from the web browser go to “http://server_url:8080/client/. The default Username is “admin” and password is “password”. For the admin user, we don’t need to provide the domain option.

Debian, libvirt, virtualization

WebVirtManager and Libvirt-KVM

It’s been almost two years since i’ve started using KVM and Libvirt for creating Virtual machines. All our production server’s are VM’s, so far KVM has never let us down. The Virt Manager is a wonderful tool taht can installed in most of the linux distributions. But when i switched to ma new MacAir, i could not find a similar tool, so i decided to use a web based tool, so that it can be used from anywhere irrespective of devices. I came across this WebVirtManager, a python and django based web app, which is very active in development, so i decided to give it a try. In my libvirt setup, i’m using LVM as my storage pooli for my VM’s, so the main thing which i wanted to check was, whether the WebVirtManager is able to create LVM’s, so that it can be used as the HDD image for my new VM’s from the WebInterface.

First, we need to install the basic dependency packages.

$ apt-get install git python-django virtinst apache2 libapache2-mod-python libapache2-mod-wsgi

Now go to libvirtd.conf, and ensure that “listen_tcp” is enabled. Also go to “/etc/default/libvirt” and add the “-l” to the “libvirtd_opts”, so that libvirt will listen on tcp. The default port is “16509”.

Now we can clone the repository from the Github.

$ git clone git://
$ cd webvirtmgr
$ ./ syncdb

While running the sync, it will ask to create a super user, so create the user, this user can be used to login to the WebVirtManager GUI. Now We can create a virtualhost in apache, and we can start the server. The Apache configurations are available in the Readme. I’ve added the below WSGI settings in my default apache sites.

WSGIScriptAlias / /var/www/webvirtmgr/wsgi/django.wsgi
Alias /static /var/www/webvirtmgr/ virtmgr/static/
<Directory /var/www/webvirtmgr/wsgi>
 Order allow,deny
Allow from all

Ensure that the directory is writable by apache user. But for testing, we can start the server from command line using the below command.

$ ./ runserver x.x.x.x:8000 (x.x.x.x - your IP address server)

So this command will start the WebvirtManager, which is listening at port “8000”. So from the Browser, we can access the url. The default usernmae and password is the one which we created during the syndb. Now before adding the connection, we need to create a user which can access the libvirt. For that we are going to use “saslpasswd2”. Ensure that the package sasl2-bin is installed in the machine.

$ saslpasswd2 -a libvirt testuser     # replace testuser is the user name.

To list all the user’s, we can use the sasldblistusers2 command.

$ sasldblistusers2 -f /etc/libvirt/passwd.db
$ testuser@cloud: userPassword        # Note that the actual user name is testuser@cloud, where cloud is the hostname of my server. This full user name has to be used for adding connections.

Now login to the WebvirManager, and click on “Add Connection”. Fill in the connection name, ip of the server, the user which we created using saslpasswd2, ie testuser@cloud and the password for that user.If everything goes fine, we can see the host connected. Now click on the “Overview”, to see the settings of the host.

Now i need to check the storage pool part. Since the storage pool is already active and running, it will get displayed at the storage pool option. If a new pool has to created, click at the “add pool” option, and select the “lvm” option, define the VolumeGroup name and the physical volumes.

I tried creating a new VM from the interface, while creating, i selected my VolumeGroup as the storage, and it sucessfully created an LVM with the specified size, and i able to continue my installtion using the vnc option avalable at the WebVirtManager.

Debian, virtualization

Virtualization with LXC

LXC or Linux Continers is is an operating system-level virtualization, using which we can run multiple isolated Linux systems on a single host. LXC relies on the cgroups functionality of the Linux Kernels. Cgroups (control groups) is a Linux kernel feature to limit, account and isolate resource usage (CPU, memory, disk I/O, etc.) of process groups. LXC does not provide a virtual machine, but rather provides a virtual environment that has its own process and network space. It is similar to a chroot, but offers much more isolation.

First, let’s install the necessary packages.

$ apt-get install lxc btrutils

By default in Ubuntu, when we install the lxc package, it will create a default bridge network called “lxcbr0”. If we don’t want to use this bridge network, we can disbale it by editing the /etc/default/lxc file. We can also create bridge networks using the “btrcl” or we can directly define the bridge networks in the interfaces file. There are a few templates, which gts shipped with the lxc package, which will be present in the /usr/share/lxc/template. I’m going to use the default template to create the containers. We can also use OPENVZ templates to create containers.

I’m going to keep my keep all my container’s files in a default path say “/lxc”

$ mkdir /lxc

Now we can create the first debian container.

$ mkdir /lxc/vm0    # where vm0 is the name of my conatiner.

$ /usr/share/lxc/templates/lxc-debian -p /lxc/vm0

Now this will install and build the necessary files for the container. If we go inside the vm0 folder, we can see two things, one is the config file, and second is the root folder of the container. This root folder will be the virtual environment for our container. Now we can edit the config file, to mention the default Network options. = # IP address should end with CIDR = 4a:59:43:49:79:bf # MAC address = br0 # name of the bridge interface = veth = veth_vm0

Now we need to add the ip to the lxc’s interface file alos, for that we need to edit the /lxc/vm0/rootfs/etc/network/interfaces file and set the ip address in it for the interface eth0
We can create a bridge interface and we can bind it with the physical interface of the host, so that the lxc will be in the same network as that of the host. If there is a virtual network already existing, for example, when we install libvirt, it will create a bridge interface called “virbr0”, or in Ubuntu the lxc package installation wil create a bridge interface called ‘lxcbr0’, we can alos use those with lxc. Or we can define a bridge interface in the “interfaces” file. Below is a configuration of creating a bridge interface.

    auto eth0
    iface eth0 inet manual

# Bridge setup
    iface br0 inet static
        bridge_ports eth0 

If a separate network has to be given for the lxc’s, then we can to go for NATing. The below configuration on the interfaces file is for the NAT enabled.

auto br0
iface br0 inet static
bridge_stp off
bridge_maxwait 5
pre-up  /usr/sbin/brctl addbr br0
post-up /usr/sbin/brctl setfd br0 0
post-up /sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

Once we are ready with the configurations, we can start our container using the below command.

$ lxc-start -n vm0 -f /lxc/vm0/config

In the NAT scenario, the lxc machines are under the “172.16” network, while the host lies in “192.168.0” network. There are some good projects which works around with lxc, vagabond is an example for that.Vagabond is a tool integrated with Chef to build local nodes easily and most importantly, quickly. Vagabond is built for Chef.

Debian, puppet, virtualization

Using Foreman With Puppet and Libvirt

TheForeman is one of the best provisioning tools available. It’s purely open-sourced. And it natively supports puppet for provisioning the nodes. Foreman can talk to libvirt also, which makes us easy to create a VM and provision it on the way. In this blog i will be explaining on how to install Foreman from the source, how to integrate it with puppet to receive the logs and facts and make Foreman to use Libvirt for building VM’s.

Setting up Foreman

First will install the basic depenencies. Since i’m using the git repository of Foreman for installation, git package has to be installed. Moreover we also need a database for Foreman. I’m going to use Mysql for that.

$ apt-get install git mysql-server ruby-mysql libmysql-ruby1.9.1 libmysqlclient-dev libvirt-dev 

Now clone the repository from github. The newer build’s works with Puppet 3.0.

$ git clone -b develop

Ensure that ”ruby and bundler” is installed in the machine.

$ bundle install --without postgresql sqlite

Now we can start configuring Foreman. Copy the sample config files.

$ cp config/settings.yaml.example config/settings.yaml
$ cp config/database.yml.example config/database.yml

Now create a database for FOreman and add the database details in the database.yml. Now add the puppet master details in the settings.yaml. Since i’m going to use the Foreman in production mode, i’ve commented out the Development and test environment setting in database.yml. Once the config files are set, we can now go ahead with db migration.

$ RAILS_ENV=production bundle exec rake db:migrate

Now we can check whether the server is fine or not by using the following command. The below command will start the Foreman with the builtin web server, and we can access the webui from http://foreman_ip:3000 in the browser. By default there is no authentication set for the WebUI. But LDAP Authentication can be set for the WebUI. Details are availabe in the foreman’s documentation.

$ RAILS_ENV=production rails server

Once the Foreman server is working fine, we can configure puppet to send its logs and facts to foreman. In the puppet clients, add report = true in the puppet.conf file. Now in the puppet master, we need to do a few stuffs.

Copy this foreman report file to puppet’s report library.

In my case it is /usr/lib/ruby/vendor_ruby/puppet/reports/ and rename it to foreman.rb. Now add reports=log, foreman in the puppet.conf file. Also add the foreman url in the foreman.rb file.

foreman_url='http://foreman:3000    # or use ip instead of foreman, if DNS/Host entry is not there for Foreman

Now for sending facts to puppet, we can put a cron job to execute the below command

$ rake puppet:import:hosts_and_facts RAILS_ENV=production

Now once the puppet clients starts running, they will send the logs to Foreman, and can be viewed in the WebUI.

Foreman and Libvirt

Now in the same machine i’ve installed libvirt and libvirt-ruby. Now create a user “foreman” and generate ssh-key for the user. Now copy the public key to the “authorized_keys” file of the root user. This is actually needed if your libvirt host is different.

Now go to the Foreman WebUI, Go to More —–> provisioning —–> Compute Resources. Now click on “New Compute Resource”, Add a name for the Resource, Select the provider as Libvirt, and URL is qemu:///system, since libvirt and foreman resides on the same system. We can also test the connection to libvirt. IF the parameters we entered are fine, Foreman can talk to libvirt directly.

Debian, virtualization

Building custom Vagrant base box

Vagrant has become a common tool used by most of the sys admins who play around with computers. It has made virtualization so easy. Seriously it’s a must have tool. Vagrant supports Puppet/Chef provisioning, which makes it even more powerfull. We just need the base boxes to play around with Vagrant. In this blog i will be explaing about creating custom Vagrant Boxes. I work for DeepRootLinux, developers of the deepOfix Mail Server, a Debian based GNU/Linux distribution. I will be using our distribution for creating base box. It will help us to deploy deepOfix VM in a faster way and we it helps us to test our custom puppet modules into our operating system.

SO first we need to create a base VM in the Virtualbox. Ensure that the network controller is set to “NAT”. For port forwarding to work properly, NAT must be used. There a few point to remember, As per the Vagrant’s Documentation, Vagrant makes some assumptions,

  • The root password is ”Vagrant
  • One user account ”vagrant” with password ”vagrant”.
  • Domain is ”
  • Hostname is ”vagrant-[os-name]”, e.g. vagrant-debian-lenny

If any different values are being used, it has to be specified in the Vagrantfile. I’ve used custom domain name and hostname and i did not mentioned it in my Vagrant file. But it did not created any problem. Anyways Vagrant is using key-based authentication for SSH. So once we SSH into the system, we will login in to the system as the Main user, in our case ”vagrant” user. So we should make this ”vagrant” user as a member of the ”sudo (super user doers)” group, so that we can use “sudo su” to switch to the root user.

Normally, using sudo will always prompt for the user password, we can remove this by modifying the /etc/sudoers file. We just need to add one line into the file ”%sudo ALL=NOPASSWD: ALL”. This will prevent password prompt for the user’s who are the member’s of the sudo group. Once the file is edited, we need to do ”/etc/init.d/sudo restart” to reflect the changes.We can verify that sudo works without a password, but logging into the sudo user account, then sudo which sudo. We should get output similar to “/usr/bin/sudo”.

Now we need to setup Virtualbox Guest Additions. So, first we need to build the necessary packages.

apt-get install linux-headers-$(uname -r) build-essential    # for root user

sudo apt-get install linux-headers-$(uname -r) build-essential    # for sudo user's

We need to insert the guest additions image by using the GUI and clicking on ”Devices” followed by ”Install Guest Additions”. And we need to mount the CDROM.

mount /dev/hd0 /media/cdrom     # where /dev/hd0 is the CDROM block device in deepOfix

And finally, run the shell script which matches our system.

sh /media/cdrom/

Since Vagrant only supports key-based authentication for SSH, we must setup the SSH user to use key-based authentication. We need to copy a public key into ”~/.ssh/authorized_keys” of the ”vagrant” user. Vagrant provides an ”insecure” pair of public and private keys which are available here. Once the public key is copied, we can shut down our VM. And we can start building our base box.

vagrant package --base <box_name>

If there is any custom option to be set like, using a specific port port forwarding, or a specific SSH keys, we can create a Vagrantfile with all custom options and we can use it during the packaging.

vagrant package --base <bxo_name> --vagrantfile Vagrantfile

If everything goes fine, it will generate a base box file. We can use this base box file anywhere with the vagrant.

Debian, virtualization

Vagrant: Make Virtualization Easier

Vagrant is a light weight virtualization tool, build over Oracle’s VirtualBox. It’s completely written in ruby and it’s very easy to install and configure. The only dependency is VirtualBox. Once VirtualBox is installed, we can either use RubyGems to install Vagrant or we can get the installer from Vagrant.

I prefer ruby gems, So

$ gem install vagrant

That’s it, it will install vagrant. Now we need to get the base box. Vagrant uses the base boxes to build the VM’s. We can download the base boxes from Vagrant Boxes. we can directly use the url to create the boxes, but it’s alwasy good if we have a downloaded copy of the base boxes.

Basically we need to follow just 3 steps. add,initialize,up. So once we have the base box just add the base box.

$ vagrant box add new_box_name our_downloaded_basebox_file

$ vagrant box add mybox

Now just do a vagrant init to create the Vagrantfile. This is the main configuration file. If we go through the Vagrantfile, we can see a bunch of options like port forward, provisioning, setting network and so on. If only one vm is required, then we just have to add the new_box name which we created at the option int the Vagrantfile. And vagrant up will start the VM.

But one of the most important feature of Vagrant is it suports multiple VM’s over one single box. But we have to define those VM’s in the Vagrantfile. Below i’ve defined two VM’s in my Vagrantfile, also comment out

config.vm.define :ubuntu do |ubuntu_config| = "precise32"
config.vm.define :puppet do |puppet_config| = "precise32"

Where ubuntu and puppet are the name of the VM’s. And precise32 is the name of the the box which i’ve created. Now, vagrant up will start all the VM’s. But we can mention the name to start a specific VM. Like vagrant up ubuntu. It will start only the ubuntu VM.

Provisioning is another important feature of Vagrant. We can use puppet,chef and shell scripts to bootstrap the vm’s. I saw @mitchellh’s talk at PuppetConf 2012, in which he mentioned about how to create a Fully Automated Puppet Master using the shell provisioner. I’ve tried this out using a shell script that will install puppet master and genrate the ssl certificates. It’s working fine, soon i will post it here.