Put the fun back into being a Sysadmin with Puppet

 

Every once in a while I come across a new tool. After a few weeks of using it I think: Darn, how was I ever able to do a good job without it? Today I’d like to introduce you to such a tool: Puppet

<insert witty remark about grown men and women playing with puppets here>

Puppet is a
configuration management framework. A what? You tell Puppet what needs to be installed and configured on your servers and Puppet does that for you. Every few minutes it can ensure that everything is still in place and running properly, and if something is broken maybe even fix it.
Puppet is very modular. You can reuse the same instructions on as many servers as you want. Adding a new server? Add a couple of lines to your manifest. Done.
Back at my previous employer local.ch I was able to run about 100 hosts more or less single-handedly and keep the dark rings under my eyes to a minimum. Puppet is like having a room full of extra sysadmins that constantly log into your servers and check if everything is still in place. They never sleep, the don’t make mistakes and they never browse Facebook. ;-) Or in other words, Puppet puts the fun back into being a Sysadmin, because it takes away the repetitive chores.

If you are already sold, you can start your journey here.

If not let me walk you trough a simple example. Installing and configuring a simple webserver.

Let puppet install the webserver for you:

package {
    "apache2":
        ensure => installed;
    }
}
Every time puppet runs it uses the package manager of your OS (for example, aptitude oder yum) to check if apache2 is installed. If not it runs the appropriate commands to get it done. Out of the box puppet knows about the default package managers on all major unix distros out there.

Let puppet ensure the webserver is running:

service {
    "apache2":
        ensure => running,
        require => Package["apache2"];
}
Every time puppet runs it checks if the service is running. Again it knows how to handle services on all major distros, but you can tweak it to your needs if required. The “require” tells Puppet, that the service only exists on the server if the package apache2 is already installed. Puppet will not run your manifest from top to bottom. It will resolve all the dependencies first and the execute the steps in a logical way.

Populate the config file of the apache server

file {
    "/etc/apache/conf/httpd.conf":
        ensure => present,
        owner => root,
        group => root,
        mode => 664,
        content => template("modules/apache/httpd.conf.erb"),
        require => Package["apache2"],
        notify => Service["apache2"];
}

Here we let Puppet deploy a template on the server. That’s neat because we can put values into the file (e.g. ip, hostnames, port) that are different on each server. Puppet compiles the template for each of your servers and compares its contents with the local copy. If there is a difference it creates a backup of the local file and deploys the new one.

Another powerful concept is introduced here as well: Notifications. If the file changes Puppet will restart the service to ensure that the config change is in effect.

Lets stop the simple example here.

Of course you can automate the same apache2 install with a simple shell script. But scripts tend to become unwieldy quickly and often can’t be reused, certainly not across different operating systems. Puppet manifests on the other hand are very easy to read and modular. One cool side effect is that your manifests become a simple form of documentation. And lets be honest for a moment: I don’t like to document ;-)

Using puppet for new servers is simple, but how can you roll it out over an existing architecture with lots of servers?

My advice is to start small. Back at local.ch I’ve started with one simple module for sudo. It became tricky to keep all these sudoers files consistent over all the servers we had. Deploying a single file with Puppet is very easy. Once you get used to Puppet you can add more and more modules quickly, without breaking things. A few other obvious candidates to start are cron (let puppet handle /etc/cron.d), the ntp clients and servers and the iptables config.

At Memonic we’ve now written over 70 modules, from apache to varnish (sorry, we don’t have one with z yet). Puppet manages almost all aspects of our servers, and making our lives so much easier. We have a simple rule: If you haven’t done it with Puppet, you haven’t done it right. ;-) Don’t forget to check out the Module Forge, a growing list of shared modules.

Another great resource and a good point to start is the Book Pulling Strings with Puppet by James Turnbull

Btw, Puppet is here to stay. Lots of large companies like Twitter, Digg and Rackspace are using it and Puppet Labs, the company behind Puppet, received some major founding last year.

This post would not be complete without mentioning a few alternatives to Puppet: Check out ChefCfengine or Bcfg2.

And thats that, you’re introduced to Puppet. The rest is up to you ;-)

 
 

3 Comments

  1. Roy says:

    Hi Toni,

    Interesting tool, thanks for the information.

    Speaking of technology, which tools you’re using for issue tracking, continuous builds, code reviews etc? Atlassian’s products? Open source?

    Thanks,
    Roy

  2. toni says:

    Hi Roy,

    Issue Tracking: Atlassian Jira (you can get it for 10 bucks a year, if your < 10 people)

    Greenhopper for Agile running Sprints

    Continuous Builds: Hudson (but it will be forked to “Jenkins” soon, you know that Oracle thing… )

    Code Review: Reviewboard, but also Atlassian Fisheye for SCM browsing and searching.

    Another worthwhile tool is Splunk> for log file handling and more.

    Cheers,
    Toni

  3. Roy says:

    Hi Toni,

    Thank you very much for your reply, I am familiar with most of the tools, splunk is the interesting among them, but the pricing scheme is outrageous, charging per gb or something like that :s.

    Anyway, if you’re attending this coming Amazee’s web monday will be happy to continue this virtual discussion in the real world :-)

    Regards,
    Roy

  4. Toni Birrer says:

    Put the fun back into being a Sysadmin with Puppet

Twitter

Twitter Updates

Pressclippings

Memonic Set by press

RSS Feed

memonic Photos

More memonic photos