Automation: No one around
The previous post outlined our architectural approach to a scalable website. This post outlines our approach to building such website. Our solution in a single word: Automation.
Some hundred years ago industrial production of goods such as cars was still a craft-based production. Ford introduced the moving assembly line and heavy automation. By standardizing productivity increased tenfold. Prices for a model T were cut from $760 to $360*.
A little less known is the information that Ford based his standardization on the work of Taylor. Taylor's gospel also known as "Taylorism" or Scientific Management standardizes tools, employs precise task allocation, does time studies (e.g., screw on each bolt in 15.2 seconds), introduces performance pay, and automates by repetition**.
So do we.
Setting up a server manually is time consuming and error prone. To install Debian – our operating system of choice – we make use of Fully Automated Installation – FAI to, well, automate server setup. FAI uses a collection of Perl and shell scripts for the installation process.
Picture by Ford
On the next level we use Puppet for automating system administration tasks. Instead of manually running through all those application setup tasks Puppet introduces a declarative language. This language provides a framework to express the relationships between servers, the services they run and the objects that compose those services.
At first it takes some time to get familiar with this language and a couple of early Puppet scripts were a bit frustrating. Yet it is worth it: The time it takes to write the puppet module that potentially runs on dozens of machines is actually often less than executing the administration task on a single server. Once you master the initial obstacles you cut huge chunks of time and pain out of the process of administrating a distributed server and services landscape.
The last part is code deployment. There we use Hudson. In a nutshell, Hudson (like a number of other systems) provides an easy-to-use so-called continuous integration system. The system does two jobs: First, it allows software developers to easily integrate changes to the software and second, it provides for an automated and continuous building and testing of software projects. Once done, Hudson / Puppet push the code to our development and live systems without human intervention.
This automated setup of our core infrastructure increases the productivity of our team dramatically. Instead of investing time at unnecessary places – server setup – we can focus on what matters most: An application that wins the minds and hearts of our users.
Next in the series how to build a scalable startup: Virtualization.
_______
*) Interestingly enough not all of the cost advantages were due to introduction of the assembly line and the involved productivity gains. A lot was owed to a focus on reduced material costs (PDF Document). The equivalent today is greatly reduced hardware costs (More on this and virtualization in the next post).
**) Taylorism would quickly become the standard for businesses worldwide. Fordism and Taylorism would be hotly contested for unfairness towards workers. But that is another story.
Comments
Yoan
@ 19.08.2009 09:53 CESTThis is all nice, but I always the problem of the chicken and egg here; there is lot to do in order to get there. Telling people go with Puppet, go with Hudson is the end of a long path of getting ready for that in terms of having all kind of tests, fixtures, migration scripts, and so on. Describing the road to get ready for that sounds more like something that can be applied to someone else, not knowning anything about that and willing to enhance his mojo.
One bonus question, why Puppet and Hudson over the existing alternatives ?
Patrice
@ 19.08.2009 12:20 CEST@Yoan
It's the basic mindset that counts and needs to be around - same as for testing itself. As you write correctly, the way to automation of course involves much more than mentioned in the post. And it's a lot easier for a startup to implement all that when doing it from scratch.
But describing the full way that doesn't help people all that much. I find that describing the final result is a good way to convince people to try it as well.
About your bonus question:
We use Puppet as we find it's easier to use and extend than cfengine - the other tool I had been familiar with. Not many other alternatives were around.
Hudson is the most powerful but still easy to use CI system we've found. We've used Atlassian Bamboo in the past but it's commercial, relatively buggy and not easy to configure automatically. We tried many systems (CruiseControl, Buildbot come to mind, there were more) and in the end we just felt best about Hudson.
