As I documented before we develop with Python. Python is an interpreted programming language that doesn’t need compilation. And yet, the Jenkins build server plays a crucial role in our development process with a full 95 builds configured at the moment. How come?

In our case the build server’s main job is testing. For each component we have an extensive suite of automated tests that are run after every commit. If all tests are successful, there are additional tasks that get executed, namely:

• Create code coverage reports.
• Create Dojo bundles for JavaScript and CSS files. I documented that extensively in the previous blog post Why we love Dojo at Memonic.
• Minify the JavaScript and CSS files.
• Upload all static files (JavaScript, CSS, images) to Amazon S3. In the live system we serve those images using Amazon CloudFront.
• Create a Python egg (Python’s equivalent to Java’s jar files or Ruby’s gem files).
• Deploy that egg to our development system.

In the next few sections I’ll explain three of those areas in a bit more detail.

Automated Testing

Most of our frontend and backend services have extensive tests. Those are very important for us to ensure a quality level without having to invest a lot of resources into manual testing.

The motivator to maintain those tests is a simple mechanism: automatic deployment is done only after all tests have passed. That means that for a developer it’s actually the easier route to fix the tests than to manually package and deploy the project.

This simple mechanism ensures that all our test suites are always green (or blue if we hadn’t installed the essential greenballs plugin).

The individual test runs are very quick because we split up our whole platform into many small components. There are 95 different projects at the moment.

One test that we added just a short time ago is for translations. In our translation system there are place holders. For example we’d write “Good morning {first_name}” in the source code. The German translator will then translate that to “Guten Morgen {first_name}”. But sometimes it happens that the translator also translates the variable name so the result is “Guten Morgen {vorname}”. That of course wreaks havoc in production and has happened several time. So eventually we ended up checking for those to make sure that all placeholders are copied untouched into the target string.

Continuous Deployment

As mentioned above we do continuous deployment to the development system. On every commit the build for the corresponding project is triggered. The deployment then consists of a few simple steps.

First we build an egg. Eggs are the Python package format. Then those packages are pushed to a web server. From that web server these packages can then be installed using easy_install or pip. And that’s the final step: a script logs into the development server, updates the package and restarts the corresponding process.

This is a process that would take a bit of time if done manually. Thanks to the automation we make sure that it happens every time somebody commits a change.

In fact this has been so successful at Memonic that my claim is this: if you currently suffer from too many red builds in your project then integrate deployment into your build system. And of course only trigger the deployment process after successful builds!

Static files on Amazon CloudFront

An important build step is uploading all static resources to Amazon S3. We then reference all of those files using a CloudFront endpoint.

The host name we use for that is s.memonic.ch. We chose a different top-level domain on purpose to avoid any pollution with cookies which can especially affect caching by proxy servers.

Because we don’t serve the JavaScript files from the same domain as the HTML files, the initial setup was a bit complicated. In the case of Dojo we had to use their cross-domain builds. But after the initial setup cost we never had
any issues with that.

Conclusion

The build process we have in place is relatively complex. Of course those steps outlined above evolved over several years and didn’t all fall into place overnight.

We have achieved the main purpose which is to maximize developer performance. Every time a developer doesn’t have to worry about how to package a service or upload files to S3 is a win.