How Ruby is beating Python in the battle for the Soul of System Administration

Roughly three years ago I purchased the book "Python for Unix and Linux System Administration" and it convinced me that Python would be the default scripting language for linux system administrators for the foreseeable future. I was working on the One Laptop Per Child project at the time, where Python was the lingua franca. It seemed that Red Hat was using Python for almost every new project. Further, Unladen Swallow was making rapid progress. I couldn't have been more wrong.

Through a combination of historical accident and some seemingly minor feature differences, ruby is inexorably becoming the dominant scripting language for linux system administration.

Before I go any farther you die-hard sysadmins are probably screaming "We already have a scripting language, Bash!" While we love bash and commandlinefu, Bash becomes a liability not an asset once your script exceeds 100 lines and a total nightmare if you need to parse or output HTML, CSV, XML, JSON, etc.

Historical Accidents

In 2004, Luke Kanies set about building Puppet, a configuration management system to improve upon the deficiencies he saw in CFEngine. From the excellent On Ruby blog

I tried to implement my idea in perl, but I just couldn’t get the class relationships to work (the attributes and resource types each needed to be classes, according to the design in my head). This was back when Python was the shiznit, so I naturally tried it, but Python just makes my eyes bleed (and no, it wasn’t the whitespace, it was things like the fact that ‘print’ was a statement instead of a function, and ‘len’ was a function instead of a method).

I had a friend who had heard Ruby was cool but hadn’t actually tried it himself. Since I was just messing around at the time, I figured I’d give it a go. Four hours in, never having seen a line of Ruby previously, I had a functional prototype.

A couple years, a number of Puppet developers felt that Puppet didn't meet their needs. They started their own configuration management tool, Chef, heavily inspired by Puppet. The biggest outward difference between Puppet and Chef is that Chef uses pure ruby as its "recipes" whereas Puppet uses its own configuration language based on ruby.

Both Puppet and Chef are seeing rapid adoption by big companies according to BusinessWeek. If you aren't yet using Puppet or Chef, you should be planning on it in the near future. Whether you choose Chef or Puppet, you are effectively scripting your infrastructure with ruby. After spending 25% of your time working with Puppet, you will be much more likely to reach for ruby for your next scripting task.

Popular Projects

Off-hand here is a non-definitive list of sysadmin/devops related projects in ruby

• puppet
• chef
• vagrant
• mcollective
• cucumber (behavior-driven testing)
• capistrano
• rake (ruby Make)
• aeolus project/openshift
• cloud foundry
• graylog2
• logstash
• travis-ci

and in Python:

• openstack
• cobbler
• fabric
• func
• buildbot

What's important here isn't the length of the respective lists but the importance of the individual projects. Chef, Puppet, and Vagrant are the new hammers and screwdrivers of system administration. If you are a sysadmin and aren't yet using these tools, don't worry, you will be sooner or later.

Openstack deserves special attention as it is a very exciting but early stage project. It could play a big role in your future as a system administrator.

Please notify me of significant devops projects that are missing from this list.

What a Girl Sysadmin Wants

Here are the features in a scripting language that a sysadmin wants

• A DSL for the problem domain
• High productivity, i.e. concise and expressive syntax
• Easy to interaction with shell commands
• Regular Expressions
• powerful one-liners

Notice that performance is not on this list of requirements. Ruby is significantly slower than Python and this was particularly true for the 1.8.* series of Matz Ruby Interpreter (MRI). However, performance just isn't critical for 90% of our work as sysadmins. We care about productivity more than we care about performance. Readability is nice but it is a distant second to productivity.

Python doesn't have regular expressions embedded in the language, probably to improve readability. It also limits the number of top-level built-in global variables. From a language design perspective, this is much cleaner. From the perspective of your average street-trained, stress-laden, vi-addicted sysadmin, it is annoying.

These top-level variables also make one-liners more concise. Here are just a few



$_ last line read from STDIN

$? last exit code from a child process

$stdin reference to stdin

$stderr reference to stdout

$stdout reference to stdout

Here is an irb session to show you these values in action

hitman@hiroko:~/pr$ irb

irb(main):001:0> %x[ ls -a ]

=> ".\n..\nfoo1\nfoo2\nfoo3\n"

irb(main):002:0> puts $?

0

=> nil

irb(main):003:0> %x[ ls xys ]

ls: cannot access xys: No such file or directory

=> ""

irb(main):004:0> puts $?

512

=> nil

"IRB?" you scoff "You will have to tear IPython from my cold dead hands."

I love IPython. I have used IPython for 4+ years and IRB doesn't hold a candle to it. That said, ruby's shortcuts are really useful, enough to compensate for IRB's shortcomings. There is something called wirble that I haven't tried yet that may make irb a lot more productive.

Here some python code to detect whether a machine is a VMware VM.



# edit: fixed python code tks to kstrauser

  import os
    if 'vmware' in os.popen('dmidecode').upper():
        print 'this is a vmware vm'
    else:
        print 'this is not a vmware vm'

Here the same code in ruby

`dmidecode`
if $_ ~= /vmware/i
    puts 'this is a vmware vm'
else
    puts 'this is not a vmware vm'


Frankly, this kind of code makes my eyes bleed. The python example is far more readable and maintainable. That said, a lot of sysadmins would appreciate the terseness of the ruby example.

Let's look at writing one-liners in both languages.

This code prints out the first 10 lines in a file using Python

python -c "import sys; sys.stdout.write(''.join(sys.stdin.readlines()[:10]))" < /path/to/your/file

here is the same in Ruby

ruby -ne 'puts $_ if $. <= 10 ' < /path/to/your/file

Compare for yourself between these collections of oneliners in python and ruby. If ruby reminds of perl, your eyes do not deceive you. In many ways it is the love child of perl and smalltalk.

DSLs FTW

Some time ago, I tried to watch all the Structure and Interpretation of Computing lectures. This endeavor failed miserably but I recall Harold Abelson saying that every large program should have its own internal DSL suited to the problem space. This is debatable in regards to the larger world of programming but I think it is very apt for system administration. We spend our entire careers with n different DSLs. Each different configuration file format is its own DSL.

If you compare rake (Ruby Make) to rails code, they look quite different, almost like different languages. If you compare fabric code to a django class, they look quite similar. This is both a strength and a liability. I am not a language lawyer but it seems much easier to create DSL's (Domain Specific Languages). Ruby certainly spawns DSLs with much greater frequency than python. No single pythonic build tool dominates the problem space like rake does in the ruby community. Most python projects seems to use setup.py for administrative tasks even though that is not its explicit purpose.

Both puppet and chef are DSLs for system administration. Capistrano is a DSL for application deployment. Ruby's operator overriding and blocks lend themselves to the easy creation of DSLs.

In Summary

Ruby's greatest strength is its amazing flexibility. There is a lot of "magic" in ruby and sometimes it is dark magic. Python intentionally has minimal magic. It's greatest strengths are the best practices it enforces across its community. These practices make Python very readable across different projects; they ensure high quality documentation; they make the standard library kick ass. But the fact is that we sysadmins need flexibility more than we need raw power or consistency. Still, these are not the real reasons that ruby is overtaking python.

Ruby is fast becoming the default scripting language for sysadmins because back in 2004, Luke Kanies looked at Python and felt ill (I had the opposite reaction). As a sysadmin you either currently are or soon will be using Puppet or Chef on a daily basis, spending a heck of a lot of time essentially coding in ruby. Personally, I much prefer writing python but am shifting to writing my scripts in ruby because I spend so much time with Chef.