Any programmer worth their salt has used and praised revision control systems. The weird thing to me though is the fact that it seems as if the majority of sysadmins havn’t embraced revision control.  If you look at what you do as a Sysadmin, it really isn’t that much different from what programmers do. I’ll bet you spend more time scripting solutions so that you don’t have to do the same task over and over again.  I’ve worked both with and without VCS’s, and I can tell you that working without them is a horrible way to work. If you don’t have some sort of VCS to put all of your scripts into you quickly end up in the untenable situation where you have random scripts laying all over the place. You’ll have some scripts that only exist on the server they were written for, some scripts sitting out on the file share, while others are just sitting on one of your admin’s computers. Or, very possibly,  just sitting on your desktop!

I’ll be the first to admit, Sysadmins generally don’t need all the fancy features of some of the modern VCS’s. But, we do need something to centrally manage our scripts and config files – these really our life blood as Sysadmins.

When looking for a solution to implement, there are a plethora of  options out there to choose from:

  • CVS
  • Subversion
  • Git
  • Mercurial
  • TFS

All of these different systems have their ups and downs, a deep dive into the details of each is well beyond the scope of this post. I’ll just leave you with this bit of advice that has always been good to me – “Look at all the options and choose what is best for you.” Personally I’ve been very happy with just plain old CVS and that has served me very well. However, like any of the systems you need to get used to its little quirks. Once you choose one and get it running the way you want, the world gets much much brighter.

Take the following scenarios as an example and learn from them – I know all of you out there have run into a similar situation in the past. These very types of situations are drawn from my own experiences as a sysadmin, and have molded , and even changed my personal view on Version Control systems.

You write a little utility script for a seriously one-off function that your boss swears up and down you’ll never have to do again.

Point 1. Yes you will need to do this again because your result will be expected at every board meeting from now to infinity.

Point 2. You are going to think of a way to make it even more awesome and will completely refactor the script to make it more awesome.

Now, lets tackle the first situation erm first.

You’ve just tossed this script together on your local machine (sans comments, or even a unified coding style of course)  you run it and get the results your boss has been clamoring for all day (he even got out the pointy stick ). Your boss is happy because he has the info he needed to make his boss happy, you are happy because he put the pointy stick away before causing permanent damage. You move on to working on more important things like figuring out why the HA cluster keeps failing over at noon on Fridays. Three months later your boss comes around 1 hour before the next board meeting asking for “that thing you did last time with the thing.” I mean you should be able to pull it right up since you did the hard work for him 3 months ago right?  Ahh you say, I still have that script! this should only take 5 mins. Of course what you forgot is that you re-imaged that laptop you had the only copy of the script on, of course you backed your laptop up like a good sysadmin before re-imaging – it’s on one of those …. usb .. drives… over … there.

Now, you poor soul if you had checked your script into your “random stuff” VCS just in case you could have just pulled the most recent version down, run the script and forgotten about it for three more months.

Next, we have the much more probable situation where you write the same script as above, and decide “I can make this more awesome!” So you start hacking away. tweaking this and that moving this, removing that. Then suddenly you realize you have completely forgotten what you just changed that makes the script drop that image of your dog into the output of everything instead of the data you need. You start pushing that code, tweaking that bit there, move this around delete that. And now … and now the script does ….. nothing. You have no way back. You need to start over.

Yet another situation where a VCS system would have saved your skin. You could just simply have reverted to the last working version and moved on from there.

Beyond those – obviously made up – situations, a few more of the added benefits of a VCS system are:

  • Centralize backups of your scripts, configs, anything you put into the VCS
  • Check and see who checked in that change that broke the production environment
  • Persist your scripts easily across machines and employees
  • Feel smug in the fact you do this when you hear of someone losing “the script”

What this is really all about is preserving your hard work. You work hard to preserve the work of others, why not work hard to preserve your own work?

  • pjz

    It’s not just scripts – config files too. Put your dns, mailserver, etc. config files under version control and never be afraid of making a change that you don’t remember how to back out. Also become able to see exactly what you’ve changed to finally get it to work.

    • Put your puppet config files in there too, and run a commit hook that does puppet-lint on them.

  • SVN works pretty well as a distributed file system too with offline access. I setup VisualSVN Server ( ) on a web server a few years back and we now use it for tracking over 2GB of office documents between 5-6 people.

  • If you work at a place with developers, use whatever they use (assuming it is something they like, and not a system they were forced to use for ‘corporate’ reasons). You’re probably already administering and backing up the machine their server runs on anyway, and then you can ask them for help with the finer points, which is a nice piece of turn-about.

  • Pingback: Tweets that mention Version Control Systems are not just for programmers - --

  • BCS

    Heck, yeh! In collage, I even keep my English homework in a VCS. Just the ability to sync files across between my laptop and desktop was worth it.

  • We’ve found that a Wiki works brilliantly for this sort of thing. It also allows you to link and integrate other customer data, provide notes and warnings to the user or subsequent modifier, give step-by-step instructions for use, and use categories to help organize what you’ve written. It also gives you nice syntax highlighting, which, at least for me, has an outsized impact on the readability of the code.

  • abel

    That is so like my desktop. What I do is select most of the files and drag them into a folder, which lets me forget about the desktop for a week or so.