At a recent talk at PICC, part of my introduction was:

“Hello and thank you for coming to our talk. This is George Beech and I am Kyle Brandt, and we are the two sysadmins at Stack Exchange…”

While “two” is the technically correct answer going by our self-anointed titles, it isn’t really the whole picture.

The Sysadmin Continuum

In reality here at Stack Exchange we all wear a lot of hats. While George and I may own the system administration tasks, we work together with our developers who are pretty accomplished sysadmins in their own right. If it were just George and I soely doing the work that could be considered system administrator work, we would probably have a pretty tough time of it at only two people.

In reality we are continuum. System administration is just a spot on the continuum where George and I spend most of our time. However, a lot of our developers spend a lot of time there and travel the same road.

Some recent examples:

  • Jarrod Dixon worked with me on developing a log daemon for inserting HAProxy syslog data into SQL Server
  • Nick Craver frequently hops on Dell and makes hardware recommendations
  • Jeff Atwood helps us with SSD research
  • Sam Saffron works on SQL restore scripts
  • Geoff Dalgas racks servers out in OR and troubleshoots our NAS
  • Ben Dumke and Jeff Atwood do some CDN research
  • Jeff Szczepanski, our Vice President of Products, analyzes packet dumps

Because of the spirit of shared responsibility, we get to take advantage of everyone’s knowledge and experience and we are all better for it.

So in the end, “How Many System Administrators does Stack Exchange Have?” is a question I can’t really answer.

  • Andy

    Does that mean the whole team have full acess to all infrastructure and servers, or are load balancers etc “sysadmin only”?

    • Hi Andy,

      The load balancers are accessible by the sysadmins and a couple of the devs — but not all the devs.  

  • I know how this works… before I got to the present gig the SA role was held variously by a lot of people. Right now I’m primary, but it also means that the ‘operational point of view’ is held by several of our developer-oriented people and that’s not a bad thing. DevOps indeed.

  • Matt

    SSD research? SSD drives are so much faster than traditional hard drives that’d probably doesn’t make sense to do too much research. you will see a big benefit regardless. I know jeff is an extreme hardware enthusiast but I gotta think time is better spent. programming.

  • Pingback: Sysadmin Sunday #31 « Boxed Ice Blog()