​Configuration management today is mess if you work in a heterogeneous platform.

There is tooling that takes a stab at it, and is getting better (from the *nix world – Puppet, CFEngine, and Chef and from the Windows world – System Center Configuration Manager, Group Policy, among other third party application deployment platforms).  These tools are all well and good, but they fall down when reaching across the OS divide.  Puppet, Chef, and CFEngine (there are others as well, but these are some of the more popular) all have some cross platform support, but it feels unnatural (especially in module or recipe development).

Why is this a mess?

Windows is traditionally described as having an API oriented management model, whereas *nix has a document based management model.

Well, that’s a load of crappy, crap, crap.  What does that actually mean?

It means that the two operating systems offer two different management models.  The two different models have different abstractions and idioms for operating system constructs.  Let’s look at a concrete example, setting a static IP on a network interface (just the rough strokes.. I’m not going to spend too much time on the minutia). As I stated before, Linux uses a document oriented management model, so to configure my network interface, I’ll edit a document or two.

The Linux (Centos) example:

  1. Find the correct interface file under /etc/sysconfig/network-scripts
  2. Open it in your text editor of choice
  3. Edit it to contain your desired settings for the network interface and save the file
  4. If you need to add/modify DNS servers, find /etc/resolv.conf
  5. When done, you can bring your interface online with a command line call to
    ifup eth0

You’ll have something that looks like this for your network configuration file:

DEVICE=eth0
IPADDR=192.168.10.80
NETMASK=255.255.255.0
NETWORK=192.168.10.0
BROADCAST=192.168.10.255
GATEWAY=192.168.10.1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
And something like this for your resolv.conf:
domain serverfault.com
search serverfault.com
nameserver 192.168.10.60
nameserver 192.168.10.61
That wasn’t so bad, and as an added benefit, they are just text files, so I could check them in to a revision control system (Versioning FTW!). Now, let’s look at what we’d need to do on the Windows side.  Since this is a blog for a community of “professional” systems administrators, we are going to dispense with any GUI example for doing this.

The Windows Server (2008 R2) example:

  1. Use WMI to retrieve the network adapter interface index.
  2. Use WMI to retrieve the network adapter configuration by the index.
  3. Set the desired IP address, gateway, and DNS servers and suffix against the WMI object.

You can use the following PowerShell commands to make those changes:

$NetworkAdapter = Get-WMIObject Win32_NetworkAdapter -filter "NetConnectionID = 'Local Area Connection'"
$NetworkAdapterConfiguration = Get-WMIObject Win32_NetworkAdapterConfiguration -filter "InterfaceIndex = $($NetworkAdapter.InterfaceIndex)"
$NetworkAdapterConfiguration.EnableStatic('192.168.10.80','255.255.255.0')
$NetworkAdapterConfiguration.SetGateways('192.168.10.1','1')
$NetworkAdapterConfiguration.SetDNSServerSearchOrder('192.168.10.60','192.168.10.61')
$NetworkAdapterConfiguration.SetDNSSuffixSearchOrder('192.168.10.60')
and

The Windows Server (2012) example:

  1. Set the desired IP address and gateway based on the interface name.
  2. Set the DNS servers and with a few more PowerShell commands.

You can use the following PowerShell commands to make those changes:

$IPAddressParameters = @{
            IPAddress = '192.168.10.80'
            InterfaceAlias = 'Local Area Connection'
            AddressFamily = 'IPv4'
            PrefixLength = 24
            DefaultGateway = '192.168.10.1'
}
Set-NetIPAddress @IPAddressParameters
Set-DNSClientServerAddress -InterfaceAlias 'Local Area Connection' -ServerAddresses '192.168.10.60','192.168.10.61'
Both of these examples are interactive commands, but I could easily save them in a file and place that under version control (and I should).

So what?

The examples don’t look all too different, but they do illustrate the difference between similar operations.  In both  examples, I end up with an artifact, but one is for a one time application of the setting (the Windows side) and the other is the setting storage location (the Centos example).

On the Centos box, we had to edit a file where the configuration was read from.  On the Windows servers, we updated settings via a WMI API (in both cases.. on Server 2012 there are more built in cmdlets, but many of them are thin wrappers over the WMI APIs) and not the actual end storage location.

This is what

Any configuration management tool that works in a cross platform capacity needs to understand these distinctions and check based on OS type which implementation to use when configuring a system.  This means for most configuration types, you’d have a big “IF” block where *nix based OS’s follow this line of processing and Windows based machines follow  the other line of processing.  This can become a maintenance nightmare as OS versions change the API on the Windows side or modify location and or layout of the configuration files on the *nix side.

And it’s even worse…

Now, what happens when you have a model that doesn’t translate across both worlds?

For example, how do manage file permissions?

Posix style permissions (used on most *nix variants) assigns permissions are nowhere as discreet as NTFS file permissions.  In addition on Windows, the file system auditing is also configured via the permissions configuration. In the reverse, on *nix files can be set as executable, where that is handled by file type mappings based on file extensions in Windows. This fragmentation leads to more complex implementations on the side of configuration management software developers or missing feature coverage.  In either case, this is a loss for the sysadmin who maintains cross platform environments.

But what if….

there was a common method of interacting with operating systems, regardless of what was running underneath? What if this method used a common transport (open standard) and communications were defined by an open standard? This is the direction Microsoft is taking with CIM and WS-Management.

CIM (Common Information Model) is a DMTF (Distributed Management Task Force) standard for describing management information for systems, networks, applications, and services.

WS-Management is another DMTF standard for management communication, focused on CIM traffic.

Microsoft has contributed to an open source project hosted by the Open Group called OMI.  OMI is a CIM server that communicates over WS-Management and is implemented to run on *nix based operating systems.

I’m personally interested in where this will go, given Microsoft’s market power (Cisco and Arista are working on incorporating OMI into their network switches).  The idea of a shared management model is very appealing to me, as I work in a cross platform environment.  I’m responsible for our Windows infrastructure, but I have to be able to work with our *nix infrastructure as well.  If I could use one model for interacting with both, that’s a huge win for me and my team.

This wouldn’t eliminate any domain specific knowledge on either OS side, as you’d still need to know what buttons to push and knobs to tweak to get things going and do some deep troubleshooting.  It does, however, make the idea standardizing how various OS components can be accessed, making basic configuration, monitoring, and troubleshooting much easier.

I’m interested because…

this pushes the implementation down to the OS provider (or the CIM provider provider) and gives vendors one target to hit for configuration standards.  In the Microsoft case, they can say “Follow this standard and any Windows system can manage you with minimal effort.”  If other OS’s support CIM and WS-MAN as well, it becomes easy to offer management interfaces there as well.

Obviously this would be huge change to the existing way of doing things for OS and application developers, not to mention systems administrators that are invested in their existing ways of doing things.

I don’t see another good alternative though, as the numbers and variety of systems continue to scale up and “cloud” becomes more of a factor in our environments, yet the number of admins is staying stagnant or being reduced.  Simplifying the management and monitoring surface makes sense in today’s and likely tomorrow’s data center landscape.

It doesn’t solve every problem and vendors can still implement vendor specific extensions (and we know how well that’s worked with SNMP).

 

NOTE – Be sure to check out some clarifications and expanded discussion in my followup post.

«
»
  • There are many tools that do this (along with all other administrative tasks which may be needed) – the 800 pound gorilla in that field is HP’s Server Automation (nee Opsware SAS).

    {disclaimer – I work for an HP partner}

    • stevenmurawski

      Hey Warren, thanks for the comment.

      I don’t want this to get into a “this” tool vs. “that” tool, but rather the standardization of a configuration and management model across platforms (network devices, Linux boxes, and Windows boxes for example).

      This is something that multiple tool vendors could leverage, rather than focusing on building specific implementations for basic functionality, they could focus on the real value adds.

      Yes, there are tools out there that can do that, but it still isn’t an easy problem. There is a cost, either in time and/or money to cross these hurdles.

      • The reason all these tools exist is because no one wants to go cross-platform in their management protocols: MS doesn’t want to support *nix, and vice-versa – everyone comes up with their own methods, and they are loath to use others.

        I only know one (third-party) tool that does that, hence the previous reference 🙂

        The other solution, of course, is to internally revert to single platform environments instead of being heterogeneous.

        • stevenmurawski

          That was the case. The OMI project I mentioned above is an effort to stretch the CIM management model across Windows to LInux and support DMTF standard interfaces that any configuration management tooling can use.

  • Nils Hildebrand

    Why should CIM achieve what SNMP did not?

    • stevenmurawski

      SNMP was pretty good at pulling monitoring data, but was tough for applying configuration. Security was a mess (it really wasn’t in focus at the time of implementation).

      CIM/WSMAN has a decent security story, supports SSL, client cert auth, etc.. And I’ll be blogging further about some of the other features of CIM.

      As to why it “should” achieve what SNMP did not, I don’t think it has any moral mandate to succeed, but I think it has a good chance in the long run. Short term, there will be a lot of organizational inertia to overcome in many camps, so we’ll see how it progresses.

      • Matt Finnigan

        I’ve been looking at CIM from time to time for years now. I know that it’s supported by Dell DRAC devices, and ESXi, both via WSMAN. It might finally be maturing out of the vaporware/too-damn-difficult phase now.

        • stevenmurawski

          That’s my hope, but we’ll see how it continues to develop.

      • Nils

        As to security – SNMPv3 is there – but no one uses it.

        I think the core-problem of SNMP is/was that vendor-specific stuff. If CIM has the same mechanisms here, there will not be that common interface. I take Cisco/SNMP as an example. Instead of using the standard-MIBs, you have to use theirs to get deeper information about configured trunks. So SNMP did standardize the way to communicate, but not how to access certain deeper (specific) information (across vendors). I see the same problem with CIM. The vendor-specific specials (in this case: OS-specific, like you outlined in your article) will develop faster than any standard.

        • stevenmurawski

          It is possible that vendor specific extensions could develop along how the Cisco MIBs developed. I’m not too familiar with how all the history developed with SNMP in that regard (just the result).

          CIM provides a way to extend the base CIM classes with vendor specific stuff, but you still can query the base CIM class and get the common information. For example Win32_LogicalDisk implements CIM_LogicalDisk. Regardless of any special vendor specific details Win32_LogicalDisk provides, it’ll offer the base CIM_LogicalDisk details and configuration options.

          The benefit for vendors implementing the standard CIM classes is that a wider array of management tooling will be able to manage/monitor it out of the box, providing more incentive to implement the special stuff.

  • Eric Eisenhart

    So, the answer to having too many standards and a fundamental difference between API-based management and document-based management is more APIs? Doesn’t seem likely to take over the world unless somebody uses it to backend a document-based system (puppet, chef, etc). Also: http://xkcd.com/927/

    • stevenmurawski

      Possibly. Again, I don’t know how this will progress, but I like the idea of there being one API implemented by operating system providers and application developers, rather than n+1 api’s/document models, with one per CM tool set plus whatever homegrown systems you want to interact with.

  • Jay Adams

    Nice article Steve. There have been a lot of standards over the years, WMI being my favorite. Hopefully this will catch on and get vendor buy-in. I’ve considered building some standardized APIs for cross platform management, but don’t want to reinvent the wheel.

    • stevenmurawski

      Thanks Jay. I hope so as well. MS has some influence in the space, but there are no guarantees. I just really like the idea of a standard interface for dealing with systems.

      I think some tooling out there tries to do that, but in each case, it is a proprietary (even if open source, it’s still particular to that product/project) protocol (some more than others).

  • Willard Dennis

    Great article Steve – it would indeed be a better world with a cross-platform config management framework. I hope that OMI takes off, but it’s up against a mountain of inertia I fear…

    Related to this article is a great DevOps Cafe podcast they did with Jeffrey Snover (Distinguished Engineer/Lead Architect Windows Server at Microsoft, and inventor of PowerShell) which can be found at http://bit.ly/VdM31p – check it out, it’s a great listen (even for the UNIX admins among us!)

    • stevenmurawski

      I love that show. Damon and John cover some great topics and guests. The show with Snover was very interesting and one of the first places you publicly started hearing about the Data Center Abstraction Layer.

  • Andrew

    Document-based is a declarative approach, and API-based is a procedural approach. Hmm. Never really thought of unix as more declarative than windows!

  • Very Exclusive Information, I read your blog and i have gained many useful points to how monitoring data. Thanks

  • booromand

    Oh. It is very good for configuration management. Thanks to this post راهبند اتوماتیککرکره برقیلامپ کم مصرفدرب ضد سرقت

  • heshmat
  • gita

    Configuration management is very important and I think this post is very useful..تست جوشتست خاک سایپانمایندگی سایپا

  • sayan
  • amir
  • sabz

    Thanks to this postنهال پسته

  • saray