Why you need Blue Herring
Configuration change monitoring for your IT operations You can't always control change, but at least you can monitor it.
The world is full of devops monitoring tools.
Nagios and New Relic are everywhere. And there are a lot of universal configuration and deployment solutions like Terraform that put all the installation and setup of systems into one super convenient umbrella. Azure has that Desired Configuration State feature that locks everything into a set of template files.
With all that going on, maybe it’s not obvious why you’d want a configuration change monitor. But here are a few use cases:
- Your environment is multi-vendor and multi-platform, and there’s no tool that watches the whole thing for you. Problems that span environments need monitoring that span environments.
- You might not control all of the resources that are relevant to your systems’ functioning. For example, there’s a vendor’s external API that might change its schema without a lot of notice.
- You have a rigorous change control process, but once in a while an application breaks in production and the problem turns out to be a change that appeared harmless when it was approved. When that happens, you’re going to want to see all the changes that might be relevant even if they were anticipated and authorized.
Bottom line, configuration changes are a really common cause of application failures and downtime. And you usually spend way more time asking “what the heck changed?” than you do actually making the fix. So we decided to focus on identifying changes in configuration because that’s where operations and devops people tend to lose a lot of time. And patience.
That’s why I created Blue Herring.
It delivers two main things for your operations teams:
- First thing, it tells you when configurations do change. That makes troubleshooting up to 64% faster when something goes wrong. Because Blue Herring scans all your stuff continually and just notes the changes in configuration, you have an instant answer to the question of “what the heck changed?” Since most things don’t change configuration very often, you don’t have to keep trimming massive log files or picking through a zillion alerts.
- Second thing, it tells you when configurations don’t change. If you’ve gone through a formal operations audit, or if you simply have things in a stable state and want to document any deviations, Blue Herring will certify that the enterprise system configuration that you’ve documented as working is still unchanged. It’s like locking in your audit results.
Blue Herring does these things in a vendor-neutral, environment-neutral way. You can drop Blue Herring into your production systems no matter what they’re running on. No matter how you’re doing deployments. No matter what other monitoring solutions are going on.
By tech people, for tech people.
I made a fancier CTO-type presentation about the business case for configuration change monitoring. Then after listening to B2B marketing people, I went and built a process consulting methodology around it. I do believe in that methodology, and you can still get it, but practitioners like you hated the idea! (You know what you’re doing, you just want the tools to make it easier.)
A few years ago, my college pal Dave Stagner shared with me the idea of “configuration change monitoring as a service”; he’s kind of a devops lifer and wanted to develop this concept together. We jammed on that until Dave got bored 🙂 and now he’s writing music instead.
I have a higher boredom threshold and I’m bad at music so here we are.
I’m interested in tech that delivers predictable business value, so past clients and employers always put me in charge of unglamorous “back end” stuff like provisioning, software distribution, data polling, schema updates, consolidation, ETL, stuff like that. Basically devops before it was called devops.
We planned the kind of tool that we would actually need and use. And now it’s ready for some early beta users.
Who’s behind the narrative voice here
Left: Peanut “Voo Voo” Boyd. Right: Mark W. “Poppy” Schumann.
You can read more about the actual team.
How it works, simplified overview.
Blue Herring’s Polymorphic Scanner runs in a continuous loop, reaching out to your defined “resources” as often as you require (although there are default intervals). When it’s time to poll a given resource, Blue Herring executes code that is specific to the type of resource (that’s the polymorphic part), extracts a “reading” of relevant configuration data, and stores it in its own database. Over time it accumulates a change history for the configuration of every “resource” and the output of that “diffing” is the key part of the deliverable. You use that to get the troubleshooting and auditing benefits.
There you go. That’s what Blue Herring is for, in devops terms.