Cavalry or cattle? Let the machine decide

Long before dashboards and decibel-loud alerts, there were watchtowers. Every kingdom worth its salt had them, men perched on hills, lighting fires to signal the moment they spotted something suspicious on the horizon. It was, in its time, a fine system. The trouble was that watchmen, being human, occasionally mistook a herd of cattle for an invading army, or a dust storm for smoke, and lit their fires anyway. Word would reach the king, the court would panic, armies would be readied, and then, some hours later, a rather embarrassed messenger would arrive to report that the enemy was, in fact, a caravan of merchants.

One imagines that kingdoms, sooner or later, learned to build a second line of watchmen, whose sole job was to verify the first. The lesson, oddly consistent across very different cultures, was that information without verification is just noise dressed up as urgency. Somebody, somewhere, needs to look twice before the beacon is lit.

I mention all this because we appear to have simply moved the watchtower indoors and called it a dashboard. The horizon is now a server; the invading army is a metric drifting somewhere it shouldn't; and the watchman has been replaced by an engineer who, in the middle of the night, is no better placed to tell cattle from cavalry than his medieval predecessor. He too gets alerted far too often, and occasionally, on the one night it truly mattered, far too late.

Which is really the whole case for AIOps, once you strip the acronym off it: someone, or even better, something ought to be doing the verifying before the alarm ever reaches a human.