Metrics That Lie

British Petroleum (BP) blew up the Texas City facility several years ago, registering billions in productive losses and eventually receiving a fine from OSHA amounting to 50 million dollars. In the five years after the incident, another three people died under their watch, and they were fined another 87 million dollars for their failure to fix original problems; and recently a jury awarded 100 million dollars to workers exposed to toxic chemicals.

There are a few possible conclusions you can draw from this, and none of them are particularly kind to BP, but regardless of those conclusions you can draw, there are a few you cannot propose. One of those conclusions that isn’t true is that every manager at BP is an evil sociopath. But if you accept that most managers don’t feel particularly pleased to have people die on their watch, why then did what led to Texas City not modify the behaviours effectively to avoid repetitive similar problems?

Part of problem is Senior Executives not getting the right information, which creates greater risk. Because performance metrics that they see focus on recordable events and loss rather than risk-management, the Board of Directors and Senior Executives frequently do not have the right information to provide effective oversight, resulting in unacceptable risk and significantly increased liability.

Typical reports to senior management are too superficial to allow meaningful involvement or critical thinking about what the reports imply. Crippled by this dearth of valid, approachable information, executives end up making choices that are, in retrospect, poor choices by any measure. Traditional safety programs that rely upon counts have no representation of implementation in their outputs, and so the executive relying on this is essentially incapable of making the decisions to enhance safe operations. Even when not faked, these counts obscure operational context to a degree where they are meaningless as analytical tools.

Metrics need to be transparent. Without the ability to see beneath raw counts, and gain a contextual understanding of what those numbers mean, risk is being misidentified, mistaken as to degree, and will mislead executives. The right information allows executive to assess risk and control measures, reducing liability and loss. Importantly, it allows them to speak directly to what matters, moving away from the superficial and meaningless attempts to demonstrate commitment to safety (the infamous safety moment that begins every meeting).

Actions speak louder than words, and the executive armed by a good metric set will always make better informed decisions. One wonders if the BP executives had those metrics prior to Texas City if the event would have been mitigated as to damages, or even avoided. Regardless of that, though, it is clear that if they had those metrics now, the problems wouldn’t be ongoing. Traditional safety approaches are why they don’t, and why they may not ever have the advantage of a clear understanding of their risks.

Metrics that lie kill people.