Data Quality is Critical

One of the revelations of our research was that the quality of data in traditional safety programs is abysmal. When we would ask safety personnel for their investigation documents, we would get documentation that often was so incomplete as to make it impossible to identify human involvement at all. In the scope of legacy data we handled, we had an average 15% rate of serious injury incidents where the injured party could not be identified. In other cases, multiple reports on the same accidents would contradict on basic levels such as classification choices, with no additional documentation available to ascertain which was correct (both were probably incorrect). The incredible data quality deficit was made worse, by an order of magnitude, by the massive amount of static documentation that was often available.

The data quantity should probably have been less surprising, given that traditional safety operates on counts alone most times, but it was amazing to encounter piles of documents about safety meetings, some with identified hazards listed, and have no associated documentation to indicate if the assigned corrective actions were ever completed. It was usually impossible to even ascertain who was at a meeting, or who was assigned to ensure the corrective action. Even when follow-up might have been done, the scale of the paper made locating the proof impossible.

The problem of generating paper is that paper is impossible to track effectively. You can not ensure it is kept, often cannot relate it to anything else, and its fundamental fault is that every page is discrete and in no way connected to the next. Even conscientious feedback gets lost when the paper is never collated, reviewed, classified, or audited.

Traditional safety is less sensitive to bad data than risk-management because it is about counts. Showing fifty inspections is fifty activities counted, and no one ever really asks you to prove they were worthwhile. In risk-management, fifty inspections with no connective substance will expose themselves as irrelevant instantly.

One of the greatest barriers so far to deploying risk-management for even the best intentioned companies has been that when they can get their data (not always as easy as it should be), they can almost never get quality data. The employee list will contain names no one has ever heard of, have missing people, have multiple spellings of the same name, etc. The occupations list will be a third the size of the employee list, with obvious spelling mistakes, and no real relationship between people and occupations (on average, clients have about 60% of listed occupations that have no apparent people employed to do those occupations). This low quality is fatal to any system that relies upon delivering upon value propositions.

Of course, in a grander scheme, the low-quality data exposed by trying to transition to risk-management should raise flags. Management quickly becomes disillusioned, if they care at all, when they discover things like Human Resources cannot give a list of employees that makes any sense at the click of a button, or that all training records are in paper form and George down the hall might have a spreadsheet he keeps that shows it in relation to the folks he knows about, which contradicts the one Bob keeps.

One of the saddest experiences we have had is having to say to companies they have no reliable data sources, because as soon as those words are used, they have one of two reactions: they bury their heads in the sand and pretend this isn’t a problem in their normal operations; or, they see it and think they cannot afford to fix the problem.

Data quality is critical for risk-management to function operationally; but, it is vital to recognise it is critical to day-to-day operations. If your information systems cannot provide reliable basic data about operations, you need to rectify that whether you want to transition to risk-management or not. The end result is the decisions you make are based on real quality information, rather than obscure guesses.