Show Notes -  The Safety of Work - Ep. 109 Do safety performance indicators mean the same thing to different stakeholders

Dr. Drew Rae and Dr. David Provan

The abstract reads:

Indicators are used by most organizations to track their safety performance. Research attention has been drawn to what makes for a good indicator (specific, proactive, etc.) and the sometimes perverse and unexpected consequences of their introduction. While previous research has demonstrated some of the complexity, uncertainties and debates that surround safety indicators in the scientific community, to date, little attention has been paid to how a safety indicator can act as a boundary object that bridges different social worlds despite being the social groups’ diverse conceptualization. We examine how a safety performance indicator is interpreted and negotiated by different social groups in the context of public procurement of critical services, specifically fixed-wing ambulance services. The different uses that the procurer and service providers have for performance data are investigated, to analyze how a safety performance indicator can act as a boundary object, and with what consequences. Moving beyond the functionality of indicators to explore the meanings ascribed by different actors, allows for greater understanding of how indicators function in and between social groups and organizations, and how safety is more fundamentally conceived and enacted. In some cases, safety has become a proxy for other risks (reputation and financial). Focusing on the symbolic equivocality of outcome indicators and even more tightly defined safety performance indicators ultimately allows a richer understanding of the priorities of each actor within a supply chain and indicates that the imposition of oversimplified indicators may disrupt important work in ways that could be detrimental to safety performance.

Discussion Points:

• What we turn into numbers in an organization
• Background of how this paper came about
• Four main groups - procurement, incoming operator, outgoing operator, pilots
• Availability is key for air ambulances
• Incentivizing availability
• Outgoing operators/providers feel they lost the contract unfairly
• The point of view of the incoming operators/providers 
• Military pilots fill in between providers
• Using numbers to show how good/bad the service is
• Pilots - caught in the middle
• Contracts always require a trade-off
• Boundary objects- what does availability mean to different people?
• Maximizing core deliverables safely
• Problems with measuring availability
• Pressure within the system
• Putting a number on performance 
• Takeaways:
• Choice of a certain metric that isn’t what you need leads to perverse behavior
• Placing indicators on things can make other things invisible
• Financial penalties tied to indicators can be counteractive
• The answer to our episode’s question – Yes, metrics on the boundaries can communicate in different directions

Quotes:

“The way in which we turn things into numbers reveals a lot about the logic that is driving the way that we act and give meaning to our actions.” - Drew

“You’ve got these different measures of the service that are vastly different, depending on what you’re counting, and what you’re looking for..” - David

“The paper never draws a final conclusion - was the service good, was the service bad?” - Drew

“The pilots are always in this sort of weird, negotiated situation, where ‘doing the right thing’ could be in either direction.” - Drew

“If someone’s promising something better, bigger, faster and cheaper, make sure you take the effort to understand how that company is going to do that….” - David 

Resources:

Link to the Paper 

The Safety of Work Podcast

The Safety of Work on LinkedIn

Feedback@safetyofwork