The Safety of Work

Ep. 125: Does ChatGPT provide good safety advice?

Nov 10, 2024

The discussion dives into the challenges of using ChatGPT for safety advice, revealing its biases and limitations. Topics like mobile phone use while driving and fall prevention for older adults highlight critical gaps in AI-generated guidance. The hosts emphasize the necessity of expert insights, especially in emergencies. They also explore data privacy concerns related to fitness apps and the complexities of burnout and workplace pressures. Ultimately, the episode serves as a cautionary tale about over-reliance on generative AI in safety matters.

59:43

Creator website

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

ChatGPT often lacks the necessary nuance and can provide oversimplified safety advice, potentially leading users to make uninformed decisions.

The absence of transparency in ChatGPT's sources raises concerns over the validity of its safety advice, risking the spread of misinformation.

Effective use of ChatGPT requires careful phrasing of queries, as the quality of its recommendations can significantly vary based on wording.

Deep dives

Concerns with AI-generated Safety Advice

The use of ChatGPT for obtaining safety advice raises significant concerns due to the risk of oversimplified or erroneous responses. ChatGPT tends to provide confident answers that may lack the necessary nuances for complex safety issues. This overconfidence can mislead users who may believe the information is more accurate than it actually is, even if it often aligns with common knowledge. Hence, it's vital for professionals and individuals to seek confirmation from credible, evidence-based sources before making safety-related decisions.

Intro

3min

Evaluating ChatGPT for Safety Advice

9min

Navigating Mobile Safety: AI Insights

4min

AI-Generated Safety Advice: Strengths and Shortcomings

16min

Navigating Suicide Prevention with ChatGPT

6min

Exploring Data Privacy in Fitness Apps

2min

Navigating Data Privacy and Machinery Safety: Understanding Key Distinctions

3min

Navigating AI for Safety Advice

18min

From discussing mobile phone use while driving to the challenges of giving advice to older adults at risk of falls, this episode covers ChatGPT’s responses to a wide range of safety topics - identifying biases, inconsistencies, and areas where ChatGPT aligns or falls short of expert advice. The broader implications of relying on ChatGPT for safety advice are examined carefully, especially in workplace settings. While ChatGPT often mirrors general lay understanding, it can overlook critical organizational responsibilities, potentially leading to oversimplified or erroneous advice. This episode underscores the importance of using AI-generated content cautiously, particularly in crafting workplace policies or addressing complex safety topics. By engaging with multiple evidence-based sources and consulting experts, organizations can better navigate the limitations of AI tools.

Discussion Points:

Drew and David discuss their own recent experience with generative AI
The multiple 15 authors are all experts, discussing the methods used
Examining the nine different question scenarios
‘Mobile phone use while driving’ results
Crowd/crush safety advice
Advice for preventing falls in older adults
Analyzing ChatGPT response formats
Exercising outdoors near traffic with asthma
Questioning ChatGPT about how to engage a distressed person who may commit suicide
Safety working ‘under high pressure’ and job demands, burnout prevention
Lack of nuance in ChatGPT
The safety of sharing personal data on fitness apps, how can it be shared safely?
Is it safe to operate heavy machinery when fatigued? Testing several ways to ask this question - sleepy, tired, fatigued
Conclusions and takeaways
The answer to our episode’s question: “AI is not currently a suitable source for writing safety guidelines or advice”
Like and follow, send us your comments and suggestions!

Quotes:

“This is one of the first papers that I've seen that actually gives us sort of fair test of ChatGPT for a realistic safety application.” - Drew

“I quite like the idea that they chose questions which may be something that a lay person or even a generalist safety practitioner might ask ChatGPT, and then they had an expert in that area to analyze the quality of the answer that was given.” - David

“I really liked the way that this paper published the transcripts of all of those interactions

with ChatGPT. So exactly what question the expert asked it, and exactly the transcript of what ChatGPT provided.”- David

“In case anyone is wondering about the evidence based advice, if you think there is a nearby terrorist attack, chat GPT's answer is consistent with the latest empirical evidence, which is run. There they go on to say that the rest of the items are essentially the standard advice that police and emergency services give.” - Drew

“[ChatGPT] seems to prioritize based on how frequently something appears rather than some sort of logical ordering or consideration of what would make sense.” - Drew

“As a supplement to an expert, it's a good way of maybe finding things that you might not have considered. But as a sole source of advice or a sole source of hazard identification or a sole position on safety, it's not where it needs to be…” - David

Resources:

The Article - The Risks Of Using ChatGPT to Obtain Common Safety-Related Information and Advice

DisasterCast Episode 54: Stadium Disasters

The Safety of Work Podcast

The Safety of Work on LinkedIn

Feedback@safetyofwork