Speaking Of Reliability: Friends Discussing Reliability Engineering Topics | Warranty | Plant Maintenance

Phantom Test Results

May 5, 2025

Dianna and Fred delve into the intriguing world of phantom test results in reliability testing. They emphasize the necessity of root cause analysis to tackle test failures effectively. The conversation touches on how faulty test rigs can produce misleading results. They share humorous anecdotes about incorrect test criteria and the real-world failures that ensue. The duo highlights the vital role of understanding customer use cases to improve testing protocols and ensure products perform reliably in actual conditions.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

Ignoring Failures Invalidates Tests

In a zero failure binomial test, simply replacing a failed unit without investigating the failure invalidates the test purpose.
Fred Schenkelberg shared a case where failed units were discarded without root cause analysis, which was a big no-no.

ANECDOTE

False Failures from Test Rigs

Fred experienced false failures caused by the test rig and improper test criteria demanding unrealistic precision.
They debugged by swapping test leads and checking component tolerances to identify software and measurement issues.

INSIGHT

Measurement Error's Hidden Impact

Measurement error can cause perfectly good units to falsely signal failure and vice versa, leading to chasing phantom results.
Test design must consider measurement system error to avoid type I and type II errors in reliability testing.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Phantom Test Results

Abstract

Dianna and Fred discuss phantom test results in reliability testing.

Key Points

Join Dianna and Fred as they discuss phantom test results and the challenges of reliability test methods.

Topics include:

The importance of root cause analysis for test failures.
How test rigs and measurement systems themselves can give false positive or false negative results.
Why understanding customer use cases is critical for effective testing.

Hear stories about phantom test results, incorrect test criteria, and unexpected product failures driven by factors not captured in standard lab tests.

Enjoy an episode of Speaking of Reliability. Where you can join friends as they discuss reliability topics. Join us as we discuss topics ranging from design for reliability techniques to field data analysis approaches.

Download Audio RSS

Show Notes

In this episode of Speaking of Reliability, Fred and Dianna continue their discussion on test methods, sharing insights and challenges they’ve encountered.

They highlight issues that can arise beyond just determining sample size, such as problems with the test method itself:

A failure to investigate why a unit failed in a binomial zero failure test, instead simply replacing it, violates the test’s purpose.
The test rig or measurement system can be the source of false failure signals. We can address this through troubleshooting the test setup.
Incorrectly setting test criteria, such as demanding unrealistic precision from components, can also lead to widespread false failures.
Measurement error is identified as a factor that must be considered when designing and running tests, as it can cause perfectly good units to signal failure and vice versa, leading to chasing phantom test results.

A significant challenge they talk about is the difficulty of replicating the complex, simultaneous stresses that products can face in real-world environments. Real world scenarios are unlike controlled lab tests, which are often sequential.

Their stories illustrate how unexpected conditions were not captured in standard testing, demonstrating the critical importance of understanding actual customer use cases and environments.

They note that without understanding use cases, even seemingly good, accelerated tests can provide misleading life predictions: those phantom test results. Using tools like FMEA and understanding customer needs are essential. They help prioritize testing and ensure results are meaningful and useful for decision-makers.

The post SOR 1067 Phantom Test Results appeared first on Accendo Reliability.