Understanding Bias in Clinical Trials: Performance and Detection

Reference: Cohen et al; SUSPECT study group. Can we avoid casting for suspected scaphoid fractures? A multicenter randomized controlled trial. J Orthop Traumatol. 2025 Date: May 1, 2025 Guest Skeptic: Dr. Matt Schmitz is an orthopedic surgeon specializing in Adolescent Sports Medicine and Young Adult Hip Preservation. He practices at the Rady Children’s Hospital in San Diego and is Professor of Orthopedics at UC San Diego. Case: A 48-year-old woman presents to the emergency department (ED) with left wrist pain after slipping on a wet pavement while walking into work. It was a FOOSH injury (fall on out-stretched hand). She did not lose consciousness and was able to drive herself to the ED, but she reports increasing wrist pain with movement. She rates the pain as 6/10 and notes it’s worse with lifting or rotating the wrist. She denies numbness, weakness, or swelling of the fingers. No previous wrist fractures or injuries. On inspection, she has no obvious deformity and minimal swelling at the wrist. The examination reveals tenderness in the anatomical snuffbox and over the scaphoid tubercle. Her range of motion is decreased due to pain, especially with radial deviation and wrist extension. Sensation and cap refill intact; radial pulse present. Plain radiographs (PA, lateral, scaphoid view) show no fracture. Background: Scaphoid fractures are a common injury seen in the ED but can represent a challenge to diagnose, even for experienced clinicians. The scaphoid injury is the most frequently fractured carpal bone, typically occurring after a fall onto an outstretched hand (FOOSH) in young, active individuals. The clinical dilemma arises when there is a high suspicion of fracture based on mechanism and physical exam (especially tenderness in the anatomical snuffbox), but the initial radiographs appear normal. An excellent SRMA on the history, physical exam, and imaging for scaphoid fractures was done by Carpenter et al AEM 2014. On SGEM#420, we wanted to know what to do with a patient who presents with a FOOSH injury and has a normal x-ray. Specifically, are there clinical exam findings that can help rule in/rule out a scaphoid fracture? The bottom line was there was no single physical examination maneuver that could reliably rule out an occult scaphoid fracture. Given this dilemma, ED physicians have erred on the side of caution. If a fracture is not radiographically visible, the patient is immobilized and referred for follow-up imaging in 10 to 14 days or orthopedic review. This conservative approach stems from the significant morbidity associated with missed scaphoid fractures, including non-union and avascular necrosis. However, this “cast first, confirm later” philosophy has led to over-treatment in most cases. Studies estimate that only 10–20% of patients with clinical suspicion of a scaphoid fracture and normal initial X-rays have a fracture confirmed on follow-up imaging. With this context, a new wave of research has emerged, questioning whether immediate casting is necessary or whether selective immobilization and early reassessment may be equally safe and more patient-centred. This ongoing debate challenges emergency physicians to balance the risks of under-treatment with the harms of unnecessary immobilization, time off work, and healthcare costs. Clinical Question: Can patients with suspected scaphoid fractures and normal initial radiographs be managed without casting, using a brief period of bandaging and reassessment? Reference: Cohen et al; SUSPECT study group. Can we avoid casting for suspected scaphoid fractures? A multicenter randomized controlled trial. J Orthop Traumatol. 2025 Population: Adults presenting to the ED with a clinical suspicion of scaphoid fracture but normal initial radiographs. Intervention: 3-day bandaging followed by reassessment. Comparison: Traditional 2-week casting with thumb spica. Outcome: Primary Outcome: Functional outcome at 3 months using the Quick Disabilities of the Arm, Shoulder, and Hand (QDASH) score. Secondary Outcomes: Patient-rated wrist evaluation, visual analog scale for pain, wrist range of motion, patient satisfaction, and complications including non-union Type of Study: Multicenter randomized controlled noninferiority trial Authors’ Conclusions: “Casting for suspected scaphoid fractures but normal initial radiographs can be avoided because bandaging seems to be an alternative treatment option when patients are reevaluated after 2 weeks.” Quality Checklist for Randomized Clinical Trials: The study population included or focused on those in the emergency department. Yes The patients were adequately randomized. Yes The randomization process was concealed. Unsure The patients were analyzed in the groups to which they were randomized. Yes The study patients were recruited consecutively (i.e. no selection bias). Unsure The patients in both groups were similar with respect to prognostic factors. Yes All participants (patients, clinicians, and outcome assessors) were unaware of group allocation. No All groups were treated equally except for the intervention. No Follow-up was complete (i.e. at least 80% for both groups). Yes All patient-important outcomes were considered. Yes The treatment effect was large enough and precise enough to be clinically significant. Yes (non-inferior) Financial Conflicts of Interest: None declared Funding of the study. “ZonMW and co-funding from CZ to the institution. Both organizations were not involved in either the study design, collection, analysis and interpretation of data, and in writing the manuscript.” Results: There were 180 patients included in the trial (91 in the bandaging group and 89 in the casting group). The median age was 45 years, with slightly more women than men, and the most common mechanism of injury (32%) was a fall from standing. Key Result: Bandaging was non-inferior to casting in terms of functional outcome at 3 months. No non-unions were reported. Primary Outcome: QDASH score at 3 months: Adjusted estimated difference = 0.30 (95% CI: 0.02–0.62), confirming noninferiority. Secondary Outcomes: Pain and function scores: No statistical differences ROM at 2 weeks: Better in the bandaging group Satisfaction: Higher in the bandaging group No increase in complications 1. Performance Bias: Neither patients nor clinicians were masked. This can introduce performance bias. This occurs when there are differences in the care provided to participants other than the intervention being studied, and those differences influence the outcomes. It often arises in trials where blinding is not possible, like in this trial. When participants and clinicians know which group a patient is in, they may unconsciously (or consciously) change how they deliver care, how closely they monitor patients, or how they report outcomes. 2. Detection Bias: This can occur when the assessment of outcomes is influenced by the knowledge of which intervention the participant received. When masking is not possible, it can lead to systematic differences in how outcomes are measured or interpreted between groups. The primary outcome in the trial was a patient-reported functional score (QDASH at 3 months), and secondary outcomes included additional subjective measures. These types of outcomes are especially vulnerable to detection bias when neither patients nor assessors are masked. If patients in the bandaging group expected to do better (less restrictive, more natural recovery), they may overreport functional improvement. If assessors expect stiffness or complications in the casting group, they may unconsciously look harder for dysfunction. Even small differences in how outcomes are measured or reported can make a noninferiority trial appear more favourable to the experimental intervention (in this case, bandaging). 3. Follow-up Bias: Patients in the bandaging group had earlier clinical contact (day 3 vs. day 14). This seems logical but from a methodological standpoint, it can introduce some concerns. Being seen sooner could enhance patient education, reassurance, or counselling. This could influence subjective outcomes like satisfaction, function, and perceived improvement. This difference in care intensity could favour the intervention group, not because the treatment is better, but because patients were monitored more closely, and early issues were managed proactively. It would have been great if both cohorts were seen on day 3 to mitigate this potential bias. 4. Non-Inferiority: The authors set the non-inferiority margin using a log-transformed threshold for the primary outcome measure. This was the QDASH score at 3 months with a natural logarithm of the margin of noninferiority = 2.0.  A margin of ln (2.0) corresponds roughly to a doubling of the QDASH score, or about 10–15 points, depending on the baseline values. This margin reflects the maximum difference between the bandaging and casting groups that would still be considered clinically acceptable—i.e., not worse enough to matter to patients. The choice of this margin is one of the most subjective and impactful parts of a non-inferiority trial. It defines the threshold for what we’re willing to accept as "not meaningfully worse." If it's set too wide, a truly worse treatment could be accepted. If it’s too narrow, useful alternatives could be rejected. The authors selected a non-inferiority margin of 7.5 points on the QuickDASH, based on 50% of the established MCID, which they cite from Vercelli et al. (2014) as 15.91 points. This is a reasonable and accepted method in non-inferiority trials; halving the MCID is a conservative way to set a threshold for "no meaningful harm." However, this MCID comes from patients undergoing physical therapy for upper-limb disorders,