SGEM#412: I Can’t Choose…from all the Head Injury Prediction Rules

Reference: Easter JS et al. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Annals of Emergency Medicine 2014. Date: July 10, 2023 Guest Skeptic: Dr. Joe Mullally is a paediatric trainee in the Welsh paediatric training program and interested in Paediatric Emergency Medicine. He is a student in the Paediatric Emergency Medicine Masters Program through Queen Mary University in London in collaboration with the Don’t Forget the Bubbles team. Dr. Joe Mullally Background: Children have big heads proportionally to their body compared to adults which makes them more at risk of traumatic brain injury (TBI). Computerized tomography (CT) is commonly used in the emergency department in the diagnosis of TBI. But we’re always trying to balance the potential harms and potential benefits in medicine. A CT scan does mean radiation to the pediatric brain which can increase the risk of leukemia or brain cancer later [1]. Thankfully, clinically important intracranial injuries are rare in children [2]. So, should we CT scan children with minor head injury? The SGEM covered pediatric concussions and head imaging in SGEM #112 and the NEXUS II Pediatric Head CT Decision Instrument in SGEM #225. Today we’re talking about three other popular clinical decision rules (PECARN, CATCH, and CHALICE). But we also want to know, how do those rules compare to physician judgement? Clinical Question: What is the diagnostic accuracy of clinical decision rules and physician judgment in identifying clinically important traumatic brain injuries (TBI) in children with minor head injury? Reference: Easter JS et al. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Annals of Emergency Medicine 2014. Population: Children less than 18 years of age presenting with head injury to a level 2 pediatric trauma center in the United States between 2012-2013. These children have to have 1) history of signs of blunt injury to the head 2) GCS scores ≥13, 3) injury within the previous 24 hours prior to presentation, 4) physician concern for potential TBI Excluded: Heightened TBI risk (GCS<13, brain tumors, ventricular shunts, on anticoagulants, or had bleeding disorders), or if they presented >24 hours after injury Intervention: CT vs no CT Comparison: Comparison of PECARN, CHALICE, CATCH, physician judgement, and physician practice Outcomes: Primary Outcome: “Clinically important TBI” defined as death from TBI, need for neurosurgery, need for intubation >24hrs for TBI, or hospital admission >2 nights for TBI. Secondary Outcomes: TBI on scan TBI requiring neurosurgery (craniotomy, elevation of skull fracture, monitoring of intracranial pressure, or intubation for elevated intracranial pressure) Type of Study: Single center prospective cohort study Authors’ Conclusions: “Of the 5 modalities described (PECARN, CATCH, CHALICE, physician judgment and physician practice), only physician practice and PECARN identified all clinically important TBIs, with PECARN being slightly more specific. CHALICE was incompletely sensitive but the most specific of all rules. CATCH was incompletely sensitive and had the poorest specificity of all modalities.” Quality Checklist for Observational Study:   Did the study address a clearly focused issue? Yes Did the authors use an appropriate method to answer their question? Yes Was the cohort recruited in an acceptable way? Yes Was the exposure accurately measured to minimize bias? Unsure  Was the outcome accurately measured to minimize bias? Unsure  Have the authors identified all-important confounding factors? Yes Was the follow up of subjects complete enough? Yes How precise are the results? Fairly precise Do you believe the results? Yes Can the results be applied to the local population? Unsure  Do the results of this study fit with other available evidence? Yes Funding of the Study. No financial conflicts of interest. Results: During the study period, 1,526 children with head injury presented to the ED, and they enrolled 1,062 (70%). The enrolled group had a median age of 6.1 years and 64% were male with 95% presenting with GCS of 15. In comparison the non-enrolled group had a median age of 5 years, 58% male, and 99% had GCS of 15. In summary, the enrolled group was a little bit older, with slightly higher proportion being male, and a bit fewer having GCS of 15. They ended up excluded another 53 mostly because they presented >24 hours from time of injury ending up with a total of 1,009 children included in the study. Key Results: Only PECARN clinical decision rule and physician practice had 100% sensitivity for detecting clinically important TBI. Fifty-two (5%) had injuries seen on CT scan. The most common were skull fractures but there were a few subarachnoid and subdural hemorrhages as well. Twenty-one (2%) had clinically important TBI. Only four (0.4%) required neurosurgical intervention. Selection Bias: This study had some inclusion criteria that may have led to selection bias. Specifically, the physician concern for potential TBI is very subjective. They also excluded anyone who was not seen within 24 hours with the reasoning that risk of clinically important TBI decreases with time. Although this may appear appropriate, we have probably encountered a child in the ED who presented greater than 24 hours from head injury. Maybe these families tried to wait it out after initial injury to avoid the ED visit, but the child continued to have persistent symptoms. What do you do if there may be physician concern for TBI in those circumstances? It’s unclear how these exclusion criteria impacted the study. Verification Bias: Verification bias occurs when only a proportion of the study participants receive confirmation of the diagnosis by the reference standard test (CT scan). Approximately half of patients were discharged after initial evaluation. A third of patients were observed for a median of three hours. The remainder underwent CT scan. Practically, we’re not arguing that every single patient in the study should have received a CT scan. The authors resorted to other methods to follow up with patients. It’s also important to realize that just because something may be radiologically significant, does not necessarily mean it is clinically significant. In this study, they noted 52 (5%) of patients who had injuries present on CT but only 21 (2%) had clinically significant TBI. Follow up obtained for 87% of patients who did not undergo CT. 57% of those were evaluated by physician and 43% were evaluated by phone. The team also looked at trauma registries and quality improvement reports to see if any patients had died. This is imperfect. Maybe the patient sought care at an institution outside of the health system they originally presented to. For more information on understanding the direction of bias in diagnostic studies we suggest you read the classic paper by Kohn et al 2013. There is also the new book by Dr. Jesse Pines on clinical decision rules and he was interviewed on an SGEM Xtra in June.  Inter-Rater Reliability: The authors looked at agreement amongst physicians on the study variables. They considered a kappa value of >0.5 to be acceptable. There are other sources that suggest a different interpretation: values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement [3]. Most of these variables and their associated kappa values are presented in Appendix 2. They looked at 180 observations total and most variables have good agreement. Worsening headache and intoxication have the lowest kappa’s of 0.49 and 0.43 respectively but there is no further discussion. GCS only had a kappa of 0.65. Previous studies show that agreement among physicians on GCS scores is quite variable [4]. In children, there are modifications to the GCS scoring based on age which poses additional challenges. This also potentially impacts the patients who were included in this study. Were there patients inappropriately excluded or inappropriately included if there was variation in how GCS was determined amongst physicians? Clinician Judgment: The authors report that physicians were trained on the data collection instrument. After assessing the patient but before obtaining results of testing, they recorded the presence of predictor variables. Is it possible that completing the data collection instrument with all the predictor variables influenced their decision because it reminded or cued them to what the predictor variable were significant? There was also discrepancy between physician estimate versus physician practice. Physicians’ estimate had sensitivity at 95% but physician practice had a sensitivity of 100% meaning they CT scanned more children compared to their estimate. We’re not sure why that is. It is also interesting to note that the specificity of physician estimate was better than physician practice. Clinical Decision “Rules”: I don’t really like the saying "clinical decision rule.” The “rule” part just feels so strict to me. We know that evidence-based medicine has three pillars, clinical judgment, scientific literature, and patient/family value and preferences. There is a lot of room for shared decision making here. How about we call these “tools” instead. They are there to help guide decision making but should not be taken as dogma. Our friend Justin Morgenstern from First10EM takes a stronger position in his blog post called: Clinical decision rules are ruining medicine  Comment on Authors’ Conclusion Compared to SGEM Conclusion: We agree with the authors’ conclusion. However,