SGEM#310: I Heard A Rumour – ER Docs are Not Great at the HINTS Exam

Date: November 30th, 2020 Reference: Ohle R et al. Can Emergency Physicians Accurately Rule Out a Central Cause of Vertigo Using the HINTS Examination? A Systematic Review and Meta-analysis. AEM 2020 Guest Skeptic: Dr. Mary McLean is an Assistant Program Director at St. John’s Riverside Hospital Emergency Medicine Residency in Yonkers, New York. She is the New York ACEP liaison for the Research and Education Committee and is a past ALL NYC EM Resident Education Fellow.  Case: A 50-year-old female presents to your community emergency department in the middle of the night with new-onset constant but mild vertigo and nausea. She has nystagmus but no other physical exam findings. You try meclizine, ondansetron, valium, and fluids, and nothing helps. Her head CT is negative (taken 3 hours after symptom onset). You’re about to call in your MRI tech from home, but then you remember reading that the HINTS exam is more sensitive than early MRI for diagnosis of posterior stroke. You wonder, “Why can’t I just rule out stroke with the HINTS exam? How hard can it be?” You perform the HINTS exam and the results are reassuring, but the patient’s symptoms persist…  Background: Up to 25% of patients presenting to the ED with acute vestibular syndrome (AVS) have a central cause of their vertigo - commonly posterior stroke. Posterior circulation strokes account for approximately up to 25% of all ischemic strokes [1]. MRI diffuse-weighted imagine (DWI) is only 77% sensitive for detecting posterior stroke when performed within 24h of symptom onset [2,3]. As an alternative diagnostic method, the HINTS exam was first established in 2009 to better differentiate central from peripheral causes of AVS [4]. But what is the HINTS exam? It’s a combination of three structured bedside assessments: the head impulse test of vestibulo-ocular reflex function, nystagmus characterization in various gaze positions, and the test of skew for ocular alignment. When used by neurologists and neuro-ophthalmologists with extensive training in these exam components, it has been found to be nearly 100% sensitive and over 90% specific for central causes of AVS [5-8]. Over the past decade, some emergency physicians have adopted this examination into their own bedside clinical assessment and documentation. We’ve used it to make decisions for our patients, particularly when MRI is not readily available. We’ve even used it to help decide whether or not to get a head CT. But we’ve done this without the extensive training undergone by neurologists and neuro-ophthalmologists, and without any evidence that the HINTS exam is diagnostically accurate in the hands of emergency physicians. Clinical Question: Can emergency physicians accurately rule out a central cause of vertigo using the HINTS examination? Reference: Ohle R et al. Can Emergency Physicians Accurately Rule Out a Central Cause of Vertigo Using the HINTS Examination? A Systematic Review and Meta-analysis. AEM 2020 Population: Adult patients presenting to an ED with AVS Exclusions: Non-peer-reviewed studies, unpublished data, retrospective studies, vertigo which stopped before or during workup, incomplete HINTS exam, or studies with data overlapping with another study used Intervention: HINTS examination by emergency physician, neurologist, or neuro-ophthalmologist Comparison: CT and/or MRI Outcome: Diagnosis of HINTS examination for central cause for AVS (i.e., posterior stroke) Authors’ Conclusions: “The HINTS examination, when used in isolation by emergency physicians, has not been shown to be sufficiently accurate to rule out a stroke in those presenting with AVS.” Quality Checklist for Systematic Review Diagnostic Studies:  The diagnostic question is clinically relevant with an established criterion standard. Unsure The search for studies was detailed and exhaustive. Yes The methodological quality of primary studies were assessed for common forms of diagnostic research bias. Yes The assessment of studies were reproducible. Yes There was low heterogeneity for estimates of sensitivity or specificity. No The summary diagnostic accuracy is sufficiently precise to improve upon existing clinical decision-making models. Unsure Key Results: They searched multiple electronic databases with no language or age restrictions and the gray literature. The authors identified 2,695 citations with five articles meeting inclusion criteria and a total of 617 patients. There were no studies that included only emergency physicians performing the HINTS examination. Essentially, the authors separated the studies into two cohorts according to the medical specialties of the HINTS examiners, and for each cohort they reported the sensitivity and specificity of the HINTS exam for diagnosis of posterior stroke. The first cohort included neurologists and neuro-ophthalmologists. The sensitivity and specificity of the HINTS examination were 96.7% (95% CI; 93.1 to 98.5) and 94.8% (95% CI; 91 to 97.1). In contrast, the second cohort (only one study) included emergency physicians and neurologists. The sensitivity and specificity were much lower at 83.3% (95% CI; 63.1 to 93.6) and 43.8% (95% CI; 36.7 to 51.2). From these results, it was deduced that emergency physicians’ participation in the latter cohort resulted in the reduced diagnostic accuracy. They did not combine the five studies into one summary result due to the heterogeneity of the included studies which was >40%.   1) Available Studies: Unfortunately, there were only five studies meeting the inclusion criteria, for a total of 617 patients. This is a known limitation of systematic reviews. Authors are limited by the available studies. 2) Biases: On the QUADAS-2 assessment, four of these studies had at least one component at high risk of bias, and three studies had unclear reports on at least one component, meaning that quality was low. Adherence to the STARD reporting guidelines was mediocre to poor overall because only two of the studies reported on most of the items in the guidelines. We will put the figure that represents the risk of bias of the included studies in the show notes. The reference standard (index test) used in these studies for all patients recruited was CT or MRI. We know CT imaging has a low sensitivity for posterior. One of the studies allowed negative head CT alone as adequate imaging to rule out posterior stroke. With such low sensitivity of CT imaging for posterior strokes, this crucial diagnosis can be missed. Even with MRI-DWI, only has a reported sensitivity of 77%. This problem in diagnostic testing studies is called the Imperfect Gold Standard Bias (Copper standard bias): It can happen if the “gold" standard is not that good of a test. Another bias identified was partial verification bias (referral or workup bias). This happens when only a certain set of patients who were suspected of having the condition are verified by the reference standard (CT or MRI). So, the AVS patients with suspected strokes with a positive HINTS exam were more likely to get advanced neuroimaging than those with a negative HINTS exam. This would increase sensitivity but decrease specificity. It is unknown if the original studies included consecutive patients or a convenience sample of patients. The later could introduce spectrum bias. Sensitivity can depend on the spectrum of disease, while specificity can depend on the spectrum of non-disease. Four out of the five studies had the ED physician identifying the patients for a referral. If patients with indeterminate or ambiguous presentations (rather than all patients presenting with AVS) were excluded this could falsely raise sensitivity. Because there were few studies it made assessing publication bias difficult. For those interested in understanding the direction of bias in studies of diagnostic test accuracy there is a fantastic article by Kohn et al AEM 2013. There is also a good book by Dr. Pines and colleagues on the topic  3) Heterogeneity: The authors used the I2 statistic to represent heterogeneity. Our overall I2 values are 53% for sensitivity and 94% for specificity likely representing moderate and considerable heterogeneity, respectively. Notably, for neurologists and neuro-ophthalmologist cohort alone, the I2 was noted to be 0, representing low or negligible heterogeneity [9]. 4) Precision and Reliability: There is poor precision overall - specifically for the cohort of emergency physicians with neurologists, the 95% confidence intervals were very wide for sensitivity (83%; 95% CI 63 to 94) and specificity (44%; 95% CI 37 to 51). The HINTS exam cannot yet be relied upon by emergency physicians as a bedside tool to rule out stroke. We simply do not have the evidence to support its adequacy as a diagnostic tool in the hands of emergency physicians, and in fact we may now have a “hint” of evidence to the contrary. I get it. We all got so excited in 2009 when we read about the HINTS exam and how well it worked for neurologists and neuro-ophthalmologists. The idea of it was spellbinding and almost hypnotic – a bedside test that was quick and free and more sensitive than early MRI. We all looked up YouTube videos on how to perform the exam, and we had to triple check how to interpret it after we got back to our desks. We dove into this too fast and too deep, before receiving structured training on this difficult exam that we thought was simple, and before learning exactly which kinds of patients it was appropriate for. We need to take a step back and be methodical, and what we really need is a large multi-center RCT on the diagnostic accuracy of the HINTS exam in the hands of emergency physicians. 5) Generalizability and Validity of Conclusions: The authors did not restrict their search to any particular language,