SGEM#299: Learning to Test for COVID19

Date: August 18th, 2020 Guest Skeptic: Dr. Corey Heitz is an emergency physician in Roanoke, Virginia. He is also the CME editor for Academic Emergency Medicine. Reference: Carpenter et al. Diagnosing COVID-19 in the Emergency Department: A Scoping Review of Clinical Exam, Labs, Imaging Accuracy and Biases. AEM August 2020 Case: You are working in the emergency department during the COVID-19 outbreak, and you see a patient with oxygen saturations of 75% on room air, a fever, and a cough. Upon review of systems, you learn that she lost her sense of taste about two days ago. Your hospital performs COVID reverse transcriptase polymerase chain reaction (rt-PCR) nasal swabs on suspected patients, so you order this test and await the results.  Background: In early 2020, a pandemic broke out with origins thought to be in the Wuhan region of China. A novel coronavirus, SARS-Co-V-2, commonly called COVID-19, rapidly spread around the world, overwhelming hospitals and medical systems, causing significant morbidity and mortality. The speed with which the outbreak occurred made identification of cases difficult, as the disease exhibited a variety of symptoms, and testing lagged spread. The US Federal Drug Administration (FDA) allowed for emergency development and use of rt-PCR assays, and dozens of companies released assay kits. Mask4All Debate I consciously have tried to avoid contributing to the COVID-19 information overload. However, I did do a CAEP Town Hall on therapeutics (SGEM Xtra: Be Skeptical) with Dr. Sean Moore and a friendly debate on mandatory universal masking in public with Dr. Joe Vipond (SGEM Xtra: Masks4All). This review discusses the diagnostic accuracy of rt-PCR for COVID-19, as well as signs, symptoms, imaging, and other laboratory tests. Clinical Question: What is the diagnostic accuracy of history, clinical examination, routine labs, rt-PCR, immunology tests and imaging tests for the emergency department diagnosis for COVID19?  Reference: Carpenter et al. Diagnosing COVID-19 in the Emergency Department: A Scoping Review of Clinical Exam, Labs, Imaging Accuracy and Biases. AEM August 2020 Population: Original research studies describing the frequency of history, physical findings, or diagnostic accuracy of history/physical findings, lab test, or imaging tests for COVID-19 Intervention: None Comparison: None Outcome: Diagnostic accuracy (sensitivity, specificity, and likelihood ratios) Dr. Chris Carpenter This is an SGEMHOP episode which means we have the lead author on the show.  Dr. Chris Carpenter is Professor of Emergency Medicine at Washington University in St. Louis and a member of their Emergency Medicine Research Core. He is a member of the SAEM Board of Directors and the former Chair of the SAEM EBM Interest Group and ACEP Geriatric Section. He is Deputy Editor-in-Chief of Academic Emergency Medicine where he is leading the development of the "Guidelines for Reasonable and Appropriate Emergency Care" (GRACE) project. He is also Associate Editor of Annals of Internal Medicine’s ACP Journal Club and the Journal of the American Geriatrics Society, and he serves on the American College of Emergency Physician's (ACEP) Clinical Policy Committee. Dr. Carpenter also wrote the book on diagnostic testing and clinical decision rules. Authors’ Conclusions: “With the exception of fever and disorders of smell/taste, history and physical exam findings are unhelpful to distinguish COVID-19 from other infectious conditions that mimic SARS-CoV-2 like influenza. Routine labs are also non-diagnostic, although lymphopenia is a common finding and other abnormalities may predict severe disease. Although rRT-PCR is the current criterion standard, more inclusive consensus-based criteria will likely emerge because of the high false-negative rate of polymerase chain reaction tests. The role of serology and CT in ED assessments remains undefined.” Quality Checklist for Systematic Review Diagnostic Studies: The diagnostic question is clinically relevant with an established criterion standard. Yes/No The search for studies was detailed and exhaustive. No The methodological quality of primary studies were assessed for common forms of diagnostic research bias. Yes The assessment of studies were reproducible. Yes There was low heterogeneity for estimates of sensitivity or specificity. No 6. The summary diagnostic accuracy is sufficiently precise to improve upon existing clinical decision-making models. No Key Results: The authors screen 1,907 citations and 87 were included in the review. None adhere to the Standards for Reporting of Diagnostic Accuracy (STARD) or the updated reporting framework for history and physical examination. Rt-PCR was used as the criterion standard for many of the studies, but none explored the possibility of false negatives. 1) PRISMA-ScR (Scoping Review): What are the differences between  PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and the PRISMA-ScR guidelines? "PRISMA provides a reproducible reporting framework for systematic review and meta-analysis authors. Multiple PRISMA extensions exist (acupuncture, harms, health equity, network meta-analysis) and in 2018 PRISMA published “scoping review” reporting methods. A scoping review differs from a systematic review in that formal quality assessment of individual diagnostic studies with QUADAS-2 is not performed.  PRISMA-ScR still requires a reproducible search strategy and synthesis of research findings.  We selected a scoping review rather than a systematic review because we had limited time to find and synthesize the studies amidst our own institution’s COVID-19 chaos, yet we wanted to draw a line in the sand for diagnostic accuracy quality reporting because we were seeing the same research biases occurring repeatedly." 2) Search: Why did you decide to exclude non-English language studies? Would there not be a benefit to the experience out of other countries (especially China), even if not published in English-language journals? "This was simply for expediency because we lacked time to find/fund a translator. You will see from the articles that we the majority of the studies were from China. This was because it was early May and there was little experience or research published from Europe or US at that time.  As described in Figure 2, we did not exclude any studies for the purpose of language. This probably reflects a bias of our search engines (PubMed and EMBASE) for Asian language journals, as well as the fact that English is increasingly the universal language for scientific reporting." 3) STARD: Can you tell us more about the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines. None of the included studies adhered to the STARD guidelines. Why are these guidelines so important to follow? "Over two decades ago, journal editors and publishers convened to create mutually agreeable reporting standards that would transcend specialities beginning with the CONSORT criteria for randomized controlled trials.  These reporting standards continue to multiple (nearly 400 now!) and are warehoused at the EQUATOR Network.   Like PRISMA for systematic reviews, STARD is the EQUATOR Network reporting standard for diagnostic studies. Unfortunately, as demonstrated in our COVID-19 scoping review, uptake of these reporting standards has been slow in emergency medicine.  In 2017, Gallo et al reporting on behalf of the Best Evidence in Emergency Medicine (BEEM) team that ~80% of a randomly selected portion of diagnostic studies from eight EM journals report about half of STARD criteria (Gallo et al 2017).  Some elements of STARD that were commonly omitted included reporting the time interval between the index test and the criterion standard, the reproducibility of the index test, harms associated with the test, 2x2 contingency tables, and test performance variability across clinicians, labs, or test interpreters.  EQUATOR Network reporting standards like STARD are imperfect, but provide a minimal basement quality standard to ensure that diagnostic investigators evaluate essential features of their research design and that journal reviewers/editors analyze those elements of the study (Carpenter and Meisel AEM 2017) ." 4) Diagnostic Biases: A core papers resident and clinicians should be familiar with is the one on various diagnostic biases (Kohn et al AEM 2013).  Let’s go through some of the common diagnostic biases and how they can impact results and specifically COVID19 testing? Spectrum Bias Spectrum Bias (Effect): Sensitivity depends on the spectrum of disease, while specificity depends on the spectrum of non-disease. So, you can falsely raise sensitivity if the clinical practice has lots of very sick people. Specificity can look great if you have no sick patients in the cohort (the worried well). How could spectrum bias impact COVID19 testing? "This is difficult to ascertain using the data provided in the research reporting of the early COVID-19 era. Investigators rarely reported distribution of disease severity (% ICU admissions, APACHE-2 scores) or baseline risk profile (frailty score, comorbid illness score) in COVID-19 positive patients nor the distribution of alternative diagnoses in COVID-19 negative patients.  Washington University is participating in a study that includes fifty emergency departments  across the United States to derive a PERC-like rule that identifies patients at low-risk of COVID-19 when testing is delayed or unavailable.  With the variability in COVID-19 prevalence compounded by fluctuating availability of criterion standard testing resources, we have noted a skew towards testing very low risk or no-risk patients, which will skew specificity upwards and leave sensitivity relatively unaffected.  Future COVID-19 diagnostic investigators (whether evaluating history,