SGEM Xtra: Are tPA Stroke Trials Really Fragile?

Date: August 18th, 2022 Reference: Munn et al. Fragility Index Meta-Analysis of Randomized Controlled Trials Shows Highly Robust Evidential Strength for Benefit of <3 Hour Intravenous Alteplase. Stroke 2022 Dr. Jeff Saver Guest: Dr. Jeff Saver is a Professor and SA Vice Chair for Clinical Research, Carol and James Collins Chair, Department of Neurology, Director of the UCLA Comprehensive Stroke and Vascular Neurology Program at the David Geffen School of Medicine at UCLA. This is an SGEM Xtra. Jeff and I have an interesting back story to how we met. I knew about Jeff from his multiple publications in the stroke literature. I did not know he knew about me until an EM physician sent me a video of a presentation that was given at an international stroke meeting. On one of the slides, Professor Daniel Fantovich and I were referred to as "Non-Expert EM Contrarians". I reached out to Jeff and we had a very good conversation. He clarified what he meant by “non-experts”: that we were not stroke neurologists or emergency physicians with subspecialty neuro expertise, such as having completed fellowship training in neurologic critical care. He did acknowledge that both Dr. Fatovich and I had expertise on critical appraisal of the medical literature. The conversation ended well with Jeff requesting one of the t-shirts I planned to make with the title of non-expert ER contrarian on the chest. Jeff recently reached out to me with his new publication called Fragility Index Meta-Analysis of Randomized Controlled Trials Shows Highly Robust Evidential Strength for Benefit of <3 Hour Intravenous Alteplase asking about my thoughts.  I thought this would be a great opportunity to dig deeper into the fragility index and have another expert in stroke neurology on the SGEM. Dr. Eddy Lang We have had a couple of individuals previously on the SGEM who strongly support the use of tPA in acute ischemic stroke (AIS). One was Dr. Eddy Lang who is a well-known Canadian researcher and emergency physician in Calgary, Alberta. Eddy appeared on the SGEM Xtra episode called the Walk of Life discussing AIS.  We had a debate on the issue of tPA for stroke published in CJEM 2020 as part of their debate series. Eddy is also the senior author on the CJEM article summarizing the Canadian Stroke Best Practice (CSBP) 2018 Guidelines. This Canadian guideline gives a level “A” recommendation for the use of tPA in AIS in patients last seen normal within 4.5 hours. “All eligible patients with disabling ischemic stroke should be offered intravenous alteplase (tPA). Eligible patients are those who can receive intravenous alteplase (tPA) within 4.5 hours” of symptom onset time or last seen normal (Evidence Level A; Section 5.3.i). We also had a neurology resident on to critically appraise a systematic review and meta-analysis of endovascular therapy plus/minus tPA as a bridging therapy (SGEM#349). A few more publications have come out since that podcast and the European Stroke Organization (ESO) recommends intravenous thrombolysis before mechanical thrombectomy in patients with acute ischemic stroke and anterior circulation large vessel occlusion. There have been several tPA skeptics on the SGEM including Dr. Hoffman, Dr. Fatovich, and Dr. Morgenstern. However, not until now have we had a stroke neurologist who is very much in support of using tPA in AIS. I think it is very important to try and mitigate against echo chambers, our own biases and listen carefully to other points views. Fragility Index Meta-Analysis of Randomized Controlled Trials Shows Highly Robust Evidential Strength for Benefit of <3 Hour Intravenous Alteplase. Jeff was asked a number of questions about his new publication. Some of the answers are listed as bullet points, but most of his responses can be heard in full by listening to the SGEM podcast: Who were your co-authors on this publication? Why did you decide to write this article? What is the fragility index (FI)? The FI is the minimum number of nonevents that when changed to events in one arm of an interventional trial or meta-analysis of trials converts the result to statistical nonsignificance. Lower FIs indicate greater fragility, higher FIs more robust results. This definition of the FI is slightly different than the one provided by Walsh et al JClinEpi 2014 because it did not mention SRMA and was only looking at RCTs. There are critics of the FI who say, among other things, it could be viewed as just restating the p-value in a different way (Dr. Ed Palmer). Medicine has this very low bar of p-value of 0.05 (95%) or two sigmas to get over to consider something “statistically significant”. In contrast, particle physics uses five sigma or 99.9999%, this is a p-value of 3×10-7, or about 1 in 3.5 million chance the data is at least as extreme as what they observed. A lot of ink has been spilled about the problems with p-values. Over 800 scientists call for the abandonment of “statistical significance”. What are your thoughts on the use/misuse of the p-value? Others have said let’s raise the bar by lowering the number we would consider statistically significant from 0.05 to 0.005 to be more certain and mitigate against things like p-hacking. Do you think we should change what is considered statistically significant by an order of magnitude to 0.005? Does the fragility index convey different information than the p-value of the test statistic? If not, how would your analysis using the cumulative FI change our confidence in the tPA evidence for acute ischemic stroke that we could not obtain from the gold standard SRMA of individual patient data like the 2014 Emberson et al publication? The FI is a summary statistic not unlike the Number Needed to Treat (NNT) with both strengths and weaknesses. A major strength of the FI is its simplicity, making complex research easier to understand. A weakness, however, is also simplicity, hiding the complexity of research, ignoring confidence intervals, and obscuring potential biases. Do you think the FI is a useful metric? Often people will criticize a trial because the FI is low. However, studies are generally powered for their primary outcome of efficacy. To be efficient, researchers estimate how many participants would be needed to observe the magnitude of effect to be statistically significant. This power calculation should be done a priori based upon the “delta” or difference between treatment and control cohorts. If a study is done correctly, most should give a result that clusters around a p-value of 0.05. Therefore, the study would be designed to have a low FI. It could be considered a circular argument to then criticize the study as being “fragile”. Another way to interpret a low FI is that the researchers did a great job estimating the number of participants necessary to answer their hypothesis. They could be congratulated for conducting a very efficient trial that was not overpowered which wastes time, resources and patients. The introduction of the fragility index paper starts by saying the era of performing randomized placebo control trials comparing tPA for AIS in < 3 hours for patients with small to medium level occlusions is over. This is because it’s the standard of care making it unethical to randomize patients to placebo. What evidence do you provide to support your position? ACEP updated their policy on stroke in 2015 with lead author Dr. Michael Brown and gave no Level “A” recommendations in their policy statement. < 3 hours: Level B recommendations. With a goal to improve functional outcomes, IV tPA should be offered and may be given to selected patients with acute ischemic stroke within 3 hours after symptom onset at institutions where systems are in place to safely administer the medication. The increased risk of symptomatic intracerebral hemorrhage (sICH) should be considered when deciding whether to administer IV tPA to patients with acute ischemic stroke. Do you have any ideas why the ACEP policy statement seems to differ from AHA, ESO and CBSP? The ethics of conducting a placebo-controlled tPA trial is an interesting question. Stroke neurologist Dr. Peter Appelros and colleagues wrote an editorial called: Ethical issues in stroke thrombolysis revisited. It was a follow-up to a bioethics paper written in 1997 by Furland and Kanoti. The original article identified five areas of concern. Appelros’ position is that the ethical issues raised over two decades ago have not been satisfactorily answered. Have you read that editorial and what are your thoughts? Standard of care is also an interesting topic. It is a legal term that has a specific definition. the reasonable degree of care a person should provide to another person, typically in a professional or medical setting.  SGEM#200 Standard of care is often discussed by  emergency physicians (Moffett and Moore WestJEM 2011). Standard of care does not necessarily mean the best care. There are many examples in the medical literature where the standard of care was not the best care. The classic story I’ve often told is about bloodletting (SGEM#200). Standard of care could be considered an argument from popularity, and I think it is better for us as scientists to look at the evidence.  Do you agree? Can you briefly describe the methods used for your fragility index study? How many studies did you find and how big was the included cohort? Using your definition of FI: The minimum number of nonevents that when changed to events in one arm of an interventional trial or meta-analysis of trials converts the result to statistical nonsignificance. In other words, the FI is the minimum number of patients who would need to have a different outcome to change the p value from <0.05 to >0.05. How many would be required to flip statistically significant to insignificant (or “positive” result to a “negative” result).