Navigating Network Meta-Analysis

Reference: Ye et al. Preoxygenation strategies before intubation in patients with acute hypoxic respiratory failure: a network meta-analysis. Frontiers in Medicine. 2025 Feb Date: June 12, 2025 Guest Skeptic:  Dr. Aine Yore is an Emergency Physician, practicing in the Seattle, Washington area for over twenty years. She is the former president of the Washington chapter of ACEP, and her career focus outside of clinical practice has been largely devoted to health care policy. Case: A 68-year-old woman presents in acute respiratory distress. She is febrile, hypoxemic, and meets criteria for sepsis. A chest x-ray reveals multilobar pneumonia. After managing her sepsis, her oxygenation remains poor, with saturations in the 88-92% range despite supplemental oxygen via a nonrebreather mask, and she now shows signs of worsening fatigue. You determine she requires endotracheal intubation, but note that she is at high risk for peri-intubation complications or even death, and wonder if there is a strategy you can utilize to reduce this risk? Background: Acute hypoxic respiratory failure (AHRF) represents a life-threatening emergency where pulmonary gas exchange becomes insufficient to maintain adequate oxygenation. It commonly arises from a variety of conditions, including pneumonia, acute respiratory distress syndrome (ARDS), sepsis, and exacerbations of chronic lung disease (ex, chronic obstructive lung disease). In such patients, intubation is often required, but the procedure itself introduces additional risk. Nearly 25% of patients undergoing emergency intubation in the context of AHRF experience profound desaturation (SpO₂ < 80%) during the procedure. Preoxygenation is a cornerstone of airway management, designed to extend the “safe apnea time” by denitrogenating the lungs and optimizing oxygen reservoirs. The aim is to minimize peri-intubation hypoxia, which is a known predictor of morbidity and mortality. Commonly used pre-oxygenation strategies include: Conventional oxygen therapy (COT), such as non-rebreather masks. High-flow nasal cannula (HFNC) provides warmed, humidified oxygen at high flow rates and can generate low levels of positive end-expiratory pressure (PEEP). Non-invasive ventilation (NIV) provides pressure support to enhance alveolar ventilation and decrease the work of breathing. Combinations of strategies like HFNC with NIV or bag-valve mask. Despite the widespread use of these techniques, clinical uncertainty persists regarding the most effective and safest strategy for preoxygenation in AHRF. This knowledge gap has led to multiple randomized controlled trials (RCTs) on the subject. Clinical Question: What is the optimal pre-oxygenation strategy in patients requiring intubation for acute hypoxic respiratory failure? Reference: Ye et al. Preoxygenation strategies before intubation in patients with acute hypoxic respiratory failure: a network meta-analysis. Frontiers in Medicine. 2025 Feb Population: Adults with AHRF defined as a respiratory rate >30/min, FiO₂ requirement ≥50% to maintain SpO₂ ≥90%, or PaO₂/FiO₂ < 300 mmHg within four hours of enrollment. Exclusions: Studies involving reviews, conference abstracts, case reports, or lacking full text. Intervention: Pre-oxygenation with Noninvasive Mechanical Ventilation, High Flow Oxygen via Nasal Cannula, or some combination of the above. Comparison: Conventional oxygen therapy (COT) or other preoxygenation. strategies. Outcome: There was no primary outcome explicitly stated. Outcomes included incidence of desaturation (SpO₂ < 80%) during intubation, lowest SpO2 during intubation, post-intubation complication rate, intensive care unit (ICU) length of stay (LOS) and ICU Mortality Type of Study: Network Meta-Analysis (NMA) Authors’ Conclusions: “Preoxygenation with HFNC appears to be the safest and most effective approach prior to intubation in patients with AHRF compared to other strategies”. Quality Checklist for Therapeutic Systematic Reviews: The clinical question is sensible and answerable. Yes The search for studies was detailed and exhaustive. Yes The primary studies were of high methodological quality. Yes The assessment of studies were reproducible. Unsure The outcomes were clinically relevant. No The treatment effect was large enough and precise enough to be clinically significant. Unsure Who funded the trial? Not stated Did the authors declare any conflicts of interest? No conflicts declared Results: Their search found 11 RCTs containing 2,874 patients with average ages ranging from mid-40s to 60s. Key Result: Outcomes: Incidence of Severe Hypoxia (SpO2 <80%): NIV>HFNC>COT, meaningful effect size Lowest SpO2 during intubation: HFNC+NIV>HFNC+COT>NIV>HFNC>COT, effect size not meaningful Post-intubation Complication Rate: HFNC>HFNC+COT>HFNC+NIV>NIV>COT, effect size not statistically significant ICU Length of Stay: HFNC>COT>NFNC+NIV>NIV, effect size not statistically significant ICU Mortality: HFNC>HFNC+NIV>HFNC+COT>NIV>COT, effect size not statistically significant 1. What Is A Network Meta-Analysis (NMA)? An NMA is an analytical method that allows for the comparison of multiple treatments simultaneously when some or all the treatments have never been directly compared in a head-to-head trial [1]. A key advantage of NMAs is that they can also rank treatments based on their effectiveness or safety. It provides outputs such as surface under the cumulative ranking (SUCRA) curves, which help identify the most effective or safest option [2]. Here’s a way to understand an NMA. Let’s say you want to compare four flavours of chewing gum: Cherry, Grape, Cheese, and Sewage. You have lots of market data comparing them, but, like the Highlander, there can be only one! But nobody has ever compared Cherry and Grape directly! And we need to prove which is best. An NMA can use a combination of direct and indirect evidence to compare them and determine the Ultimate Champion. Direct evidence would include the head-to-head comparisons. Cherry was a lot better than Sewage, of course. And Grape was marginally better than Cheese, and everything was better than Sewage, which is just objectively bad. The NMA will indirectly compare Cherry to Grape and can give you a reasonable sense of confidence as to which is better. 2. How Do You Critically Appraise An NMA? It is like the structured critical appraisal used to probe an SRMA for its validity.  There are quality checklists for NMA (PRISMA and CINeMA) [3,4]. What’s the PICO question? How exhaustive was the search? What was the quality of the included studies (Risk of bias assessment) Transitivity assumption? What statistical model was used (Bayesian/Frequentist), and what was the heterogeneity? How precise were the results? Was the effect size clinically relevant? Were there any COIs?  One thing specific to NMAs is transitivity, which is different than heterogeneity? Heterogeneity refers to statistical variability in results among studies comparing the same interventions. In contrast, transitivity is the idea that we can validly compare two treatments indirectly through a common comparator [5]. Heterogeneity is assessed, not globally, but within each treatment arm (direct comparisons). If there were three studies comparing Grape to Cheese flavoured gum, those studies themselves need to be similar. There are some highly quantitative statistical tools for this, and also some that are more vibe-based, as in this study. Additionally, an NMA requires internal cross-checking to ensure there is little inconsistency for the results to be valid. If Grape scored higher than Cheese flavour, and Cherry scored higher than Grape, yet Cheese scored higher than Cherry, the data is inconsistent, and an NMA may not be able to provide valid indirect evidence. The tests for inconsistency are also technical, and there are multiple methods of performing them. Assuming that your data is not too heterogeneous, and no inconsistencies are found, you can compute the rank order. This is done with a tool called SUCRA – Surface Under Cumulative Ranking Curve. This gives you a percentage of how likely a given flavour is to be the best. It’s normalized, so the percentages will not add up to 100%. In our Gum Challenge, Cherry might score 90% and Grape 75%, Cheese 10%, and Sewage 1%. What this tells you is that both Cherry and Grape are pretty good, but there’s a small margin between the two. First, you assess heterogeneity within each direct comparison. Then you consider whether the network appears transitive by comparing PICO elements. Finally, you check for inconsistency, which is the statistical signal that transitivity might not hold. 3. Effect Sizes vs Rank Order: One key advantage of NMAs is that they not only estimate effect size but are also able to rank the efficacy and/or safety of an intervention. Ranking of treatments can provide clinicians, guideline writers and policy-makers choices based on the probability of each intervention being the most effective or safest option. Ranking can also be a weakness of the NMA method if there is over-interpretation of the rank order. This may lead to a conclusion which is inaccurate. It doesn’t just matter that a given treatment was better than another, but by how much. Both Grape and Cheese flavours taste better than sewage, but let’s be honest, Cheese-flavoured gum is only a little better than sewage, and you may not capture that effect size if you only look at the rank orders. The study we are reviewing today looks at several different treatments that were not different from placebo (in this case, conventional oxygen therapy), yet it generated rank orders and included these results in its conclusions. And in the one outcome to truly have a meaningful statistical effect size,