Trial Design is the biggest problem with Evidence Based Medicine

Trial design is the biggest problem with Evidence Based Medicine in the Intensive Care Unit. Paul Young wants to change that paradigm completely. He argues for research as we know it to change and to focus on clinical care with systemised and optimised treatments that reliably improves outcomes over time for all patients. Mortality measured at a particular time point (landmark mortality) is often regarded as the gold standard outcome for randomised controlled trials in Intensive Care Medicine. An important limitation of many Intensive Care Medicine trials is that they hypothesize large and potentially implausible reductions in absolute mortality. This is a major problem in trial design for two reasons. Firstly, it makes false negative trial results more likely. Secondly, the less plausible a postulated mortality reduction is the more likely that a statistically significant mortality difference will represent a false positive. This is because a p-value is defined as the probability of finding a result equal to or more extreme than that actually observed, under the assumption that the null hypothesis is true. This means that the greater the pre-trial chance or prior probability that the null hypothesis is correct, the lower the chance that a p-value below a particular significance threshold will represent a true positive. In Paul’s words, p-values suck! The biggest single problem with the current evidence base is that most hypotheses being tested have low prior probability. This leads to the two most likely results being 1) no difference or 2) a false positive. We need a new research paradigm to address this problem, particularly in relation to the fundamentals of Intensive Care Medicine. Paul argues for an approach based on the Bayesian approach, utilising big data sources about patients under current care and then randomising treatment in real time. Intensive Care therapy is fundamentally about providing supportive care, including airway support, oxygen therapy, ventilation therapy and haemodynamic support amongst others. These treatments can be uncertain. Under such conditions of uncertainty and idiosyncratic practice variation, treatment should be randomised. Randomised treatment is likely to be the best treatment is these situations due to inherent cognitive biases. Using this data, Paul stipulates that we will rapidly learn from every patient, ensuring improved outcomes for each subsequent patient. As knowledge grow, prior probability can be adjusted, skewing the randomisation process. This continues until the treatment has either proven effective, or been shown to be equivalent. Every patient contributes information that enhances the care of every subsequent patient. And, importantly, p-values no longer matter. Effective research is important. As Paul states, “Optimising ICU care is a priority for global public health.” By increasing the probability that patients will get the treatments that work even before we know what those treatments are the Intensive Care Unit can not only save money, but most importantly and critically, save lives. That is the way research should work.

For more head to: codachange.org/podcasts/