Audio long read: Medicine is plagued by untrustworthy clinical trials. How many studies are faked or flawed?
Aug 25, 2023
auto_awesome
The podcast discusses the prevalence of flawed or faked clinical trials in medicine and the potential dangers they pose. It explores specific examples of flawed trials in maternal health and the impact of fraudulent studies on clinical practice. The podcast also highlights the challenges of reproducibility and research integrity in medicine, emphasizing the need for improved oversight and checks to ensure trustworthiness in medical research.
A significant number of clinical trial studies in medical journals are plagued by flawed data or fabrication, raising concerns about the trustworthiness of medical research. An analysis conducted by John Carlyle, an editor at the Journal Anesthesia, revealed that 44% of the trials he reviewed contained some form of flawed data, while 26% were deemed impossible to trust due to widespread problems or fake data. These trials were referred to as 'zombie trials' because they appeared to be genuine, but further scrutiny exposed their lack of reliability. The issue extends to different medical fields, with some researchers estimating that up to a third of trials in certain areas are fraudulent or unreliable. The problem is further exacerbated when these flawed trials are included in meta-analyses and systematic reviews, leading to potentially erroneous conclusions influencing medical guidelines and clinical practices.
Examples of Fraudulent Trials and Impacts on Clinical Practice
Several examples showcase the impact of fraudulent trials on clinical practice and patient outcomes. Julio Cruz, a Brazilian neurosurgeon, published key trials on high doses of a sugary solution reducing death after head injuries, which were later retracted due to doubts about their authenticity. The work of Yoshihiro Sato, a Japanese bone health researcher who fabricated data in dozens of trials, resulted in widespread repercussions. For instance, 27 of Sato's retracted trials were cited in systematic reviews and clinical guidelines, potentially leading to ineffective treatments for osteoporosis. Another case highlights the influence of untrustworthy trials on decision-making. Trials into whether to administer steroids to people undergoing a caesarean section birth yielded conflicting results, with potential harm from steroids being downplayed due to flawed or fraudulent trials. These examples demonstrate the need for improved scrutiny and removal of unreliable trials from systematic reviews.
Addressing the Issue of Unreliable Trials and Ensuring Trustworthiness
The prevalence of flawed clinical trials necessitates better practices for ensuring trustworthiness. While multiple trustworthiness protocols exist, their adoption varies among medical journals. Some researchers advocate for routine request of raw individual participant data (IPD) in order to thoroughly assess the quality and authenticity of trials. However, journals typically do not require IPD sharing from authors. Implementing comprehensive trustworthiness checks, including scrutiny of data and statistics, funding sources, trial registration, and methodological plausibility, is a step towards addressing the problem. Reviewers should also be more diligent in identifying and excluding untrustworthy trials from systematic reviews, and journals should expedite the correction or retraction of articles when concerns are raised. These measures are crucial to improving the reliability of medical research and preventing the propagation of inaccurate or potentially harmful information.
Investigations suggest that, in some fields, at least one-quarter of clinical trials might be problematic or even entirely made up. Faked or unreliable trials are dangerous, as they could end up being included in the reviews that help inform clinical treatments. However, the extent of the problem in unclear, and many researchers urge stronger scrutiny.