E. BENOIT1, S. F. Monaghan1, T. Kheirbek1, A. H. Stephen1, S. N. Leuckel1, M. D. Connolly1, D. Heffernan1, C. A. Adams1, W. G. Cioffi1 1Brown University School Of Medicine,Division Of Trauma And Surgical Critical Care/Department Of Surgery,Providence, RI, USA
Introduction:
Randomized controlled trials (RCTs) remain the gold standard for determining the efficacy of novel therapies. Despite improvements in medical care over time, few RCTs have demonstrated a mortality benefit in trauma. Because trials may fail to reach significance for reasons other than ineffective interventions, we hypothesized that characteristics of RCT design, such as the estimated event rate in the control group & the predicted treatment effect, may influence the results of trauma studies.
Methods:
We reviewed PubMed, CENTRAL, and clinicaltrials.gov for all randomized controlled trials in adult trauma patients from 1988-2018. We included trials that used mortality as a primary or secondary endpoint. We excluded feasibility trials & those that focused on traumatic brain injury & burns. We collected data on study populations including average age, mechanism of injury & injury severity score (ISS). For those trials that reported sample size assessment, we examined the difference between predicted & observed values for control group mortality & treatment effect. For statistically significant trials, we calculated how many patients would need to switch groups for the results to lose significance (Fragility Index). We also evaluated early termination of trials.
Results:
We identified 31 RCTs, of which only 5 (16%) demonstrated a statistically significant mortality benefit. Of the 31 trials, 17 reported predicted control group mortality. Trials with an accurate control mortality (ratio of observed to predicted control mortality >80%) were more likely to achieve significance than trials that overestimated control mortality (4/6 vs 1/11, p=0.028). Of the 16 trials that reported estimated treatment effects, no trial had a greater treatment effect than predicted. The mean predicted treatment effect was 38.3% while the mean observed effect was 7.6%. Of the trials that demonstrated a mortality benefit, 3 of 5 had a Fragility Index ≤ 1. Early termination occurred in 12 (38.7%) trials, and half of these were due to insufficient events to determine a difference between treatment & control groups. Despite data spanning many years, we identified no trend towards decreased control group mortality over time; however there were differences among populations.
Conclusion:
Overestimation of control mortality rate & treatment effect contributes to underpowered trials that fail to reach significance, and this may unnecessarily exclude novel therapies. As the standard of care improves, demonstrating a mortality benefit becomes more challenging. Simply increasing sample size is insufficient. Patient selection based on mortality risk to enrich the control event rate coupled with conservative estimates of treatment effect may optimize a trial’s chance of demonstrating a significant result. In an era of evidence based medicine, we need better designed clinical trials to asess interventions that decrease the risk of death after trauma.