Are large randomised controlled trials in severe sepsis and septic shock statistically disadvantaged by repeated inadvertent underestimates of required sample size?


Wong, JLC; Mason, AJ; Gordon, AC; Brett, SJ; (2018) Are large randomised controlled trials in severe sepsis and septic shock statistically disadvantaged by repeated inadvertent underestimates of required sample size? BMJ open, 8 (8). e020068. ISSN 2044-6055 DOI: https://doi.org/10.1136/bmjopen-2017-020068

[img]
Preview
Text - Published Version
License:

Download (772kB) | Preview

Abstract

We sought to understand why randomised controlled trials in septic shock have failed to demonstrate effectiveness in the face of improving overall outcomes for patients and seemingly promising results of early phase trials of interventions. We performed a retrospective analysis of large critical care trials of severe sepsis and septic shock. Data were collected from the primary trial manuscripts, prepublished statistical plans or by direct communication with corresponding authors. Critical care randomised control trials in severe sepsis and septic shock. 14 619 patients randomised in 13 trials published between 2005 and 2015, enrolling greater than 500 patients and powered to a primary outcome of mortality. Multiple interventions including the evaluation of treatment strategies and novel therapeutics. Our primary outcome measure was the difference between the anticipated and actual control arm mortality. Secondary analysis examined the actual effect size and the anticipated effect size employed in sample size calculation. In this post hoc analysis of 13 trials with 14 619 patients randomised, we highlight a global tendency to overestimate control arm mortality in estimating sample size (absolute difference 9.8%, 95% CI -14.7% to -5.0%, p<0.001). When we compared anticipated and actual effect size of a treatment, there was also a substantial overestimation in proposed values (absolute difference 7.4%, 95% CI -9.0% to -5.8%, p<0.0001). An interpretation of our results is that trials are consistently underpowered in the planning phase by employing erroneous variables to calculate a satisfactory sample size. Our analysis cannot establish if, given a larger sample size, a trial would have had a positive result. It is disappointing so many promising phase II results have not translated into durable phase III outcomes. It is possible that our current framework has biased us towards discounting potentially life-saving treatments.

Item Type: Article
Faculty and Department: Faculty of Public Health and Policy > Dept of Health Services Research and Policy
PubMed ID: 30158216
URI: http://researchonline.lshtm.ac.uk/id/eprint/4649119

Statistics


Download activity - last 12 months
Downloads since deposit
4Downloads
9Hits
Accesses by country - last 12 months
Accesses by referrer - last 12 months
Impact and interest
Additional statistics for this record are available via IRStats2

Actions (login required)

Edit Item Edit Item