Systematic reviews incorporating evidence from nonrandomized study designs: reasons for caution when estimating health effects.

Reeves, BC; van Binsbergen, J; van Weel, C; (2005) Systematic reviews incorporating evidence from nonrandomized study designs: reasons for caution when estimating health effects. European journal of clinical nutrition, 59 Suppl 1. S155-61. ISSN 0954-3007 DOI:

Full text not available from this repository.


Systematic reviews that include nonrandomized studies (NRS) face a number of logistical challenges. However, the greatest threat to the validity of such reviews arises from the differing susceptibility of randomized controlled trials (RCTs) and NRS to selection bias. Groups compared in NRS are unlikely to be balanced because of the reasons leading study participants to adopt different health behaviours or to be treated differentially. Researchers can try to minimize the susceptibility of NRS to selection bias both at the design stage, for example, by matching participants on key prognostic factors, and during data analysis, for example, by regression modelling. However, because of logistical difficulties in matching, imperfect knowledge about the relationships between prognostic factors and between prognostic factors and outcome, and measurement limitations, it is inevitable that estimates of effect size derived from NRS will be confounded to some extent. Researchers, reviewers and users of evidence alike need to be aware of the consequences of residual confounding. In poor quality RCTs, selection bias tends to favour the new treatment being evaluated. Selection bias need not necessarily lead to systematic bias in favour of one treatment but, even if it acts in an unpredictable way, it will still give rise to additional, nonstatistical uncertainty bias around the estimate of effect size. Systematic reviews of NRS studies run the risk of compounding these biases. Nutritional choices and uptake of health education about nutrition are very likely to be associated with potential confounding factors. Therefore, pooled estimates of the effects of nutritional exposures and their confidence intervals are likely to be misleading; reviewers need to take into account both systematic and uncertainty bias.

Item Type: Article
Faculty and Department: Faculty of Public Health and Policy > Dept of Health Services Research and Policy
PubMed ID: 16052184
Web of Science ID: 231545400041


Download activity - last 12 months
Downloads since deposit
Accesses by country - last 12 months
Accesses by referrer - last 12 months
Impact and interest
Additional statistics for this record are available via IRStats2

Actions (login required)

Edit Item Edit Item