What errors do peer reviewers detect, and does training improve their ability to detect them?


Schroter, S; Black, N; Evans, S; Godlee, F; Osorio, L; Smith, R; (2008) What errors do peer reviewers detect, and does training improve their ability to detect them? Journal of the Royal Society of Medicine, 101 (10). pp. 507-14. ISSN 0141-0768 DOI: https://doi.org/10.1258/jrsm.2008.080062

Full text not available from this repository. (Request a copy)

Abstract

OBJECTIVE: To analyse data from a trial and report the frequencies with which major and minor errors are detected at a general medical journal, the types of errors missed and the impact of training on error detection. DESIGN: 607 peer reviewers at the BMJ were randomized to two intervention groups receiving different types of training (face-to-face training or a self-taught package) and a control group. Each reviewer was sent the same three test papers over the study period, each of which had nine major and five minor methodological errors inserted. SETTING: BMJ peer reviewers. MAIN OUTCOME MEASURES: The quality of review, assessed using a validated instrument, and the number and type of errors detected before and after training. RESULTS: The number of major errors detected varied over the three papers. The interventions had small effects. At baseline (Paper 1) reviewers found an average of 2.58 of the nine major errors, with no notable difference between the groups. The mean number of errors reported was similar for the second and third papers, 2.71 and 3.0, respectively. Biased randomization was the error detected most frequently in all three papers, with over 60% of reviewers rejecting the papers identifying this error. Reviewers who did not reject the papers found fewer errors and the proportion finding biased randomization was less than 40% for each paper. Conclusions: Editors should not assume that reviewers will detect most major errors, particularly those concerned with the context of study. Short training packages have only a slight impact on improving error detection.

Item Type: Article
Faculty and Department: Faculty of Epidemiology and Population Health > Dept of Medical Statistics
Faculty of Public Health and Policy
Faculty of Public Health and Policy > Dept of Health Services Research and Policy
Research Centre: Centre for Global Non-Communicable Diseases (NCDs)
PubMed ID: 18840867
Web of Science ID: 260447800010
URI: http://researchonline.lshtm.ac.uk/id/eprint/6900

Statistics


Download activity - last 12 months
Downloads since deposit
0Downloads
299Hits
Accesses by country - last 12 months
Accesses by referrer - last 12 months
Impact and interest
Additional statistics for this record are available via IRStats2

Actions (login required)

Edit Item Edit Item