BACKGROUND: Viewing Plasmodium in Romanovsky-stained blood has long been considered the gold standard for diagnosis and a cornerstone in management of the disease. This method however, requires a subjective evaluation by trained, experienced diagnosticians and establishing proficiency of diagnosis is fraught with many challenges. Reported here is an evaluation of a diagnostic system (a "device" consisting of a microscope, a scanner, and a computer algorithm) that evaluates scanned images of standard Giemsa-stained slides and reports species and parasitaemia. METHODS: The device was challenged with two independent tests: a 55 slide, expert slide reading test the composition of which has been published by the World Health Organization ("WHO55" test), and a second test in which slides were made from a sample of consenting subjects participating in a malaria incidence survey conducted in Equatorial Guinea (EGMIS test). These subjects' blood was tested by malaria RDT as well as having the blood smear diagnosis unequivocally determined by a worldwide panel of a minimum of six reference microscopists. Only slides with unequivocal microscopic diagnoses were used for the device challenge, n = 119. RESULTS: On the WHO55 test, the device scored a "Level 4" using the WHO published grading scheme. Broken down by more traditional analysis parameters this result was translated to 89% and 70% sensitivity and specificity, respectively. Species were correctly identified in 61% of the slides and the quantification of parasites fell within acceptable range of the validated parasitaemia in 10% of the cases. On the EGMIS test it scored 100% and 94% sensitivity/specificity, with 64% of the species correct and 45% of the parasitaemia within an acceptable range. A pooled analysis of the 174 slides used for both tests resulted in an overall 92% sensitivity and 90% specificity with 61% species and 19% quantifications correct. CONCLUSIONS: In its current manifestation, the device performs at a level comparable to that of many human slide readers. Because its use requires minimal additional equipment and it uses standard stained slides as starting material, its widespread adoption may eliminate the current uncertainty about the quality of microscopic diagnoses worldwide.