OBJECTIVE: Somatic symptoms unexplained by disease are common in all medical settings. The process of identifying such patients requires a clinical assessment often supported by clinical tests. Such assessments are time-consuming and expensive. Consequently the observation that such patients tend to report a greater number of symptom has led to the use of self-rated somatic symptom counts as a simpler and cheaper diagnostic aid and proxy measure for epidemiological surveys. However, despite their increasing popularity there is little evidence to support their validity. METHODS: We tested the score on a commonly used self-rated symptom questionnaire- the Patient Health Questionnaire (PHQ 15) (plus enhanced iterations including an additional 10 items on specific neurological symptoms and an additional 5 items on mental state) for diagnostic sensitivity and specificity against a medical assessment (with 18 months follow-up) in a prospective cohort study of 3781 newly attending patients at neurology clinics in Scotland, UK. RESULTS: We found 1144/3781 new outpatients had symptoms that were unexplained by disease. The patients with symptoms unexplained by disease reported higher symptoms count scores (PHQ 15: 5.6 (95% CI 5.4 to 5.8) vs 4.2 (4.1 to 4.4) p<0.0001). However, the PHQ15 performed little better than chance in its ability to identify patients with symptoms unexplained by disease. The findings with the enhanced scales were similar. CONCLUSIONS: Self-rated symptom count scores should not be used to identify patients with symptoms unexplained by disease.