OBJECTIVES: To evaluate the psychometric properties (and identify specific anomalies to be resolved) of urinary and sexual function scales of the Symptom Tracking and Reporting (STAR) instrument for use in clinical practice with individual men using Rasch analysis. DESIGN: Prospective cohort study. SETTING: 9 UK surgery centres in secondary care. PARTICIPANTS: 403 men diagnosed with prostate cancer and completed at least one questionnaire immediately before and at 1 or 3 months after radical prostatectomy. PRIMARY AND SECONDARY OUTCOMES: STAR instrument before surgery and 1 and 3 months afterwards. RESULTS: Neither scale fitted the Rasch model (both scales p<0.001). Both urinary (seven items) and sexual function (six items) had disordered thresholds, suggesting response categories are not working as intended. Both scales (three urinary items; five sexual function items) showed problems with item fit (large fit residuals, significant χ2, an inspection of item characteristic curves). Both scales showed items that were unstable over time (differential item functioning (DIF) by time). Both scales (four pairs of items in each scale) showed local response dependency (residual correlations >0.2 above the average). Internal consistency was acceptable at the group level for both scales. Targeting was poor for both scales, indicating an inadequate match between the location of items and the distribution of the patients, suggesting that the underlying constructs that the scales purport to measure are not clear. CONCLUSION: Using Rasch analysis as a diagnostic tool, we identified that both the urinary and the sexual function scales have issues that need to be resolved before STAR can be used with confidence in clinical practice. The sexual function scale, in particular, is unlikely to provide precise estimates for the outcomes experienced by men after radical prostatectomy. These results demonstrate the need to evaluate the suitability of any patient-reported outcome measure before implementation in routine clinical practice, preferably using modern psychometric methods.