OBJECTIVE: The best practice tariff for hip and knee replacement in the English National Health Service (NHS) rewards providers based on improvements in patient-reported outcome measures (PROMs) collected before and after surgery. Providers only receive a bonus if at least 50% of their patients complete the preoperative questionnaire. We determined how many providers failed to meet this threshold prior to the policy introduction and assessed longitudinal stability of participation rates. DESIGN: Retrospective observational study using data from Hospital Episode Statistics and the national PROM programme from April 2009 to March 2012. We calculated participation rates based on either (a) all PROM records or (b) only those that could be linked to inpatient records; constructed confidence intervals around rates to account for sampling variation; applied precision weighting to allow for volume; and applied risk adjustment. SETTING: NHS hospitals and private providers in England. PARTICIPANTS: NHS patients undergoing elective unilateral hip and knee replacement surgery. MAIN OUTCOME MEASURES: Number of providers with participation rates statistically significantly below 50%. RESULTS: Crude rates identified many providers that failed to achieve the 50% threshold but there were substantially fewer after adjusting for uncertainty and precision. While important, risk adjustment required restricting the analysis to linked data. Year-on-year correlation between provider participation rates was moderate. CONCLUSIONS: Participation rates have improved over time and only a small number of providers now fall below the threshold, but administering preoperative questionnaires remains problematic in some providers. We recommend that participation rates are based on linked data and take into account sampling variation.