AIMS: To compare the British Isles Lupus Assessment Group (BILAG) 2004, the Safety of Estrogens in Lupus Erythematosus National Assessment (SELENA) flare index (SFI) and physician's global assessment (PGA) in assessing flares of disease activity in patients with systemic lupus erythematosus (SLE). METHODS: Sixteen patients with active SLE were assessed by a panel of 16 rheumatologists. The order in which the patients were seen was randomised using a 4×4 Latin square design. Each patient's flare status was determined at each assessment using the BILAG 2004 activity index; the SFI and a PGA. A group of five specialists designated each patient into severe, moderate, mild or no flare categories. RESULTS: The rate of complete agreement (95% CI) of the four individual examining physicians for any flare versus no flare was 81% (55% to 94%), 75% (49% to 90%) and 75% (49% to 90%) for the BILAG 2004 index, SELENA flare instrument and PGA, respectively. The overall agreement between flare defined by BILAG 2004 and the SFI was 81% and when type of flare was considered was 52%. Intraclass correlation coefficients (95% CI), as a measure of internal reliability, were 0.54 (0.32 to 0.78) for BILAG 2004 flare compared with 0.21 (0.08 to 0.48) for SELENA flare and 0.18 (0.06 to 0.45) for PGA. Severe flare was associated with good agreement between the indices but mild/moderate flare was much less consistent. CONCLUSIONS: The assessment of flare in patients with SLE is challenging. No flare and severe flare are identifiable but further work is needed to optimise the accurate 'capture' of mild and moderate flares.