OBJECTIVE: Gray matter (GM) atrophy occurs in all multiple sclerosis (MS) phenotypes. We investigated whether there is a spatiotemporal pattern of GM atrophy that is associated with faster disability accumulation in MS. METHODS: We analyzed 3,604 brain high-resolution T1-weighted magnetic resonance imaging scans from 1,417 participants: 1,214 MS patients (253 clinically isolated syndrome [CIS], 708 relapsing-remitting [RRMS], 128 secondary-progressive [SPMS], and 125 primary-progressive [PPMS]), over an average follow-up of 2.41 years (standard deviation [SD] = 1.97), and 203 healthy controls (HCs; average follow-up = 1.83 year; SD = 1.77), attending seven European centers. Disability was assessed with the Expanded Disability Status Scale (EDSS). We obtained volumes of the deep GM (DGM), temporal, frontal, parietal, occipital and cerebellar GM, brainstem, and cerebral white matter. Hierarchical mixed models assessed annual percentage rate of regional tissue loss and identified regional volumes associated with time-to-EDSS progression. RESULTS: SPMS showed the lowest baseline volumes of cortical GM and DGM. Of all baseline regional volumes, only that of the DGM predicted time-to-EDSS progression (hazard ratio = 0.73; 95% confidence interval, 0.65, 0.82; p < 0.001): for every standard deviation decrease in baseline DGM volume, the risk of presenting a shorter time to EDSS worsening during follow-up increased by 27%. Of all longitudinal measures, DGM showed the fastest annual rate of atrophy, which was faster in SPMS (-1.45%), PPMS (-1.66%), and RRMS (-1.34%) than CIS (-0.88%) and HCs (-0.94%; p < 0.01). The rate of temporal GM atrophy in SPMS (-1.21%) was significantly faster than RRMS (-0.76%), CIS (-0.75%), and HCs (-0.51%). Similarly, the rate of parietal GM atrophy in SPMS (-1.24-%) was faster than CIS (-0.63%) and HCs (-0.23%; all p values <0.05). Only the atrophy rate in DGM in patients was significantly associated with disability accumulation (beta = 0.04; p < 0.001). INTERPRETATION: This large, multicenter and longitudinal study shows that DGM volume loss drives disability accumulation in MS, and that temporal cortical GM shows accelerated atrophy in SPMS than RRMS. The difference in regional GM atrophy development between phenotypes needs to be taken into account when evaluating treatment effect of therapeutic interventions. Ann Neurol 2018;83:210-222.