Dense mammographic parenchymal patterns are associated with an increased risk of breast cancer. Certain features of body size have been found to be associated with breast cancer risk, but less is known about their relation to breast density. We investigated the association of birth size, childhood growth and life-course changes in body size with Wolfe grade in 1298 perimenopausal women from a British cohort of women born in 1946. The cohort benefits from repeated measures of body size in childhood and adulthood. We obtained mammograms for 90% of women who at age 53 years reported having previously had a mammogram. We found no associations with birth weight or maximum attained height. Body mass index (BMI) at age 53 years and breast size were independently and inversely associated with Wolfe grade (P-value for trend <0.001 for both). Women who reached puberty later were at a greater odds of a higher Wolfe grade than women who had an earlier puberty (odds ratio associated with a 1 year delay in menarche 1.14, 95% CI: 1.01-1.27, adjusted for BMI and breast size at mammography). A higher BMI at any age during childhood or adult life was associated with a reduction in the odds of a higher Wolfe grade, after controlling for breast size and BMI at mammography, for example, standardised odds ratio for height at age 7 was 0.72 (95% CI: 0.64, 0.81). These findings reveal the importance of taking life-course changes in body size, and not just contemporaneous measures, into account when using mammographic density as an intermediate marker for risk of breast cancer.