IMPORTANCE: Midlife vascular risk burden is associated with late-life dementia. Less is known about if and how risk exposure in early adulthood influences late-life brain health. OBJECTIVE: To determine the associations between vascular risk in early adulthood, midlife, and late life with late-life brain structure and pathology using measures of white matter-hyperintensity volume, β-amyloid load, and whole-brain and hippocampal volumes. DESIGN, SETTING, AND PARTICIPANTS: This prospective longitudinal cohort study, Insight 46, is part of the Medical Research Council National Survey of Health and Development, which commenced in 1946. Participants had vascular risk factors evaluated at ages 36 years (early adulthood), 53 years (midlife), and 69 years (early late life). Participants were assessed with multimodal magnetic resonance imaging and florbetapir-amyloid positron emission tomography scans between May 2015 and January 2018 at University College London. Participants with at least 1 available imaging measure, vascular risk measurements at 1 or more points, and no dementia were included in analyses. EXPOSURES: Office-based Framingham Heart study-cardiovascular risk scores (FHS-CVS) were derived at ages 36, 53, and 69 years using systolic blood pressure, antihypertensive medication usage, smoking, diabetic status, and body mass index. Analysis models adjusted for age at imaging, sex, APOE genotype, socioeconomic position, and, where appropriate, total intracranial volume. MAIN OUTCOMES AND MEASURES: White matter-hyperintensity volume was generated from T1/fluid-attenuated inversion recovery scans using an automated technique and whole-brain volume and hippocampal volume were generated from automated in-house pipelines; β-amyloid status was determined using a gray matter/eroded subcortical white matter standardized uptake value ratio threshold of 0.61. RESULTS: A total of 502 participants were assessed as part of Insight 46, and 463 participants (236 male [51.0%]) with at least 1 available imaging measure (mean [SD] age at imaging, 70.7 [0.7] years; 83 β-amyloid positive [18.2%]) who fulfilled eligibility criteria were included. Among them, FHS-CVS increased with age (36 years: median [interquartile range], 2.7% [1.5%-3.6%]; 53 years: 10.9% [6.7%-15.6%]; 69 years: 24.3% [14.9%-34.9%]). At all points, these scores were associated with smaller whole-brain volumes (36 years: β coefficient per 1% increase, -3.6 [95% CI, -7.0 to -0.3]; 53 years: -0.8 [95% CI, -1.5 to -0.08]; 69 years: -0.6 [95% CI, -1.1 to -0.2]) and higher white matter-hyperintensity volume (exponentiated coefficient: 36 years, 1.09 [95% CI, 1.01-1.18]; 53 years, 1.02 [95% CI, 1.00-1.04]; 69 years, 1.01 [95% CI, 1.00-1.02]), with largest effect sizes at age 36 years. At no point were FHS-CVS results associated with β-amyloid status. CONCLUSIONS AND RELEVANCE: Higher vascular risk is associated with smaller whole-brain volume and greater white matter-hyperintensity volume at age 69 to 71 years, with the strongest association seen with early adulthood vascular risk. There was no evidence that higher vascular risk influences amyloid deposition, at least up to age 71 years. Reducing vascular risk with appropriate interventions should be considered from early adulthood to maximize late-life brain health.