Few studies can address how adulthood cognitive trajectories relate to brain health in 70-year-olds. Participants (n = 468, 49% female) from the 1946 British birth cohort underwent 18F-Florbetapir PET/MRI. Cognitive function was measured in childhood (age 8 years) and across adulthood (ages 43, 53, 60-64 and 69 years) and was examined in relation to brain health markers of β-amyloid (Aβ) status, whole brain and hippocampal volume, and white matter hyperintensity volume (WMHV). Taking into account key contributors of adult cognitive decline including childhood cognition, those with greater Aβ and WMHV at age 70 years had greater decline in word-list learning memory in the preceding 26 years, particularly after age 60. In contrast, those with smaller whole brain and hippocampal volume at age 70 years had greater decline in processing search speed, subtly manifest from age 50 years. Subtle changes in memory and processing speed spanning 26 years of adulthood were associated with markers of brain health at 70 years of age, consistent with detectable prodromal cognitive effects in early older age.