Objective: The apolipoprotein E (APOE) ε4 allele is the most important genetic risk factor for late-onset Alzheimer’s disease. Many ε4 carriers, however, never develop Alzheimer’s disease. The purpose of this study is to characterize the variability in phenotypic expression of the ε4 allele, as measured by the longitudinal trajectory of cognitive test scores and MRI brain volumes, in cognitively intact elders. Method: Healthy older adults, ages 65–85, participated in a 5-year longitudinal study that included structural MRI and cognitive testing administered at baseline and at 1.5 and 5 years postenrollment. Participants included 22 ε4 noncarriers, 15 ε4 carriers who experienced a decline in cognition over the 5-year interval, and 11 ε4 carriers who remained cognitively stable. Results: No baseline cognitive or volumetric group differences were observed. Compared to noncarriers, declining ε4 carriers had significantly greater rates of atrophy in left (p = .001, Cohen’s d = .691) and right (p = .003, d = .622) cortical gray matter, left (p = .003, d = .625) and right (p = .020, d = .492) hippocampi, and greater expansion of the right inferior lateral ventricle (p < .001, d = .751) over 5 years. Conclusions: This study illustrates the variability in phenotypic expression of the ε4 allele related to neurodegeneration. Specifically, only those individuals who exhibited longitudinal declines in cognitive function experienced concomitant changes in brain volume. Future research is needed to better understand the biological and lifestyle factors that may influence the expression of the ε4 allele. (PsycINFO Database Record (c) 2018 APA, all rights reserved)