We recently reported that alpha-2 macroglobulin (A2M) is a biomarker of neuronal injury in Alzheimer's disease (AD) and identified a network of nine genes co-expressed with A2M in the brain. This network includes the gene encoding SPARCL1, a protein implicated in synaptic maintenance. Here, we examine whether SPARCL1 is associated with longitudinal changes in brain structure and function in older individuals at risk for AD in the Baltimore Longitudinal Study of Aging. Using data from the Gene-Tissue Expression Project, we first identified two single nucleotide polymorphisms (SNPs), rs9998212 and rs7695558, associated with lower brain SPARCL1 gene expression. We then analyzed longitudinal trajectories of cognitive performance in 591 participants who remained cognitively normal (average follow-up interval: 11.8 years) and 129 subjects who eventually developed MCI or AD (average follow-up interval: 9.4 years). Cognitively normal minor allele carriers of rs7695558 who developed incident AD showed accelerated memory loss prior to disease onset. Next, we compared longitudinal changes in brain volumes (MRI; n = 120 participants; follow-up = 6.4 years; 826 scans) and resting-state cerebral blood flow (rCBF; 15O-water PET; n = 81 participants; follow-up = 7.7 years; 664 scans) in cognitively normal participants. Cognitively normal minor allele carriers of rs9998212 showed accelerated atrophy in several global, lobar, and regional brain volumes. Minor allele carriers of both SNPs showed longitudinal changes in rCBF in several brain regions, including those vulnerable to AD pathology. Our findings suggest that SPARCL1 accelerates AD pathogenesis and thus link neuroinflammation with widespread changes in brain structure and function during aging.