Alterations in basal cortisol secretion and feedback sensitivity are reported in aging. However, it is not known whether these hypothalamus-pituitary-adrenal (HPA) axis alterations are related to structural brain changes. This study was designed to investigate these relationships in the human. Nine young (24.0 ± 1.2 years; mean ± SE; range: 19–30) and 11 older (69.0 ± 1.8 years; range: 59–76) men, in addition to having standardized magnetic resonance imaging of their brains, were given 0.5 mg/kg cortisol or placebo intravenously in a double-blind, crossover study. As expected, older men had significantly smaller volumes for all brain regions. Although the groups did not differ in baseline HPA axis activity, there were significant and specific relationships between the brain volumes and the baseline measures of HPA activity. Namely, for young and older subjects combined and after controlling for age and cerebral vault size, hippocampal volumes were inversely associated with 24-hour urinary cortisol and basal corticotropin (ACTH) levels, and the anterior cingulate gyrus volume was negatively correlated with baseline ACTH. Elderly subjects had a slower decrease in ACTH levels (percent of baseline level) during the first 30 min after cortisol administration. However, no associations were observed between the ACTH feedback indices and any brain measure. This report, although based on a small number of subjects, supports previous studies showing a blunted ACTH fast feedback during normal aging. Hippocampal atrophy appears to be related to increased basal measures of HPA axis activity, but not to fast ACTH feedback. It remains possible that age-associated changes in fast feedback may be related to changes to other brain sites, such as hypothalamus or pituitary.