The objective of this study was to investigate cross-sectional and longitudinal associations between declines in sexual activity and function and health outcomes in a large population-based sample of older adults. Data were from 2577 men and 3195 women aged ≥ 50 years participating in the English Longitudinal Study of Ageing. Past-year changes in sexual desire, frequency of sexual activity, and ability to have an erection (men)/become sexually aroused (women) were assessed at baseline by self-completion questionnaire. Health outcomes (self-rated health, limiting long-standing illness, doctor-diagnosed diseases of the vascular system, and cancer) were self-reported at baseline (2012/2013) and 4-year follow-up (2016/2017). Data were analyzed using logistic regression, adjusted for sociodemographics, health behaviors, and depressive symptoms. Prospectively, men who reported a decline in sexual desire had higher odds of incident limiting long-standing illness (OR 1.41, 95% CI 1.04–1.91) and incident cancer (OR 1.63, 95% CI 1.06–2.50) than those who maintained their sexual desire. Men who reported a decline in the frequency of sexual activities had higher odds of deterioration in self-rated health (OR 1.47, 95% CI 1.04–2.08) and incident limiting long-standing illness (OR 1.69, 95% CI 1.20–2.37). In women, a decline in frequency of sexual activities was associated with deterioration of self-rated health (OR 1.64, 95% CI 1.07–2.51). Erectile dysfunction was longitudinally associated with poorer health outcomes including incident cancer (OR 1.73, 95% CI 1.11–2.70), coronary heart disease (OR 2.29, 95% CI 1.29–4.07), and fair/poor self-rated health (OR 1.66, 95% CI 1.19–2.32). Practitioners should be mindful that a decline in sexual activity, desire, or function in older age may be an important indicator of future adverse health outcomes.