Background As the age of a population increases, so too does the rate of disability. In addition, disability is likely to be more common in rural compared with urban areas. The present study aimed to examine the influence of rapid population changes in terms of age and rural/urban residence on the prevalence of disability. Methods Data from the 1987 and 2006 China Sampling Surveys on Disability were used to estimate the impacts of rapid ageing and the widening urban-rural gap on the prevalence of disability. Stratum specific rates of disability were estimated by 5-year age-group and type of residence. The decomposition of rates method was used to calculate the rate difference for each stratum between the two surveys. Results The crude disability rate increased from 4.89% in 1987 to 6.39% in 2006, a 1.5% increase over the 19 year period. However, after the compositional effects from the overall rates of changing age-structure in 1987 and 2006 were eliminated by standardization, the disability rate in 1987 was 6.13%, which is higher than that in 2006 (5.91%). While in 1987 the excess due to rural residence compared with urban was 1.5% by 2006, suggesting a widening disparity by type of residence. When rates were decomposed, the bulk of the disability could be attributed to ageing, and very little to rural residence. However, a wider gap in prevalence between rural and urban areas could be observed in some age groups by 2006. Conclusion The increasing number of elderly disabled persons in China and the widening discrepancy of disability prevalence between urban and rural areas may indicate that the most important priorities for disability prevention in China are to reinforce health promotion in older adults and improve health services in rural communities.