The excellent electrical, optical and mechanical properties of graphene have driven the search to find methods for its large-scale production, but established procedures (such as mechanical exfoliation or chemical vapour deposition) are not ideal for the manufacture of processable graphene sheets. An alternative method is the reduction of graphene oxide, a material that shares the same atomically thin structural framework as graphene, but bears oxygen-containing functional groups. Here we use molecular dynamics simulations to study the atomistic structure of progressively reduced graphene oxide. The chemical changes of oxygen-containing functional groups on the annealing of graphene oxide are elucidated and the simulations reveal the formation of highly stable carbonyl and ether groups that hinder its complete reduction to graphene. The calculations are supported by infrared and X-ray photoelectron spectroscopy measurements. Finally, more effective reduction treatments to improve the reduction of graphene oxide are proposed.