We investigate the carrier mobility in mono- and bi-layer graphene with a top HfO2 dielectric, as a function of the HfO2 film thickness and temperature. The results show that the carrier mobility decreases during the deposition of the first 2-4 nm of top dielectric and remains constant for thicker layers. The carrier mobility shows a relatively weak dependence on temperature indicating that phonon scattering does not play a dominant role in controlling the carrier mobility. The data strongly suggest that fixed charged impurities located in close proximity to the graphene are responsible for the mobility degradation.