The observation of the superfluid to Mott insulator phase transition of ultracold atoms in optical lattices was an enabling discovery in experimental many-body physics, providing the first tangible example of a quantum phase transition (one that occurs even at zero temperature) in an ultracold atomic gas. For a trapped gas, the spatially varying local chemical potential gives rise to multiple quantum phases within a single sample, complicating the interpretation of bulk measurements. Here we report spatially resolved, in-situ imaging of a two-dimensional ultracold atomic gas as it crosses the superfluid to Mott insulator transition, providing direct access to individual characteristics of the insulating, superfluid and normal phases. We present results for the local compressibility in all phases, observing a strong suppression in the insulator domain and suppressed density fluctuations for the Mott insulator, in accordance with the fluctuation-dissipation theorem. Furthermore, we obtain a direct measure of the finite temperature of the system. Taken together, these methods enable a complete characterization of multiple phases in a strongly correlated Bose gas, and of the interplay between quantum and thermal fluctuations in the quantum critical regime.