The evolution of a driven quantum system is said to be adiabatic whenever the state of the system stays close to an instantaneous eigenstate of its time-dependent Hamiltonian. The celebrated quantum adiabatic theorem ensures that such pure state adiabaticity can be maintained with arbitrary accuracy, provided one chooses a small enough driving rate. Here, we extend the notion of quantum adiabaticity to closed quantum systems initially prepared at finite temperature. In this case adiabaticity implies that the (mixed) state of the system stays close to a quasi-Gibbs state diagonal in the basis of the instantaneous eigenstates of the Hamiltonian. We prove a sufficient condition for the finite temperature adiabaticity. Remarkably, it implies that the finite temperature adiabaticity can be more robust than the pure state adiabaticity, particularly in many-body systems. We present an example of a many-body system where, in the thermodynamic limit, the finite temperature adiabaticity is maintained, while the pure state adiabaticity breaks down.