Recent theoretical studies have suggested that a magnetic field may play a crucial role in the first star formation in the universe. However, the influence of the magnetic field on the first star formation has yet to be understood well. In this study, we perform three-dimensional magnetohydrodynamic simulations taking into account all the relevant cooling processes and non-equilibrium chemical reactions up to the protostar density, in order to study the collapse of magnetized primordial gas cores with self-consistent thermal evolution. Our results show that the thermal evolution of the central core is hardly affected by a magnetic field because magnetic forces do not prevent the contraction along the field lines. We also find that the magnetic braking extracts the angular momentum from the core and suppresses fragmentation depending on the initial strength of the magnetic field. The angular momentum transport by the magnetic outflows is less effective than that by the magnetic braking because the outflows are launched only in a late phase of the collapse. Our results indicate that the magnetic effects become important for the field strength $B\gt 10^{-8}(n_{\rm H}/1\ \rm cm^{-3})^{2/3}\ \rm G$, where nH is the number density, during the collapse phase. Finally, we compare our results with simulations using a barotropic approximation and confirm that this approximation is reasonable at least for the collapse phase. Nevertheless, self-consistent treatment of the thermal and chemical processes is essential for extending simulations to the accretion phase, in which radiative feedback by protostars plays a crucial role.