We provide an extensive study of the sub-ohmic spin-boson model with power law density of states J(\omega)=\omega^s (with 0<s<1), focusing on the equilibrium dynamics of the three possible spin components, from very weak dissipation to the quantum critical regime. Two complementary methods, the bosonic Numerical Renormalization Group (NRG) and Majorana diagrammatics, are used to explore the physical properties in a wide range of parameters. We show that the bosonic self-energy is the crucial ingredient for the description of critical fluctuations, but that many-body vertex corrections need to be incorporated as well in order to obtain quantitative agreement of the diagrammatics with the numerical simulations. Our results suggest that the out-of-equilibrium dynamics in dissipative models beyond the Bloch-Redfield regime should be reconsidered in the long-time limit. Regarding also the spin-boson Hamiltonian as a toy model of quantum criticality, some of the insights gained here may be relevant for field theories of electrons coupled to bosons in higher dimensions.