The counter-intuitive rotational motion of the propeller of a devil's stick when the agitator is rubbed against the pylon has long intrigued performers, audiences and scientists. The apparently unrelated phenomenon of self-stabilization of a Kapitza pendulum at the inverted position, once again an amazing lecture demonstration, has on the other hand been subjected to years of investigation and research. Here we show that the two systems are in fact identical, and obtain a previously unreported, friction-stabilized synchronously rotating state in the Kapitza pendulum, which in fact explains the rotational motion of the Devil's stick.