In this short note, we construct Rota-Baxter algebras by idempotent-like elements and show that every finite dimensional Hopf algebra admits nontrivial Rota-Baxter algebra structures and tridendriform algebra structures. Several concrete examples are provided, including finite quantum groups.