Background: No randomized study exists comparing the effects of different modes of androgen substitution on bone mineral density (BMD). Methods: We performed a prospective, randomized, trial assigning 53 hypogonadal men to the following treatment groups: mesterolone 100 mg p.o. daily, testosterone undecanoate 160 mg p.o. daily, testosterone enanthate 250 mg i.m. every 21 days, or a single subcutaneous implantation of 1,200 mg crystalline testosterone. The BMD was determined by peripheral quantitative computed tomography. Results: At baseline, men with secondary hypogonadism (n = 33) had a lower BMD (–1.52 ± 0.23 SDS; Z-scores) than men with primary hypogonadism (n = 20, –0.87 ± 0.23 SDS, p < 0.01). In men with primary hypogonadism, the BMD increased dose dependently (crystalline testosterone +7.0 ± 1.3%, testosterone enanthate +4.8 ± 0.2%, testosterone undecanoate +3.4 ± 2.5%, mesterolone +0.8 ± 1.6%) after 6 months of therapy. Only secondary hypogonadal men treated with testosterone enanthate experienced an increase of the BMD. Conclusions: In primary hypogonadal men the BMD responds dose dependently to testosterone substitution, whereas in secondary hypogonadism only testosterone enanthate treatment significantly increased the BMD.