Structural gender differences in bone mass - characterized by wider but not thicker bones - are generally attributed to opposing sex steroid actions in men and women. Recent findings have redefined the traditional concept of sex hormones as the main regulators of skeletal sexual dimorphism. GH-IGF1 action is likely to be the most important determinant of sex differences in bone mass. Estrogens limit periosteal bone expansion but stimulate endosteal bone apposition in females, whereas androgens stimulate radial bone expansion in males. Androgens not only act directly on bone through the androgen receptor (AR) but also activate estrogen receptor-α or -β (ERα or ERβ) following aromatization into estrogens. Both the AR and ERα pathways are needed to optimize radial cortical bone expansion, whereas AR signaling alone is the dominant pathway for normal male trabecular bone development. Estrogen/ERα-mediated effects in males may - at least partly - depend on interaction with IGF1. In addition, sex hormones and their receptors have an impact on the mechanical sensitivity of the growing skeleton. AR and ERβ signaling may limit the osteogenic response to loading in males and females respectively, while ERα may stimulate the response of bone to mechanical stimulation in the female skeleton. Overall, current evidence suggests that skeletal sexual dimorphism is not just the end result of differences in sex steroid secretion between the sexes, but depends on gender differences in GH-IGF1 and mechanical sensitivity to loading as well.