Osteoporosis, fracture, large-scale craniofacial defects and osteonecrosis are hot topics and are still underdiagnosed and undertreated in the clinic. It is urgent to understand the molecular mechanisms corresponding to the regulation of bone formation. CMTM3 (CKLF-like MARVEL transmembrane domain containing 3) connects the classic chemokine to the transmembrane 4 superfamily and plays an important role in intracellular vesicles transport, EGF receptor function maintenance and cancer development. However, its expression and function in bone remain unclear. In this paper, we found that the bone volume/total volume, trabecular number, trabecular thickness and bone surface area/bone volume of Cmtm3 KO mice increased significantly, and trabecular separation and trabecular pattern factor decreased in Cmtm3 KO mice compared with WT mice by microcomputed tomography. Moreover, the bone mineral content, bone mineral density, ultimate force and stiffness were also increased in Cmtm3 KO mice. Using in vitro analysis, we showed that CMTM3 expression decreases during the differentiation of hBMSCs to osteoblasts. Knockdown of CMTM3 promoted ALP and mineralization of hBMSCs and facilitated osteoblastic differentiation with increasing RUNX2 expression. However, overexpression of CMTM3 got the opposite results. These results proved that CMTM3 was essential for osteogenic differentiation. In addition, knockdown of CMTM3 enhanced p-Erk1/2, but had no significant effect on p-Akt or p-STAT3 in hBMSCs and MC3T3-E1 cells. Taken together, our results indicated that Erk1/2 and RUNX2 pathways mediated by CMTM3 were involved in the process of osteogenic differentiation, and CMTM3 might be a new potential target in the treatment of bone formation-related disease.