Very preterm (VP; <32 weeks gestation) and/or very low birth weight (VLBW; <1500 g) birth has been associated with an increased risk of adverse motor and cognitive outcomes that may persist into adulthood. The aim of this study was to determine whether motor development in the first five years of life is associated with motor and cognitive outcomes in adulthood. A prospective observational study in Germany followed 260 VP/VLBW and 229 term-born individuals from birth into adulthood. Early motor trajectories (i.e., high and low degree of motor difficulties) were determined from neurological examinations from birth to 56 months. Adult motor and cognitive outcomes were determined from information from multiple instruments and IQ tests, respectively. Associations of VP/VLBW birth and early motor difficulties on adult outcomes were assessed using regression analyses. VP/VLBW individuals had an increased risk for early motor difficulties (Relative Risk: 11.77, 95% confidence interval (CI): 4.28, 32.35). Early motor difficulties were associated with poorer motor competence in adulthood ( β = 0.22, p < 0.001), independent of VP/VLBW birth. Adult IQ was predicted by VP/VLBW ( β = −0.12, p < 0.05) and child IQ ( β = 0.51, p < 0.001), while early motor difficulties ceased to be associated with adult IQ once participants with a neurological impairment were excluded ( β = 0.02, p > 0.05). Motor problems in childhood were homotypically associated with poorer motor competence in adulthood. Similarly, early cognitive problems were homotypically associated with adult cognitive outcomes. Thus, both motor and cognitive function should be assessed in routine follow-up during childhood.