Functional neuroimaging data have pointed to the activation of a fronto-parietal network during calculation tasks, the activity of which is modulated by arithmetic operation and arithmetical competence. As the cortical brain regions of this network are distant, it is crucial to investigate the white matter connections between them and to examine how these connections are related to different arithmetic operations and individual differences in arithmetical competence. By using diffusion tensor imaging (DTI) tractography in eighteen 12-year-olds, we tested whether white matter pathways connecting these distant regions were related to children's arithmetical competence and how this association was modulated by operation. For each child, we delineated the three subcomponents of the arcuate fasciculus, a bundle of pathways linking frontal and temporo-parietal regions that are commonly active during calculation tasks. Fractional anisotropy in the left anterior portion of the arcuate fasciculus was positively correlated with addition and multiplication, but not with subtraction and division, suggesting a specific role of this left anterior segment in the solution of those problems that are expected to be solved with fact retrieval. The observed correlation was not explained by age, intelligence and working memory. Follow-up control analyses using different types of reading measures revealed that the observed correlation only disappeared when measures that draw heavily on phonological processing, such as non-word reading, were controlled for, suggesting that the association between the left arcuate fasciculus-anterior and addition/multiplication reflects the involvement of phonological processing. These results are the first to demonstrate that individual differences in fronto-parietal white matter are associated with arithmetical competence in typically developing children of a very narrow age range and indicate that this association is modulated by arithmetic operation.