The moral force of impartiality (i.e. the equal treatment of all human beings) is imperative for providing justice and fairness. Yet, in reality many people become partial during intergroup interactions; they demonstrate a preferential treatment of ingroup members and a discriminatory treatment of outgroup members. Some people, however, do not show this intergroup bias. The underlying sources of these inter-individual differences are poorly understood. Here we demonstrate that the larger the gray matter volume and thickness of the dorsomedial prefrontal cortex (DMPFC), the more individuals in the role of an uninvolved third-party impartially punish outgroup and ingroup perpetrators. Moreover, we show evidence for a possible mechanism that explains the impact of DMPFC's gray matter volume on impartiality, namely perspective-taking. Large gray matter volume of DMPFC seems to facilitate equal perspective-taking of all sides, which in turn leads to impartial behavior. This is the first evidence demonstrating that brain structure of the DMPFC constitutes an important source underlying an individual's propensity for impartiality. Copyright © 2013 Elsevier Inc. All rights reserved.