Neuroticism is a fundamental personality trait associated with proneness to feel negative affect. Here we ask how Neuroticism influences the neural response to positive and negative social interactions and how Neuroticism modulates the effect of intranasal oxytocin (OT) and vasopressin (AVP) on the neural response to social interactions. In a double-blind, placebo-controlled study, 153 male participants were randomized to receive 24 IU intranasal OT, 20 IU AVP or placebo. Afterwards, they were imaged with fMRI while playing an iterated Prisoner's Dilemma Game. On a different day, subjects completed the NEO personality inventory to measure Neuroticism. Neuroticism was positively correlated with the neural response to negative social interactions in the anterior cingulate cortex/medial prefrontal cortex and with the neural response to positive social interactions in the insula, indicating that Neuroticism modulates neuropsychological processing of both negative and positive social interactions. Neuroticism did not modulate the effect of intranasal OT treatment on the neural response to either positive or negative social interactions. On the other hand, AVP treatment significantly interacted with Neuroticism to modulate the BOLD response to both positive and negative social interactions. Specifically, AVP increased anterior cingulate cortex/medial prefrontal cortex and lateral temporal lobe responses to negative social interactions to a greater extent in participants scoring high rather than low on Neuroticism. AVP also increased the insula response to positive social interactions to a greater extent in participants scoring high rather than low on Neuroticism. These results imply that AVP may increase emotion regulation in response to negative social interactions and the salience of positive social interactions to a greater extent in individuals high compared to low in Neuroticism. The current findings urge caution against uniform clinical application of nonapeptides and suggest that their efficacy may vary as a function of personality.