Social memory plays a pivotal role in social behaviors, from mating behaviors to cooperative behaviors based on reciprocal altruism. More specifically, social/person recognition memory is supposed, by behavioral-economic and game-theoretic analysis, to be required for tit-for-tat like cooperative behaviors to evolve under the N-person iterated prisoner's dilemma game condition. Meanwhile, humans are known to show a social stress response during face-to-face social interactions, which might affect economic behaviors. Furthermore, it is known that there are individual differences in a social stress response, which might be reflected in individual differences in various types of economic behaviors, partially via different capacities of social memory. In the present study, we investigated the acute effects of social stress-induced free cortisol (a stress hormone) elevation on hippocampus-dependent social memory by utilizing the Trier social stress test (consisting of a public speech and a mental arithmetic task). We also examine the correlation between an economic behavior-related personality trait (i.e., general trust scale) and social stress-induced cortisol elevations. We found that (1) social stress acutely impairs social memory during social interaction and (2) interpersonal trust reduces social stress response. Together, interpersonal trust may modulate economic behaviors via stress hormone's action on social cognition-related brain regions.