Research on the cognitive processing of facial attractiveness has mainly focused on adult faces. Recent studies have revealed that the cognitive processing of facial attractiveness in infant faces is not the same as that in adult faces. Therefore, it is necessary to summarize the evidence on the processing of facial attractiveness in each kind of face and compare their underlying mechanisms. In this paper, we first reviewed studies on the cognitive processing of facial attractiveness in adult faces, including attentional and mnemonic processing, and then discussed the underlying mechanisms. Afterward, studies on facial attractiveness in infant faces were reviewed, and the underlying mechanisms were also discussed. Direct comparisons between the two kinds of cognitive processing were subsequently made. The results showed that the mechanisms for the processing of attractiveness in adult faces and infant faces are mainly motivated by the perspectives of mate selection and raising offspring, respectively, in evolutionary psychology. Finally, directions for future research are proposed.