The ability to recognize a previously experienced stimulus is supported by two processes: recollection of the stimulus in the context of other information associated with the experience, and a sense of familiarity with the features of the stimulus. Although familiarity and recollection are functionally distinct, there is considerable debate about how these kinds of memory are supported by regions in the medial temporal lobes (MTL). Here, we review evidence for the distinction between recollection and familiarity and then consider the evidence regarding the neural mechanisms of these processes. Evidence from neuropsychological, neuroimaging, and neurophysiological studies of humans, monkeys, and rats indicates that different subregions of the MTL make distinct contributions to recollection and familiarity. The data suggest that the hippocampus is critical for recollection but not familiarity. The parahippocampal cortex also contributes to recollection, possibly via the representation and retrieval of contextual (especially spatial) information, whereas perirhinal cortex contributes to and is necessary for familiarity-based recognition. The findings are consistent with an anatomically guided hypothesis about the functional organization of the MTL and suggest mechanisms by which the anatomical components of the MTL interact to support the phenomenology of recollection and familiarity.