Multitarget drug discovery is one of the hottest topics and most active fields in the search for new molecules against Alzheimer’s disease (AD). Over the last 20 years, many promising multitarget-directed ligands (MTDLs) have been identified and developed at a pre-clinical level. However, how to design them in a rational way remains the most fundamental challenge of medicinal chemists. This is related to the foundational question of achieving an optimized activity towards multiple targets of interest, while preserving drug-like properties. In this respect, large hybrid molecules and small fragments are poles apart. In this review article, our aim is to appraise what we have accomplished in the development of both hybrid- and fragment-like molecules directed to diverse AD targets ( i.e., acetylcholinesterase, NMDA receptors, metal chelation, BACE-1 and GSK-3β). In addition, we attempt to highlight what are the persistent needs that deserve to be improved and cared for, with the ultimate goal of moving an MTDL to AD clinical studies.