Alzheimer’s disease (AD) clinical trials, focused on disease modifying drugs and conducted in patients with mild to moderate AD, as well as prodromal (early) AD, have failed to reach efficacy endpoints in improving cognitive function in most cases to date or have been terminated due to adverse events. Drugs that have reached clinical stage were reviewed using web resources (such as clinicaltrials.gov, alzforum.org, company press releases, and peer reviewed literature) to identify late stage (Phase II and Phase III) efficacy clinical trials and summarize reasons for their failure. For each drug, only the latest clinical trials and ongoing trials that aimed at improving cognitive function were included in the analysis. Here we highlight the potential reasons that have hindered clinical success, including clinical trial design and choice of outcome measures, heterogeneity of patient populations, difficulties in diagnosing and staging the disease, drug design, mechanism of action, and toxicity related to the long-term use. We review and suggest approaches for AD clinical trial design aimed at improving our ability to identify novel therapies for this devastating disease.