Both caffeic acid and 3,4-dihydroxyphenyllactic acid (danshensu) are synthesized through two distinct routs of the shikimic acid biosynthesis pathway. In many plants, especially the rosemary and sage family of Lamiaceae, these two compounds are joined through an ester linkage to form rosmarinic acid (RA). A further structural diversity of RA derivatives in some plants such as Salvia miltiorrhiza Bunge is a form of RA dimer, salvianolic acid-B (SA-B), that further give rise to diverse salvianolic acid derivatives. This review provides a comprehensive perspective on the chemistry and pharmacology of these compounds related to their potential therapeutic applications to dementia. The two common causes of dementia, Alzheimer’s disease (AD) and stroke, are employed to scrutinize the effects of these compounds in vitro and in animal models of dementia. Key pharmacological mechanisms beyond the common antioxidant and anti-inflammatory effects of polyphenols are highlighted with emphasis given to amyloid beta (Aβ) pathologies among others and neuronal regeneration from stem cells.