Marijuana and hashish consist of at least 66 distinctive plant-derived (phyto-) cannabinoid compounds, with tetrahydrocannabinoids proving the most effective phytocannabinoid psychotropically. Despite the known pharmacological effects of phytocannabinoids, their role in controlling the cell survival/death decision in cells of the CNS continues to be unravelled. In this review, we examine the influence of phytocannabinoids on neural cell fate, with particular emphasis on how the time of marijuana exposure (neonatal vs. pubertal vs. adult) might influence the neurotoxic activities of phytocannabinoid compounds. Evidence in the literature indicates that exposure to phytocannabinoids during the prenatal period, in addition to the adolescent period, can alter the temporally ordered sequence of events that occur during neurotransmitter development, in addition to negatively impacting neural cell survival and maturation. Regarding the effect of marijuana consumption on brain composition in adults the evidence is contradictory. Exposure to marijuana during pregnancy might impact negatively on brain structure in the first years of postnatal life. Furthermore, early-onset (before age 17) marijuana use might also have damaging effects on brain composition. The neonatal and immature CNS is more susceptible to phytocannabinoid damage. In the adult CNS the data are conflicting and the continued development of methods to assess whether marijuana consumption results in brain atrophy or morphometric changes will determine if structural changes are occurring.