Dinoflagellates are a trophically diverse group of protists with photosynthetic and non-photosynthetic members that appears to incorporate and lose endosymbionts relatively easily. To trace the gain and loss of plastids in dinoflagellates, we have sequenced the nuclear small subunit rRNA gene of 28 photosynthetic and four non-photosynthetic species, and produced phylogenetic trees with a total of 81 dinoflagellate sequences. Patterns of plastid gain, loss, and replacement were plotted onto this phylogeny. With the exception of the apparently early-diverging Syndiniales and Noctilucales, all non-photosynthetic dinoflagellates are very likely to have had photosynthetic ancestors with peridinin-containing plastids. The same is true for all dinoflagellates with plastids other than the peridinin-containing plastid: their ancestors have replaced one type of plastid for another, in some cases most likely through a non-photosynthetic intermediate. Eight independent instances of plastid loss and three of replacement can be inferred from existing data, but as more non-photosynthetic lineages are characterized these numbers will surely grow.