Next-generation sequencing (NGS) has become a powerful tool in phylogenetic and evolutionary studies. Here we applied NGS to recover two ribosomal RNA genes (18S and 28S) from 16 species and 15 mitochondrial genomes from 16 species of scale worms representing six families in the suborder Aphroditiformia (Phyllodocida, Annelida), a complex group of polychaetes characterized by the presence of dorsal elytra or scales. The phylogenetic relationship of the several groups of scale worms remains unresolved due to insufficient taxon sampling and low resolution of individual gene markers. Phylogenetic tree topology based on mitochondrial genomes is comparable with that based on concatenated sequences from two mitochondrial genes (cox1 and 16S) and two ribosomal genes (18S and 28S) genes, but has higher statistical support for several clades. Our analyses show that Aphroditiformia is monophyletic, indicating the presence of elytra is an apomorphic trait. Eulepethidae and Aphroditidae together form the sister group to all other families in this suborder, whereas Acoetidae is sister to Iphionidae. Polynoidae is monophyletic, but within this family the deep-sea subfamilies Branchinotogluminae and Macellicephalinae are paraphyletic. Mitochondrial genomes in most scale-worm families have a conserved gene order, but within Polynoidae there are two novel arrangement patterns in the deep-sea clade. Mitochondrial protein-coding genes in polynoids as a whole have evolved under strong purifying selection, but substitution rates in deep-sea species are much higher than those in shallow-water species, indicating that purifying selection is relaxed in deep-sea polynoids. There are positive selected amino acids for some mitochondrial genes of the deep-sea clade, indicating they may involve in the adaption of deep-sea polynoids. Overall, our study (1) provided more evidence for reconstruction of the phylogeny of Aphroditiformia, (2) provided evidence to refute the assumption that mitochondrial gene order in Errantia is conserved, and (3) indicated that the deep-sea extreme environment may have affected the mitochondrial genome evolution rate and gene order arrangement in Polynoidae.