Arachis hypogaea is an allotetraploid species with low genetic variability. Its closest relatives, all of the genus Arachis, are important sources of alleles for peanut breeding. However, a better understanding of the genome constitution of the species and of the relationships among taxa is needed for the effective use of the secondary gene pool of Arachis. In the present work, we focused on all 11 non-A genome (or B genome sensu lato) species of Arachis recognized so far. Detailed karyotypes were developed by heterochromatin detection and mapping of the 5S and the 18S-25S rRNA using FISH. On the basis of outstanding differences observed in the karyotype structures, we propose segregating the non-A genome taxa into three genomes: B sensu stricto (s.s.), F and K. The B genome s.s. is deprived of centromeric heterochromatin and is homologous to one of the A. hypogaea complements. The other two genomes have centromeric bands on most of the chromosomes, but differ in the amount and distribution of heterochromatin. This organization is supported by previously published data on molecular markers, cross compatibility assays and bivalent formation at meiosis in interspecific hybrids. The geographic structure of the karyotype variability observed also reflects that each genome group may constitute lineages that have evolved through independent evolutionary pathways. In the present study, we confirmed that Arachis ipaensis was the most probable B genome donor for A. hypogaea, and we identified a group of other closely related species. The data provided here will facilitate the identification of the most suitable species for the development of prebreeding materials for further improvement of cultivated peanut.