We revisit ground states of spinor Bose-Einstein condensates with a Rashba spin-orbit coupling, and find that votices show up as a direct consequence of spontaneous symmetry breaking into a combined gauge, spin, and space rotation symmetry, which determines the vortex-core spin state at the rotating center. For the continuous combined symmetry, the total spin rotation about the rotating axis is restricted to \(2\pi\), whereas for the discrete combined symmetry, we further need 2F quantum numbers to characterize the total spin rotation for the spin-\(F\) system. For lattice phases we find that in the ground state the topological charge for each unit cell vanishes. However, we find two types of highly symmetric lattices with a nontrivial topological charge in the spin-\(\frac{1}{2}\) system based on the symmetry classification, and show that they are skyrmion crystals.