The two-dimensional SU(N) quantum antiferromagnet, a generalization of the quantum Heisenberg model, is investigated by quantum Monte Carlo simulations. The ground state for \(N\le 4\) is found to be of the N\'eel type with broken SU(N) symmetry, whereas it is of the Spin-Peierls type for \(N\ge 5\) with broken lattice translational invariance. No intermediate spin-liquid phase was observed in contrast to previous numerical simulations on smaller lattices [Santoro et al., Phys. Rev. Lett. {\bf 83} 3065 (1999)].