Genomics-assisted breeding methods have been rapidly developed with novel technologies such as next-generation sequencing, genomic selection and genome-wide association study. However, phenotyping is still time consuming and is a serious bottleneck in genomics-assisted breeding. In this study, we established a high-throughput phenotyping system for sorghum plant height and its response to nitrogen availability; this system relies on the use of unmanned aerial vehicle (UAV) remote sensing with either an RGB or near-infrared, green and blue (NIR-GB) camera. We evaluated the potential of remote sensing to provide phenotype training data in a genomic prediction model. UAV remote sensing with the NIR-GB camera and the 50th percentile of digital surface model, which is an indicator of height, performed well. The correlation coefficient between plant height measured by UAV remote sensing (PH UAV) and plant height measured with a ruler (PH R) was 0.523. Because PH UAV was overestimated (probably because of the presence of taller plants on adjacent plots), the correlation coefficient between PH UAV and PH R was increased to 0.678 by using one of the two replications (that with the lower PH UAV value). Genomic prediction modeling performed well under the low-fertilization condition, probably because PH UAV overestimation was smaller under this condition due to a lower plant height. The predicted values of PH UAV and PH R were highly correlated with each other ( r = 0.842). This result suggests that the genomic prediction models generated with PH UAV were almost identical and that the performance of UAV remote sensing was similar to that of traditional measurements in genomic prediction modeling. UAV remote sensing has a high potential to increase the throughput of phenotyping and decrease its cost. UAV remote sensing will be an important and indispensable tool for high-throughput genomics-assisted plant breeding.