The relationships between energy use in food systems, food system productivity, and energy resource constraints are complex. Moreover, ongoing changes in food production and consumption norms concurrent with urbanization, globalization, and demographic changes underscore the importance of energy use in food systems as a food security concern. Here, we review the current state of knowledge with respect to the energy intensity of agriculture and food systems. We highlight key drivers and trends in food system energy use along with opportunities for and constraints on improved efficiencies. In particular, we point toward a current dearth of research with respect to the energy performance of food systems in developing countries and provide a cautionary note vis-à-vis increasing food system energy dependencies in the light of energy price volatility and concerns as to long-term fossil energy availabilities.