Body-in-the-loop optimization algorithms have the capability to automatically tune the parameters of robotic prostheses and exoskeletons to minimize the metabolic energy expenditure of the user. However, current body-in-the-loop algorithms rely on indirect calorimetry to obtain measurements of energy cost, which are noisy, sparsely sampled, time-delayed, and require wearing a respiratory mask. To improve these algorithms, the goal of this work is to predict a user's steady-state energy cost quickly and accurately using physiological signals obtained from portable, wearable sensors. In this paper, we quantified physiological signal salience to discover which signals, or groups of signals, have the best predictive capability when estimating metabolic energy cost. We collected data from 10 healthy individuals performing 6 activities (walking, incline walking, backward walking, running, cycling, and stair climbing) at various speeds or intensities. Subjects wore a suite of physiological sensors that measured breath frequency and volume, limb accelerations, lower limb EMG, heart rate, electrodermal activity, skin temperature, and oxygen saturation; indirect calorimetry was used to establish the 'ground truth' energy cost for each activity. Evaluating Pearson's correlation coefficients and single and multiple linear regression models with cross validation (leave-one- subject-out and leave-one- task-out), we found that 1) filtering the accelerations and EMG signals improved their predictive power, 2) global signals (e.g., heart rate, electrodermal activity) were more sensitive to unknown subjects than tasks, while local signals (e.g., accelerations) were more sensitive to unknown tasks than subjects, and 3) good predictive performance was obtained combining a small number of signals (4-5) from multiple sensor modalities. NEW & NOTEWORTHY In this paper, we systematically compare a large set of physiological signals collected from portable sensors and determine which sensor signals contain the most salient information for predicting steady-state metabolic energy cost, robust to unknown subjects or tasks. This information, together with the comprehensive data set that is published in conjunction with this paper, will enable researchers and clinicians across many fields to develop novel algorithms to predict energy cost from wearable sensors.