Background: Health literacy (HL) is an important factor in improving health inequalities in poor and marginalized groups. Assessing comprehensive HL is critical. In this study, we validated the use of a comprehensive short-form HL survey tool (HLS-SF12) and examined the determinants of HL among people in rural areas. Methods: A cross-sectional study was conducted in July 2019 on 440 people residing in mountainous areas in Vietnam. Health literacy was measured using the HLS-SF12. Personal characteristics were also collected. We analyzed data using confirmatory factor analysis, internal consistency analysis, and regression analysis. Results: The questionnaire demonstrated a good construct validity with satisfactory goodness-of-fit indices and item-scale convergent validity. The tool was reliable and homogeneous with Cronbach’s alpha = 0.79, with no floor/ceiling effects. People who were married had lower HL (regression coefficient B = −3.12; 95% confidence interval (CI) = −5.69, −0.56; p = 0.017) compared with those who never married. Higher education attainment (B = 3.41 to 10.44; p < 0.001), a better ability to pay for medication (B = 4.17 to 9.89; p < 0.001), and a tendency to view health-related TV/radio more often (B = 5.23 to 6.15; p < 0.001) were associated with higher HL. Conclusions: The HLS-SF12 is a valid survey tool for the evaluation of HL in rural populations. A number of personal characteristics were strongly associated with HL.