Item Response Theory (IRT) is being increasingly used to develop and evaluate outcome measures. However, many pain measures, including those that assess pain quality, have yet to be evaluated from the IRT perspective. The current study evaluated the scales of a commonly used measure of pain quality (the Pain Quality Assessment Scale, or PQAS) using IRT analyses in 3 samples of patients with chronic pain. The findings indicated variability in the precision of the scales, suggesting that all 3 of the PQAS scales are precise when pain is severe and that the Paroxysmal and Deep scales but not necessarily the Surface scale are precise when pain is of moderate or lower severity. In addition, 2 potential problems with the 11 (ie, 0 to 10) response levels used for the PQAS items were identified: (1) a high degree of overlap between adjacent response levels and (2) a lack of interval scaling. Research is needed to determine the extent to which these problems do, or do not, threaten the validity of the PQAS items and scales as outcome measures in pain clinical trials.