The aim of ultrasonic non-destructive evaluation includes the detection and characterization of defects, and an understanding of the nature of defects is essential for the assessment of structural integrity in safety critical systems. In general, the defect characterization challenge involves an estimation of defect parameters from measured data. In this paper, we explore the extent to which defects can be characterized by their ultrasonic scattering behaviour. Given a number of ultrasonic measurements, we show that characterization information can be extracted by projecting the measurement onto a parametric manifold in principal component space. We show that this manifold represents the entirety of the characterization information available from far-field harmonic ultrasound. We seek to understand the nature of this information and hence provide definitive statements on the defect characterization performance that is, in principle, extractable from typical measurement scenarios. In experiments, the characterization problem of surface-breaking cracks and the more general problem of elliptical voids are studied, and a good agreement is achieved between the actual parameter values and the characterization results. The nature of the parametric manifold enables us to explain and quantify why some defects are relatively easy to characterize, whereas others are inherently challenging.