The aim of this work is to automate the interpretation of ultrasonic images during the non-destructive testing (NDT) technique called time-of-flight diffraction (TOFD) to aid in decision making. In this paper, the mathematical morphology approach is used to extract relevant pixels corresponding to the presence of a discontinuity, and a pattern recognition technique is used to characterize the discontinuity. The watershed technique is exploited to determine the region of interest and image background is removed using an erosion process, thereby improving the detection of connected shapes present in the image. Remaining shapes, are finally reduced to curves using a skeletonization technique. In the case of crack defects, the curve formed by such pixels has a parabolic form that can be automatically detected using the randomized Hough transform.