In this article, we explore the form of evaluation put forward by guidelines used in the health sciences for appraising qualitative research and we begin to articulate an alternative posture. Most guidelines are derivative of the modes of assessment developed by clinical epidemiologists as part of the promotion of evidence-based medicine (EBM). They are predominantly proceduralist in orientation, equating quality with the proper execution of research techniques. We argue that this form of judgment assumes a fixed relationship between research practice and knowledge generated, and tends to over-simplify and standardize the complex and non-formulaic nature of qualitative inquiry. A concern with methods as objects of judgment in and of themselves restricts the reader's field of vision to the research process and diverts attention away from the analytic content of the research. We propose an alternative 'substantive' perspective that focuses on the analysis put forward, and regards methods as resources for engaging with and understanding the substantive findings and topic of inquiry. An important challenge is to find a way to embody such a form of judgment in practical assessment tools.