Hazards of inference: the active control investigation.

Abstract

A finding of no significant difference in seizure incidence and/or severity among groups of epileptic patients prospectively randomized to treatment with either a standard or experimental antiepileptic drug is often advanced as compelling proof of the latter's anticonvulsant efficacy. Regrettably, the finding of no difference does not, on its own at least, support such a conclusion. A finding of no difference between treatments has numerous explanations that have nothing whatsoever to do with the pharmacological actions of the experimental drug. To be clear, it is possible to argue that a pharmacological effect is the most plausible explanation for a finding of no difference in an active control study, but this argument is only supportable if it is based on additional information gathered from sources external to the study. In most circumstances, the essential external evidence is not available, and the interpretation of the null difference as a positive result turns entirely upon the truth or falsity of the sanguine assumptions made. There is a risk in inferring efficacy from the seemingly consistent results of a series of active control studies that fail to find differences. Despite the long-recognized deficiencies in the active control study design, it remains popular.