Many oncologists believe that patients with cancer who enroll in clinical trials have better outcomes than those who do not enroll. We aimed to assess the empirical evidence that such a trial effect exists. We developed a conceptual framework for comparison of trial and non-trial patients. We then did a comprehensive literature search to identify studies that compared outcomes between these groups. We critically evaluated these studies to assess whether they provide valid and generalizable support for a trial effect. We identified 26 comparisons, from 24 published articles, of outcomes among cancer patients enrolled and not enrolled in clinical trials. 21 comparisons used retrospective cohort designs. 14 comparisons provided some evidence that patients enrolled in trials have improved outcomes. However, strategies to control for potential confounding factors were inconsistent and frequently inadequate. Only eight comparisons restricted non-trial patients to those meeting trial eligibility criteria. Of these, three noted better outcomes in trial patients than in non-trial patients. Children with cancer, patients with haematological malignant disease, and patients treated before 1986 were disproportionately represented in positive studies. Despite widespread belief that enrollment in clinical trials leads to improved outcomes in patients with cancer, there are insufficient data to conclude that such a trial effect exists. Until such data are available, patients with cancer should be encouraged to enroll in clinical trials on the basis of trials' unquestioned role in improving treatment for future patients.