Many computer software developers and vendors claim that their systems can directly improve clinical decisions. As for other health care interventions, such claims should be based on careful trials that assess their effects on clinical performance and, preferably, patient outcomes. To systematically review controlled clinical trials assessing the effects of computer-based clinical decision support systems (CDSSs) on physician performance and patient outcomes. We updated earlier reviews covering 1974 to 1992 by searching the MEDLINE, EMBASE, INSPEC, SCISEARCH, and the Cochrane Library bibliographic databases from 1992 to March 1998. Reference lists and conference proceedings were reviewed and evaluators of CDSSs were contacted. Studies were included if they involved the use of a CDSS in a clinical setting by a health care practitioner and assessed the effects of the system prospectively with a concurrent control. The validity of each relevant study (scored from 0-10) was evaluated in duplicate. Data on setting, subjects, computer systems, and outcomes were abstracted and a power analysis was done on studies with negative findings. A total of 68 controlled trials met our criteria, 40 of which were published since 1992. Quality scores ranged from 2 to 10, with more recent trials rating higher (mean, 7.7) than earlier studies (mean, 6.4) (P<.001). Effects on physician performance were assessed in 65 studies and 43 found a benefit (66%). These included 9 of 15 studies on drug dosing systems, 1 of 5 studies on diagnostic aids, 14 of 19 preventive care systems, and 19 of 26 studies evaluating CDSSs for other medical care. Six of 14 studies assessing patient outcomes found a benefit. Of the remaining 8 studies, only 3 had a power of greater than 80% to detect a clinically important effect. Published studies of CDSSs are increasing rapidly, and their quality is improving. The CDSSs can enhance clinical performance for drug dosing, preventive care, and other aspects of medical care, but not convincingly for diagnosis. The effects of CDSSs on patient outcomes have been insufficiently studied.