The Suitability of Potential Organ Donors Using Real Case-Scenarios; Do we Need to Create a “Donor Board” Process for Donors Perceived as Unlikely Suitable?

Background Rater agreement is important in clinical research, and Cohen’s Kappa is a widely used method for assessing inter-rater reliability; however, there are well documented statistical problems associated with the measure. In order to assess its utility, we evaluated it against Gwet’s AC1 and compared the results. Methods This study was carried out across 67 patients (56% males) aged 18 to 67, with a mean SD of 44.13 ± 12.68 years. Nine raters (7 psychiatrists, a psychiatry resident and a social worker) participated as interviewers, either for the first or the second interviews, which were held 4 to 6 weeks apart. The interviews were held in order to establish a personality disorder (PD) diagnosis using DSM-IV criteria. Cohen’s Kappa and Gwet’s AC1 were used and the level of agreement between raters was assessed in terms of a simple categorical diagnosis (i.e., the presence or absence of a disorder). Data were also compared with a previous analysis in order to evaluate the effects of trait prevalence. Results Gwet’s AC1 was shown to have higher inter-rater reliability coefficients for all the PD criteria, ranging from .752 to 1.000, whereas Cohen’s Kappa ranged from 0 to 1.00. Cohen’s Kappa values were high and close to the percentage of agreement when the prevalence was high, whereas Gwet’s AC1 values appeared not to change much with a change in prevalence, but remained close to the percentage of agreement. For example a Schizoid sample revealed a mean Cohen’s Kappa of .726 and a Gwet’s AC1of .853 , which fell within the different level of agreement according to criteria developed by Landis and Koch, and Altman and Fleiss. Conclusions Based on the different formulae used to calculate the level of chance-corrected agreement, Gwet’s AC1 was shown to provide a more stable inter-rater reliability coefficient than Cohen’s Kappa. It was also found to be less affected by prevalence and marginal probability than that of Cohen’s Kappa, and therefore should be considered for use with inter-rater reliability analysis.

Clinical vignette-based surveys have been used for more than 30 years to measure variation in physicians' approaches to the diagnosis and treatment of patients with similar health problems. Vignettes offer advantages over medical record reviews, analysis of claims data, and standardized patients. A vignette-based survey can be completed more quickly than a record review or standardized patient program. Research has shown that vignette-based surveys produce better measures of quality of care than medical record reviews when used to measure differential diagnosis, selection of tests, and treatment decisions. Although standardized patients are preferred when measuring communication and physical examination skills, vignettes are more cost-effective than standardized patients when assessing clinical physicians' decision making. Vignettes offer better opportunities to isolate physicians' decision making and to control case-mix variation than do analyses of claims data sets. Clinical vignette-based surveys are simple and economical tools that can be used to characterize physicians' practice variation.

This document was developed through the collaborative efforts of the Society of Critical Care Medicine, the American College of Chest Physicians, and the Association of Organ Procurement Organizations. Under the auspices of these societies, a multidisciplinary, multi-institutional task force was convened, incorporating expertise in critical care medicine, organ donor management, and transplantation. Members of the task force were divided into 13 subcommittees, each focused on one of the following general or organ-specific areas: death determination using neurologic criteria, donation after circulatory death determination, authorization process, general contraindications to donation, hemodynamic management, endocrine dysfunction and hormone replacement therapy, pediatric donor management, cardiac donation, lung donation, liver donation, kidney donation, small bowel donation, and pancreas donation. Subcommittees were charged with generating a series of management-related questions related to their topic. For each question, subcommittees provided a summary of relevant literature and specific recommendations. The specific recommendations were approved by all members of the task force and then assembled into a complete document. Because the available literature was overwhelmingly comprised of observational studies and case series, representing low-quality evidence, a decision was made that the document would assume the form of a consensus statement rather than a formally graded guideline. The goal of this document is to provide critical care practitioners with essential information and practical recommendations related to management of the potential organ donor, based on the available literature and expert consensus.