Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies.

Please see later in the article for the Editors' Summary

Related collections

Most cited references 80

Record: found
Abstract: found
Article: not found

REporting recommendations for tumour MARKer prognostic studies (REMARK)

L M McShane, D Altman, W Sauerbrei … (2005)

Despite years of research and hundreds of reports on tumour markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodological problems have been cited to explain these discrepancies. Unfortunately, many tumour marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalisability of the study results. The development of guidelines for the reporting of tumour marker studies was a major recommendation of the US National Cancer Institute and the European Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics in 2000. Similar to the successful CONSORT initiative for randomised trials and the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply.

0 comments Cited 555 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Using the outcome for imputation of missing predictor values was preferred.

Karel Moons, Rogier A. R. T. Donders, Theo Stijnen … (2006)

Epidemiologic studies commonly estimate associations between predictors (risk factors) and outcome. Most software automatically exclude subjects with missing values. This commonly causes bias because missing values seldom occur completely at random (MCAR) but rather selectively based on other (observed) variables, missing at random (MAR). Multiple imputation (MI) of missing predictor values using all observed information including outcome is advocated to deal with selective missing values. This seems a self-fulfilling prophecy. We tested this hypothesis using data from a study on diagnosis of pulmonary embolism. We selected five predictors of pulmonary embolism without missing values. Their regression coefficients and standard errors (SEs) estimated from the original sample were considered as "true" values. We assigned missing values to these predictors--both MCAR and MAR--and repeated this 1,000 times using simulations. Per simulation we multiple imputed the missing values without and with the outcome, and compared the regression coefficients and SEs to the truth. Regression coefficients based on MI including outcome were close to the truth. MI without outcome yielded very biased--underestimated--coefficients. SEs and coverage of the 90% confidence intervals were not different between MI with and without outcome. Results were the same for MCAR and MAR. For all types of missing values, imputation of missing predictor values using the outcome is preferred over imputation without outcome and is no self-fulfilling prophecy.

0 comments Cited 333 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

European system for cardiac operative risk evaluation (EuroSCORE).

S Nashef, F. Roques, P. Michel … (1999)

To construct a scoring system for the prediction of early mortality in cardiac surgical patients in Europe on the basis of objective risk factors. The EuroSCORE database was divided into developmental and validation subsets. In the former, risk factors deemed to be objective, credible, obtainable and difficult to falsify were weighted on the basis of regression analysis. An additive score of predicted mortality was constructed. Its calibration and discrimination characteristics were assessed in the validation dataset. Thresholds were defined to distinguish low, moderate and high risk groups. The developmental dataset had 13,302 patients, calibration by Hosmer Lemeshow Chi square was (8) = 8.26 (P 200 micromol/l (2), active endocarditis (3) and critical preoperative state (3). Cardiac factors were unstable angina on intravenous nitrates (2), reduced left ventricular ejection fraction (30-50%: 1, 60 mmHg (2). Operation-related factors were emergency (2), other than isolated coronary surgery (2), thoracic aorta surgery (3) and surgery for postinfarct septal rupture (4). The scoring system was then applied to three risk groups. The low risk group (EuroSCORE 1-2) had 4529 patients with 36 deaths (0.8%), 95% confidence limits for observed mortality (0.56-1.10) and for expected mortality (1.27-1.29). The medium risk group (EuroSCORE 3-5) had 5977 patients with 182 deaths (3%), observed mortality (2.62-3.51), predicted (2.90-2.94). The high risk group (EuroSCORE 6 plus) had 4293 patients with 480 deaths (11.2%) observed mortality (10.25-12.16), predicted (10.93-11.54). Overall, there were 698 deaths in 14,799 patients (4.7%), observed mortality (4.37-5.06), predicted (4.72-4.95). EuroSCORE is a simple, objective and up-to-date system for assessing heart surgery, soundly based on one of the largest, most complete and accurate databases in European cardiac surgical history. We recommend its widespread use.

0 comments Cited 269 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Journal

Journal ID (nlm-ta): PLoS Med

Journal ID (iso-abbrev): PLoS Med

Journal ID (publisher-id): PLoS

Journal ID (pmc): plosmed

Title: PLoS Medicine

Publisher: Public Library of Science (San Francisco, USA )

ISSN (Print): 1549-1277

ISSN (Electronic): 1549-1676

Publication date Collection: October 2014

Publication date (Electronic): 14 October 2014

Volume: 11

Issue: 10

Electronic Location Identifier: e1001744

Affiliations

[1 ]Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht, The Netherlands

[2 ]Department of Primary Care Health Sciences, New Radcliffe House, University of Oxford, Oxford, United Kingdom

[3 ]Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Road, Oxford, United Kingdom

Author notes

* E-mail: k.g.m.moons@ 123456umcutrecht.nl

The authors have declared that no competing interests exist.

Wrote the first draft of the manuscript: KGMM JAHdG GSC. Wrote the paper: KGMM JAHdG WB YV SM DGA JBR GSC. ICMJE criteria for authorship read and met: KGMM JAHdG WB YV SM DGA JBR GSC. Agree with manuscript results and conclusions: KGMM JAHdG WB YV SM DGA JBR GSC.

¶ Moons and de Groot contributed equally to this work and are joint first authors.

Article

Publisher ID: PMEDICINE-D-14-00436

DOI: 10.1371/journal.pmed.1001744

PMC ID: 4196729

PubMed ID: 25314315

SO-VID: b96bb314-04c1-4cab-ba7c-c6e695e0c59e

License:

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

History

Page count

Pages: 12

Funding

We gratefully acknowledge financial contribution by the Netherlands Organisation for Scientific Research (project 918.10.615). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist

Read this article at

Abstract

Related collections

Reproducible research article collection

Most cited references 80

REporting recommendations for tumour MARKer prognostic studies (REMARK)

Using the outcome for imputation of missing predictor values was preferred.

European system for cardiac operative risk evaluation (EuroSCORE).

Author and article information

Journal

Affiliations

Author notes

Article

History

Page count

Funding

Categories

Comments

Comment on this article

Cited by 561

Most referenced authors 2,823