Quality of reporting in systematic reviews of adverse events: systematic review

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Objectives To examine the quality of reporting of harms in systematic reviews, and to determine the need for a reporting guideline specific for reviews of harms.

Design Systematic review.

Data sources Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE).

Review methods Databases were searched for systematic reviews having an adverse event as the main outcome, published from January 2008 to April 2011. Adverse events included an adverse reaction, harms, or complications associated with any healthcare intervention. Articles with a primary aim to investigate the complete safety profile of an intervention were also included. We developed a list of 37 items to measure the quality of reporting on harms in each review; data were collected as dichotomous outcomes (“yes” or “no” for each item).

Results Of 4644 reviews identified, 309 were systematic reviews or meta-analyses primarily assessing harms (13 from CDSR; 296 from DARE). Despite a short time interval, the comparison between the years of 2008 and 2010-11 showed no difference on the quality of reporting over time (P=0.079). Titles in fewer than half the reviews (proportion of reviews 0.46 (95% confidence interval 0.40 to 0.52)) did not mention any harm related terms. Almost one third of DARE reviews (0.26 (0.22 to 0.31)) did not clearly define the adverse events reviewed, nor did they specify the study designs selected for inclusion in their methods section. Almost half of reviews (n=170) did not consider patient risk factors or length of follow-up when reviewing harms of an intervention. Of 67 reviews of complications related to surgery or other procedures, only four (0.05 (0.01 to 0.14)) reported professional qualifications of the individuals involved. The overall, unweighted, proportion of reviews with good reporting was 0.56 (0.55 to 0.57); corresponding proportions were 0.55 (0.53 to 0.57) in 2008, 0.55 (0.54 to 0.57) in 2009, and 0.57 (0.55 to 0.58) in 2010-11.

Conclusion Systematic reviews compound the poor reporting of harms data in primary studies by failing to report on harms or doing so inadequately. Improving reporting of adverse events in systematic reviews is an important step towards a balanced assessment of an intervention.

Related collections

Most cited references 26

Record: found
Abstract: not found
Article: not found

Consort 2010 statement: extension to cluster randomised trials.

, K. Campbell, Doug G. Altman … (2011)

0 comments Cited 273 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas.

J P Ioannidis, J. C. Lau (2015)

Randomized trials with adequate sample size offer an opportunity to assess the safety of new medications in a controlled setting; however, generalizable data on drug safety reporting are sparse. To scrutinize the completeness of safety reporting in randomized trials. Survey of safety reporting in 192 randomized drug trials 7 diverse topics with sample sizes of at least 100 patients and at least 50 patients in a study arm (N = 130074 patients). Trial reports were identified from comprehensive meta-analyses in 7 medical areas. Adequate reporting of specific adverse effects and frequency and reasons for withdrawals due to toxic effects; article space allocated to safety reporting and predictors of such reporting. Severity of clinical adverse effects and laboratory-determined toxicity was adequately defined in only 39% and 29% of trial reports, respectively. Only 46% of trials stated the frequency of specific reasons for discontinuation of study treatment due to toxicity. For these 3 parameters, there was significant heterogeneity in rates of adequate reporting across topics (P =.003, P<.001, and P =.02, respectively). Overall, the median space allocated to safety results was 0.3 page. A similar amount of space was devoted to contributor names and affiliations (P =.16). On average, the percentage of space devoted to safety in the results section was 9.3% larger in trials involving dose comparisons than in those that did not (P<.001) and 3.8% smaller in trials reporting statistically significant results for efficacy outcomes (P =.047). The quality and quantity of safety reporting vary across medical areas, study designs, and settings but they are largely inadequate. Current standards for safety reporting in randomized trials should be revised to address this inadequacy.

0 comments Cited 78 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Systematic reviews of adverse effects: framework for a structured approach

Yoon K Loke, Deirdre Price, Andrew Herxheimer … (2007)

Background As every healthcare intervention carries some risk of harm, clinical decision making needs to be supported by a systematic assessment of the balance of benefit to harm. A systematic review that considers only the favourable outcomes of an intervention, without also assessing the adverse effects, can mislead by introducing a bias favouring the intervention. Much of the current guidance on systematic reviews is directed towards the evaluation of effectiveness; but this differs in important ways from the methods used in assessing the safety and tolerability of an intervention. A detailed discussion of why, how and when to include adverse effects in a systematic review, is required. Methods This discussion paper, which presupposes a basic knowledge of systematic review methodology, was developed by consensus among experienced reviewers, members of the Adverse Effects Subgroup of The Cochrane Collaboration, and supplemented by a consultation of content experts in reviews methodology, as well as those working in drug safety. Results A logical framework for making decisions in reviews that incorporate adverse effects is provided. We explore situations where a comprehensive investigation of adverse effects is warranted and suggest strategies to identify practicable and clinically useful outcomes. The advantages and disadvantages of including observational and experimental study designs are reviewed. The consequences of including separate studies for intended and unintended effects are explained. Detailed advice is given on designing electronic searches for studies with adverse effects data. Reviewers of adverse effects are given general guidance on the assessment of study bias, data collection, analysis, presentation and the interpretation of harms in a systematic review. Conclusion Readers need to be able to recognize how strategic choices made in the review process determine what harms are found, and how the findings may affect clinical decisions. Researchers undertaking a systematic review that incorporates adverse effect data should understand the rationale for the suggested methods and be able to implement them in their review.

0 comments Cited 66 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

: Role: PhD candidate

: Role: MRC fellow in health services research

: Role: lecturer

: Role: senior research fellow

: Role: assistant professor

: Role: clinical professor

: Role: clinical senior lecturer

: Role: professor

Journal

Journal ID (nlm-ta): BMJ

Journal ID (iso-abbrev): BMJ

Journal ID (publisher-id): bmj

Title: BMJ : British Medical Journal

Publisher: BMJ Publishing Group Ltd.

ISSN (Print): 0959-8138

ISSN (Electronic): 1756-1833

Publication date Collection: 2014

Publication date (Print): 2014

Publication date (Electronic): 8 January 2014

Volume: 348

Electronic Location Identifier: f7668

Affiliations

[1 ]Department of Pediatrics, 4-548 Edmonton Clinic Health Academy, University of Alberta, Edmonton, Canada

[2 ]Centre for Reviews and Dissemination, University of York, York, UK

[3 ]Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Chengguan District, Lanzhou, Gansu, People’s Republic of China

[4 ]School of Life Sciences, University of Westminster, London, UK

[5 ]Department of Pediatrics, 4-472 Edmonton Clinic Health Academy, University of Alberta, Canada

[6 ]Stollery Children’s Hospital, Department of Pediatrics, University of Alberta, Edmonton, Canada

[7 ]University of East Anglia Medical School, Norwich, UK

[8 ]Department of Pediatrics, University of Alberta, Canada, T5K 0L4

Author notes

Correspondence to: S Vohra svohra@ 123456ualberta.ca

Article

Publisher ID: zorl015187

DOI: 10.1136/bmj.f7668

PMC ID: 3898583

PubMed ID: 24401468

SO-VID: f0d959bc-580c-4927-b0a3-0ad8cde728df

License:

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.

Quality of reporting in systematic reviews of adverse events: systematic review

Read this article at

Abstract

Related collections

Wikipedia Quality

Most cited references 26

Consort 2010 statement: extension to cluster randomised trials.

Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas.

Systematic reviews of adverse effects: framework for a structured approach

Author and article information

Contributors

Journal

Affiliations

Author notes

Article

History

Categories

Comments

Comment on this article

Similar content 293

Cited by 79

Most referenced authors 1,044