Benchmarked performance charts using principal components analysis to improve the effectiveness of feedback for audit data in HIV care

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Feedback tools for clinical audit data that compare site-specific results to average performance over all sites can be useful for quality improvement. Proposed tools should be simple and clearly benchmark the site’s performance, so that a relevant action plan can be directly implemented to improve patient care services. We aimed to develop such a tool in order to feedback data to UK HIV clinics participating in the 2015 British HIV Association (BHIVA) audit assessing compliance with the 2011 guidelines for routine investigation and monitoring of adult HIV-1- infected individuals.

Methods

HIV clinic sites were asked to provide data on a random sample of 50–100 adult patients attending for HIV care during 2014 and/or 2015 by completing a self-audit spreadsheet. Outcomes audited included the proportion of patients with recorded resistance testing, viral load monitoring, adherence assessment, medications, hepatitis testing, vaccination management, risk assessments, and sexual health screening. For each outcome we benchmarked the proportion for a specific site against the average performance. We produced performance charts for each site using boxplots for the outcomes. We also used the mean and differences from the mean performance to produce a dashboard for each site. We used principal components analysis to group correlated outcomes and simplify the dashboard.

Results

The 106 sites included in the study provided information on a total of 7768 patients. Outcomes capturing monitoring of treatment of HIV-infection showed high performance across the sites, whereas testing for hepatitis, and risk assessment for cardiovascular disease and smoking, management of flu vaccination, sexual health screening, and cervical cytology for women were very variable across sites. The principal components analysis reduced the original 12 outcomes to four factors that represented HIV care, hepatitis testing, other screening tests, and resistance testing. These provided simplified measures of adherence to guidelines which were presented as a 4 bar dashboard of performance.

Conclusion

Our dashboard performance charts provide easily digestible visual summaries of locally relevant audit data that are benchmarked against the overall mean and can be used to improve feedback to HIV services. Feedback from clinicians indicated that they found these charts acceptable and useful.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2426-6) contains supplementary material, which is available to authorized users.

Related collections

Most cited references 24

Record: found
Abstract: found
Article: not found

Audit and feedback: effects on professional practice and healthcare outcomes.

Noah Ivers, Gro Jamtvedt, Signe Flottorp … (2012)

Audit and feedback is widely used as a strategy to improve professional practice either on its own or as a component of multifaceted quality improvement interventions. This is based on the belief that healthcare professionals are prompted to modify their practice when given performance feedback showing that their clinical practice is inconsistent with a desirable target. Despite its prevalence as a quality improvement strategy, there remains uncertainty regarding both the effectiveness of audit and feedback in improving healthcare practice and the characteristics of audit and feedback that lead to greater impact. To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes and to examine factors that may explain variation in the effectiveness of audit and feedback. We searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2010, Issue 4, part of The Cochrane Library. www.thecochranelibrary.com, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 10 December 2010); MEDLINE, Ovid (1950 to November Week 3 2010) (searched 09 December 2010); EMBASE, Ovid (1980 to 2010 Week 48) (searched 09 December 2010); CINAHL, Ebsco (1981 to present) (searched 10 December 2010); Science Citation Index and Social Sciences Citation Index, ISI Web of Science (1975 to present) (searched 12-15 September 2011). Randomised trials of audit and feedback (defined as a summary of clinical performance over a specified period of time) that reported objectively measured health professional practice or patient outcomes. In the case of multifaceted interventions, only trials in which audit and feedback was considered the core, essential aspect of at least one intervention arm were included. All data were abstracted by two independent review authors. For the primary outcome(s) in each study, we calculated the median absolute risk difference (RD) (adjusted for baseline performance) of compliance with desired practice compliance for dichotomous outcomes and the median percent change relative to the control group for continuous outcomes. Across studies the median effect size was weighted by number of health professionals involved in each study. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient, and risk of bias within the trial itself. We also conducted exploratory analyses to assess the role of context and the targeted clinical behaviour. Quantitative (meta-regression), visual, and qualitative analyses were undertaken to examine variation in effect size related to these factors. We included and analysed 140 studies for this review. In the main analyses, a total of 108 comparisons from 70 studies compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. After excluding studies at high risk of bias, there were 82 comparisons from 49 studies featuring dichotomous outcomes, and the weighted median adjusted RD was a 4.3% (interquartile range (IQR) 0.5% to 16%) absolute increase in healthcare professionals' compliance with desired practice. Across 26 comparisons from 21 studies with continuous outcomes, the weighted median adjusted percent change relative to control was 1.3% (IQR = 1.3% to 28.9%). For patient outcomes, the weighted median RD was -0.4% (IQR -1.3% to 1.6%) for 12 comparisons from six studies reporting dichotomous outcomes and the weighted median percentage change was 17% (IQR 1.5% to 17%) for eight comparisons from five studies reporting continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats, and when it includes both explicit targets and an action plan. In addition, the effect size varied based on the clinical behaviour targeted by the intervention. Audit and feedback generally leads to small but potentially important improvements in professional practice. The effectiveness of audit and feedback seems to depend on baseline performance and how the feedback is provided. Future studies of audit and feedback should directly compare different ways of providing feedback.

0 comments Cited 396 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: not found
Article: not found

Clinical guidelines: Potential benefits, limitations, and harms of clinical guidelines

S Woolf, R Grol, A. Hutchinson … (1999)

0 comments Cited 199 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Funnel plots for comparing institutional performance.

D J Spiegelhalter (2005)

'Funnel plots' are recommended as a graphical aid for institutional comparisons, in which an estimate of an underlying quantity is plotted against an interpretable measure of its precision. 'Control limits' form a funnel around the target outcome, in a close analogy to standard Shewhart control charts. Examples are given for comparing proportions and changes in rates, assessing association between outcome and volume of cases, and dealing with over-dispersion due to unmeasured risk factors. We conclude that funnel plots are flexible, attractively simple, and avoid spurious ranking of institutions into 'league tables'. Copyright 2004 John Wiley & Sons, Ltd.

0 comments Cited 105 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Skevi Michael:

ORCID: http://orcid.org/0000-0003-4575-946X

+44(0)117331703 , skevi.michael@bris.ac.uk

Mark Gompels: Mark.Gompels@nbt.nhs.uk

Caroline Sabin: c.sabin@ucl.ac.uk

Hilary Curtis: hilary@regordane.net

Margaret T. May: margaret.may@bristol.ac.uk

Journal

Journal ID (nlm-ta): BMC Health Serv Res

Journal ID (iso-abbrev): BMC Health Serv Res

Title: BMC Health Services Research

Publisher: BioMed Central (London )

ISSN (Electronic): 1472-6963

Publication date (Electronic): 24 July 2017

Publication date PMC-release: 24 July 2017

Publication date Collection: 2017

Volume: 17

Electronic Location Identifier: 506

Affiliations

[1 ]ISNI 0000 0004 1936 7603, GRID grid.5337.2, School of Mathematics, , University of Bristol, ; University Walk, Bristol, BS8 1TW UK

[2 ]ISNI 0000 0004 0380 7221, GRID grid.418484.5, , North Bristol NHS Trust, ; Bristol, UK

[3 ]ISNI 0000000121901201, GRID grid.83440.3b, Research Department of Infection & Population Health, , UCL, Royal Free Hospital, ; London, UK

[4 ]ISNI 0000 0001 2116 3923, GRID grid.451056.3, , National Institute of Health Research (NIHR) Health Protection Research Unit (HPRU) in Blood Borne and Sexually Transmitted Infections, ; London, UK

[5 ]ISNI 0000 0000 9724 5581, GRID grid.470770.3, , British HIV Association, ; London, UK

[6 ]ISNI 0000 0004 1936 7603, GRID grid.5337.2, School of Social and Community Medicine, , University of Bristol, ; Bristol, UK

[7 ]ISNI 0000 0001 2116 3923, GRID grid.451056.3, , National Institute of Health Research (NIHR) Health Protection Research Unit (HPRU) in Evaluation of Interventions, ; London, UK

Author information

Skevi Michael http://orcid.org/0000-0003-4575-946X

Article

Publisher ID: 2426

DOI: 10.1186/s12913-017-2426-6

PMC ID: 5525257

PubMed ID: 28738800

SO-VID: 672e83a5-d773-447e-81ce-ec3ce0ca6b6d

License:

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

History

Date received : 14 September 2016

Date accepted : 30 June 2017

Funding

Funded by: FundRef http://dx.doi.org/10.13039/501100000864, British HIV Association;

Funded by: National Institute for Health Research Health Protection Research Units (NIHR HPRUs)

Custom metadata

ScienceOpen disciplines: Health & Social care

Data availability:

ScienceOpen disciplines: Health & Social care

Comments

Comment on this article

scite_

Cited by 2

See all cited by

Most referenced authors 154

See all reference authors

Benchmarked performance charts using principal components analysis to improve the effectiveness of feedback for audit data in HIV care

Read this article at

Abstract

Background

Methods

Results

Conclusion

Electronic supplementary material

Related collections

UCL: UN SDG 05 Gender Equality

Most cited references 24

Audit and feedback: effects on professional practice and healthcare outcomes.

Clinical guidelines: Potential benefits, limitations, and harms of clinical guidelines

Funnel plots for comparing institutional performance.

Author and article information

Contributors

Journal

Affiliations

Author information

Article

History

Funding

Categories

Custom metadata

Comments

Comment on this article

Similar content 51

Cited by 2

Most referenced authors 154