13
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Clinical performance comparators in audit and feedback: a review of theory and evidence

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Audit and feedback (A&F) is a common quality improvement strategy with highly variable effects on patient care. It is unclear how A&F effectiveness can be maximised. Since the core mechanism of action of A&F depends on drawing attention to a discrepancy between actual and desired performance, we aimed to understand current and best practices in the choice of performance comparator.

          Methods

          We described current choices for performance comparators by conducting a secondary review of randomised trials of A&F interventions and identifying the associated mechanisms that might have implications for effective A&F by reviewing theories and empirical studies from a recent qualitative evidence synthesis.

          Results

          We found across 146 trials that feedback recipients’ performance was most frequently compared against the performance of others (benchmarks; 60.3%). Other comparators included recipients’ own performance over time (trends; 9.6%) and target standards (explicit targets; 11.0%), and 13% of trials used a combination of these options. In studies featuring benchmarks, 42% compared against mean performance. Eight (5.5%) trials provided a rationale for using a specific comparator. We distilled mechanisms of each comparator from 12 behavioural theories, 5 randomised trials, and 42 qualitative A&F studies.

          Conclusion

          Clinical performance comparators in published literature were poorly informed by theory and did not explicitly account for mechanisms reported in qualitative studies. Based on our review, we argue that there is considerable opportunity to improve the design of performance comparators by (1) providing tailored comparisons rather than benchmarking everyone against the mean, (2) limiting the amount of comparators being displayed while providing more comparative information upon request to balance the feedback’s credibility and actionability, (3) providing performance trends but not trends alone, and (4) encouraging feedback recipients to set personal, explicit targets guided by relevant information.

          Electronic supplementary material

          The online version of this article (10.1186/s13012-019-0887-1) contains supplementary material, which is available to authorized users.

          Related collections

          Most cited references80

          • Record: found
          • Abstract: found
          • Article: not found

          Audit and feedback: effects on professional practice and health care outcomes.

          Audit and feedback continues to be widely used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines. Yet, audit and feedback has not consistently been found to be effective. To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes. We searched the Cochrane Effective Practice and Organisation of Care Group's register and pending file up to January 2004. Randomised trials of audit and feedback (defined as any summary of clinical performance over a specified period of time) that reported objectively measured professional practice in a healthcare setting or healthcare outcomes. Two reviewers independently extracted data and assessed study quality. Quantitative (meta-regression), visual and qualitative analyses were undertaken. For each comparison we calculated the risk difference (RD) and risk ratio (RR), adjusted for baseline compliance when possible, for dichotomous outcomes and the percentage and the percent change relative to the control group average after the intervention, adjusted for baseline performance when possible, for continuous outcomes. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: the type of intervention (audit and feedback alone, audit and feedback with educational meetings, or multifaceted interventions that included audit and feedback), the intensity of the audit and feedback, the complexity of the targeted behaviour, the seriousness of the outcome, baseline compliance and study quality. Thirty new studies were added to this update, and a total of 118 studies are included. In the primary analysis 88 comparisons from 72 studies were included that compared any intervention in which audit and feedback is a component compared to no intervention. For dichotomous outcomes the adjusted risk difference of compliance with desired practice varied from - 0.16 (a 16 % absolute decrease in compliance) to 0.70 (a 70% increase in compliance) (median = 0.05, inter-quartile range = 0.03 to 0.11) and the adjusted risk ratio varied from 0.71 to 18.3 (median = 1.08, inter-quartile range = 0.99 to 1.30). For continuous outcomes the adjusted percent change relative to control varied from -0.10 (a 10 % absolute decrease in compliance) to 0.68 (a 68% increase in compliance) (median = 0.16, inter-quartile range = 0.05 to 0.37). Low baseline compliance with recommended practice and higher intensity of audit and feedback were associated with larger adjusted risk ratios (greater effectiveness) across studies. Audit and feedback can be effective in improving professional practice. When it is effective, the effects are generally small to moderate. The relative effectiveness of audit and feedback is likely to be greater when baseline adherence to recommended practice is low and when feedback is delivered more intensively.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Meta-analysis: audit and feedback features impact effectiveness on care quality.

            Audit and feedback (A&F) has long been used to improve quality of care, albeit with variable results. This meta-analytic study tested whether Feedback Intervention Theory, a framework from industrial/organizational psychology, explains the observed variability in health care A&F research. studies cited by Jamtvedt's 2006 Cochrane systematic review of A&F, followed by database searches using the Cochrane review's search strategy to identify more recent studies. Cochrane review criteria, plus: presence of a treatment group receiving only A & F; a control group receiving no intervention; a quantitatively measurable outcome; minimum n of 10 per arm; sufficient statistics for effect size calculations. Moderators: presence of discouragement and praise; correct solution, attainment level, velocity, frequency, and normative information; feedback format (verbal, textual, graphic, public, computerized, group vs. individual); goal setting activity. meta-analytic procedures using the Hedges-Olkin method. Of 519 studies initially identified, 19 met all inclusion criteria. Studies were most often excluded due to the lack of a feedback-only arm. A&F has a modest, though significant positive effect on quality outcomes (d = 0.40, 95% confidence interval = +/-0.20); providing specific suggestions for improvement, written, and more frequent feedback strengthened this effect, whereas graphical and verbal feedback attenuated this effect. A&F effectiveness is improved when feedback is delivered with specific suggestions for improvement, in writing, and frequently. Other feedback characteristics could also potentially improve effectiveness; however, research with stricter experimental controls is needed to identify the specific feedback characteristics that maximize its effectiveness.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes

              Self-appraisal has repeatedly been shown to be inadequate as a mechanism for performance improvement. This has placed greater emphasis on understanding the processes through which self-perception and external feedback interact to influence professional development. As feedback is inevitably interpreted through the lens of one’s self-perceptions it is important to understand how learners interpret, accept, and use feedback (or not) and the factors that influence those interpretations. 134 participants from 8 health professional training/continuing competence programs were recruited to participate in focus groups. Analyses were designed to (a) elicit understandings of the processes used by learners and physicians to interpret, accept and use (or not) data to inform their perceptions of their clinical performance, and (b) further understand the factors (internal and external) believed to influence interpretation of feedback. Multiple influences appear to impact upon the interpretation and uptake of feedback. These include confidence, experience, and fear of not appearing knowledgeable. Importantly, however, each could have a paradoxical effect of both increasing and decreasing receptivity. Less prevalent but nonetheless important themes suggested mechanisms through which cognitive reasoning processes might impede growth from formative feedback. Many studies have examined the effectiveness of feedback through variable interventions focused on feedback delivery. This study suggests that it is equally important to consider feedback from the perspective of how it is received. The interplay observed between fear, confidence, and reasoning processes reinforces the notion that there is no simple recipe for the delivery of effective feedback. These factors should be taken into account when trying to understand (a) why self-appraisal can be flawed, (b) why appropriate external feedback is vital (yet can be ineffective), and (c) why we may need to disentangle the goals of performance improvement from the goals of improving self-assessment.
                Bookmark

                Author and article information

                Contributors
                w.t.gude@amc.uva.nl
                benjamin.brown@manchester.ac.uk
                sabine.vanderveer@manchester.ac.uk
                heather.colquhoun@utoronto.ca
                noahivers@gmail.com
                jbrehaut@ohri.ca
                zachll@umich.edu
                chris.armitage@manchester.ac.uk
                n.f.keizer@amc.uva.nl
                niels.peek@manchester.ac.uk
                Journal
                Implement Sci
                Implement Sci
                Implementation Science : IS
                BioMed Central (London )
                1748-5908
                24 April 2019
                24 April 2019
                2019
                : 14
                : 39
                Affiliations
                [1 ]ISNI 0000000084992262, GRID grid.7177.6, Department of Medical Informatics, Amsterdam UMC, Amsterdam Public Health Research Institute, , University of Amsterdam, ; Amsterdam, The Netherlands
                [2 ]ISNI 0000000121662407, GRID grid.5379.8, NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, Manchester Academic Health Science Centre, , The University of Manchester, ; Manchester, UK
                [3 ]ISNI 0000000121662407, GRID grid.5379.8, Centre for Health Informatics, Division of Informatics, Imaging and Data Sciences, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, , The University of Manchester, ; Manchester, UK
                [4 ]ISNI 0000 0001 2157 2938, GRID grid.17063.33, Occupational Science and Occupational Therapy, , University of Toronto, ; Toronto, Ontario Canada
                [5 ]ISNI 0000 0001 2157 2938, GRID grid.17063.33, Family and Community Medicine, , Women’s College Hospital, University of Toronto, ; Toronto, Ontario Canada
                [6 ]ISNI 0000 0000 9606 5108, GRID grid.412687.e, Clinical Epidemiology Program, , Ottawa Hospital Research Institute, ; Ottawa, Ontario Canada
                [7 ]ISNI 0000 0001 2182 2255, GRID grid.28046.38, School of Epidemiology and Public Health, , University of Ottawa, ; Ottawa, Ontario Canada
                [8 ]ISNI 0000 0004 1936 9000, GRID grid.21925.3d, Center for Health Informatics for the Underserved, Department of Biomedical Informatics, , University of Pittsburgh, ; Pittsburgh, PA USA
                [9 ]ISNI 0000000121662407, GRID grid.5379.8, Manchester Centre for Health Psychology, Division of Psychology and Mental Health, , The University of Manchester, ; Manchester, UK
                [10 ]ISNI 0000000121662407, GRID grid.5379.8, NIHR Manchester Biomedical Research Centre, Manchester Academic Health Science Centre, , The University of Manchester, ; Manchester, UK
                Author information
                http://orcid.org/0000-0001-7941-5281
                Article
                887
                10.1186/s13012-019-0887-1
                6480497
                31014352
                8f492e15-77b7-4e32-a97f-2026c6da466c
                © The Author(s). 2019

                Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

                History
                : 9 January 2019
                : 1 April 2019
                Funding
                Funded by: NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre
                Categories
                Research
                Custom metadata
                © The Author(s) 2019

                Medicine
                benchmarking,medical audit,feedback,quality improvement
                Medicine
                benchmarking, medical audit, feedback, quality improvement

                Comments

                Comment on this article