+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Looking both ways: a review of methods for assessing research impacts on policy and the policy utilisation of research


      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.



          Measuring the policy and practice impacts of research is becoming increasingly important. Policy impacts can be measured from two directions – tracing forward from research and tracing backwards from a policy outcome. In this review, we compare these approaches and document the characteristics of studies assessing research impacts on policy and the policy utilisation of research.


          Keyword searches of electronic databases were conducted in December 2016. Included studies were published between 1995 and 2016 in English and reported methods and findings of studies measuring policy impacts of specified health research, or research use in relation to a specified health policy outcome, and reviews reporting methods of research impact assessment. Using an iterative data extraction process, we developed a framework to define the key elements of empirical studies (assessment reason, assessment direction, assessment starting point, unit of analysis, assessment methods, assessment endpoint and outcomes assessed) and then documented the characteristics of included empirical studies according to this framework.


          We identified 144 empirical studies and 19 literature reviews. Empirical studies were derived from two parallel streams of research of equal size, which we termed ‘research impact assessments’ and ‘research use assessments’. Both streams provided insights about the influence of research on policy and utilised similar assessment methods, but approached measurement from opposite directions. Research impact assessments predominantly utilised forward tracing approaches while the converse was true for research use assessments. Within each stream, assessments focussed on narrow or broader research/policy units of analysis as the starting point for assessment, each with associated strengths and limitations. The two streams differed in terms of their relative focus on the contributions made by specific research (research impact assessments) versus research more generally (research use assessments) and the emphasis placed on research and the activities of researchers in comparison to other factors and actors as influencers of change.


          The Framework presented in this paper provides a mechanism for comparing studies within this broad field of research enquiry. Forward and backward tracing approaches, and their different ways of ‘looking’, tell a different story of research-based policy change. Combining approaches may provide the best way forward in terms of linking outcomes to specific research, as well as providing a realistic picture of research influence.

          Electronic supplementary material

          The online version of this article (10.1186/s12961-018-0310-4) contains supplementary material, which is available to authorized users.

          Related collections

          Most cited references 29

          • Record: found
          • Abstract: not found
          • Article: not found

          Reforming the health sector in developing countries: the central role of policy analysis

            • Record: found
            • Abstract: found
            • Article: not found

            The terrain of health policy analysis in low and middle income countries: a review of published literature 1994–2007

            This article provides the first ever review of literature analysing the health policy processes of low and middle income countries (LMICs). Based on a systematic search of published literature using two leading international databases, the article maps the terrain of work published between 1994 and 2007, in terms of policy topics, lines of inquiry and geographical base, as well as critically evaluating its strengths and weaknesses. The overall objective of the review is to provide a platform for the further development of this field of work. From an initial set of several thousand articles, only 391 were identified as relevant to the focus of inquiry. Of these, 164 were selected for detailed review because they present empirical analyses of health policy change processes within LMIC settings. Examination of these articles clearly shows that LMIC health policy analysis is still in its infancy. There are only small numbers of such analyses, whilst the diversity of policy areas, topics and analytical issues that have been addressed across a large number of country settings results in a limited depth of coverage within this body of work. In addition, the majority of articles are largely descriptive in nature, limiting understanding of policy change processes within or across countries. Nonetheless, the broad features of experience that can be identified from these articles clearly confirm the importance of integrating concern for politics, process and power into the study of health policy. By generating understanding of the factors influencing the experience and results of policy change, such analysis can inform action to strengthen future policy development and implementation. This article, finally, outlines five key actions needed to strengthen the field of health policy analysis within LMICs, including capacity development and efforts to generate systematic and coherent bodies of work underpinned by both the intent to undertake rigorous analytical work and concern to support policy change.
              • Record: found
              • Abstract: not found
              • Article: not found

              The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster?

               B R Martin (2011)

                Author and article information

                61419494244 , 61266764806 , rnewson@uni.sydney.edu.au
                Health Res Policy Syst
                Health Res Policy Syst
                Health Research Policy and Systems
                BioMed Central (London )
                25 June 2018
                25 June 2018
                : 16
                [1 ]ISNI 0000 0004 1936 834X, GRID grid.1013.3, Sydney School of Public Health, , The University of Sydney, ; Charles Perkins Centre D17, Level 6 Hub, Sydney, NSW 2006 Australia
                [2 ]ISNI 0000 0004 0402 6494, GRID grid.266886.4, School of Medicine Sydney, University of Notre Dame Australia, ; 160 Oxford St, Darlinghurst, 2010 Australia
                © The Author(s). 2018

                Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

                Funded by: FundRef http://dx.doi.org/10.13039/501100000925, National Health and Medical Research Council;
                Award ID: 1024291
                Award Recipient :
                Custom metadata
                © The Author(s) 2018


                Comment on this article