+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Using the Nine Common Themes of Good Practice checklist as a tool for evaluating the research priority setting process of a provincial research and program evaluation program


      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.



          Given the context-specific nature of health research prioritization and the obligation to effectively allocate resources to initiatives that will achieve the greatest impact, evaluation of priority setting processes can refine and strengthen such exercises and their outcomes. However, guidance is needed on evaluation tools that can be applied to research priority setting. This paper describes the adaption and application of a conceptual framework to evaluate a research priority setting exercise operating within the public health sector in Ontario, Canada.


          The Nine Common Themes of Good Practice checklist, described by Viergever et al. (Health Res Policy Syst 8:36, 2010) was used as the conceptual framework to evaluate the research priority setting process developed for the Locally Driven Collaborative Projects (LDCP) program in Ontario, Canada. Multiple data sources were used to inform the evaluation, including a review of selected priority setting approaches, surveys with priority setting participants, document review, and consultation with the program advisory committee.


          The evaluation assisted in identifying improvements to six elements of the LDCP priority setting process. The modifications were aimed at improving inclusiveness, information gathering practices, planning for project implementation, and evaluation. In addition, the findings identified that the timing of priority setting activities and level of control over the process were key factors that influenced the ability to effectively implement changes.


          The findings demonstrate the novel adaptation and application of the ‘Nine Common Themes of Good Practice checklist’ as a tool for evaluating a research priority setting exercise. The tool can guide the development of evaluation questions and enables the assessment of key constructs related to the design and delivery of a research priority setting process.

          Electronic supplementary material

          The online version of this article (doi:10.1186/s12961-016-0092-5) contains supplementary material, which is available to authorized users.

          Related collections

          Most cited references22

          • Record: found
          • Abstract: found
          • Article: not found

          A checklist for health research priority setting: nine common themes of good practice

          Health research priority setting processes assist researchers and policymakers in effectively targeting research that has the greatest potential public health benefit. Many different approaches to health research prioritization exist, but there is no agreement on what might constitute best practice. Moreover, because of the many different contexts for which priorities can be set, attempting to produce one best practice is in fact not appropriate, as the optimal approach varies per exercise. Therefore, following a literature review and an analysis of health research priority setting exercises that were organized or coordinated by the World Health Organization since 2005, we propose a checklist for health research priority setting that allows for informed choices on different approaches and outlines nine common themes of good practice. It is intended to provide generic assistance for planning health research prioritization processes. The checklist explains what needs to be clarified in order to establish the context for which priorities are set; it reviews available approaches to health research priority setting; it offers discussions on stakeholder participation and information gathering; it sets out options for use of criteria and different methods for deciding upon priorities; and it emphasizes the importance of well-planned implementation, evaluation and transparency.
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            How Are Health Research Priorities Set in Low and Middle Income Countries? A Systematic Review of Published Reports

            Background Priority setting is increasingly recognised as essential for directing finite resources to support research that maximizes public health benefits and drives health equity. Priority setting processes have been undertaken in a number of low- and middle-income country (LMIC) settings, using a variety of methods. We undertook a critical review of reports of these processes. Methods and Findings We searched electronic databases and online for peer reviewed and non-peer reviewed literature. We found 91 initiatives that met inclusion criteria. The majority took place at the global level (46%). For regional or national initiatives, most focused on Sub Saharan Africa (49%), followed by East Asia and Pacific (20%) and Latin America and the Caribbean (18%). A quarter of initiatives aimed to cover all areas of health research, with a further 20% covering communicable diseases. The most frequently used process was a conference or workshop to determine priorities (24%), followed by the Child Health and Nutrition Initiative (CHNRI) method (18%). The majority were initiated by an international organization or collaboration (46%). Researchers and government were the most frequently represented stakeholders. There was limited evidence of any implementation or follow-up strategies. Challenges in priority setting included engagement with stakeholders, data availability, and capacity constraints. Conclusions Health research priority setting (HRPS) has been undertaken in a variety of LMIC settings. While not consistently used, the application of established methods provides a means of identifying health research priorities in a repeatable and transparent manner. In the absence of published information on implementation or evaluation, it is not possible to assess what the impact and effectiveness of health research priority setting may have been.
              • Record: found
              • Abstract: not found
              • Article: not found

              On being a good listener: setting priorities for applied health services research.


                Author and article information

                647-260-7145 , rebecca.mador@oahpp.ca
                647-981-1913 , kornas.kathy@gmail.com
                647-260-7195 , anne.simard@oahpp.ca
                905-470-8995 , vharoun@oltca.com
                Health Res Policy Syst
                Health Res Policy Syst
                Health Research Policy and Systems
                BioMed Central (London )
                23 March 2016
                23 March 2016
                : 14
                [ ]Public Health Ontario, Santé publique Ontario, 480 University Avenue, Suite 300, Toronto, ON M5G 1V2 Canada
                [ ]Ontario Long-Term Care Association, 425 University Avenue, Suite 500, Toronto, ON M5G 1T6 Canada
                © Mador et al. 2016

                Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

                Custom metadata
                © The Author(s) 2016

                Health & Social care
                evaluation,priority setting,public health,research priorities
                Health & Social care
                evaluation, priority setting, public health, research priorities


                Comment on this article