34
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years' Evaluation in Afghanistan

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Anbrasi Edward and colleagues report the results of a balanced scorecard performance system used to examine 29 key performance indicators over a 5-year period in Afghanistan, between 2004 and 2008.

          Abstract

          Background

          In 2004, Afghanistan pioneered a balanced scorecard (BSC) performance system to manage the delivery of primary health care services. This study examines the trends of 29 key performance indicators over a 5-year period between 2004 and 2008.

          Methods and Findings

          Independent evaluations of performance in six domains were conducted annually through 5,500 patient observations and exit interviews and 1,500 provider interviews in >600 facilities selected by stratified random sampling in each province. Generalized estimating equation (GEE) models were used to assess trends in BSC parameters. There was a progressive improvement in the national median scores scaled from 0–100 between 2004 and 2008 in all six domains: patient and community satisfaction of services (65.3–84.5, p<0.0001); provider satisfaction (65.4–79.2, p<0.01); capacity for service provision (47.4–76.4, p<0.0001); quality of services (40.5–67.4, p<0.0001); and overall vision for pro-poor and pro-female health services (52.0–52.6). The financial domain also showed improvement until 2007 (84.4–95.7, p<0.01), after which user fees were eliminated. By 2008, all provinces achieved the upper benchmark of national median set in 2004.

          Conclusions

          The BSC has been successfully employed to assess and improve health service capacity and service delivery using performance benchmarking during the 5-year period. However, scorecard reconfigurations are needed to integrate effectiveness and efficiency measures and accommodate changes in health systems policy and strategy architecture to ensure its continued relevance and effectiveness as a comprehensive health system performance measure. The process of BSC design and implementation can serve as a valuable prototype for health policy planners managing performance in similar health care contexts.

          Please see later in the article for the Editors' Summary

          Editors' Summary

          Background

          Traditionally, the performance of a health system (the complete network of health care agencies, facilities, and providers in a defined geographical region) has been measured in terms of health outcomes: how many people have been treated, how many got better, and how many died. But, nowadays, with increased demand for improved governance and accountability, policy makers are seeking comprehensive performance measures that show in detail how innovations designed to strengthen health systems are affecting service delivery and health outcomes. One such performance measure is the “balanced scorecard,” an integrated management and measurement tool that enables organizations to clarify their vision and strategy and translate them into action. The balanced scorecard—essentially a list of key performance indicators and performance benchmarks in several domains—was originally developed for industry but is now becoming a popular strategic management tool in the health sector. For example, balanced scorecards have been successfully integrated into the Dutch and Italian public health care systems.

          Why Was This Study Done?

          Little is known about the use of balanced scorecards in the national public health care systems of developing countries but the introduction of performance management into health system reform in fragile states in particular (developing countries where the state fails to perform the fundamental functions necessary to meet its citizens' basic needs and expectations) could help to promote governance and leadership, and facilitate essential policy changes. One fragile state that has introduced the balanced scorecard system for public health care management is Afghanistan, which emerged from decades of conflict in 2002 with some of the world's worst health indicators. To deal with an extremely high burden of disease, the Ministry of Public Health (MOPH) designed a Basic Package of Health Services (BPHS), which is delivered by nongovernmental organizations and MOPH agencies. In 2004, the MOPH introduced the National Health Service Performance Assessment (NHSPA), an annual country-wide assessment of service provision and patient satisfaction and pioneered a balanced scorecard, which uses data collected in the NHSPA, to manage the delivery of primary health care services. In this study, the researchers examine the trends between 2004 and 2008 of the 29 key performance indicators in six domains included in this balanced scorecard, and consider the potential and limitations of the scorecard as a management tool to measure and improve health service delivery in Afghanistan and other similar countries.

          What Did the Researchers Do and Find?

          Each year of the study, a random sample of 25 facilities (district hospitals and comprehensive and basic health centers) in 28 of Afghanistan's 34 provinces was chosen (one province did not have functional facilities in 2004 and the other five missing provinces were inaccessible because of ongoing conflicts). NHSPA surveyors collected approximately 5,000 patient observations, 5,000 exit interviews with patients or their caregivers, and 1,500 health provider interviews by observing consultations involving five children under 5 years old and five patients over 5 years old in each facility. The researchers then used this information to evaluate the key performance indicators in the balanced scorecard and a statistical method called generalized estimating equation modeling to assess trends in these indicators. They report that there was a progressive improvement in national average scores in all six domains (patients and community satisfaction with services, provider satisfaction, capacity for service provision, quality of services, overall vision for pro-poor and pro-female health services, and financial systems) between 2004 and 2008.

          What Do These Findings Mean?

          These findings suggest that the balanced scorecard was successfully used to improve health system capacity and service delivery through performance benchmarking over the 5-year study period. Importantly, the use of the balanced scorecard helped to show the effects of investments, facilitate policy change, and create a more evidence-based decision-making culture in Afghanistan's primary health care system. However, the researchers warn that the continuing success of the balanced scorecard in Afghanistan will depend on its ability to accommodate changes in health systems policy. Furthermore, reconfigurations of the scorecard are needed to include measures of the overall effectiveness and efficiency of the health system such as mortality rates. More generally, the researchers conclude that the balanced scorecard offers a promising measure of health system performance that could be used to examine the effectiveness of health care strategies and innovations in other fragile and developing countries.

          Additional Information

          Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001066.

          Related collections

          Most cited references44

          • Record: found
          • Abstract: found
          • Article: not found

          Benchmarking of performance of Mexican states with effective coverage.

          Benchmarking of the performance of states, provinces, or districts in a decentralised health system is important for fostering of accountability, monitoring of progress, identification of determinants of success and failure, and creation of a culture of evidence. The Mexican Ministry of Health has, since 2001, used a benchmarking approach based on the WHO concept of effective coverage of an intervention, which is defined as the proportion of potential health gain that could be delivered by the health system to that which is actually delivered. Using data collection systems, including state representative examination surveys, vital registration, and hospital discharge registries, we have monitored the delivery of 14 interventions for 2005-06. Overall effective coverage ranges from 54.0% in Chiapas, a poor state, to 65.1% in the Federal District. Effective coverage for maternal and child health interventions is substantially higher than that for interventions that target other health problems. Effective coverage for the lowest wealth quintile is 52% compared with 61% for the highest quintile. Effective coverage is closely related to public-health spending per head across states; this relation is stronger for interventions that are not related to maternal and child health than those for maternal and child health. Considerable variation also exists in effective coverage at similar amounts of spending. We discuss the implications of these issues for the further development of the Mexican health-information system. Benchmarking of performance by measuring effective coverage encourages decision-makers to focus on quality service provision, not only service availability. The effective coverage calculation is an important device for health-system stewardship. In adopting this approach, other countries should select interventions to be measured on the basis of the criteria of affordability, effect on population health, effect on health inequalities, and capacity to measure the effects of the intervention. The national institutions undertaking this benchmarking must have the mandate, skills, resources, and independence to succeed.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Innovations in Rwanda's health system: looking to the future.

            Rwanda is making substantial progress towards improvement of health and is working towards achievement of the Millennium Development Goals, which is a challenging task because the country has had genocide in 1994, has few natural resources, is landlocked, and has high population growth. Like many impoverished sub-Saharan countries, Rwanda's health system has had an uncoordinated plethora of donors, shortage of health staff, inequity of access, and poor quality of care in health facilities. This report describes three health system developments introduced by the Rwandan government that are improving these barriers to care-ie, the coordination of donors and external aid with government policy, and monitoring the effectiveness of aid; a country-wide independent community health insurance scheme; and the introduction of a performance-based pay initiative. If these innovations are successful, they might be of interest to other sub-Saharan countries. However, Rwanda still does not have sufficient financial resources for health and will need additional external aid for some time to attain the Millennium Development Goals.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Where giving birth is a forecast of death: maternal mortality in four districts of Afghanistan, 1999-2002.

              Maternal mortality in Afghanistan is uniformly identified as an issue of primary public-health importance. To guide the implementation of reproductive-health services, we examined the numbers, causes, and preventable factors for maternal deaths among women in four districts. We did a retrospective cohort study of women of reproductive age (15-49 years) who died between March 21, 1999, and March 21, 2002, in four selected districts in four provinces: Kabul city, Kabul province (urban); Alisheng district, Laghman province (semirural); Maywand, Kandahar province (rural); and Ragh, Badakshan province (rural, most remote). Deaths among women of reproductive age were identified through a survey of all households in randomly selected villages and investigated through verbal-autopsy interviews of family members. In a population of 90 816, 357 women of reproductive age died; 154 deaths were related to complications during pregnancy, childbirth, or the puerperal period. Most maternal deaths were caused by ante-partum haemorrhage, except in Ragh, where a greater proportion of women died of obstructed labour. All measures of maternal risk were high, especially in the more remote areas; the maternal mortality ratio (per 100,000 livebirths) was 418 (235-602) in Kabul, 774 (433-1115) in Alisheng, 2182 (1451-2913) in Maywand, and 6507 (5026-7988) in Ragh. In the two rural sites, no woman who died was assisted by a skilled birth attendant. Maternal mortality in Afghanistan is high and becomes significantly greater with increasing remoteness. Deaths could be averted if complications were prevented through optimisation of general health status and if complications that occurred were treated to reduce their severity--efforts that require a multisectoral approach to increase availability and accessibility of health care.
                Bookmark

                Author and article information

                Contributors
                Role: Academic Editor
                Journal
                PLoS Med
                PLoS
                plosmed
                PLoS Medicine
                Public Library of Science (San Francisco, USA )
                1549-1277
                1549-1676
                July 2011
                July 2011
                26 July 2011
                : 8
                : 7
                : e1001066
                Affiliations
                [1 ]Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America
                [2 ]Indian Institute of Public Health, Public Health Foundation of India, New Delhi, India
                [3 ]Office of the President of Afghanistan, Kabul, Afghanistan
                [4 ]Ministry of Public Health, Kabul, Afghanistan
                London School of Hygiene and Tropical Medicine, United Kingdom
                Author notes

                Conceived and designed the experiments: AE BK FK ASS GMB DHP. Performed the experiments: AE BK. Analyzed the data: BK AE. Contributed reagents/materials/analysis tools: AE BK GMB DHP. Wrote the first draft of the manuscript: AE. Contributed to the writing of the manuscript: AE BK DHP FK ASS GMB. ICMJE criteria for authorship read and met: AE BK FK ASS GMB DHP. Agree with manuscript results and conclusions: AE BK FK ASS GMB DHP.

                Article
                PMEDICINE-D-11-00085
                10.1371/journal.pmed.1001066
                3144209
                21814499
                357c3e9f-c3bd-4b0e-a572-635a17723d59
                Edward et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
                History
                : 12 January 2011
                : 14 June 2011
                Page count
                Pages: 9
                Categories
                Research Article
                Medicine
                Global Health
                Public Health

                Medicine
                Medicine

                Comments

                Comment on this article

                scite_

                Similar content21

                Cited by47

                Most referenced authors301