Blog
About

2
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: not found

      External Validity and Policy Adaptation: From Impact Evaluation to Policy Design

      1

      The World Bank Research Observer

      Oxford University Press (OUP)

      Read this article at

      ScienceOpenPublisher
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy’s design and implementation? This paper suggests that failures of external validity (both in transporting and scaling up policy) can be understood as arising from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. The paper surveys existing approaches to analyzing external validity, and suggests that there has been more focus on the generalizability of impact evaluation results than on the applicability of evidence to specific contexts. To help fill this gap, the study develops a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength of evidence on the policy from other contexts and the policy maker’s information about the local context.

          Related collections

          Most cited references 24

          • Record: found
          • Abstract: found
          • Article: not found

          Issues and challenges in the design of culturally adapted evidence-based interventions.

          This article examines issues and challenges in the design of cultural adaptations that are developed from an original evidence-based intervention (EBI). Recently emerging multistep frameworks or stage models are examined, as these can systematically guide the development of culturally adapted EBIs. Critical issues are also presented regarding whether and how such adaptations may be conducted, and empirical evidence is presented regarding the effectiveness of such cultural adaptations. Recent evidence suggests that these cultural adaptations are effective when applied with certain subcultural groups, although they are less effective when applied with other subcultural groups. Generally, current evidence regarding the effectiveness of cultural adaptations is promising but mixed. Further research is needed to obtain more definitive conclusions regarding the efficacy and effectiveness of culturally adapted EBIs. Directions for future research and recommendations are presented to guide the development of a new generation of culturally adapted EBIs.
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Mechanism Experiments and Policy Evaluations

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Site Selection Bias in Program Evaluation

               Hunt Allcott (2015)
                Bookmark

                Author and article information

                Journal
                The World Bank Research Observer
                Oxford University Press (OUP)
                0257-3032
                1564-6971
                August 2020
                August 01 2020
                August 01 2019
                August 2020
                August 01 2020
                August 01 2019
                : 35
                : 2
                : 158-191
                Affiliations
                [1 ]University of Oxford, Blavatnik School of Government
                Article
                10.1093/wbro/lky010
                © 2019

                Comments

                Comment on this article