36
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: not found
      • Article: not found

      Contextual sensitivity in scientific reproducibility

      Read this article at

      ScienceOpenPublisherPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          In recent years, scientists have paid increasing attention to reproducibility. For example, the Reproducibility Project, a large-scale replication attempt of 100 studies published in top psychology journals found that only 39% could be unambiguously reproduced. There is a growing consensus among scientists that the lack of reproducibility in psychology and other fields stems from various methodological factors, including low statistical power, researcher's degrees of freedom, and an emphasis on publishing surprising positive results. However, there is a contentious debate about the extent to which failures to reproduce certain results might also reflect contextual differences (often termed "hidden moderators") between the original research and the replication attempt. Although psychologists have found extensive evidence that contextual factors alter behavior, some have argued that context is unlikely to influence the results of direct replications precisely because these studies use the same methods as those used in the original research. To help resolve this debate, we recoded the 100 original studies from the Reproducibility Project on the extent to which the research topic of each study was contextually sensitive. Results suggested that the contextual sensitivity of the research topic was associated with replication success, even after statistically adjusting for several methodological characteristics (e.g., statistical power, effect size). The association between contextual sensitivity and replication success did not differ across psychological subdisciplines. These results suggest that researchers, replicators, and consumers should be mindful of contextual factors that might influence a psychological process. We offer several guidelines for dealing with contextual sensitivity in reproducibility.

          Related collections

          Most cited references 45

          • Record: found
          • Abstract: not found
          • Article: not found

          One Hundred Years of Social Psychology Quantitatively Described.

            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Investigating Variation in Replicability

            Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, 10 effects replicated consistently. One effect – imagined contact reducing prejudice – showed weak support for replicability. And two effects – flag priming influencing conservatism and currency priming influencing system justification – did not replicate. We compared whether the conditions such as lab versus online or US versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Experiencing physical warmth promotes interpersonal warmth.

              "Warmth" is the most powerful personality trait in social judgment, and attachment theorists have stressed the importance of warm physical contact with caregivers during infancy for healthy relationships in adulthood. Intriguingly, recent research in humans points to the involvement of the insula in the processing of both physical temperature and interpersonal warmth (trust) information. Accordingly, we hypothesized that experiences of physical warmth (or coldness) would increase feelings of interpersonal warmth (or coldness), without the person's awareness of this influence. In study 1, participants who briefly held a cup of hot (versus iced) coffee judged a target person as having a "warmer" personality (generous, caring); in study 2, participants holding a hot (versus cold) therapeutic pad were more likely to choose a gift for a friend instead of for themselves.
                Bookmark

                Author and article information

                Journal
                Proceedings of the National Academy of Sciences
                Proc Natl Acad Sci USA
                Proceedings of the National Academy of Sciences
                0027-8424
                1091-6490
                June 07 2016
                June 07 2016
                June 07 2016
                May 23 2016
                : 113
                : 23
                : 6454-6459
                Article
                10.1073/pnas.1521897113
                4988618
                27217556
                bbbc2de3-bcf1-481e-8997-ac56039d6919
                © 2016

                Comments

                Comment on this article