10
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      On reproducibility and replicability: Arguing for open science practices and methodological improvements at the South African Journal of Industrial Psychology

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          PROBLEMATISATION: In recent years, psychology has been going through a crisis of sorts. Research methods and practices have come under increased scrutiny, with many issues identified as negatively contributing to low replicability and reproducibility of psychological research. IMPLICATIONS: As a consequence, researchers are increasingly called upon to overhaul and improve their research process. Various stakeholders within the scientific community are arguing for more openness and rigor within industrial and organisational (I-O) psychological research. A lack of transparency and openness further fuels criticisms as to the credibility and trustworthiness of I-O psychology which negatively affects the evidence-based practices which it supports. Furthermore, traditional gate-keepers such as grant agencies, professional societies and journals, are adapting their policies, reflecting an effort to curtail these trends. PURPOSE: The purpose of this opinion paper is, therefore, to stimulate an open dialogue with the South African Journal of Industrial Psychology (SAJIP) contributing authors, its editorial board and readership about the challenges associated with the replication crisis in psychology. Furthermore, it attempts to discuss how the identified issues affect I-O psychology and how these could be managed through open science practices and other structural improvements within the SAJIP. RECOMMENDATIONS: We enumerate several easily implementable open science practices, methodological improvements and editorial policy enhancements to enhance credibility and transparency within the SAJIP. Relying on these, we recommend changes to the current practices that can be taken up by researchers and the SAJIP to improve reproducibility and replicability in I-O psychological science.

          Related collections

          Most cited references55

          • Record: found
          • Abstract: found
          • Article: not found

          Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

          Being able to replicate scientific findings is crucial for scientific progress1-15. We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 201516-36. The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies. We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators. Consistent with these results, the estimated true-positive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility. Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Lower bounds on sample size in structural equation modeling

              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency

              Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.
                Bookmark

                Author and article information

                Contributors
                Role: ND
                Role: ND
                Journal
                sajip
                SA Journal of Industrial Psychology
                SA j. ind. Psychol.
                Society for Industrial and Organisational Psychology of South Africa (SIOPSA) (Johannesburg, Gauteng, South Africa )
                0258-5200
                2071-0763
                2019
                : 45
                : 1
                : 1-10
                Affiliations
                [03] Vanderbijlpark orgnameNorth-West University orgdiv1Faculty of Humanities South Africa
                [01] Eindhoven orgnameUniversity of Eindhoven orgdiv1Department of Human Performance Management Netherlands
                [02] Louvain-la-Neuve orgnameUniversity of Louvain orgdiv1Psychological Sciences Research Institute Belgium
                Article
                S2071-07632019000100021
                10.4102/sajip.v45i0.1607
                7ee2a4d7-fd36-4c30-addb-a18f4f04df0c

                This work is licensed under a Creative Commons Attribution 4.0 International License.

                History
                : 23 April 2019
                : 10 November 2018
                Page count
                Figures: 0, Tables: 0, Equations: 0, References: 80, Pages: 10
                Product

                SciELO South Africa

                Categories
                Original Research

                organisational psychology,academic publishing,industrial psychology,reproducibility,replication,Open science

                Comments

                Comment on this article