+1 Recommend
0 collections
      • Record: found
      • Abstract: not found
      • Article: not found

      Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results

      1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 4 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 5 , 27 , 10 , 28 , 29 , 30 , 15 , 26 , 31 , 32 , 33 , 16 , 34 , 16 , 16 , 35 , 36 , 37 , 16 , 4 , 38 , 24 , 39 , 25 , 37 , 40 , 41 , 42 , 43 , 44 , 4 , 16 , 21 , 4 , 45 , 46 , 19 , 3 , 47

      Advances in Methods and Practices in Psychological Science

      SAGE Publications

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Related collections

          Most cited references 30

          • Record: found
          • Abstract: found
          • Article: not found

          False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant.

          In this article, we accomplish two things. First, we show that despite empirical psychologists' nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.
            • Record: found
            • Abstract: found
            • Article: not found

            The new statistics: why and how.

             Geoff Cumming (2013)
            We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research literature is incomplete and untrustworthy, we need new requirements to ensure research integrity. These include prespecification of studies whenever possible, avoidance of selection and other inappropriate data-analytic practices, complete reporting, and encouragement of replication. Second, in response to renewed recognition of the severe flaws of null-hypothesis significance testing (NHST), we need to shift from reliance on NHST to estimation and other preferred techniques. The new statistics refers to recommended practices, including estimation based on effect sizes, confidence intervals, and meta-analysis. The techniques are not new, but adopting them widely would be new for many researchers, as well as highly beneficial. This article explains why the new statistics are important and offers guidance for their use. It describes an eight-step new-statistics strategy for research with integrity, which starts with formulation of research questions in estimation terms, has no place for NHST, and is aimed at building a cumulative quantitative discipline.
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Investigating Variation in Replicability

              Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, 10 effects replicated consistently. One effect – imagined contact reducing prejudice – showed weak support for replicability. And two effects – flag priming influencing conservatism and currency priming influencing system justification – did not replicate. We compared whether the conditions such as lab versus online or US versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect.

                Author and article information

                Advances in Methods and Practices in Psychological Science
                Advances in Methods and Practices in Psychological Science
                SAGE Publications
                August 23 2018
                August 23 2018
                : 251524591774764
                [1 ]Organisational Behaviour, University of Sussex Business School
                [2 ]Organisational Behaviour Area, INSEAD Asia Campus
                [3 ]Department of Psychology, University of Virginia
                [4 ]Department of Philosophy, Sociology, Education and Applied Psychology, University of Padua
                [5 ]Department of Psychology, University of Cologne
                [6 ]Department of Management, University of Cincinnati
                [7 ]Department of Management, Faculty of Business Administration, University of Economics, Prague
                [8 ]Department of Management and Marketing, Hong Kong Polytechnic University
                [9 ]Department of Psychology, University of Liverpool
                [10 ]Department of Economics, Stockholm School of Economics
                [11 ]Department of Psychology, Linnaeus University
                [12 ]School of Public Health, University of Hong Kong
                [13 ]Berkeley Institute for Data Science, University of California, Berkeley
                [14 ]Department of Psychology, College of Staten Island, City University of New York
                [15 ]Department of Psychology, New York University
                [16 ]Faculty of Economics and Business, University of Groningen
                [17 ]Division of Neuroscience and Experimental Psychology, University of Manchester
                [18 ]Westat, Rockville, Maryland
                [19 ]Department of Marketing and Supply Chain Management, Temple University
                [20 ]Department of Management and Organizations, Kellogg School of Management, Northwestern University
                [21 ]Department of Psychology, University of Zurich
                [22 ]Washington, D.C.
                [23 ]School of Mathematics and Statistics, University of Sheffield
                [24 ]Swedish Institute for Social Research (SOFI), Stockholm University
                [25 ]Department of Psychology, Ludwig-Maximilians-Universität München
                [26 ]College of Liberal Arts, University of Minnesota
                [27 ]School of Management, Xiamen University
                [28 ]College of Business, Oregon State University
                [29 ]Department of Psychology, Federal University of Santa Catarina
                [30 ]School of Business, University of Washington Bothell
                [31 ]School of Mathematical Sciences, Queen Mary University of London
                [32 ]School of Psychology, University of Nottingham
                [33 ]Department of Psychology, Northwestern University
                [34 ]School of Psychology, Cardiff University
                [35 ]Department of Economics, University of Maryland
                [36 ]Department of Economics, Brigham Young University
                [37 ]Department of Psychology, Loyola University Maryland
                [38 ]Rotman School of Management, University of Toronto
                [39 ]Department of Social Sciences and Cultural Studies, Institute of Sociology, Justus Liebig University, Giessen
                [40 ]United States Military Academy at West Point
                [41 ]Department of Marketing and Management, SUNY Oswego
                [42 ]John Molson School of Business, Concordia University
                [43 ]Lehrstuhl für Soziologie, insb. Sozialstrukturanalyse, Otto-Friedrich-Universität Bamberg
                [44 ]Department of Psychology, University of Sheffield
                [45 ]Department of Psychological Methods, University of Amsterdam
                [46 ]Poznań, Poland
                [47 ]Center for Open Science, Charlottesville, Virginia
                © 2018



                Comment on this article