19
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: not found
      • Article: not found

      Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results

      1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 4 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 5 , 27 , 10 , 28 , 29 , 30 , 15 , 26 , 31 , 32 , 33 , 16 , 34 , 16 , 16 , 35 , 36 , 37 , 16 , 4 , 38 , 24 , 39 , 25 , 37 , 40 , 41 , 42 , 43 , 44 , 4 , 16 , 21 , 4 , 45 , 46 , 19 , 3 , 47

      Advances in Methods and Practices in Psychological Science

      SAGE Publications

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Related collections

          Most cited references 24

          • Record: found
          • Abstract: found
          • Article: not found

          An Agenda for Purely Confirmatory Research.

          The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss an uncomfortable fact that threatens the core of psychology's academic enterprise: almost without exception, psychologists do not commit themselves to a method of data analysis before they see the actual data. It then becomes tempting to fine tune the analysis to the data in order to obtain a desired result-a procedure that invalidates the interpretation of the common statistical tests. The extent of the fine tuning varies widely across experiments and experimenters but is almost impossible for reviewers and readers to gauge. To remedy the situation, we propose that researchers preregister their studies and indicate in advance the analyses they intend to conduct. Only these analyses deserve the label "confirmatory," and only for these analyses are the common statistical tests valid. Other analyses can be carried out but these should be labeled "exploratory." We illustrate our proposal with a confirmatory replication attempt of a study on extrasensory perception.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            The Rules of the Game Called Psychological Science.

            If science were a game, a dominant rule would probably be to collect results that are statistically significant. Several reviews of the psychological literature have shown that around 96% of papers involving the use of null hypothesis significance testing report significant outcomes for their main results but that the typical studies are insufficiently powerful for such a track record. We explain this paradox by showing that the use of several small underpowered samples often represents a more efficient research strategy (in terms of finding p < .05) than does the use of one larger (more powerful) sample. Publication bias and the most efficient strategy lead to inflated effects and high rates of false positives, especially when researchers also resorted to questionable research practices, such as adding participants after intermediate testing. We provide simulations that highlight the severity of such biases in meta-analyses. We consider 13 meta-analyses covering 281 primary studies in various fields of psychology and find indications of biases and/or an excess of significant results in seven. These results highlight the need for sufficiently powerful replications and changes in journal policies.
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              The police officer's dilemma: Using ethnicity to disambiguate potentially threatening individuals.

                Bookmark

                Author and article information

                Journal
                Advances in Methods and Practices in Psychological Science
                Advances in Methods and Practices in Psychological Science
                SAGE Publications
                2515-2459
                2515-2467
                August 23 2018
                August 23 2018
                : 251524591774764
                Affiliations
                [1 ]Organisational Behaviour, University of Sussex Business School
                [2 ]Organisational Behaviour Area, INSEAD Asia Campus
                [3 ]Department of Psychology, University of Virginia
                [4 ]Department of Philosophy, Sociology, Education and Applied Psychology, University of Padua
                [5 ]Department of Psychology, University of Cologne
                [6 ]Department of Management, University of Cincinnati
                [7 ]Department of Management, Faculty of Business Administration, University of Economics, Prague
                [8 ]Department of Management and Marketing, Hong Kong Polytechnic University
                [9 ]Department of Psychology, University of Liverpool
                [10 ]Department of Economics, Stockholm School of Economics
                [11 ]Department of Psychology, Linnaeus University
                [12 ]School of Public Health, University of Hong Kong
                [13 ]Berkeley Institute for Data Science, University of California, Berkeley
                [14 ]Department of Psychology, College of Staten Island, City University of New York
                [15 ]Department of Psychology, New York University
                [16 ]Faculty of Economics and Business, University of Groningen
                [17 ]Division of Neuroscience and Experimental Psychology, University of Manchester
                [18 ]Westat, Rockville, Maryland
                [19 ]Department of Marketing and Supply Chain Management, Temple University
                [20 ]Department of Management and Organizations, Kellogg School of Management, Northwestern University
                [21 ]Department of Psychology, University of Zurich
                [22 ]Washington, D.C.
                [23 ]School of Mathematics and Statistics, University of Sheffield
                [24 ]Swedish Institute for Social Research (SOFI), Stockholm University
                [25 ]Department of Psychology, Ludwig-Maximilians-Universität München
                [26 ]College of Liberal Arts, University of Minnesota
                [27 ]School of Management, Xiamen University
                [28 ]College of Business, Oregon State University
                [29 ]Department of Psychology, Federal University of Santa Catarina
                [30 ]School of Business, University of Washington Bothell
                [31 ]School of Mathematical Sciences, Queen Mary University of London
                [32 ]School of Psychology, University of Nottingham
                [33 ]Department of Psychology, Northwestern University
                [34 ]School of Psychology, Cardiff University
                [35 ]Department of Economics, University of Maryland
                [36 ]Department of Economics, Brigham Young University
                [37 ]Department of Psychology, Loyola University Maryland
                [38 ]Rotman School of Management, University of Toronto
                [39 ]Department of Social Sciences and Cultural Studies, Institute of Sociology, Justus Liebig University, Giessen
                [40 ]United States Military Academy at West Point
                [41 ]Department of Marketing and Management, SUNY Oswego
                [42 ]John Molson School of Business, Concordia University
                [43 ]Lehrstuhl für Soziologie, insb. Sozialstrukturanalyse, Otto-Friedrich-Universität Bamberg
                [44 ]Department of Psychology, University of Sheffield
                [45 ]Department of Psychological Methods, University of Amsterdam
                [46 ]Poznań, Poland
                [47 ]Center for Open Science, Charlottesville, Virginia
                Article
                10.1177/2515245917747646
                © 2018

                Comments

                Comment on this article