9
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Which method is optimal for estimating variance components and their variability in generalizability theory? evidence form a set of unified rules for bootstrap method

      research-article
      * ,
      PLOS ONE
      Public Library of Science

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Objective

          The purpose of this study is to compare the performance of the four estimation methods (traditional method, jackknife method, bootstrap method, and MCMC method), find the optimal one, and make a set of unified rules for Bootstrap.

          Methods

          Based on four types of simulated data (normal, dichotomous, polytomous, and skewed data), this study estimates and compares the estimated variance components and their variability of the four estimation methods when using a p×i design in generalizability theory. The estimated variance components are vc.p, vc.i and vc.pi and the variability of estimated variance components are their estimated standard errors (SE(vc.p), SE(vc.i) and SE(vc.pi)) and confidence intervals (CI(vc.p), CI(vc.i) and CI(vc.pi)).

          Results

          For the normal data, all the four methods can accurately estimate the variance components and their variability. For the dichotomous data, the |RPB| of SE (vc.i) of traditional method is 128.5714, the |RPB| of SE (vc.i), SE (vc.pi) and CI (vc.i) of jackknife method are 42.8571, 43.6893 and 40.5000, which are larger than 25 and not accurate. For the polytomous data, the |RPB| of SE (vc.i) and CI (vc.i) of MCMC method are 59.6612 and 45.2500, which are larger than 25 and not accurate. For the skewed data, the |RPB| of SE (vc.p), SE (vc.i) and SE (vc. pi) of traditional method and MCMC method are over 25, which are not accurate. Only the bootstrap method can estimate variance components and their variability accurately across different data distribution. Nonetheless, the divide-and-conquer strategy must be used when adopting the bootstrap method.

          Conclusions

          The bootstrap method is optimal among the four methods and shows the cross-distribution superiority over the other three methods. However, a set of unified rules for the divide-and-conquer strategy need to be recommended for the bootstrap method, which is optimal when boot-p for p (person), boot-pi for i (item), and boot-i for pi (person × item).

          Related collections

          Most cited references35

          • Record: found
          • Abstract: not found
          • Article: not found

          Bias and confidence in not quite large samples

            Bookmark
            • Record: found
            • Abstract: not found
            • Book: not found

            Generalizability Theory

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses

                Bookmark

                Author and article information

                Contributors
                Role: ConceptualizationRole: Data curationRole: Formal analysisRole: InvestigationRole: MethodologyRole: Project administrationRole: SoftwareRole: SupervisionRole: ValidationRole: Writing – original draftRole: Writing – review & editing
                Role: Editor
                Journal
                PLoS One
                PLoS One
                plos
                PLOS ONE
                Public Library of Science (San Francisco, CA USA )
                1932-6203
                14 July 2023
                2023
                : 18
                : 7
                : e0288069
                Affiliations
                [001] School of Psychology, Center for Studies of Psychological Application, Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, 510631, China
                Zhejiang Normal University, CHINA
                Author notes

                Competing Interests: The authors have declared that no competing interests exist.

                Author information
                https://orcid.org/0000-0002-2526-8682
                Article
                PONE-D-23-10415
                10.1371/journal.pone.0288069
                10348584
                37450506
                72ede376-2fb0-4294-8c70-5c78929b2777
                © 2023 Guangming Li

                This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                : 6 April 2023
                : 17 June 2023
                Page count
                Figures: 0, Tables: 6, Pages: 18
                Funding
                Funded by: funder-id http://dx.doi.org/10.13039/501100003453, Natural Science Foundation of Guangdong Province;
                Award ID: 2021A1515012516
                Award Recipient :
                Funded by: Characteristic Innovation Project of Colleges and Universities in Guangdong Province (Philosophy and Social Science of Educational Science)
                Award ID: 2021WTSCX020
                Award Recipient :
                This research was supported in part by Grant No. 2021A1515012516 from the Natural Science Foundation of Guangdong Province and Grant No. 2021WTSCX020 from the Characteristic Innovation Project of Colleges and Universities in Guangdong Province (Philosophy and Social Science of Educational Science).
                Categories
                Research Article
                Physical Sciences
                Mathematics
                Probability Theory
                Probability Distribution
                Normal Distribution
                Physical Sciences
                Mathematics
                Probability Theory
                Statistical Distributions
                Physical Sciences
                Mathematics
                Probability Theory
                Probability Distribution
                Skewness
                Physical Sciences
                Mathematics
                Statistics
                Statistical Data
                Research and analysis methods
                Mathematical and statistical techniques
                Statistical methods
                Monte Carlo method
                Physical sciences
                Mathematics
                Statistics
                Statistical methods
                Monte Carlo method
                Physical Sciences
                Mathematics
                Probability Theory
                Markov Models
                Research and Analysis Methods
                Simulation and Modeling
                Physical Sciences
                Mathematics
                Statistics
                Statistical Theories
                Custom metadata
                All relevant data are within the paper and its Supporting Information files.

                Uncategorized
                Uncategorized

                Comments

                Comment on this article