2
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Better null models for assessing predictive accuracy of disease models

      research-article
      1 , 2 , * , , 3
      PLOS ONE
      Public Library of Science

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Null models provide a critical baseline for the evaluation of predictive disease models. Many studies consider only the grand mean null model (i.e. R 2) when evaluating the predictive ability of a model, which is insufficient to convey the predictive power of a model. We evaluated ten null models for human cases of West Nile virus (WNV), a zoonotic mosquito-borne disease introduced to the United States in 1999. The Negative Binomial, Historical (i.e. using previous cases to predict future cases) and Always Absent null models were the strongest overall, and the majority of null models significantly outperformed the grand mean. The length of the training timeseries increased the performance of most null models in US counties where WNV cases were frequent, but improvements were similar for most null models, so relative scores remained unchanged. We argue that a combination of null models is needed to evaluate the forecasting performance of predictive models for infectious diseases and the grand mean is the lowest bar.

          Related collections

          Most cited references30

          • Record: found
          • Abstract: not found
          • Article: not found

          A simple sequentially rejective multiple test procedure

            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Origin of the West Nile virus responsible for an outbreak of encephalitis in the northeastern United States.

            In late summer 1999, an outbreak of human encephalitis occurred in the northeastern United States that was concurrent with extensive mortality in crows (Corvus species) as well as the deaths of several exotic birds at a zoological park in the same area. Complete genome sequencing of a flavivirus isolated from the brain of a dead Chilean flamingo (Phoenicopterus chilensis), together with partial sequence analysis of envelope glycoprotein (E-glycoprotein) genes amplified from several other species including mosquitoes and two fatal human cases, revealed that West Nile (WN) virus circulated in natural transmission cycles and was responsible for the human disease. Antigenic mapping with E-glycoprotein-specific monoclonal antibodies and E-glycoprotein phylogenetic analysis confirmed these viruses as WN. This North American WN virus was most closely related to a WN virus isolated from a dead goose in Israel in 1998.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Experimental Infection of North American Birds with the New York 1999 Strain of West Nile Virus

              To evaluate transmission dynamics, we exposed 25 bird species to West Nile virus (WNV) by infectious mosquito bite. We monitored viremia titers, clinical outcome, WNV shedding (cloacal and oral), seroconversion, virus persistence in organs, and susceptibility to oral and contact transmission. Passeriform and charadriiform birds were more reservoir competent (a derivation of viremia data) than other species tested. The five most competent species were passerines: Blue Jay (Cyanocitta cristata), Common Grackle (Quiscalus quiscula), House Finch (Carpodacus mexicanus), American Crow (Corvus brachyrhynchos), and House Sparrow (Passer domesticus). Death occurred in eight species. Cloacal shedding of WNV was observed in 17 of 24 species, and oral shedding in 12 of 14 species. We observed contact transmission among four species and oral in five species. Persistent WNV infections were found in tissues of 16 surviving birds. Our observations shed light on transmission ecology of WNV and will benefit surveillance and control programs.
                Bookmark

                Author and article information

                Contributors
                Role: ConceptualizationRole: Data curationRole: Formal analysisRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: SoftwareRole: ValidationRole: VisualizationRole: Writing – original draftRole: Writing – review & editing
                Role: ConceptualizationRole: Formal analysisRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: SupervisionRole: VisualizationRole: Writing – original draftRole: Writing – review & editing
                Role: Editor
                Journal
                PLoS One
                PLoS One
                plos
                PLOS ONE
                Public Library of Science (San Francisco, CA USA )
                1932-6203
                5 May 2023
                2023
                : 18
                : 5
                : e0285215
                Affiliations
                [1 ] Division of Infectious Diseases, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America
                [2 ] Department of Atmospheric and Environmental Sciences, University at Albany, SUNY, Albany, NY, United States of America
                [3 ] Department of Ecology and Evolutionary Biology, University of California, Santa Cruz, Santa Cruz, CA, United States of America
                Gulf University for Science and Technology, KUWAIT
                Author notes

                Competing Interests: The authors have declared that no competing interests exist.

                Author information
                https://orcid.org/0000-0001-5256-6274
                Article
                PONE-D-22-33184
                10.1371/journal.pone.0285215
                10162537
                37146010
                ba7ac567-aa99-4ae6-ae34-9f96c671131f
                © 2023 Keyel, Kilpatrick

                This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                : 2 December 2022
                : 17 April 2023
                Page count
                Figures: 3, Tables: 5, Pages: 11
                Funding
                Funded by: funder-id http://dx.doi.org/10.13039/100000030, Centers for Disease Control and Prevention;
                Award ID: 1U01CK000509-01
                Funded by: funder-id http://dx.doi.org/10.13039/100006492, Division of Intramural Research, National Institute of Allergy and Infectious Diseases;
                Award ID: R01AI168097
                Award Recipient :
                Funded by: funder-id http://dx.doi.org/10.13039/100000001, National Science Foundation;
                Award ID: DEB 1911853
                Award Recipient :
                Funded by: funder-id http://dx.doi.org/10.13039/100000001, National Science Foundation;
                Award ID: DEB-1717498
                Award Recipient :
                Funded by: funder-id http://dx.doi.org/10.13039/100000001, National Science Foundation;
                Award ID: CNH-1115069
                Award Recipient :
                This publication was supported by cooperative agreement 1U01CK000509-01, funded by the Centers for Disease Control and Prevention and by the National Institutes of Health grant R01AI168097 [ACK] and National Science Foundation grants DEB 1911853, DEB-1717498 and CNH-1115069 [AMK]. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
                Categories
                Research Article
                Research and Analysis Methods
                Mathematical and Statistical Techniques
                Mathematical Models
                Biology and life sciences
                Organisms
                Viruses
                RNA viruses
                Flaviviruses
                West Nile virus
                Biology and life sciences
                Microbiology
                Medical microbiology
                Microbial pathogens
                Viral pathogens
                Flaviviruses
                West Nile virus
                Medicine and health sciences
                Pathology and laboratory medicine
                Pathogens
                Microbial pathogens
                Viral pathogens
                Flaviviruses
                West Nile virus
                Biology and life sciences
                Organisms
                Viruses
                Viral pathogens
                Flaviviruses
                West Nile virus
                Physical Sciences
                Mathematics
                Algebra
                Polynomials
                Binomials
                Research and Analysis Methods
                Mathematical and Statistical Techniques
                Statistical Methods
                Forecasting
                Physical Sciences
                Mathematics
                Statistics
                Statistical Methods
                Forecasting
                Medicine and Health Sciences
                Epidemiology
                People and places
                Geographical locations
                North America
                United States
                Physical Sciences
                Mathematics
                Probability Theory
                Probability Distribution
                Medicine and Health Sciences
                Pathology and Laboratory Medicine
                Pathogens
                Custom metadata
                The human neuroinvasive West Nile virus case data set used here is maintained by the Centers for Disease Control Division of Vector-borne Disease ( dvbid2@ 123456cdc.gov ) and is available to qualified researchers, subject to a data use agreement. County population data were obtained from the US Census Bureau and are available in a ready-to-use format in the census.data object from www.github.com/akeyel/wnvdata.

                Uncategorized
                Uncategorized

                Comments

                Comment on this article