9
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Significance

          Accurate prediction of the size and timing of infectious disease outbreaks could help public health officials in planning an appropriate response. This paper compares approaches developed by five different research groups to forecast seasonal influenza outbreaks in real time in the United States. Many of the models show more accurate forecasts than a historical baseline. A major impediment to predictive ability was the real-time accuracy of available data. The field of infectious disease forecasting is in its infancy and we expect that innovation will spur improvements in forecasting in the coming years.

          Abstract

          Influenza infects an estimated 9–35 million individuals each year in the United States and is a contributing cause for between 12,000 and 56,000 deaths annually. Seasonal outbreaks of influenza are common in temperate regions of the world, with highest incidence typically occurring in colder and drier months of the year. Real-time forecasts of influenza transmission can inform public health response to outbreaks. We present the results of a multiinstitution collaborative effort to standardize the collection and evaluation of forecasting models for influenza in the United States for the 2010/2011 through 2016/2017 influenza seasons. For these seven seasons, we assembled weekly real-time forecasts of seven targets of public health interest from 22 different models. We compared forecast accuracy of each model relative to a historical baseline seasonal average. Across all regions of the United States, over half of the models showed consistently better performance than the historical baseline when forecasting incidence of influenza-like illness 1 wk, 2 wk, and 3 wk ahead of available data and when forecasting the timing and magnitude of the seasonal peak. In some regions, delays in data reporting were strongly and negatively associated with forecast accuracy. More timely reporting and an improved overall accessibility to novel and traditional data sources are needed to improve forecasting accuracy and its integration with real-time public health decision making.

          Related collections

          Most cited references31

          • Record: found
          • Abstract: found
          • Article: not found

          Influenza Virus Transmission Is Dependent on Relative Humidity and Temperature

          Introduction Influenza A virus, of the family Orthomyxoviridae, carries an RNA genome consisting of eight segments of negative-stranded RNA. This genome encodes one or two non-structural proteins and nine structural proteins, which, together with a host cell–derived lipid envelope, comprise the influenza virus particle. Influenza virus causes widespread morbidity and mortality among human populations worldwide: in the United States alone, an average of 41,400 deaths and 1.68 million hospitalizations [1] are attributed to influenza each year. In temperate regions like the United States, this impact is felt predominantly during the winter months; that is, epidemics recur with a highly predictable seasonal pattern. In northern latitudes, influenza viruses circulate from November to March, while in the southern hemisphere influenza occurs primarily from May to September [2]. Tropical regions, by contrast, experience influenza throughout the year, although increased incidence has been correlated with rainy seasons [2,3]. Despite extensive documentation of the seasonal cycles of influenza and curiosity as to their causes, little concrete data is available to indicate why influenza virus infections peak in the wintertime. Theories to explain the seasonal variation of influenza have therefore proliferated over the years (reviewed in [4]). Current hypotheses include fluctuations in host immune competence mediated by seasonal factors such as melatonin [5] and vitamin D [6] levels; seasonal changes in host behavior, such as school attendance, air travel [7], and indoor crowding during cold or rainy weather; and environmental factors, including temperature [8], relative humidity (RH), and the direction of air movement in the upper atmosphere [9]. In early studies using mouse-adapted strains of influenza virus, experiments performed in the winter months yielded a transmission rate of 58.2%; in contrast, a rate of only 34.1% was observed in the summer months [10]. While these data suggested that the seasonal influences acting on humans also affect laboratory mice, no mechanism to explain the observations was identified. Herein, we directly tested the hypotheses that ambient air temperature and RH impact the efficiency with which influenza virus is spread. As a mammalian animal model we used Hartley strain guinea pigs, which we have recently shown to be highly susceptible to infection with human influenza viruses [11]. Importantly, we also found that naïve guinea pigs readily become infected when exposed to inoculated guinea pigs, unlike mice, which do not efficiently transmit influenza virus [11]. Thus, by housing infected and naïve guinea pigs together in an environmental chamber, we were able to assess the efficiency of transmission under conditions of controlled RH and temperature. Our data show that both RH and temperature do indeed affect the frequency of influenza virus transmission among guinea pigs, although via apparently differing mechanisms. Results Twenty replicate experiments were performed in which all factors remained constant except for the RH and/or temperature inside the environmental chamber. Each experiment involved eight guinea pigs, and transmission under each set of conditions was assessed in duplicate. The arrangement of animals in the environmental chamber is illustrated in Figure 1. Virus contained in nasal wash samples collected on alternating days post-inoculation (p.i.) was titrated by plaque assay to determine the infection status of each animal. Serum samples were collected from each animal prior to infection and on day 17 p.i., and seroconversion was assessed by hemagglutination inhibition assay (results in Table S1). Figure 1 Arrangement of Infected and Exposed Guinea Pigs in Environmental Chamber In each experiment, eight animals were housed in a Caron 6030 environmental chamber. Each guinea pig was placed in its own cage, and two cages were positioned on each shelf. Naïve animals were placed behind infected animals, such that the direction of airflow was toward the naïve animals. The cages used were open to airflow through the top and one side, both of which were covered by wire mesh. Although infected and exposed guinea pigs were placed in pairs, air flowed freely between shelves, allowing transmission to occur from any infected to any naïve animal. In general, the behavior (level of activity, food and water consumption, symptoms of infection) of guinea pigs was not observed to change with the ambient relative humidity. Likewise, animals housed at 5 °C behaved in a similar manner to those housed at 20 °C. Guinea pigs kept at 30 °C consumed more water than those housed under cooler conditions, and appeared lethargic. Consistent with our previous observations [11], influenza virus–infected guinea pigs did not display detectable symptoms of disease (e.g., weight loss, fever, sneezing, coughing) during the experiments described. Transmission Efficiency Is Dependent on Relative Humidity The results of transmission experiments performed at 20 °C and five different RHs (20%, 35%, 50%, 65%, and 80%) indicated that the efficiency of aerosol spread of influenza virus varied with RH. Transmission was highly efficient (occurred to three or four of four exposed guinea pigs) at low RH values of 20% or 35%. At an intermediate RH of 50%, however, only one of four naïve animals contracted infection. Three of four exposed guinea pigs were infected at 65% RH, while no transmission was observed at a high RH of 80% (Figure 2). Where transmission was observed, the kinetics with which infection was detected in each exposed animal varied between and within experiments. To an extent, we believe this variation is due to the stochastic nature of infection. However, while most infection events were the product of primary transmission from an inoculated animal, others could be the result of secondary transmission from a previously infected, exposed guinea pig. With the exception of the lack of transmission at 80% RH, the observed relationship between transmission and RH is similar to that between influenza virus stability in an aerosol and RH [12], suggesting that at 20 °C the sensitivity of transmission to humidity is due largely to virus stability. Figure 2 Transmission of Influenza Virus from Guinea Pig to Guinea Pig Is Dependent on Relative Humidity Titers of influenza virus in nasal wash samples are plotted as a function of day p.i. Overall transmission rate and the RH and temperature conditions of each experiment are stated underneath the graph. Titers from intranasally inoculated guinea pigs are represented as dashed lines; titers from exposed guinea pigs are shown with solid lines. Virus titrations were performed by plaque assay on Madin Darby canine kidney cells. Transmission Efficiency Is Inversely Correlated with Temperature To test whether cold temperatures would increase transmission, the ambient temperature in the chamber was lowered to 5 °C and experiments were performed at 35%–80% RH. Overall, transmission was more efficient at 5 °C: 75%–100% transmission occurred at 35% and 50% RH, and 50% transmission was observed at 65% and 80% RH (Figure 3A–3H). The statistical significance of differences in transmission rates at 5 °C compared to 20 °C was assessed using the Fisher's exact test. While at 35% and 65% RH the difference was not found to be significant, at both 50% and 80% RH, transmissibility at 5 °C was found to be greater than that at 20 °C (p 20 °C) and either intermediate (50%) or high (80%) RHs. Materials and Methods Virus. Influenza A/Panama/2007/99 virus (Pan/99; H3N2) was kindly supplied by Adolfo García-Sastre and was propagated in Madin Darby canine kidney cells. Animals. Female Hartley strain guinea pigs weighing 300–350 g were obtained from Charles River Laboratories. Animals were allowed free access to food and water and kept on a 12-h light/dark cycle. Guinea pigs were anesthetized for the collection of blood and of nasal wash samples, using a mixture of ketamine (30 mg/kg) and xylazine (2 mg/kg), administered intramuscularly. All procedures were performed in accordance with the Institutional Animal Care and Used Committee guidelines. During guinea pig transmission experiments, strict measures were followed to prevent aberrant cross-contamination between cages: sentinel animals were handled before inoculated animals, gloves were changed between cages, and work surfaces were sanitized between guinea pigs. Transmission experiments. The term “aerosol” is used herein to describe respiratory droplets of all sizes. The term “droplet nuclei” is used to refer to droplets that remain airborne (typically less than 5 μm in diameter). Each transmission experiment involved eight guinea pigs. On day 0, four of the eight guinea pigs were inoculated intranasally with 103 PFU of influenza A/Panama/2007/99 virus (150 μl per nostril in phosphate buffered saline [PBS] supplemented with 0.3% bovine serum albumin [BSA]) and housed in a separate room from the remaining animals. At 24 h p.i., each of the eight guinea pigs was placed in a “transmission cage”, a standard rat cage (Ancare R20 series) with an open wire top, which has been modified by replacing one side panel with a wire grid. The transmission cages were then placed into the environmental chamber (Caron model 6030) with two cages per shelf, such that the wire grids opposed each other (Figure 1). In this arrangement, the guinea pigs cannot come into physical contact with each other. Each infected animal was paired on a shelf with a naïve animal. The guinea pigs were housed in this way for 7 d, after which they were removed from the chamber and separated. On day 2 p.i. (day 1 post-exposure) and every second day thereafter up to day 12 p.i., nasal wash samples were collected from anesthetized guinea pigs by instilling 1 ml of PBS-BSA into the nostrils and collecting the wash in a Petri dish. Titers in nasal wash samples were determined by plaque assay of 10-fold serial dilutions on Madin Darby canine kidney cells. Serum samples were collected from each animal prior to infection and on day 17 post-infection, and seroconversion was assessed by hemagglutination inhibition assay. All transmission experiments reported herein were performed between September 2006 and April 2007. Analysis of expression levels of mediators of innate immunity. Guinea pigs were inoculated with 103 PFU of Pan/99 virus intranasally and immediately housed under the appropriate conditions (5 °C or 20 °C and 35% RH). At days 1, 2, 3, 5, and 7 post-infection, three guinea pigs were killed and their nasal turbinates removed. Tissues were placed immediately in RNAlater reagent (Qiagen), and stored at 4 °C for 1 to 5 d. RNA was extracted from equivalent masses of tissue using the RNAeasy Protect Mini kit (Qiagen) and subjected to DNAse treatment (Qiagen). One microgram of RNA was subjected to reverse transcription using MMLV reverse transcriptase (Roche). One microlitre of the resultant product was used as the template in a SYBR green (Invitrogen) real-time PCR assay (Roche Light Cycler 480) with Ampli-taq Gold polymerase (Perkin-Elmer). Primers used were as follows: β-actin f AAACTGGAACGGTGAAGGTG; β-actin r CTTCCTCTGTGGAGGAGTGG; Mx1 f CATCCCYTTGrTCATCCAGT; Mx1 r CATCCCyTTGRTCATCCAGT; MDA-5 f GAGCCAGAGCTGATGARAGC; MDA-5 r TCTTATGWGCATACTCCTCTGG; IL-1β f GAAGAAGAGCCCATCGTCTG; IL-1β r CATGGGTCAGACAACACCAG; RANTES f GCAATGCTAGCAGCTTCTCC; RANTES r TTGCCTTGAAAGATGTGCTG; TLR3 f TAACCACGCACTCTGTTTGC; TLR3 r ACAGTATTGCGGGATCCAAG; TNFα f TTCCGGGCAGATCTACTTTG; TNFα r TGAACCAGGAGAAGGTGAGG; MCP-1 f ATTGCCAAACTGGACCAGAG; MCP-1 r CTACGGTTCTTGGGGTCTTG; MCP-3 f TCATTGCAGTCCTTCTGTGC; MCP-3 r TAGTCTCTGCACCCGAATCC; IFNγ f GACCTGAGCAAGACCCTGAG; IFNγ r TGGCTCAGAATGCAGAGATG; STAT1 f AAGGGGCCATCACATTCAC; STAT1 r GCTTCCTTTGGCCTGGAG; TBK1 f CAAGAAACTyTGCCwCAGAAA; TBK1 r AGGCCACCATCCAykGTTA; IRF5 f CAAACCCCGaGAGAAGAAG; IRF5 r CTGCTGGGACtGCCAGA; IRF7 f TGCAAGGTGTACTGGGAGGT; IRF7 r TCACCAGGATCAGGGTCTTC (where R = A or G, Y = C or T, W = A or T, K = T or G). Primer sequences were based either on guinea pig mRNA sequences available in GenBank (MCP1, MCP3, IL-1b, IFNγ, RANTES, TLR3, TNFα, and β-actin), or on the consensus sequence of all species available in GenBank (Mx1, MDA-5, IRF5, IRF7, STAT1, and TBK1). Sequencing of each PCR product indicated that all primer pairs were specific for the expected transcript. Reactions were performed in duplicate and normalized by dividing the mean value of the cycle threshold (Ct) of β-actin expressed as an exponent of 2 (2Ct) by the mean value of 2Ct for the target gene. The fold-induction over the mock-infected was then calculated by dividing the normalized value by the normalized mock value. Data is represented in Figure 5 as the mean of three like samples (nasal turbinates harvested on the same day p.i. from three guinea pigs) ± standard deviation. Statistical analyses. Statistical analyses were performed using GraphPad Prism 5 software. Supporting Information Table S1 Seroconversion of Inoculated and Exposed Guinea Pigs Results of hemagglutination inhibition tests for each transmission experiment are shown. (58 KB DOC) Click here for additional data file. Accession Numbers The GenBank (http://www.ncbi.nlm.nih.gov/Genbank/index.html) accession numbers of guinea pig genes used for primer design are as follows: β-actin (AF508792.1); IFNγ (AY151287.1); IL-1β (AF119622); MCP-1 (L04985); MCP-3 (AB014340); RANTES (CPU77037); TLR3 (DQ415679.1); and TNFα (CPU77036).
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Estimating the impact of school closure on influenza transmission from Sentinel data.

            The threat posed by the highly pathogenic H5N1 influenza virus requires public health authorities to prepare for a human pandemic. Although pre-pandemic vaccines and antiviral drugs might significantly reduce illness rates, their stockpiling is too expensive to be practical for many countries. Consequently, alternative control strategies, based on non-pharmaceutical interventions, are a potentially attractive policy option. School closure is the measure most often considered. The high social and economic costs of closing schools for months make it an expensive and therefore controversial policy, and the current absence of quantitative data on the role of schools during influenza epidemics means there is little consensus on the probable effectiveness of school closure in reducing the impact of a pandemic. Here, from the joint analysis of surveillance data and holiday timing in France, we quantify the role of schools in influenza epidemics and predict the effect of school closure during a pandemic. We show that holidays lead to a 20-29% reduction in the rate at which influenza is transmitted to children, but that they have no detectable effect on the contact patterns of adults. Holidays prevent 16-18% of seasonal influenza cases (18-21% in children). By extrapolation, we find that prolonged school closure during a pandemic might reduce the cumulative number of cases by 13-17% (18-23% in children) and peak attack rates by up to 39-45% (47-52% in children). The impact of school closure would be reduced if it proved difficult to maintain low contact rates among children for a prolonged period.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found
              Is Open Access

              The RAPIDD ebola forecasting challenge: Synthesis and lessons learnt.

              Infectious disease forecasting is gaining traction in the public health community; however, limited systematic comparisons of model performance exist. Here we present the results of a synthetic forecasting challenge inspired by the West African Ebola crisis in 2014-2015 and involving 16 international academic teams and US government agencies, and compare the predictive performance of 8 independent modeling approaches. Challenge participants were invited to predict 140 epidemiological targets across 5 different time points of 4 synthetic Ebola outbreaks, each involving different levels of interventions and "fog of war" in outbreak data made available for predictions. Prediction targets included 1-4 week-ahead case incidences, outbreak size, peak timing, and several natural history parameters. With respect to weekly case incidence targets, ensemble predictions based on a Bayesian average of the 8 participating models outperformed any individual model and did substantially better than a null auto-regressive model. There was no relationship between model complexity and prediction accuracy; however, the top performing models for short-term weekly incidence were reactive models with few parameters, fitted to a short and recent part of the outbreak. Individual model outputs and ensemble predictions improved with data accuracy and availability; by the second time point, just before the peak of the epidemic, estimates of final size were within 20% of the target. The 4th challenge scenario - mirroring an uncontrolled Ebola outbreak with substantial data reporting noise - was poorly predicted by all modeling teams. Overall, this synthetic forecasting challenge provided a deep understanding of model performance under controlled data and epidemiological conditions. We recommend such "peace time" forecasting challenges as key elements to improve coordination and inspire collaboration between modeling groups ahead of the next pandemic threat, and to assess model forecasting accuracy for a variety of known and hypothetical pathogens.
                Bookmark

                Author and article information

                Journal
                Proc Natl Acad Sci U S A
                Proc. Natl. Acad. Sci. U.S.A
                pnas
                pnas
                PNAS
                Proceedings of the National Academy of Sciences of the United States of America
                National Academy of Sciences
                0027-8424
                1091-6490
                19 February 2019
                15 January 2019
                15 January 2019
                : 116
                : 8
                : 3146-3154
                Affiliations
                [1] aDepartment of Biostatistics and Epidemiology, University of Massachusetts-Amherst, Amherst, MA 01003;
                [2] bComputer Science Department, Carnegie Mellon University, Pittsburgh, PA, 15213;
                [3] cDepartment of Integrative Biology, University of Texas at Austin, Austin, TX 78712;
                [4] dDepartment of Environmental Health Sciences, Columbia University, New York, NY 10032;
                [5] eInfluenza Division, Centers for Disease Control and Prevention, Atlanta, GA 30333;
                [6] fStatistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM 87545;
                [7] gDepartment of Mathematics and Statistics, Mount Holyoke College, South Hadley, MA 01075;
                [8] hDivision of Vector-Borne Diseases, Centers for Disease Control and Prevention, San Juan, PR 00920;
                [9] iMachine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213
                Author notes
                1To whom correspondence should be addressed. Email: nick@ 123456schoolph.umass.edu .

                Edited by Sebastian Funk, London School of Hygiene & Tropical Medicine, London, United Kingdom, and accepted by Editorial Board Member Diane E. Griffin December 10, 2018 (received for review July 24, 2018)

                Author contributions: N.G.R., L.C.B., S.J.F., S.K., C.J.M., E.M., D.O., E.L.R., A.T., T.K.Y., M.B., M.A.J., R.R., and J.S. designed research; N.G.R., L.C.B., S.J.F., S.K., C.J.M., E.M., D.O., E.L.R., A.T., and T.K.Y. performed research; N.G.R., C.J.M., and E.M. analyzed data; and N.G.R. wrote the paper.

                Author information
                http://orcid.org/0000-0003-3503-9899
                http://orcid.org/0000-0003-1969-3778
                http://orcid.org/0000-0002-6298-0185
                http://orcid.org/0000-0002-5090-7722
                Article
                201812594
                10.1073/pnas.1812594116
                6386665
                30647115
                c23c09a5-62b8-44d3-ae9f-0279b8abe5ea
                Copyright © 2019 the Author(s). Published by PNAS.

                This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).

                History
                Page count
                Pages: 9
                Funding
                Funded by: HHS | NIH | National Institute of General Medical Sciences (NIGMS) 100000057
                Award ID: R35GM119582
                Award Recipient : Nicholas G. Reich Award Recipient : Sasikiran Kandula Award Recipient : Evan Moore Award Recipient : Abhinav Tushar Award Recipient : Jeffrey Shaman
                Funded by: DOD | Defense Advanced Research Projects Agency (DARPA) 100000185
                Award ID: D16AP00144
                Award Recipient : Nicholas G. Reich Award Recipient : Evan L Ray
                Funded by: DOD | Defense Threat Reduction Agency (DTRA) 100000774
                Award ID: HDTRA1-18-C-0008
                Award Recipient : Logan C Brooks Award Recipient : Teresa K Yamana Award Recipient : Roni Rosenfeld Award Recipient : Jeffrey Shaman
                Funded by: Foundation for the National Institutes of Health (FNIH) 100000009
                Award ID: 5U54GM088491
                Award Recipient : Logan C Brooks Award Recipient : Roni Rosenfeld
                Funded by: National Science Foundation (NSF) 100000001
                Award ID: 0946825
                Award ID: DGE-1252522
                Award ID: DGE-1745016
                Award Recipient : Logan C Brooks
                Funded by: Uptake Technologies
                Award ID: NA
                Award Recipient : Logan C Brooks Award Recipient : Roni Rosenfeld
                Funded by: HHS | NIH | National Institute of General Medical Sciences (NIGMS) 100000057
                Award ID: GM110748
                Award Recipient : Nicholas G. Reich Award Recipient : Sasikiran Kandula Award Recipient : Evan Moore Award Recipient : Abhinav Tushar Award Recipient : Jeffrey Shaman
                Funded by: DOD | Defense Threat Reduction Agency (DTRA) 100000774
                Award ID: HDTRA1-15-C-0018
                Award Recipient : Logan C Brooks Award Recipient : Teresa K Yamana Award Recipient : Roni Rosenfeld Award Recipient : Jeffrey Shaman
                Categories
                PNAS Plus
                Biological Sciences
                Medical Sciences
                Physical Sciences
                Statistics
                From the Cover
                PNAS Plus

                influenza,forecasting,statistics,infectious disease,public health

                Comments

                Comment on this article