Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
; Ray, EL; Lopez, VK; Bracher, J; Brennen, A; Castro Rivadeneira, AJ; Gerding, A; Gneiting, T; House, KH; Huang, Y; +285 more...Jayawardena, D; Kanji, AH; Khandelwal, A; Le, K; Mühlemann, A; Niemi, J; Shah, A; Stark, A; Wang, Y; Wattanachit, N; Zorn, MW; Gu, Y; Jain, S; Bannur, N; Deva, A; Kulkarni, M; Merugu, S; Raval, A; Shingi, S; Tiwari, A; White, J; Abernethy, NF; Woody, S; Dahan, M; Fox, S; Gaither, K; Lachmann, M; Meyers, LA; Scott, JG; Tec, M; Srivastava, A; George, GE; Cegan, JC; Dettwiller, ID; England, WP; Farthing, MW; Hunter, RH; Lafferty, B; Linkov, I; Mayo, ML; Parno, MD; Rowland, MA; Trump, BD; Zhang-James, Y; Chen, S; Faraone, SV; Hess, J; Morley, CP; Salekin, A; Wang, D; Corsetti, SM; Baer, TM; Eisenberg, MC; Falb, K; Huang, Y; Martin, ET; McCauley, E; Myers, RL; Schwarz, T; Sheldon, D; Gibson, GC; Yu, R; Gao, L; Ma, Y; Wu, D; Yan, X; Jin, X; Wang, Y; Chen, Y; Guo, L; Zhao, Y; Gu, Q; Chen, J; Wang, L; Xu, P; Zhang, W; Zou, D; Biegel, H; Lega, J; McConnell, S; Nagraj, V; Guertin, SL; Hulme-Lowe, C; Turner, SD; Shi, Y; Ban, X; Walraven, R; Hong, Q; Kong, S; van de Walle, A; Turtle, JA; Ben-Nun, M; Riley, S; Riley, P; Koyluoglu, U; DesRoches, D; Forli, P; Hamory, B; Kyriakides, C; Leis, H; Milliken, J; Moloney, M; Morgan, J; Nirgudkar, N; Ozcan, G; Piwonka, N; Ravi, M; Schrader, C; Shakhnovich, E; Siegel, D; Spatz, R; Stiefeling, C; Wilkinson, B; Wong, A; Cavany, S; España, G; Moore, S; Oidtman, R; Perkins, A; Kraus, D; Kraus, A; Gao, Z; Bian, J; Cao, W; Lavista Ferres, J; Li, C; Liu, T; Xie, X; Zhang, S; Zheng, S; Vespignani, A; Chinazzi, M; Davis, JT; Mu, K; Pastore Y Piontti, A; Xiong, X; Zheng, A; Baek, J; Farias, V; Georgescu, A; Levi, R; Sinha, D; Wilde, J; Perakis, G; Bennouna, MA; Nze-Ndong, D; Singhvi, D; Spantidakis, I; Thayaparan, L; Tsiourvas, A; Sarker, A; Jadbabaie, A; Shah, D; Della Penna, N; Celi, LA; Sundar, S; Wolfinger, R; Osthus, D; Castro, L; Fairchild, G; Michaud, I; Karlen, D; Kinsey, M; Mullany, LC; Rainwater-Lovett, K; Shin, L; Tallaksen, K; Wilson, S; Lee, EC; Dent, J; Grantz, KH; Hill, AL; Kaminsky, J; Kaminsky, K; Keegan, LT; Lauer, SA; Lemaitre, JC; Lessler, J; Meredith, HR; Perez-Saez, J; Shah, S; Smith, CP; Truelove, SA; Wills, J; Marshall, M; Gardner, L; Nixon, K; Burant, JC; Wang, L; Gao, L; Gu, Z; Kim, M; Li, X; Wang, G; Wang, Y; Yu, S; Reiner, RC; Barber, R; Gakidou, E; Hay, SI; Lim, S; Murray, C; Pigott, D; Gurung, HL; Baccam, P; Stage, SA; Suchoski, BT; Prakash, BA; Adhikari, B; Cui, J; Rodríguez, A; Tabassum, A; Xie, J; Keskinocak, P; Asplund, J; Baxter, A; Oruc, BE; Serban, N; Arik, SO; Dusenberry, M; Epshteyn, A; Kanal, E; Le, LT; Li, C; Pfister, T; Sava, D; Sinha, R; Tsai, T; Yoder, N; Yoon, J; Zhang, L; Abbott, S; Bosse, NI; Funk, S; Hellewell, J; Meakin, SR; Sherratt, K; Zhou, M; Kalantari, R; Yamana, TK; Pei, S; Shaman, J; Li, ML; Bertsimas, D; Skali Lami, O; Soni, S; Tazi Bouardi, H; Ayer, T; Adee, M; Chhatwal, J; Dalgic, OO; Ladd, MA; Linas, BP; Mueller, P; Xiao, J; Wang, Y; Wang, Q; Xie, S; Zeng, D; Green, A; Bien, J; Brooks, L; Hu, AJ; Jahja, M; McDonald, D; Narasimhan, B; Politsch, C; Rajanala, S; Rumack, A; Simon, N; Tibshirani, RJ; Tibshirani, R; Ventura, V; Wasserman, L; O'Dea, EB; Drake, JM; Pagano, R; Tran, QT; Ho, LST; Huynh, H; Walker, JW; Slayton, RB; Johansson, MA; Biggerstaff, M; Reich, NG and
(2022)
Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
Proceedings of the National Academy of Sciences, 119 (15).
e2113561119-.
ISSN 0027-8424
DOI: 10.1073/pnas.2113561119
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.
-
picture_as_pdf - Cramer_etal_2021_Evaluation-of-individual-and-ensemble.pdf
-
subject - Published Version
-
- Available under Creative Commons: Attribution 4.0