Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
; Ray, Evan L; Lopez, Velma K; Bracher, Johannes; Brennen, Andrea; Castro Rivadeneira, Alvaro J; Gerding, Aaron; Gneiting, Tilmann; House, Katie H; Huang, Yuxin; +285 more...Jayawardena, Dasuni; Kanji, Abdul H; Khandelwal, Ayush; Le, Khoa; Mühlemann, Anja; Niemi, Jarad; Shah, Apurv; Stark, Ariane; Wang, Yijin; Wattanachit, Nutcha; Zorn, Martha W; Gu, Youyang; Jain, Sansiddh; Bannur, Nayana; Deva, Ayush; Kulkarni, Mihir; Merugu, Srujana; Raval, Alpan; Shingi, Siddhant; Tiwari, Avtansh; White, Jerome; Abernethy, Neil F; Woody, Spencer; Dahan, Maytal; Fox, Spencer; Gaither, Kelly; Lachmann, Michael; Meyers, Lauren Ancel; Scott, James G; Tec, Mauricio; Srivastava, Ajitesh; George, Glover E; Cegan, Jeffrey C; Dettwiller, Ian D; England, William P; Farthing, Matthew W; Hunter, Robert H; Lafferty, Brandon; Linkov, Igor; Mayo, Michael L; Parno, Matthew D; Rowland, Michael A; Trump, Benjamin D; Zhang-James, Yanli; Chen, Samuel; Faraone, Stephen V; Hess, Jonathan; Morley, Christopher P; Salekin, Asif; Wang, Dongliang; Corsetti, Sabrina M; Baer, Thomas M; Eisenberg, Marisa C; Falb, Karl; Huang, Yitao; Martin, Emily T; McCauley, Ella; Myers, Robert L; Schwarz, Tom; Sheldon, Daniel; Gibson, Graham Casey; Yu, Rose; Gao, Liyao; Ma, Yian; Wu, Dongxia; Yan, Xifeng; Jin, Xiaoyong; Wang, Yu-Xiang; Chen, YangQuan; Guo, Lihong; Zhao, Yanting; Gu, Quanquan; Chen, Jinghui; Wang, Lingxiao; Xu, Pan; Zhang, Weitong; Zou, Difan; Biegel, Hannah; Lega, Joceline; McConnell, Steve; Nagraj, VP; Guertin, Stephanie L; Hulme-Lowe, Christopher; Turner, Stephen D; Shi, Yunfeng; Ban, Xuegang; Walraven, Robert; Hong, Qi-Jun; Kong, Stanley; van de Walle, Axel; Turtle, James A; Ben-Nun, Michal; Riley, Steven; Riley, Pete; Koyluoglu, Ugur; DesRoches, David; Forli, Pedro; Hamory, Bruce; Kyriakides, Christina; Leis, Helen; Milliken, John; Moloney, Michael; Morgan, James; Nirgudkar, Ninad; Ozcan, Gokce; Piwonka, Noah; Ravi, Matt; Schrader, Chris; Shakhnovich, Elizabeth; Siegel, Daniel; Spatz, Ryan; Stiefeling, Chris; Wilkinson, Barrie; Wong, Alexander; Cavany, Sean; España, Guido; Moore, Sean; Oidtman, Rachel; Perkins, Alex; Kraus, David; Kraus, Andrea; Gao, Zhifeng; Bian, Jiang; Cao, Wei; Lavista Ferres, Juan; Li, Chaozhuo; Liu, Tie-Yan; Xie, Xing; Zhang, Shun; Zheng, Shun; Vespignani, Alessandro; Chinazzi, Matteo; Davis, Jessica T; Mu, Kunpeng; Pastore Y Piontti, Ana; Xiong, Xinyue; Zheng, Andrew; Baek, Jackie; Farias, Vivek; Georgescu, Andreea; Levi, Retsef; Sinha, Deeksha; Wilde, Joshua; Perakis, Georgia; Bennouna, Mohammed Amine; Nze-Ndong, David; Singhvi, Divya; Spantidakis, Ioannis; Thayaparan, Leann; Tsiourvas, Asterios; Sarker, Arnab; Jadbabaie, Ali; Shah, Devavrat; Della Penna, Nicolas; Celi, Leo A; Sundar, Saketh; Wolfinger, Russ; Osthus, Dave; Castro, Lauren; Fairchild, Geoffrey; Michaud, Isaac; Karlen, Dean; Kinsey, Matt; Mullany, Luke C; Rainwater-Lovett, Kaitlin; Shin, Lauren; Tallaksen, Katharine; Wilson, Shelby; Lee, Elizabeth C; Dent, Juan; Grantz, Kyra H; Hill, Alison L; Kaminsky, Joshua; Kaminsky, Kathryn; Keegan, Lindsay T; Lauer, Stephen A; Lemaitre, Joseph C; Lessler, Justin; Meredith, Hannah R; Perez-Saez, Javier; Shah, Sam; Smith, Claire P; Truelove, Shaun A; Wills, Josh; Marshall, Maximilian; Gardner, Lauren; Nixon, Kristen; Burant, John C; Wang, Lily; Gao, Lei; Gu, Zhiling; Kim, Myungjin; Li, Xinyi; Wang, Guannan; Wang, Yueying; Yu, Shan; Reiner, Robert C; Barber, Ryan; Gakidou, Emmanuela; Hay, Simon I; Lim, Steve; Murray, Chris; Pigott, David; Gurung, Heidi L; Baccam, Prasith; Stage, Steven A; Suchoski, Bradley T; Prakash, B Aditya; Adhikari, Bijaya; Cui, Jiaming; Rodríguez, Alexander; Tabassum, Anika; Xie, Jiajia; Keskinocak, Pinar; Asplund, John; Baxter, Arden; Oruc, Buse Eylul; Serban, Nicoleta; Arik, Sercan O; Dusenberry, Mike; Epshteyn, Arkady; Kanal, Elli; Le, Long T; Li, Chun-Liang; Pfister, Tomas; Sava, Dario; Sinha, Rajarishi; Tsai, Thomas; Yoder, Nate; Yoon, Jinsung; Zhang, Leyou; Abbott, Sam; Bosse, Nikos I; Funk, Sebastian; Hellewell, Joel; Meakin, Sophie R; Sherratt, Katharine; Zhou, Mingyuan; Kalantari, Rahi; Yamana, Teresa K; Pei, Sen; Shaman, Jeffrey; Li, Michael L; Bertsimas, Dimitris; Skali Lami, Omar; Soni, Saksham; Tazi Bouardi, Hamza; Ayer, Turgay; Adee, Madeline; Chhatwal, Jagpreet; Dalgic, Ozden O; Ladd, Mary A; Linas, Benjamin P; Mueller, Peter; Xiao, Jade; Wang, Yuanjia; Wang, Qinxia; Xie, Shanghong; Zeng, Donglin; Green, Alden; Bien, Jacob; Brooks, Logan; Hu, Addison J; Jahja, Maria; McDonald, Daniel; Narasimhan, Balasubramanian; Politsch, Collin; Rajanala, Samyak; Rumack, Aaron; Simon, Noah; Tibshirani, Ryan J; Tibshirani, Rob; Ventura, Valerie; Wasserman, Larry; O'Dea, Eamon B; Drake, John M; Pagano, Robert; Tran, Quoc T; Ho, Lam Si Tung; Huynh, Huong; Walker, Jo W; Slayton, Rachel B; Johansson, Michael A; Biggerstaff, Matthew; and Reich, Nicholas G
(2022)
Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
Proceedings of the National Academy of Sciences, 119 (15).
e2113561119-.
ISSN 0027-8424
DOI: 10.1073/pnas.2113561119
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.
-
picture_as_pdf - Cramer_etal_2021_Evaluation-of-individual-and-ensemble.pdf
-
subject - Published Version
-
- Available under Creative Commons: Attribution 4.0