Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
;
Evan L
Ray
;
Velma K
Lopez
;
Johannes
Bracher
;
Andrea
Brennen
;
Alvaro J
Castro Rivadeneira
;
Aaron
Gerding
;
Tilmann
Gneiting
;
Katie H
House
;
Yuxin
Huang
;
+285 more...
Dasuni
Jayawardena
;
Abdul H
Kanji
;
Ayush
Khandelwal
;
Khoa
Le
;
Anja
Mühlemann
;
Jarad
Niemi
;
Apurv
Shah
;
Ariane
Stark
;
Yijin
Wang
;
Nutcha
Wattanachit
;
Martha W
Zorn
;
Youyang
Gu
;
Sansiddh
Jain
;
Nayana
Bannur
;
Ayush
Deva
;
Mihir
Kulkarni
;
Srujana
Merugu
;
Alpan
Raval
;
Siddhant
Shingi
;
Avtansh
Tiwari
;
Jerome
White
;
Neil F
Abernethy
;
Spencer
Woody
;
Maytal
Dahan
;
Spencer
Fox
;
Kelly
Gaither
;
Michael
Lachmann
;
Lauren Ancel
Meyers
;
James G
Scott
;
Mauricio
Tec
;
Ajitesh
Srivastava
;
Glover E
George
;
Jeffrey C
Cegan
;
Ian D
Dettwiller
;
William P
England
;
Matthew W
Farthing
;
Robert H
Hunter
;
Brandon
Lafferty
;
Igor
Linkov
;
Michael L
Mayo
;
Matthew D
Parno
;
Michael A
Rowland
;
Benjamin D
Trump
;
Yanli
Zhang-James
;
Samuel
Chen
;
Stephen V
Faraone
;
Jonathan
Hess
;
Christopher P
Morley
;
Asif
Salekin
;
Dongliang
Wang
;
Sabrina M
Corsetti
;
Thomas M
Baer
;
Marisa C
Eisenberg
;
Karl
Falb
;
Yitao
Huang
;
Emily T
Martin
;
Ella
McCauley
;
Robert L
Myers
;
Tom
Schwarz
;
Daniel
Sheldon
;
Graham Casey
Gibson
;
Rose
Yu
;
Liyao
Gao
;
Yian
Ma
;
Dongxia
Wu
;
Xifeng
Yan
;
Xiaoyong
Jin
;
Yu-Xiang
Wang
;
YangQuan
Chen
;
Lihong
Guo
;
Yanting
Zhao
;
Quanquan
Gu
;
Jinghui
Chen
;
Lingxiao
Wang
;
Pan
Xu
;
Weitong
Zhang
;
Difan
Zou
;
Hannah
Biegel
;
Joceline
Lega
;
Steve
McConnell
;
VP
Nagraj
;
Stephanie L
Guertin
;
Christopher
Hulme-Lowe
;
Stephen D
Turner
;
Yunfeng
Shi
;
Xuegang
Ban
;
Robert
Walraven
;
Qi-Jun
Hong
;
Stanley
Kong
;
Axel
van de Walle
;
James A
Turtle
;
Michal
Ben-Nun
;
Steven
Riley
;
Pete
Riley
;
Ugur
Koyluoglu
;
David
DesRoches
;
Pedro
Forli
;
Bruce
Hamory
;
Christina
Kyriakides
;
Helen
Leis
;
John
Milliken
;
Michael
Moloney
;
James
Morgan
;
Ninad
Nirgudkar
;
Gokce
Ozcan
;
Noah
Piwonka
;
Matt
Ravi
;
Chris
Schrader
;
Elizabeth
Shakhnovich
;
Daniel
Siegel
;
Ryan
Spatz
;
Chris
Stiefeling
;
Barrie
Wilkinson
;
Alexander
Wong
;
Sean
Cavany
;
Guido
España
;
Sean
Moore
;
Rachel
Oidtman
;
Alex
Perkins
;
David
Kraus
;
Andrea
Kraus
;
Zhifeng
Gao
;
Jiang
Bian
;
Wei
Cao
;
Juan
Lavista Ferres
;
Chaozhuo
Li
;
Tie-Yan
Liu
;
Xing
Xie
;
Shun
Zhang
;
Shun
Zheng
;
Alessandro
Vespignani
;
Matteo
Chinazzi
;
Jessica T
Davis
;
Kunpeng
Mu
;
Ana
Pastore Y Piontti
;
Xinyue
Xiong
;
Andrew
Zheng
;
Jackie
Baek
;
Vivek
Farias
;
Andreea
Georgescu
;
Retsef
Levi
;
Deeksha
Sinha
;
Joshua
Wilde
;
Georgia
Perakis
;
Mohammed Amine
Bennouna
;
David
Nze-Ndong
;
Divya
Singhvi
;
Ioannis
Spantidakis
;
Leann
Thayaparan
;
Asterios
Tsiourvas
;
Arnab
Sarker
;
Ali
Jadbabaie
;
Devavrat
Shah
;
Nicolas
Della Penna
;
Leo A
Celi
;
Saketh
Sundar
;
Russ
Wolfinger
;
Dave
Osthus
;
Lauren
Castro
;
Geoffrey
Fairchild
;
Isaac
Michaud
;
Dean
Karlen
;
Matt
Kinsey
;
Luke C
Mullany
;
Kaitlin
Rainwater-Lovett
;
Lauren
Shin
;
Katharine
Tallaksen
;
Shelby
Wilson
;
Elizabeth C
Lee
;
Juan
Dent
;
Kyra H
Grantz
;
Alison L
Hill
;
Joshua
Kaminsky
;
Kathryn
Kaminsky
;
Lindsay T
Keegan
;
Stephen A
Lauer
;
Joseph C
Lemaitre
;
Justin
Lessler
;
Hannah R
Meredith
;
Javier
Perez-Saez
;
Sam
Shah
;
Claire P
Smith
;
Shaun A
Truelove
;
Josh
Wills
;
Maximilian
Marshall
;
Lauren
Gardner
;
Kristen
Nixon
;
John C
Burant
;
Lily
Wang
;
Lei
Gao
;
Zhiling
Gu
;
Myungjin
Kim
;
Xinyi
Li
;
Guannan
Wang
;
Yueying
Wang
;
Shan
Yu
;
Robert C
Reiner
;
Ryan
Barber
;
Emmanuela
Gakidou
;
Simon I
Hay
;
Steve
Lim
;
Chris
Murray
;
David
Pigott
;
Heidi L
Gurung
;
Prasith
Baccam
;
Steven A
Stage
;
Bradley T
Suchoski
;
B Aditya
Prakash
;
Bijaya
Adhikari
;
Jiaming
Cui
;
Alexander
Rodríguez
;
Anika
Tabassum
;
Jiajia
Xie
;
Pinar
Keskinocak
;
John
Asplund
;
Arden
Baxter
;
Buse Eylul
Oruc
;
Nicoleta
Serban
;
Sercan O
Arik
;
Mike
Dusenberry
;
Arkady
Epshteyn
;
Elli
Kanal
;
Long T
Le
;
Chun-Liang
Li
;
Tomas
Pfister
;
Dario
Sava
;
Rajarishi
Sinha
;
Thomas
Tsai
;
Nate
Yoder
;
Jinsung
Yoon
;
Leyou
Zhang
;
Sam
Abbott
;
Nikos I
Bosse
;
Sebastian
Funk
;
Joel
Hellewell
;
Sophie R
Meakin
;
Katharine
Sherratt
;
Mingyuan
Zhou
;
Rahi
Kalantari
;
Teresa K
Yamana
;
Sen
Pei
;
Jeffrey
Shaman
;
Michael L
Li
;
Dimitris
Bertsimas
;
Omar
Skali Lami
;
Saksham
Soni
;
Hamza
Tazi Bouardi
;
Turgay
Ayer
;
Madeline
Adee
;
Jagpreet
Chhatwal
;
Ozden O
Dalgic
;
Mary A
Ladd
;
Benjamin P
Linas
;
Peter
Mueller
;
Jade
Xiao
;
Yuanjia
Wang
;
Qinxia
Wang
;
Shanghong
Xie
;
Donglin
Zeng
;
Alden
Green
;
Jacob
Bien
;
Logan
Brooks
;
Addison J
Hu
;
Maria
Jahja
;
Daniel
McDonald
;
Balasubramanian
Narasimhan
;
Collin
Politsch
;
Samyak
Rajanala
;
Aaron
Rumack
;
Noah
Simon
;
Ryan J
Tibshirani
;
Rob
Tibshirani
;
Valerie
Ventura
;
Larry
Wasserman
;
Eamon B
O'Dea
;
John M
Drake
;
Robert
Pagano
;
Quoc T
Tran
;
Lam Si Tung
Ho
;
Huong
Huynh
;
Jo W
Walker
;
Rachel B
Slayton
;
Michael A
Johansson
;
Matthew
Biggerstaff
;
Nicholas G
Reich
;
(2022)
Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
Proceedings of the National Academy of Sciences, 119 (15).
e2113561119-.
ISSN 0027-8424
DOI: 10.1073/pnas.2113561119
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.
-
picture_as_pdf - Cramer_etal_2021_Evaluation-of-individual-and-ensemble.pdf
-
subject - Published Version
-
- Available under Creative Commons: Attribution 4.0