TY - JOUR
T1 - Searching for scientific evidence in a pandemic
T2 - An overview of TREC-COVID
AU - Roberts, Kirk
AU - Alam, Tasmeer
AU - Bedrick, Steven
AU - Demner-Fushman, Dina
AU - Lo, Kyle
AU - Soboroff, Ian
AU - Voorhees, Ellen
AU - Wang, Lucy Lu
AU - Hersh, William R.
N1 - Publisher Copyright:
© 2021
PY - 2021/9
Y1 - 2021/9
N2 - We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics (sets of related queries) were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging topics at that state of the still-emerging pandemic. This paper provides a comprehensive overview of the structure and results of TREC-COVID. Specifically, the paper provides details on the background, task structure, topic structure, corpus, participation, pooling, assessment, judgments, results, top-performing systems, lessons learned, and benchmark datasets.
AB - We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics (sets of related queries) were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging topics at that state of the still-emerging pandemic. This paper provides a comprehensive overview of the structure and results of TREC-COVID. Specifically, the paper provides details on the background, task structure, topic structure, corpus, participation, pooling, assessment, judgments, results, top-performing systems, lessons learned, and benchmark datasets.
KW - COVID-19
KW - Information retrieval
KW - Pandemics
KW - TREC-COVID
UR - http://www.scopus.com/inward/record.url?scp=85111810459&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85111810459&partnerID=8YFLogxK
U2 - 10.1016/j.jbi.2021.103865
DO - 10.1016/j.jbi.2021.103865
M3 - Article
C2 - 34245913
AN - SCOPUS:85111810459
SN - 1532-0464
VL - 121
JO - Journal of Biomedical Informatics
JF - Journal of Biomedical Informatics
M1 - 103865
ER -