"Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source" by Alberto Barrón-Cedeño

Article

Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Alberto Barrón-Cedeño, Alma Mater Studiorum Università di Bologna
Firoj Alam, Qatar Computing Research Institute
Andrea Galassi, Alma Mater Studiorum Università di Bologna
Giovanni Da San Martino, Università degli Studi di Padova
Preslav Nakov, Mohamed Bin Zayed University of Artificial Intelligence
Tamer Elsayed, Qatar University
Dilshod Azizov, Qatar University
Tommaso Caselli, Rijksuniversiteit Groningen
Gullal S. Cheema, Forschungszentrum L3S
Fatima Haouari, Qatar University
Maram Hasanain, Qatar Computing Research Institute
Mucahid Kutlu, TOBB University of Economics and Technology
Chengkai Li, The University of Texas at Arlington
Federico Ruggeri, Alma Mater Studiorum Università di Bologna
Julia Maria Struß, Fachhochschule Potsdam
Wajdi Zaghouani, Hamad Bin Khalifa University

Link

Document Type

Conference Proceeding

Abstract

We describe the sixth edition of the CheckThat! lab, part of the 2023 Conference and Labs of the Evaluation Forum (CLEF). The five previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, verifying whether a claim was fact-checked before, supporting evidence retrieval, and claim verification. In this sixth edition, we zoom into some new problems and for the first time we offer five tasks in seven languages: Arabic, Dutch, English, German, Italian, Spanish, and Turkish. Task 1 asks to determine whether an item —text or text plus image— is check-worthy. Task 2 aims to predict whether a sentence from a news article is subjective or not. Task 3 asks to assess the political bias of the news at the article and at the media outlet level. Task 4 focuses on the factuality of reporting of news media. Finally, Task 5 looks at identifying authorities in Twitter that could help verify a given target claim. For a second year, CheckThat! was the most popular lab at CLEF-2023 in terms of team registrations: 127 teams. About one-third of them (a total of 37) actually participated.

DOI

10.1007/978-3-031-42448-9_20

Publication Date

9-11-2023

Keywords

Authority Finding,
Check-Worthiness,
Fact Checking,
Factuality of Reporting,
Political Bias,
Subjectivity

Disciplines

Comments

IR conditions: non-described

Citation Information

A. Barrón-Cedeño et al., “Overview of the clef–2023 checkthat! lab on checkworthiness, subjectivity, political bias, factuality, and authority of news articles and their source,” Lecture Notes in Computer Science, pp. 251–275, Sept 2023. doi:10.1007/978-3-031-42448-9_20