Skip to main content
Article
Overview of the CLEF-2023 CheckThat! Lab Task 1 on Check-Worthiness of Multimodal and Multigenre Content
CEUR Workshop Proceedings
  • Firoj Alam, Qatar Computing Research Institute
  • Alberto Barrón-Cedeño, Alma Mater Studiorum Università di Bologna
  • Gullal S. Cheema, Forschungszentrum L3S
  • Gautam Kishore Shahi, Universität Duisburg-Essen
  • Sherzod Hakimov, Universität Potsdam
  • Maram Hasanain, Qatar Computing Research Institute
  • Chengkai Li, The University of Texas at Arlington
  • Rubén Míguez, Newtral Media Audiovisual
  • Hamdy Mubarak, Qatar Computing Research Institute
  • Wajdi Zaghouani, Hamad Bin Khalifa University
  • Preslav Nakov, Mohamed Bin Zayed University of Artificial Intelligence
Document Type
Conference Proceeding
Abstract

We present an overview of CheckThat! Lab’s 2023 Task 1, which is part of CLEF-2023. Task 1 asks to determine whether a text item, or a text coupled with an image, is check-worthy. This task places a special emphasis on COVID-19, political debates and transcriptions, and it is conducted in three languages: Arabic, English, and Spanish. A total of 15 teams participated, and most submissions managed to achieve significant improvements over the baselines using Transformer-based models. Out of these, seven teams participated in the multimodal subtask (1A), and 12 teams participated in the Multigenre subtask (1B), collectively submitting 155 official runs for both subtasks. Across both subtasks, approaches that targeted multiple languages, either individually or in conjunction, generally achieved the best performance. We provide a description of the dataset and the task setup, including the evaluation settings, and we briefly overview the participating systems. As is customary in the CheckThat! lab, we have release all datasets from the lab as well as the evaluation scripts to the research community. This will enable further research on finding relevant check-worthy content that can assist various stakeholders such as fact-checkers, journalists, and policymakers.

Publication Date
9-1-2023
Keywords
  • Check-worthiness,
  • fact-checking,
  • multilinguality,
  • multimodality
Comments

Archived thanks to CEUR-WS.org

Open Access

License: CC by 4.0

Uploaded: April 03, 2024

Citation Information
Firoj Alam, Alberto Barrón-Cedeño, Gullal S. Cheema, Gautam Kishore Shahi, et al.. "Overview of the CLEF-2023 CheckThat! Lab Task 1 on Check-Worthiness of Multimodal and Multigenre Content" CEUR Workshop Proceedings Vol. 3497 (2023) p. 219 - 235 ISSN: 16130073
Available at: http://works.bepress.com/preslav-nakov/33/