Publications

Here we include the publications that overview the overall 2022 edition of the CheckThat! lab. If you are interested in one specific approach, visit the CheckThat! section at the CLEF 2022 CEUR proceedings

ECIR 2022 paper

Nakov et al. The CLEF-2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection. In Advances in Information Retrieval. ECIR 2022. Lecture Notes in Computer Science, vol 13186. Springer, Cham.

Overview of the whole CheckThat! 2022 (CLEF 2022 LNCS)

Nakov et al. Overview of the CLEF–2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. Lecture Notes in Computer Science, vol 13390. Springer, Cham.

Overview of Task 1

Nakov et al. Overview of the CLEF-2022 CheckThat! Lab Task 1 on Identifying Relevant Claims in Tweets. In Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. Bologna, Italy

Overview of Task 2

Nakov et al. Overview of the CLEF-2022 CheckThat! Lab Task 2 on Detecting Previously Fact-Checked Claims. In Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. Bologna, Italy

Overview of Task 3

Köhler etal. Overview of the CLEF-2022 CheckThat! Lab: Task 3 on Fake News Detection. In Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. Bologna, Italy

Main Proceedings: Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum

• Task 1 paper: http://ceur-ws.org/Vol-3180/paper-28.pdf
• Task 2 paper: http://ceur-ws.org/Vol-3180/paper-29.pdf
• Task 3 paper: http://ceur-ws.org/Vol-3180/paper-30.pdf

Lab program

This year, lab activities will run on 6 and 7 September. All times follow CET

Plenary talk overview: 6th September, 10:00-12:30

Session 1: 7th September, 8:50-10:20 (Chair: Alberto Barrón-Cedeño)

Session 2: 7th September, 13:30-15:00 (Chair: Giovanni Da San Martino)

Session 3: 7th September, 15:30-17:00 (Chair: Preslav Nakov)

Giovanni Zagni is the Director of Pagella Politica and of facta.

Plenary talk closing: 8th September, 15.30-17.00

Recent Updates

• July 14, 2022: Program overview posted
• May 16, 2022: Leaderboard for task 3 is made public
• May 12, 2022: Leaderboards for task 1 and task 2 is made public
• May 9, 2022: Test set with gold label for task 1 and task 2 released
• Submission deadline for Task-3 extended: May 9, 2022 (AoE)
• May 3, 2022: Task-3 submission site released
• April 29, 2022: Task-3 test data released
• April 23, 2022: Task-2 submission site released
• April 22, 2022: Task-1 submission site released
• April 22, 2022: Test data released
• March 24, 2022: Baseline system for task 3 available on https://zenodo.org/record/6362498
• February 24, 2022: Training data released
• September 14, 2021: Website is up!

Lab registration

CLOSED on April 22, 2022: To register in the CheckThat! lab, please visit: https://clef2022-labs-registration.dei.unipd.it/registrationForm.php

Leaderboards

Kindly check the following leaderboards:

• Task 1: http://shorturl.at/nuCOS
• Task 2: http://shorturl.at/dzX28
• Task 3: http://shorturl.at/knBCG

Submission sites

To submit your results on the test data, kindly use the following leaderboard:

• Task 1: https://codalab.lisn.upsaclay.fr/competitions/4230
• Task 2: https://codalab.lisn.upsaclay.fr/competitions/4260
• Task 3: https://codalab.lisn.upsaclay.fr/competitions/4633

Instructions to prepare your paper

The title of paper should be in the following format:
<Team Name> at CheckThat! 2022: <Name of the title>

For example: LamdaX at CheckThat! 2022: Ensemble Models for Checkworthy Tweet Identification

Cite: Please check the citation section to cite the shared task papers.

Templates: The working notes are prepared using the CEUR-WS templates (both Word and LaTex) available at: https://drive.google.com/drive/folders/1atq0bC9hSDMicqm79Jeu6TWOR-q_BCRQ?usp=sharing

More information: https://clef2022.clef-initiative.eu/index.php?page=Pages/instructions_for_authors.html

Paper submission: https://www.easychair.org/my/conference?conf=clef2022

Datasets

You can check out the main repo containing all the scripts and data, CheckThat! Lab-2022. Liks to the Individual task dataset pages can be found below

• Task 1: https://gitlab.com/checkthat_lab/clef2022-checkthat-lab/clef2022-checkthat-lab/-/tree/main/task1
• Task 2: https://gitlab.com/checkthat_lab/clef2022-checkthat-lab/clef2022-checkthat-lab/-/tree/main/task2
• Task 3: https://zenodo.org/record/6508748

Important dates

All times are Any where On Earth (AOE).

• 15 November 2021: Registration opens
• 22 April 2022: Registration closes and test set data released
• 6 May 2022: End of the evaluation cycle (test set submission closes)
• 9 May 2022: Results released
• 27 May 2022: Submission of participant papers [CEUR-WS]
• 13 June 2022: Deadline for reviews of participant papers
• 1 July 2022: Deadline for submission camera-ready version of the participant papers
• 18-22 July 2022: Preview of working notes by authors and lab organizers [CEUR-WS]
• 5-8 September 2022: CLEF 2022 Conference

Discussion group

Please join our discussion group clef-factcheck@googlegroups.com to receive announcements and participate in discussions.

Organizers

• Preslav Nakov, Qatar Computing Research Institute, HBKU
• Alberto Barrón-Cedeño, Università di Bologna
• Giovanni Da San Martino, University of Padova
• Firoj Alam, Qatar Computing Research Institute, HBKU
• Julia Maria Struß, Potsdam University of Applied Sciences
• Thomas Mandl, University of Hildesheim
• Rubén Míguez, Newtral Media Audiovisual, Spain
• Tommaso Caselli, University of Groningen
• Mucahid Kutlu, TOBB Economy and Technology University
• Wajdi Zaghouani, HBKU
• Chengkai Li, University of Texas at Arlington
• Shaden Shaar, Cornell University
• Gautam Kishore Shahi, University of Duisburg-Essen
• Hamdy Mubarak, Qatar Computing Research Institute, HBKU
• Alex Nikolov, Sofia University
• Nikolay Babulkov, Sofia University
• Yavuz Selim Kartal, TOBB Economy and Technology University
• Javier Beltr ́an, Newtral Media Audiovisual, Spain
• Juliane Köhler, Potsdam University of Applied Sciences
• Michael Wiegand, University of Klagenfurt
• Melanie Siegel, Darmstadt University of Applied Sciences

Citation