Best of labs: Monday 18th September, Conference session 2, 13:00-14:30
Title
Fight Against Misinformation on Social Media: Detecting Attention-Worthy and Harmful Tweets and Verifiable and Check-Worthy Claims. Ahmet Bahadır Eyuboğlu, Bahadir Altun, Mustafa Bora Arslan, Ekrem Sonmezer and Mucahid Kutlu.
Fraunhofer SIT at CheckThat! 2023: Enhancing the Detection of Multimodal and Multigenre Check Worthiness Using Optical Character Recognition and Model Souping (on site)
10:20
1
OpenFact at CheckThat! 2023: Head-to-Head GPT vs. BERT - A Comparative Study of Transformers Language Models for the Detection of Check-worthy Claims (on site)
10:35
1
Zhaw-cai at CheckThat! 2023: Ensembling using kernel averaging (online)
10:50
1
ES-VRAI at CheckThat! 2023: Analyzing Ch eckworthiness in Multimodal and Multigenre Contents through Fusion and Sampling Approaches (on site)
Session 2: Thursday 21st September, 11:30 to 13:00 (Chair: Andrea Galassi)
Time
Task
Title
11:30
2
Accenture at CheckThat! 2023: Impact of back-translation on subjectivity detection (on site)
11:45
3
Accenture at CheckThat! 2023: Learning to detect factuality levels of news sources (on site)
12:00
2
Thesis Titan at CheckThat! 2023 Task 2: Language-Specific Fine-tuning of mDeBERTaV3 for Subjectivity Detection (online)
12:15
2
TUDublin at CheckThat! 2023: ChatGPT for data augmentation (on site)
12:30
3
TOBB ETU at CheckThat! 2023: Utilizing ChatGPT to Detect Subjective Statements and Political Bias (online)
12:45
5
ES-VRAI at CheckThat! 2023: Leveraging Bio and Lists Information for Enhanced Rumor Verification in Twitter (on site)
3 May 2023: Beginning of the evaluation cycle (test sets release)
10 May 2023 (23:59 AOE): End of the evaluation cycle (run submission)
5 June 2023: Deadline for the submission of working notes
23 June 2023: Notification of acceptance of working notes
7 July 2023: Deadline for submission of camera-ready working notes
July 2023: Preview of working notes
18-21 September: CLEF 2023 Conference in Thessaloniki
Proceedings
Instructions to Prepare Your Paper
The title of paper should be in the following format:
<Team Name> at CheckThat! 2023: <Descriptive title>
For example, team LamdaX would their title as follows:
LamdaX at CheckThat! 2023: Cool Models for Checkworthiness Identification
Templates: The working notes are prepared using the CEUR-WS templates.
Instructions and templates are both available at the CLEF 2023
website.
Overview paper: We are working on the preparation of the overview papers
ourselves (one for the full lab and one per each task). Please stay tuned for
the details. Meanwhile, you can consider to cite the ECIR 2023 paper about the
lab.
Recent Updates
13rd Aug, 2023: The programme has been posted.
18th May, 2023: Leaderboards and test gold standards for all tasks
have been published. Check on the Task section of this website and
the corresponding git projects.
10th May, 2023: Submissions for all tasks are allowed **until 11th May
(23:59 AOE)** due to problems during the submission by a number of teams.
Notice that runs submitted after the original deadline (10th May, 23:59 AOE)
**will be flagged accordingly** in the official leaderboards and in the overview
papers. If you **do NOT want your submission to be flagged as late**, do submit
by 10 May 2023 (23:59 AOE), as originally scheduled!
3rd May, 2023: The test sets for all tasks have been
released. Predict!
28th April May, 2023: The registration to the shared task
was closed.
Task 1: Firoj Alam, Qatar Computing Research Institute, HBKU, Qatar
Task 2: Andrea Galassi, Università di Bologna, Italy
Task 3: Giovanni Da San Martino, Università di Padova, Italy
Task 4: Preslav Nakov, MBZUAI, UAE
Task 5: Tamer Elsayed, Qatar University, Qatar
PC chairs
Tamer Elsayed, Qatar University, Qatar
Fatima Haouari, Qatar University, Qatar
Citation
There are various papers associated with this edition of the lab. So far, the
ECIR 2023 is the only one published (at the bottom). Details for the papers
specific to each task as well as an overall overview will be posted here as they
come out.
Bib entries for each paper are included here. For your convenience, the
bib file
is available as well.
CLEF 2023 LNCS
Cite this paper to refer to the whole CheckThat! 2023 lab.
As of 23rd May, we are still defining this paper. Stay tuned!
CLEF 2023 Working notes
Task 1
Cite this paper to refer to Task 1 of the CheckThat! 2023 lab.
Alam, Barrón-Cedeño, Cheema, Hakimov, Hasanain, Li, Míguez, Mubarak, Shahi,
Zaghouani, and Nakovet. Overview of the CLEF-2023 CheckThat! Lab Task 1 on
Check-Worthiness in Multimodal and Multigenre Content. In Proceedings of the
Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum.
Thessaloniki, Greece
@InProceedings{clef-checkthat:2023:task1,
author = {
and Alam, Firoj
and Barr\'{o}n-Cede\~{n}o, Alberto
and Cheema, Gullal S.
and Hakimov, Sherzod
and Hasanain, Maram
and Li, Chengkai
and M\'{\i}guez, Rub\'{e}n
and Mubarak, Hamdy,
and Shahi, Gautam Kishore
and Zaghouani, Wajdi
and Nakov, Preslav},
title = "Overview of the {CLEF}-2023 {CheckThat}! Lab Task 1 on
Check-Worthiness in Multimodal and Multigenre Content",
year = {2023},
booktitle = "Working Notes of CLEF 2023---Conference and Labs of the
Evaluation Forum",
series = {CLEF~'2023},
address = {Thessaloniki, Greece},
}
Task 2
Cite this paper to refer to Task 2 of the CheckThat! 2023 lab.
Overview of the CLEF-2023 CheckThat! Lab Task 2 on Subjectivity in News
Articles.
Galassi, Ruggeri, Barrónn-Cedeño, Alam, Caselli, Kutlu, Struss, Antici,
Hasanain, Köhler, Korre, Leistra, Muti, Siegel, Turkmen, Wiegand, and
Zaghouani. In Proceedings of the Working Notes of CLEF 2023 - Conference and
Labs of the Evaluation Forum. Thessaloniki, Greece
As of 23rd May, we are still defining this paper. Stay tuned!
@InProceedings{clef-checkthat:2023:task2,
author = {Andrea Galassi
and Federico Ruggeri
and Alberto Barr\'{o}n-Cede\~{n}o
and Firoj Alam
and Tommaso Caselli
and Mucahid Kutlu
and Julia Maria Struss
and Francesco Antici
and Maram Hasanain
and Juliane Köhler
and Katerina Korre
and Folkert Leistra
and Arianna Muti
and Melanie Siegel
and Mehmet Deniz Turkmen
and Michael Wiegand
and Wajdi Zaghouani},
title = "Overview of the {CLEF}-2023 {CheckThat}! Lab Task 2 on Subjectivity
in News Articles",
year = {2023},
booktitle = "Working Notes of CLEF 2023--Conference and Labs of the
Evaluation Forum",
series = {CLEF~'2023},
address = {Thessaloniki, Greece},
}
Task 3
Cite this paper to refer to Task 3 of the CheckThat! 2023 lab.
Da San Martino, Alam, Hasanain, Nandi, Azizov and Nakov. Overview of the
CLEF-2023 CheckThat! Lab Task 3 on Political Bias of News Articles and News
Media. In Proceedings of the Working Notes of CLEF 2023-Conference and Labs of
the Evaluation Forum. Thessaloniki, Greece
@InProceedings{clef-checkthat:2023:task3,
author = {Da San Martino, Giovanni
and Alam, Firoj
and Hasanain, Maram
and Nandi, Rabindra Nath
and Azizov, Dilshod
and Nakov, Preslav},
title = "Overview of the {CLEF}-2023 {CheckThat}! Lab Task 3 on Political
Bias of News Articles and News Media",
year = {2023},
booktitle = "Working Notes of CLEF 2023--Conference and Labs of the
Evaluation Forum",
series = {CLEF~'2023},
address = {Thessaloniki, Greece},
}
Task 4
Cite this paper to refer to Task 4 of the CheckThat! 2023 lab.
Nakov, Alam, Da San Martino, Hasanain, Nandi, Azizov, and Panayotov. Overview of
the CLEF-2023 CheckThat! Lab Task 4 on Factuality of Reporting of News Media. In
Proceedings of the Working Notes of CLEF 2023-Conference and Labs of the
Evaluation Forum. Thessaloniki, Greece
@InProceedings{clef-checkthat:2023:task4,
author = {Nakov, Preslav
and Alam, Firoj
and Da San Martino, Giovanni
and Hasanain, Maram
and Nandi, Rabindra Nath
and Azizov, Dilshod
and Panayotov, Panayot},
title = "Overview of the {CLEF}-2023 {CheckThat}! Lab Task 4 on Factuality
of Reporting of News Media",
year = {2023},
booktitle = "Working Notes of CLEF 2023--Conference and Labs of the
Evaluation Forum",
series = {CLEF~'2023},
address = {Thessaloniki, Greece},
}
Task 5
Cite this paper to refer to Task 5 of the CheckThat! 2023 lab.
Haouari, Sheikh Ali and Elsayed. Overview of the CLEF-2023 CheckThat! Lab Task
5 on Authority Finding in Twitter. In Proceedings of the Working Notes of CLEF
2023-Conference and Labs of the Evaluation Forum. Thessaloniki, Greece
@InProceedings{clef-checkthat:2023:task5,
author = {Haouari, Fatima
and Sheikh Ali, Zien
and Elsayed, Tamer
},
title = "Overview of the {CLEF}-2023 {CheckThat}! Lab Task 5 on Authority
Finding in Twitter",
year = {2023},
booktitle = "Working Notes of CLEF 2023--Conference and Labs of the
Evaluation Forum",
series = {CLEF~'2023},
address = {Thessaloniki, Greece},
}
ECIR 2023
So far this is the only paper published already. You can cite it when referring
to the whole lab.
[1]
Barrón-Cedeño, Alam, Caselli, Da San Martino, Elsayed, Galassi, Haouari, Ruggeri,
Struss, Nandi, Cheema, Azizov, Nakov (2023).
The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias,
Factuality, and Authority. In: Advances in Information Retrieval. ECIR 2023.
Lecture Notes in Computer Science, vol 13982. Springer, Cham.
https://doi.org/10.1007/978-3-031-28241-6_59
@InProceedings{10.1007/978-3-031-28241-6_59,
author="Barr{\'o}n-Cede{\~{n}}o, Alberto
and Alam, Firoj
and Caselli, Tommaso
and Da San Martino, Giovanni
and Elsayed, Tamer
and Galassi, Andrea
and Haouari, Fatima
and Ruggeri, Federico
and Struss, Julia Maria
and Nandi, Rabindra Nath
and Cheema, Gullal S.
and Azizov, Dilshod
and Nakov, Preslav",
title="The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority",
editor="Kamps, Jaap
and Goeuriot, Lorraine
and Crestani, Fabio
and Maistro, Maria
and Joho, Hideo
and Davis, Brian
and Gurrin, Cathal
and Kruschwitz, Udo
and Caputo, Annalina",
booktitle="Advances in Information Retrieval",
year="2023",
publisher="Springer Nature Switzerland",
address="Cham",
pages="506--517",
isbn="978-3-031-28241-6"
}
