CheckThat! Lab at CLEF 2025

Home

Editions

Tasks

Contents

Task 1: Subjectivity

Definition

Systems are challenged to distinguish whether a sentence from a news article expresses the subjective view of the author behind it or presents an objective view on the covered topic instead.

This is a binary classification tasks in which systems have to identify whether a text sequence (a sentence or a paragraph) is subjective (SUBJ) or objective (OBJ).

The task comprises three settings:

  • Monolingual: train and test on data in a given language L
  • Multilingual: train and test on data comprising several languages
  • Zero-shot: train on several languages and test on unseen languages

Datasets

We provide training data in five languages:

Note: Test datasets will be made available once the evaluation cycle starts.

Note: For multilingual and zero-shot settings, all language-specific training data can be used.

Annotation Guidelines

Information regarding the annotation guidelines can be found in the following paper: A Corpus for Sentence-Level Subjectivity Detection on English News Articles.

Evaluation

The official evaluation is macro-averaged F1 between the two classes.

Submission

TBA

Leaderboard

TBA

Organizers

  • Federico Ruggeri, DISI, University of Bologna, Italy,
  • Arianna Muti, MilaNLP, Bocconi, Milan
  • Katerina Korre, DIT, University of Bologna, Italy

Note: Language-specific curators will be announced once the evaluation cycle ended.

Contact

For queries, please join the Slack channel

Alternatively, please send an email to: clef-factcheck@googlegroups.com