CheckThat! Lab at CLEF 2023





Task 3: Political Bias of News Articles and News Media


The goal of the task is to detect political bias of news reporting at the article and at the media level. This is an ordinal classification task and is offered in English. It includes two subtasks:

  • Subtask 3A: Given an article, classify its political leaning as left, center or right.
  • Subtask 3B: Given the news article(s) a news outlet (e.g.,, predict the overall political bias of that news outlet as left, center or right leaning.


Subtask 3A and 3B datasets


This is an ordinal classification task. We use mean absolute error as the official measure for both subtasks.


Scorers, Format Checkers, and Baseline Scripts

All scripts can be found on gitlab at CheckThat! Lab Task 3 repository

Submission guidelines

  • Make sure that you create one account single account for your team, and submit runs through that account only.
  • The last file submitted to the leaderboard will be considered as the final submission.
  • The file with your predictions should be called subtask3[A|B].tsv where A or B refer to the specific subtask. Get sure to set .tsv as the file extension; otherwise, you will get an error on the leaderboard. For instance, a submission file for task 3A should be subtask3B.tsv.
  • You have to zip the tsv, zip subtask3A.tsv and submit it through the codalab page.
  • You have to include the team name and the description of your method each submission. Your team name must EXACTLY match the one used during the CLEF registration.
  • You are allowed to submit max 200 submissions per day for each subtask.
  • We will keep the leaderboard private until the end of the submission period, hence, results will not be available upon submission. All results will be available after the evaluation period.

Submission Site

Task 3: Codalab


In both Tasks 3A and 3B the baseline is a random system.

Task 3A

Team MAE
1 Accenture 0.473
2 TOBB ETU 0.646
3 KUCST 0.736
4 Awakened 0.752
5 Baseline 0.877

Task 3B


Team MAE
1 Accenture 0.549
2 Awakened 0.765
3 Baseline 0.902


  • Giovanni Da San Martino, University of Padova
  • Firoj Alam, Qatar Computing Research Institute, HBKU
  • Preslav Nakov, Mohamed bin Zayed University of Artificial Intelligence
  • Maram Hasanain, Qatar Computing Research Institute, HBKU
  • Rabindra Nath Nandi, BJIT Limited
  • Dilshod Azizov, Mohamed bin Zayed University of Artificial Intelligence


Please join on Slack channel for any queries:

Slack Channel:

Or send an email to: