Task 3: Political Bias of News Articles and News Media
Definition
The goal of the task is to detect political bias of news reporting at the article and at the media level. This is an ordinal classification task and is offered in English. It includes two subtasks:
- Subtask 3A: Given an article, classify its political leaning as left,
center or right.
- Subtask 3B: Given the news article(s) a news outlet (e.g., www.cnn.com), predict the overall political bias of that news outlet as left, center or right leaning.
Evaluation
This is an ordinal classification task. We use mean absolute error as the official measure for both subtasks.
Submission
All scripts can be found on gitlab at CheckThat! Lab Task 3 repository
Submission guidelines
- Make sure that you create one account single account for your team, and submit runs through that account only.
- The last file submitted to the leaderboard will be considered as the final submission.
- The file with your predictions should be called
subtask3[A|B].tsv where A or B refer to the specific subtask. Get sure to set .tsv as the file extension; otherwise, you will get an error on the leaderboard. For instance, a submission file for task 3A should be subtask3B.tsv.
- You have to zip the tsv,
zip subtask3A.zip subtask3A.tsv and submit it through the codalab page.
- You have to include the team name and the description of your method each submission. Your team name must EXACTLY match the one used during the CLEF registration.
- You are allowed to submit max 200 submissions per day for each subtask.
- We will keep the leaderboard private until the end of the submission period, hence, results will not be available upon submission. All results will be available after the evaluation period.
Submission Site
Task 3: Codalab
Leaderboard
In both Tasks 3A and 3B the baseline is a random system.
Task 3A
|
Team |
MAE |
| 1 |
Accenture |
0.473 |
| 2 |
TOBB ETU |
0.646 |
| 3 |
KUCST |
0.736 |
| 4 |
Awakened |
0.752 |
| 5 |
Baseline |
0.877 |
Task 3B
Random
|
Team |
MAE |
| 1 |
Accenture |
0.549 |
| 2 |
Awakened |
0.765 |
| 3 |
Baseline |
0.902 |
Organizers
- Giovanni Da San Martino, University of Padova
- Firoj Alam, Qatar Computing Research Institute, HBKU
- Preslav Nakov, Mohamed bin Zayed University of Artificial Intelligence
- Maram Hasanain, Qatar Computing Research Institute, HBKU
- Rabindra Nath Nandi, BJIT Limited
- Dilshod Azizov, Mohamed bin Zayed University of Artificial Intelligence
Please join on Slack channel for any queries: https://join.slack.com/t/newworkspace-qgd1635/shared_invite/zt-1k9lnm0ys-aMeEmUtGY0tfW5HbkXHPJg
Slack Channel: https://ct-23-participants.slack.com
Or send an email to: clef-factcheck@googlegroups.com
