Task 5: Authority Finding in Twitter
In this task, the systems are challenged to retrieve a set of authority Twitter
accounts for a given rumor propagating in Twitter.
Given a tweet stating a rumor, a model has to retrieve a ranked list of
authority Twitter accounts that can help verify the rumor; i.e. they may tweet
evidence that supports or denies the rumor.
This task is offered in Arabic.
The training dataset is available here
It comprises 150 rumors expressed in tweets associated with 1,044 authorities’
Twitter accounts, a user collection of 395,231 Twitter accounts, and about 1.2M
unique Twitter lists.
The official evaluation measure is P@5 to evaluate how systems are able to
retrieve authorities at the top of a short retrieved list.
Other measures, such as P@1 and nDCG@5, will also be used.
Scorers, Format Checkers, and Baseline Scripts
All scripts can be found on gitlab at CheckThat! Lab Task 5 repository
To submit your runs (the output of your system), please use the form sent to all
participants of Task 5 over email for each run. Your runs must be in the output
format specified in the task repository. The submission form will be closed on
Each team can submit up to 3 runs based in the following guidelines:
- For each run, you will have to explicitly indicate if it is primary or
- One and only one run must be primary and it will be used for main
comparison to other teams.
- For each run, you will have to explicitly indicate if external data was used
- Maximum of one run can use external data.
The official metric is P@5
||+ bigIR (Hybrid3)
||+ bigIR (Hybrid1)
||+ bigIR (Hybrid2)
* Submissions with a + sign include task organisers
- Tamer Elsayed, Qatar University, Qatar
- Fatima Haouari, Qatar University, Qatar
- Zien Sheikh Ali, Qatar University, Qatar