Definition

Source Retrieval for Scientific Web Claims: Given a social media post that contains a scientific claim and an implicit reference to a scientific paper (mentions it without a URL), retrieve the mentioned paper from a pool of candidate papers.

The data comprises English, German, and French social media posts.

Datasets

The data will be provided in three different languages: English, French and German.

Examples

Tweets with implicit references:

Language	Tweet Text	Publication ID
EN	Peer-reviewed in the New England Journal of Medicine regarding Delta (B.1.617.2): •Pfizer is ~90% effective •AstraZeneca is ~70% effective. This falls in line with vaccine efficacy of other variants. Yes, the vaccines ARE indeed effective against Delta.	123456
EN	Published in the journal Antiviral Research, the study from Monash University showed that a single dose of Ivermectin could stop the coronavirus growing in cell culture -- effectively eradicating all genetic material of the virus within two days.	234567
DE	Die in der Fachzeitschrift Antiviral Research veröffentlichte Studie der Monash University zeigte, dass eine einzige Dosis Ivermectin das Wachstum des Coronavirus in Zellkulturen stoppen kann und das gesamte genetische Material des Virus innerhalb von zwei Tagen effektiv zerstört.	234567
FR	L'outil génétique CRISPR-Cas9 a été utilisé pour permettre à la provitamine D3 de s'accumuler dans les feuilles & fruits de la tomate. Une fois exposée aux UV, elle s'est bien convertie en vitamine D (dont 1 milliard de personnes est carencé ds le monde)	345678

Collection of Publications:

Publication ID	Study Title	Study Date	Study Venue + Authors	Study Abstract
123456	Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant	21-07-2021	New England Journal of Medicine Jamie Lopez Bernal, Nick Andrews, Charlotte Gower, Eileen Gallagher, Ruth Simmons, Simon Thelwall, Julia Stowe, Elise Tessier, [...]	BACKGROUND: The B.1.617.2 (delta) variant of the severe acute respiratory syndrome coronavirus 2 [...]
234567	The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro	03-04-2020	Antiviral Research Caly, Leon; Druce, Julian D.; Catton, Mike G.; Jans, David A.; Wagstaff, Kylie M.	Although several clinical trials are now underway to test possible therapies, the worldwide response to the COVID-19 outbreak has been largely limited to monitoring/containment [...]
345678	Biofortified tomatoes provide a new route to vitamin D sufficiency	23-05-2022	Nature Plants Jie Li, John Innes Centre, Norwich Research Park, Norwich, UK; Aurelia Scarano, Institute of [...]	[...] We have engineered the accumulation of provitamin D3 in tomato by genome editing, modifying a duplicated section of phytosterol biosynthesis in Solanaceous plants [...]

Evaluation

This task is a retrieval task and will be evaluated by the MRR@5 (Mean Reciprocal Rank) score. The task will be evaluated separately for each language.

Submission

Scorer, Format Checker, and Baseline Scripts

TBA

Submission Site

TBA

Submission Guidelines

TBA

Leaderboard

TBA

Organizers

Dimitar Dimtrov, GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany
Katarina Boland, Heinrich-Heine-University, Düsseldorf, Germany
Konstantin Todorov, LIRMM, CNRS, University of Montpellier, Montpellier, France
Salim Hafid, médialab Sciences Po, Paris, France
Sebastian Schellhammer, GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany
Stefan Dietze, GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany
Yavuz Selim Kartal, GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany

Contact

Alternatively, please send an email to: clef-factcheck@googlegroups.com

CheckThat! Lab at CLEF 2026

Contents