Home
Editions
Tasks
Contents
Task 1: Source Retrieval for Scientific Web Claims
Source Retrieval for Scientific Web Claims: Given a social media post that contains a scientific claim and an implicit reference to a scientific paper (mentions it without a URL), retrieve the mentioned paper from a pool of candidate papers.
The data comprises English, German, and French social media posts.
The data for Task 1 can be found in this Hugging Face repo: https://huggingface.co/datasets/sschellhammer/CT26_Task1_SourceRetrievalForScientificWebClaims
Please fill out the Declaration of Commitment on the Hugging Face repo to get access to the datasets.
The repo includes query sets for training and development in three languages: English, German, and French, as well as a single collection set that is shared across all languages and all data splits.
Ensure that your runtime is authenticated with a Hugging Face account that has been granted access to the datasets.
Tweets with implicit references:
| Language | Tweet Text | Publication ID |
|---|---|---|
| EN | Peer-reviewed in the New England Journal of Medicine regarding Delta (B.1.617.2): •Pfizer is ~90% effective •AstraZeneca is ~70% effective. This falls in line with vaccine efficacy of other variants. Yes, the vaccines ARE indeed effective against Delta. |
123456 |
| EN | Published in the journal Antiviral Research, the study from Monash University showed that a single dose of Ivermectin could stop the coronavirus growing in cell culture -- effectively eradicating all genetic material of the virus within two days. | 234567 |
| DE | Die in der Fachzeitschrift Antiviral Research veröffentlichte Studie der Monash University zeigte, dass eine einzige Dosis Ivermectin das Wachstum des Coronavirus in Zellkulturen stoppen kann und das gesamte genetische Material des Virus innerhalb von zwei Tagen effektiv zerstört. | 234567 |
| FR | L'outil génétique CRISPR-Cas9 a été utilisé pour permettre à la provitamine D3 de s'accumuler dans les feuilles & fruits de la tomate. Une fois exposée aux UV, elle s'est bien convertie en vitamine D (dont 1 milliard de personnes est carencé ds le monde) |
345678 |
Collection of Publications:
| Publication ID | Study Title | Study Venue + Authors | Study Abstract |
|---|---|---|---|
| 123456 | Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant | New England Journal of Medicine Jamie Lopez Bernal, Nick Andrews, Charlotte Gower, Eileen Gallagher, Ruth Simmons, Simon Thelwall, Julia Stowe, Elise Tessier, [...] |
BACKGROUND: The B.1.617.2 (delta) variant of the severe acute respiratory syndrome coronavirus 2 [...] |
| 234567 | The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro | Antiviral Research Caly, Leon; Druce, Julian D.; Catton, Mike G.; Jans, David A.; Wagstaff, Kylie M. |
Although several clinical trials are now underway to test possible therapies, the worldwide response to the COVID-19 outbreak has been largely limited to monitoring/containment [...] |
| 345678 | Biofortified tomatoes provide a new route to vitamin D sufficiency | Nature Plants Jie Li, John Innes Centre, Norwich Research Park, Norwich, UK; Aurelia Scarano, Institute of [...] |
[...] We have engineered the accumulation of provitamin D3 in tomato by genome editing, modifying a duplicated section of phytosterol biosynthesis in Solanaceous plants [...] |
This task is a retrieval task and will be evaluated by the MRR@5 (Mean Reciprocal Rank) score. The task will be evaluated separately for each language.
A scorer and a baseline script can be found here.
TBA
TBA
TBA
For any questions, please contact us on Discord or via email at sebastian.schellhammer@gesis.org