Details
| Originalsprache | Englisch |
|---|---|
| Titel des Sammelwerks | Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Seiten | 650-660 |
| Seitenumfang | 11 |
| ISBN (elektronisch) | 9798400704314 |
| Publikationsstatus | Veröffentlicht - 11 Juli 2024 |
| Veranstaltung | 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024 - Washington, USA / Vereinigte Staaten Dauer: 14 Juli 2024 → 18 Juli 2024 |
Abstract
With the growth of misinformation on the web, automated fact checking has garnered immense interest for detecting growing misinformation and disinformation. Current systems have made significant advancements in handling synthetic claims sourced from Wikipedia, and noteworthy progress has been achieved in addressing real-world claims that are verified by fact-checking organizations as well. We compile and release QuanTemp, a diverse, multi-domain dataset focused exclusively on numerical claims, encompassing comparative, statistical, interval, and temporal aspects, with detailed metadata and an accompanying evidence collection. This addresses the challenge of verifying real-world numerical claims, which are complex and often lack precise information, a gap not filled by existing works that mainly focus on synthetic claims. We evaluate and quantify these gaps in existing solutions for the task of verifying numerical claims. We also evaluate claim decomposition based methods, numerical understanding based natural language inference (NLI) models and our best baselines achieves a macro-F1 of 58.32. This demonstrates that QuanTemp serves as a challenging evaluation set for numerical claim verification.
ASJC Scopus Sachgebiete
- Informatik (insg.)
- Information systems
- Informatik (insg.)
- Software
Zitieren
- Standard
- Harvard
- Apa
- Vancouver
- BibTex
- RIS
Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2024. S. 650-660.
Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung › Peer-Review
}
TY - GEN
T1 - QuanTemp
T2 - 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024
AU - Venktesh, V.
AU - Anand, Abhijit
AU - Anand, Avishek
AU - Setty, Vinay
N1 - Publisher Copyright: © 2024 Owner/Author.
PY - 2024/7/11
Y1 - 2024/7/11
N2 - With the growth of misinformation on the web, automated fact checking has garnered immense interest for detecting growing misinformation and disinformation. Current systems have made significant advancements in handling synthetic claims sourced from Wikipedia, and noteworthy progress has been achieved in addressing real-world claims that are verified by fact-checking organizations as well. We compile and release QuanTemp, a diverse, multi-domain dataset focused exclusively on numerical claims, encompassing comparative, statistical, interval, and temporal aspects, with detailed metadata and an accompanying evidence collection. This addresses the challenge of verifying real-world numerical claims, which are complex and often lack precise information, a gap not filled by existing works that mainly focus on synthetic claims. We evaluate and quantify these gaps in existing solutions for the task of verifying numerical claims. We also evaluate claim decomposition based methods, numerical understanding based natural language inference (NLI) models and our best baselines achieves a macro-F1 of 58.32. This demonstrates that QuanTemp serves as a challenging evaluation set for numerical claim verification.
AB - With the growth of misinformation on the web, automated fact checking has garnered immense interest for detecting growing misinformation and disinformation. Current systems have made significant advancements in handling synthetic claims sourced from Wikipedia, and noteworthy progress has been achieved in addressing real-world claims that are verified by fact-checking organizations as well. We compile and release QuanTemp, a diverse, multi-domain dataset focused exclusively on numerical claims, encompassing comparative, statistical, interval, and temporal aspects, with detailed metadata and an accompanying evidence collection. This addresses the challenge of verifying real-world numerical claims, which are complex and often lack precise information, a gap not filled by existing works that mainly focus on synthetic claims. We evaluate and quantify these gaps in existing solutions for the task of verifying numerical claims. We also evaluate claim decomposition based methods, numerical understanding based natural language inference (NLI) models and our best baselines achieves a macro-F1 of 58.32. This demonstrates that QuanTemp serves as a challenging evaluation set for numerical claim verification.
KW - claim decomposition
KW - fact-checking
KW - numerical claims
UR - http://www.scopus.com/inward/record.url?scp=85200538956&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2403.1716
DO - 10.48550/arXiv.2403.1716
M3 - Conference contribution
AN - SCOPUS:85200538956
SP - 650
EP - 660
BT - Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval
Y2 - 14 July 2024 through 18 July 2024
ER -