Details
| Original language | English |
|---|---|
| Title of host publication | SIGIR 2021 |
| Subtitle of host publication | Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Pages | 2575-2579 |
| Number of pages | 5 |
| ISBN (electronic) | 9781450380379 |
| Publication status | Published - 11 Jul 2021 |
| Event | 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021 - Virtual, Online, Canada Duration: 11 Jul 2021 → 15 Jul 2021 |
Publication series
| Name | SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval |
|---|
Abstract
The World Wide Web and social media platforms have become popular sources for news and information. Typically, multimodal information, e.g., image and text is used to convey information more effectively and to attract attention. While in most cases image content is decorative or depicts additional information, it has also been leveraged to spread misinformation and rumors in recent years. In this paper, we present a web-based demo application that automatically quantifies the cross-modal relations of entities∼(persons, locations, and events) in image and text. The applications are manifold. For example, the system can help users to explore multimodal articles more efficiently, or can assist human assessors and fact-checking efforts in the verification of the credibility of news stories, tweets, or other multimodal documents.
Keywords
- cross-modal consistency, deep learning, image-text-relations, multimodal documents
ASJC Scopus subject areas
- Computer Science(all)
- Software
- Computer Science(all)
- Computer Graphics and Computer-Aided Design
- Computer Science(all)
- Information Systems
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
SIGIR 2021: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021. p. 2575-2579 3462796 (SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - QuTI! Quantifying Text-Image Consistency in Multimodal Documents
AU - Springstein, Matthias
AU - Müller-Budack, Eric
AU - Ewerth, Ralph
PY - 2021/7/11
Y1 - 2021/7/11
N2 - The World Wide Web and social media platforms have become popular sources for news and information. Typically, multimodal information, e.g., image and text is used to convey information more effectively and to attract attention. While in most cases image content is decorative or depicts additional information, it has also been leveraged to spread misinformation and rumors in recent years. In this paper, we present a web-based demo application that automatically quantifies the cross-modal relations of entities∼(persons, locations, and events) in image and text. The applications are manifold. For example, the system can help users to explore multimodal articles more efficiently, or can assist human assessors and fact-checking efforts in the verification of the credibility of news stories, tweets, or other multimodal documents.
AB - The World Wide Web and social media platforms have become popular sources for news and information. Typically, multimodal information, e.g., image and text is used to convey information more effectively and to attract attention. While in most cases image content is decorative or depicts additional information, it has also been leveraged to spread misinformation and rumors in recent years. In this paper, we present a web-based demo application that automatically quantifies the cross-modal relations of entities∼(persons, locations, and events) in image and text. The applications are manifold. For example, the system can help users to explore multimodal articles more efficiently, or can assist human assessors and fact-checking efforts in the verification of the credibility of news stories, tweets, or other multimodal documents.
KW - cross-modal consistency
KW - deep learning
KW - image-text-relations
KW - multimodal documents
UR - http://www.scopus.com/inward/record.url?scp=85111632068&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2104.13748
DO - 10.48550/arXiv.2104.13748
M3 - Conference contribution
AN - SCOPUS:85111632068
T3 - SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
SP - 2575
EP - 2579
BT - SIGIR 2021
T2 - 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021
Y2 - 11 July 2021 through 15 July 2021
ER -