Details
| Original language | English |
|---|---|
| Title of host publication | MMPT 2021 |
| Subtitle of host publication | Proceedings of the 2021 Workshop on Multi-Modal Pre-Training for Multimedia Understanding |
| Pages | 37-45 |
| Number of pages | 9 |
| ISBN (electronic) | 9781450385305 |
| Publication status | Published - 27 Aug 2021 |
| Event | 1st International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding, MMPT 2021 - Taipei, Taiwan Duration: 21 Aug 2021 → … |
Publication series
| Name | MMPT 2021 - Proceedings of the 2021 Workshop on Multi-Modal Pre-Training for Multimedia Understanding |
|---|
Abstract
Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts. In this paper, we present a comprehensive experimental evaluation and comparison with six state-of-the-art methods, from which we have re-implemented one of them. In addition, we investigate different textual and visual feature embeddings that cover different aspects of the content, as well as the recently introduced multimodal CLIP embeddings. Experimental results are presented for two different publicly available benchmark datasets of tweets and corresponding images. In contrast to the evaluation methodology of previous work, we introduce a reproducible and fair evaluation scheme to make results comparable. Finally, we conduct an error analysis to outline the limitations of the methods and possibilities for the future work.
Keywords
- computer vision, information retrieval, multimodal sentiment analysis, natural language processing, social media, transformer models
ASJC Scopus subject areas
- Computer Science(all)
- Computer Networks and Communications
- Computer Science(all)
- Hardware and Architecture
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
MMPT 2021: Proceedings of the 2021 Workshop on Multi-Modal Pre-Training for Multimedia Understanding. 2021. p. 37-45 (MMPT 2021 - Proceedings of the 2021 Workshop on Multi-Modal Pre-Training for Multimedia Understanding).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods
AU - Cheema, Gullal S.
AU - Hakimov, Sherzod
AU - Müller-Budack, Eric
AU - Ewerth, Ralph
N1 - Funding Information: This work has been financially supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no 812997.
PY - 2021/8/27
Y1 - 2021/8/27
N2 - Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts. In this paper, we present a comprehensive experimental evaluation and comparison with six state-of-the-art methods, from which we have re-implemented one of them. In addition, we investigate different textual and visual feature embeddings that cover different aspects of the content, as well as the recently introduced multimodal CLIP embeddings. Experimental results are presented for two different publicly available benchmark datasets of tweets and corresponding images. In contrast to the evaluation methodology of previous work, we introduce a reproducible and fair evaluation scheme to make results comparable. Finally, we conduct an error analysis to outline the limitations of the methods and possibilities for the future work.
AB - Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts. In this paper, we present a comprehensive experimental evaluation and comparison with six state-of-the-art methods, from which we have re-implemented one of them. In addition, we investigate different textual and visual feature embeddings that cover different aspects of the content, as well as the recently introduced multimodal CLIP embeddings. Experimental results are presented for two different publicly available benchmark datasets of tweets and corresponding images. In contrast to the evaluation methodology of previous work, we introduce a reproducible and fair evaluation scheme to make results comparable. Finally, we conduct an error analysis to outline the limitations of the methods and possibilities for the future work.
KW - computer vision
KW - information retrieval
KW - multimodal sentiment analysis
KW - natural language processing
KW - social media
KW - transformer models
UR - http://www.scopus.com/inward/record.url?scp=85114774687&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2106.08829
DO - 10.48550/arXiv.2106.08829
M3 - Conference contribution
AN - SCOPUS:85114774687
T3 - MMPT 2021 - Proceedings of the 2021 Workshop on Multi-Modal Pre-Training for Multimedia Understanding
SP - 37
EP - 45
BT - MMPT 2021
T2 - 1st International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding, MMPT 2021
Y2 - 21 August 2021
ER -