Details
| Original language | English |
|---|---|
| Pages (from-to) | 19-33 |
| Number of pages | 15 |
| Journal | International Journal of Multimedia Information Retrieval |
| Volume | 8 |
| Issue number | 1 |
| Early online date | 23 Jan 2019 |
| Publication status | Published - 7 Mar 2019 |
Abstract
Exoticism is the charm of the unfamiliar or something remote. It has received significant interest in different kinds of arts, but although visual concept classification in images and videos for semantic multimedia retrieval has been researched for years, the visual concept of exoticism has not been investigated yet from a computational perspective. In this paper, we present the first approach to automatically classify images as exotic or non-exotic. We have gathered two large datasets that cover exoticism in a general as well as a concept-specific way. The datasets have been annotated in a crowdsourcing approach. To circumvent cultural differences in the annotation, only North American crowdworkers are employed for this task. Two deep learning architectures to learn the concept of exoticism are evaluated. Besides deep learning features, we also investigate the usefulness of hand-crafted features, which are combined with deep features in our proposed fusion-based approach. Different machine learning models are compared with the fusion-based approach, which is the best performing one, reaching an accuracy over 83% and 91% on two different datasets. Comprehensive experimental results provide insights into which features contribute at most to recognizing exoticism. The estimation of image exoticism could be applied in fields like advertising and travel suggestions, as well as to increase serendipity and diversity of recommendations and search results.
Keywords
- Exoticism, Image retrieval, Visual concept classification
ASJC Scopus subject areas
- Computer Science(all)
- Information Systems
- Engineering(all)
- Media Technology
- Social Sciences(all)
- Library and Information Sciences
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: International Journal of Multimedia Information Retrieval, Vol. 8, No. 1, 07.03.2019, p. 19-33.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Mining exoticism from visual content with fusion-based deep neural networks
AU - Ceroni, Andrea
AU - Ma, Chenyang
AU - Ewerth, Ralph
PY - 2019/3/7
Y1 - 2019/3/7
N2 - Exoticism is the charm of the unfamiliar or something remote. It has received significant interest in different kinds of arts, but although visual concept classification in images and videos for semantic multimedia retrieval has been researched for years, the visual concept of exoticism has not been investigated yet from a computational perspective. In this paper, we present the first approach to automatically classify images as exotic or non-exotic. We have gathered two large datasets that cover exoticism in a general as well as a concept-specific way. The datasets have been annotated in a crowdsourcing approach. To circumvent cultural differences in the annotation, only North American crowdworkers are employed for this task. Two deep learning architectures to learn the concept of exoticism are evaluated. Besides deep learning features, we also investigate the usefulness of hand-crafted features, which are combined with deep features in our proposed fusion-based approach. Different machine learning models are compared with the fusion-based approach, which is the best performing one, reaching an accuracy over 83% and 91% on two different datasets. Comprehensive experimental results provide insights into which features contribute at most to recognizing exoticism. The estimation of image exoticism could be applied in fields like advertising and travel suggestions, as well as to increase serendipity and diversity of recommendations and search results.
AB - Exoticism is the charm of the unfamiliar or something remote. It has received significant interest in different kinds of arts, but although visual concept classification in images and videos for semantic multimedia retrieval has been researched for years, the visual concept of exoticism has not been investigated yet from a computational perspective. In this paper, we present the first approach to automatically classify images as exotic or non-exotic. We have gathered two large datasets that cover exoticism in a general as well as a concept-specific way. The datasets have been annotated in a crowdsourcing approach. To circumvent cultural differences in the annotation, only North American crowdworkers are employed for this task. Two deep learning architectures to learn the concept of exoticism are evaluated. Besides deep learning features, we also investigate the usefulness of hand-crafted features, which are combined with deep features in our proposed fusion-based approach. Different machine learning models are compared with the fusion-based approach, which is the best performing one, reaching an accuracy over 83% and 91% on two different datasets. Comprehensive experimental results provide insights into which features contribute at most to recognizing exoticism. The estimation of image exoticism could be applied in fields like advertising and travel suggestions, as well as to increase serendipity and diversity of recommendations and search results.
KW - Exoticism
KW - Image retrieval
KW - Visual concept classification
UR - http://www.scopus.com/inward/record.url?scp=85060720565&partnerID=8YFLogxK
U2 - 10.1007/s13735-018-00165-4
DO - 10.1007/s13735-018-00165-4
M3 - Article
AN - SCOPUS:85060720565
VL - 8
SP - 19
EP - 33
JO - International Journal of Multimedia Information Retrieval
JF - International Journal of Multimedia Information Retrieval
SN - 2192-6611
IS - 1
ER -