TY - GEN

T1 - Automatic and Semi-automatic Augmentation of Migration Related Semantic Concepts for Visual Media Retrieval

AU - Galanopoulos, Damianos

AU - Elejalde, Erick

AU - Pournaras, Alexandros

AU - Niederée, Claudia

AU - Mezaris, Vasileios

N1 - Funding Information: This work was supported by the EU Horizon 2020 programme under grant agreement 832921 (MIRROR).

PY - 2021/10/19

Y1 - 2021/10/19

N2 - Understanding the factors related to migration, such as perceptions about routes and target countries, is critical for border agencies and society altogether. A systematic analysis of communication and news channels, such as social media, can improve our understanding of such factors. Videos and images play a critical role in social media as they have significant impact on perception manipulation and misinformation campaigns. However, more research is needed in the identification of semantically relevant visual content for specific queried concepts. Furthermore, an important problem to overcome in this area is the lack of annotated datasets that could be used to create and test accurate models. A recent study proposed a novel video representation and retrieval approach that effectively bridges the gap between a substantiated domain understanding - encapsulated into textual descriptions of Migration Related Semantic Concepts (MRSCs) - and the expression of such concepts in a video. In this work, we build on this approach and propose an improved procedure for the crucial step of the concept labels' textual augmentation, which contributes towards the full automation of the pipeline. We assemble the first, to the best of our knowledge, migration-related videos and images dataset and we experimentally assess our method on it.

AB - Understanding the factors related to migration, such as perceptions about routes and target countries, is critical for border agencies and society altogether. A systematic analysis of communication and news channels, such as social media, can improve our understanding of such factors. Videos and images play a critical role in social media as they have significant impact on perception manipulation and misinformation campaigns. However, more research is needed in the identification of semantically relevant visual content for specific queried concepts. Furthermore, an important problem to overcome in this area is the lack of annotated datasets that could be used to create and test accurate models. A recent study proposed a novel video representation and retrieval approach that effectively bridges the gap between a substantiated domain understanding - encapsulated into textual descriptions of Migration Related Semantic Concepts (MRSCs) - and the expression of such concepts in a video. In this work, we build on this approach and propose an improved procedure for the crucial step of the concept labels' textual augmentation, which contributes towards the full automation of the pipeline. We assemble the first, to the best of our knowledge, migration-related videos and images dataset and we experimentally assess our method on it.

KW - image/video retrieval

KW - migration related semantic concepts

KW - semantic queries

UR - http://www.scopus.com/inward/record.url?scp=85121420882&partnerID=8YFLogxK

U2 - 10.1145/3472720.3483618

DO - 10.1145/3472720.3483618

M3 - Conference contribution

AN - SCOPUS:85121420882

T3 - OASIS 2021 - Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks

SP - 33

EP - 38

BT - OASIS 2021 - Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks

T2 - 2021 Workshop on Open Challenges in Online Social Networks, OASIS 2021, held in conjunction with the 2021 ACM Conference on Hypertext and Social Media, ACM HT 2021

Y2 - 30 August 2021 through 30 August 2021

ER -