Details
| Original language | English |
|---|---|
| Pages (from-to) | 319-333 |
| Number of pages | 15 |
| Journal | International Journal on Digital Libraries |
| Volume | 23 |
| Issue number | 4 |
| Early online date | 10 Sept 2022 |
| Publication status | Published - Dec 2022 |
Abstract
Video retrieval methods, e.g., for visual concept classification, person recognition, and similarity search, are essential to perform fine-grained semantic search in large video archives. However, such retrieval methods often have to be adapted to the users’ changing search requirements: which concepts or persons are frequently searched for, what research topics are currently important or will be relevant in the future? In this paper, we present VIVA, a software tool for building content-based video retrieval methods based on deep learning models. VIVA allows non-expert users to conduct visual information retrieval for concepts and persons in video archives and to add new people or concepts to the underlying deep learning models as new requirements arise. For this purpose, VIVA provides a novel semi-automatic data acquisition workflow including a web crawler, image similarity search, as well as review and user feedback components to reduce the time-consuming manual effort for collecting training samples. We present experimental retrieval results using VIVA for four use cases in the context of a historical video collection of the German Broadcasting Archive based on about 34,000 h of television recordings from the former German Democratic Republic (GDR). We evaluate the performance of deep learning models built using VIVA for 91 GDR specific concepts and 98 personalities from the former GDR as well as the performance of the image and person similarity search approaches.
Keywords
- Deep learning, German broadcasting archive, Video mining, Video retrieval, Visual information retrieval
ASJC Scopus subject areas
- Social Sciences(all)
- Library and Information Sciences
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: International Journal on Digital Libraries, Vol. 23, No. 4, 12.2022, p. 319-333.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - VIVA
T2 - visual information retrieval in video archives
AU - Mühling, Markus
AU - Korfhage, Nikolaus
AU - Pustu-Iren, Kader
AU - Bars, Joanna
AU - Knapp, Mario
AU - Bellafkir, Hicham
AU - Vogelbacher, Markus
AU - Schneider, Daniel
AU - Hörth, Angelika
AU - Ewerth, Ralph
AU - Freisleben, Bernd
N1 - Funding Information: This work is financially supported by the German Research Foundation (DFG project number 388420599).
PY - 2022/12
Y1 - 2022/12
N2 - Video retrieval methods, e.g., for visual concept classification, person recognition, and similarity search, are essential to perform fine-grained semantic search in large video archives. However, such retrieval methods often have to be adapted to the users’ changing search requirements: which concepts or persons are frequently searched for, what research topics are currently important or will be relevant in the future? In this paper, we present VIVA, a software tool for building content-based video retrieval methods based on deep learning models. VIVA allows non-expert users to conduct visual information retrieval for concepts and persons in video archives and to add new people or concepts to the underlying deep learning models as new requirements arise. For this purpose, VIVA provides a novel semi-automatic data acquisition workflow including a web crawler, image similarity search, as well as review and user feedback components to reduce the time-consuming manual effort for collecting training samples. We present experimental retrieval results using VIVA for four use cases in the context of a historical video collection of the German Broadcasting Archive based on about 34,000 h of television recordings from the former German Democratic Republic (GDR). We evaluate the performance of deep learning models built using VIVA for 91 GDR specific concepts and 98 personalities from the former GDR as well as the performance of the image and person similarity search approaches.
AB - Video retrieval methods, e.g., for visual concept classification, person recognition, and similarity search, are essential to perform fine-grained semantic search in large video archives. However, such retrieval methods often have to be adapted to the users’ changing search requirements: which concepts or persons are frequently searched for, what research topics are currently important or will be relevant in the future? In this paper, we present VIVA, a software tool for building content-based video retrieval methods based on deep learning models. VIVA allows non-expert users to conduct visual information retrieval for concepts and persons in video archives and to add new people or concepts to the underlying deep learning models as new requirements arise. For this purpose, VIVA provides a novel semi-automatic data acquisition workflow including a web crawler, image similarity search, as well as review and user feedback components to reduce the time-consuming manual effort for collecting training samples. We present experimental retrieval results using VIVA for four use cases in the context of a historical video collection of the German Broadcasting Archive based on about 34,000 h of television recordings from the former German Democratic Republic (GDR). We evaluate the performance of deep learning models built using VIVA for 91 GDR specific concepts and 98 personalities from the former GDR as well as the performance of the image and person similarity search approaches.
KW - Deep learning
KW - German broadcasting archive
KW - Video mining
KW - Video retrieval
KW - Visual information retrieval
UR - http://www.scopus.com/inward/record.url?scp=85137839339&partnerID=8YFLogxK
U2 - 10.1007/s00799-022-00337-y
DO - 10.1007/s00799-022-00337-y
M3 - Article
AN - SCOPUS:85137839339
VL - 23
SP - 319
EP - 333
JO - International Journal on Digital Libraries
JF - International Journal on Digital Libraries
SN - 1432-5012
IS - 4
ER -