Details
| Original language | English |
|---|---|
| Title of host publication | Digital Libraries for Open Knowledge |
| Subtitle of host publication | 23rd International Conference on Theory and Practice of Digital Libraries, TPDL 2019, Proceedings |
| Editors | Antoine Doucet, Antoine Isaac, Koraljka Golub, Trond Aalberg, Adam Jatowt |
| Publisher | Springer Verlag |
| Pages | 107-114 |
| Number of pages | 8 |
| Edition | 1. |
| ISBN (electronic) | 978-3-030-30760-8 |
| ISBN (print) | 978-3-030-30759-2 |
| Publication status | Published - 30 Aug 2019 |
| Event | 23rd International Conference on Theory and Practice of Digital Libraries, TPDL 2019 - Oslo, Norway Duration: 9 Sept 2019 → 12 Sept 2019 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 11799 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (electronic) | 1611-3349 |
Abstract
Video indexing approaches such as visual concept classification and person recognition are essential to enable fine-grained semantic search in large-scale video archives such as the historical video collection of the former German Democratic Republic (GDR) maintained by the German Broadcasting Archive (DRA). Typically, a lexicon of visual concepts has to be defined for semantic search. But the definition of visual concepts can be more or less subjective due to individually differing judgments of annotators, which may have an impact on training data quality for supervised machine learning methods. In this paper, we analyze the inter-coder agreement on historical TV data of the former GDR for visual concept classification and person recognition. The inter-coder agreement is evaluated for a group of expert as well as non-expert annotators. Furthermore, correlations between visual recognition performance and inter-annotator agreement are measured. In this context, information about training dataset size and agreement are used to predict average precision for concept classification. Finally, the impact of expert vs. non-expert annotations on person recognition is analyzed.
Keywords
- Historical videos, Inter-coder agreement, Performance prediction, Person identification, Visual concept classification
ASJC Scopus subject areas
- Mathematics(all)
- Theoretical Computer Science
- Computer Science(all)
- General Computer Science
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
Digital Libraries for Open Knowledge: 23rd International Conference on Theory and Practice of Digital Libraries, TPDL 2019, Proceedings. ed. / Antoine Doucet; Antoine Isaac; Koraljka Golub; Trond Aalberg; Adam Jatowt. 1. ed. Springer Verlag, 2019. p. 107-114 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11799 LNCS).
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Investigating Correlations of Inter-coder Agreement and Machine Annotation Performance for Historical Video Data
AU - Pustu-Iren, Kader
AU - Mühling, Markus
AU - Korfhage, Nikolaus
AU - Bars, Joanna
AU - Bernhöft, Sabrina
AU - Hörth, Angelika
AU - Freisleben, Bernd
AU - Ewerth, Ralph
N1 - Funding information: This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)-project number 388420599.
PY - 2019/8/30
Y1 - 2019/8/30
N2 - Video indexing approaches such as visual concept classification and person recognition are essential to enable fine-grained semantic search in large-scale video archives such as the historical video collection of the former German Democratic Republic (GDR) maintained by the German Broadcasting Archive (DRA). Typically, a lexicon of visual concepts has to be defined for semantic search. But the definition of visual concepts can be more or less subjective due to individually differing judgments of annotators, which may have an impact on training data quality for supervised machine learning methods. In this paper, we analyze the inter-coder agreement on historical TV data of the former GDR for visual concept classification and person recognition. The inter-coder agreement is evaluated for a group of expert as well as non-expert annotators. Furthermore, correlations between visual recognition performance and inter-annotator agreement are measured. In this context, information about training dataset size and agreement are used to predict average precision for concept classification. Finally, the impact of expert vs. non-expert annotations on person recognition is analyzed.
AB - Video indexing approaches such as visual concept classification and person recognition are essential to enable fine-grained semantic search in large-scale video archives such as the historical video collection of the former German Democratic Republic (GDR) maintained by the German Broadcasting Archive (DRA). Typically, a lexicon of visual concepts has to be defined for semantic search. But the definition of visual concepts can be more or less subjective due to individually differing judgments of annotators, which may have an impact on training data quality for supervised machine learning methods. In this paper, we analyze the inter-coder agreement on historical TV data of the former GDR for visual concept classification and person recognition. The inter-coder agreement is evaluated for a group of expert as well as non-expert annotators. Furthermore, correlations between visual recognition performance and inter-annotator agreement are measured. In this context, information about training dataset size and agreement are used to predict average precision for concept classification. Finally, the impact of expert vs. non-expert annotations on person recognition is analyzed.
KW - Historical videos
KW - Inter-coder agreement
KW - Performance prediction
KW - Person identification
KW - Visual concept classification
UR - http://www.scopus.com/inward/record.url?scp=85072871625&partnerID=8YFLogxK
U2 - 10.48550/arXiv.1907.10450
DO - 10.48550/arXiv.1907.10450
M3 - Conference contribution
AN - SCOPUS:85072871625
SN - 978-3-030-30759-2
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 107
EP - 114
BT - Digital Libraries for Open Knowledge
A2 - Doucet, Antoine
A2 - Isaac, Antoine
A2 - Golub, Koraljka
A2 - Aalberg, Trond
A2 - Jatowt, Adam
PB - Springer Verlag
T2 - 23rd International Conference on Theory and Practice of Digital Libraries, TPDL 2019
Y2 - 9 September 2019 through 12 September 2019
ER -