Details
| Original language | English |
|---|---|
| Title of host publication | MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia |
| Pages | 1107-1116 |
| Number of pages | 10 |
| ISBN (electronic) | 9781450392037 |
| Publication status | Published - Oct 2022 |
Abstract
Gesture as language of non-verbal communication has been theoretically established since the 17th century. However, its relevance for the visual arts has been expressed only sporadically. This may be primarily due to the sheer overwhelming amount of data that traditionally had to be processed by hand. With the steady progress of digitization, though, a growing number of historical artifacts have been indexed and made available to the public, creating a need for automatic retrieval of art-historical motifs with similar body constellations or poses. Since the domain of art differs significantly from existing real-world data sets for human pose estimation due to its style variance, this presents new challenges. In this paper, we propose a novel approach to estimate human poses in art-historical images. In contrast to previous work that attempts to bridge the domain gap with pre-trained models or through style transfer, we suggest semi-supervised learning for both object and keypoint detection. Furthermore, we introduce a novel domain-specific art data set that includes both bounding box and keypoint annotations of human figures. Our approach achieves significantly better results than methods that use pre-trained models or style transfer.
Keywords
- art history, human pose estimation, semi-supervised learning, style transfer
ASJC Scopus subject areas
- Computer Science(all)
- Software
- Computer Science(all)
- Artificial Intelligence
- Computer Science(all)
- Human-Computer Interaction
- Computer Science(all)
- Computer Graphics and Computer-Aided Design
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia. 2022. p. 1107-1116.
Research output: Chapter in book/report/conference proceeding › Conference contribution › Research › peer review
}
TY - GEN
T1 - Semi-supervised Human Pose Estimation in Art-historical Images
AU - Springstein, Matthias
AU - Schneider, Stefanie
AU - Althaus, Christian
AU - Ewerth, Ralph
N1 - Funding Information: This work was partly funded by the German Research Foundation (DFG) under project number 415796915. We thank the participants in the retrieval study for their valuable contributions.
PY - 2022/10
Y1 - 2022/10
N2 - Gesture as language of non-verbal communication has been theoretically established since the 17th century. However, its relevance for the visual arts has been expressed only sporadically. This may be primarily due to the sheer overwhelming amount of data that traditionally had to be processed by hand. With the steady progress of digitization, though, a growing number of historical artifacts have been indexed and made available to the public, creating a need for automatic retrieval of art-historical motifs with similar body constellations or poses. Since the domain of art differs significantly from existing real-world data sets for human pose estimation due to its style variance, this presents new challenges. In this paper, we propose a novel approach to estimate human poses in art-historical images. In contrast to previous work that attempts to bridge the domain gap with pre-trained models or through style transfer, we suggest semi-supervised learning for both object and keypoint detection. Furthermore, we introduce a novel domain-specific art data set that includes both bounding box and keypoint annotations of human figures. Our approach achieves significantly better results than methods that use pre-trained models or style transfer.
AB - Gesture as language of non-verbal communication has been theoretically established since the 17th century. However, its relevance for the visual arts has been expressed only sporadically. This may be primarily due to the sheer overwhelming amount of data that traditionally had to be processed by hand. With the steady progress of digitization, though, a growing number of historical artifacts have been indexed and made available to the public, creating a need for automatic retrieval of art-historical motifs with similar body constellations or poses. Since the domain of art differs significantly from existing real-world data sets for human pose estimation due to its style variance, this presents new challenges. In this paper, we propose a novel approach to estimate human poses in art-historical images. In contrast to previous work that attempts to bridge the domain gap with pre-trained models or through style transfer, we suggest semi-supervised learning for both object and keypoint detection. Furthermore, we introduce a novel domain-specific art data set that includes both bounding box and keypoint annotations of human figures. Our approach achieves significantly better results than methods that use pre-trained models or style transfer.
KW - art history
KW - human pose estimation
KW - semi-supervised learning
KW - style transfer
UR - http://www.scopus.com/inward/record.url?scp=85146864522&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2207.02976
DO - 10.48550/arXiv.2207.02976
M3 - Conference contribution
SP - 1107
EP - 1116
BT - MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia
ER -