TY - JOUR

T1 - Domain Adaptation for Head Pose Estimation Using Relative Pose Consistency

AU - Kuhnke, Felix

AU - Ostermann, Jorn

PY - 2023/7

Y1 - 2023/7

N2 - Head pose estimation plays a vital role in biometric systems related to facial and human behavior analysis. Typically, neural networks are trained on head pose datasets. Unfortunately, manual or sensor-based annotation of head pose is impractical. A solution is synthetic training data generated from 3D face models, which can provide an infinite number of perfect labels. However, computer generated images only provide an approximation of real-world images, leading to a performance gap between training and application domain. Therefore, there is a need for strategies that allow simultaneous learning on labeled synthetic data and unlabeled real-world data to overcome the domain gap. In this work we propose relative pose consistency, a semi-supervised learning strategy for head pose estimation based on consistency regularization. Consistency regularization enforces consistent network predictions under random image augmentations, including pose-preserving and pose-altering augmentations. We propose a strategy to exploit the relative pose introduced by pose-altering augmentations between augmented image pairs, to allow the network to benefit from relative pose labels during training on unlabeled data. We evaluate our approach in a domain-adaptation scenario and in a commonly used cross-dataset scenario. Furthermore, we reproduce related works to enforce consistent evaluation protocols and show that for both scenarios we outperform SOTA.

AB - Head pose estimation plays a vital role in biometric systems related to facial and human behavior analysis. Typically, neural networks are trained on head pose datasets. Unfortunately, manual or sensor-based annotation of head pose is impractical. A solution is synthetic training data generated from 3D face models, which can provide an infinite number of perfect labels. However, computer generated images only provide an approximation of real-world images, leading to a performance gap between training and application domain. Therefore, there is a need for strategies that allow simultaneous learning on labeled synthetic data and unlabeled real-world data to overcome the domain gap. In this work we propose relative pose consistency, a semi-supervised learning strategy for head pose estimation based on consistency regularization. Consistency regularization enforces consistent network predictions under random image augmentations, including pose-preserving and pose-altering augmentations. We propose a strategy to exploit the relative pose introduced by pose-altering augmentations between augmented image pairs, to allow the network to benefit from relative pose labels during training on unlabeled data. We evaluate our approach in a domain-adaptation scenario and in a commonly used cross-dataset scenario. Furthermore, we reproduce related works to enforce consistent evaluation protocols and show that for both scenarios we outperform SOTA.

KW - Behavioral sciences

KW - Consistency Regularization

KW - Deep Learning

KW - Domain Adaptation

KW - Feature extraction

KW - Head Pose Estimation

KW - Pose estimation

KW - Task analysis

KW - Three-dimensional displays

KW - Training

KW - Training data

KW - consistency regularization

KW - deep learning

KW - domain adaptation

KW - Head pose estimation

UR - http://www.scopus.com/inward/record.url?scp=85147281573&partnerID=8YFLogxK

U2 - 10.1109/TBIOM.2023.3237039

DO - 10.1109/TBIOM.2023.3237039

M3 - Article

AN - SCOPUS:85147281573

VL - 5

SP - 348

EP - 359

JO - IEEE Transactions on Biometrics, Behavior, and Identity Science

JF - IEEE Transactions on Biometrics, Behavior, and Identity Science

IS - 3

ER -