TY - JOUR

T1 - Self-supervised learning of face appearances in tv casts and movies

AU - Ewerth, Ralph

AU - Mühling, Markus

AU - Freisleben, Bernd

N1 - Funding Information: This work is financially supported by the Deutsche Forschungsgemeinschaft (SFB/FK 615, Projekt MT). We would like to thank the reviewers for their valuable comments which helped to improve the quality of the paper.

PY - 2007/6/1

Y1 - 2007/6/1

N2 - Retrieving information about the occurrences of persons in a video is an important task in many video indexing and retrieval applications. The problem is to answer the question 'In which shots and scenes does person X appear?'. In this paper, we present an automatic video annotation system with respect to a person's appearance based on state-of-the-art algorithms for face detection, tracking and recognition. In contrast to many related approaches, knowledge about the persons in a given video is not assumed in advance. Adaboost is employed after an initial clustering of faces to select the best features describing a person's face. These features are then used to train new classifiers based only on the faces extracted from the video under consideration. Several possibilities to train Adaboost and Support Vector Machine (ensemble) classifiers directly on a video are compared. Finally, experimental results demonstrate the effectiveness of correcting in-plane face rotation and of the employed self-supervised learning method.

AB - Retrieving information about the occurrences of persons in a video is an important task in many video indexing and retrieval applications. The problem is to answer the question 'In which shots and scenes does person X appear?'. In this paper, we present an automatic video annotation system with respect to a person's appearance based on state-of-the-art algorithms for face detection, tracking and recognition. In contrast to many related approaches, knowledge about the persons in a given video is not assumed in advance. Adaboost is employed after an initial clustering of faces to select the best features describing a person's face. These features are then used to train new classifiers based only on the faces extracted from the video under consideration. Several possibilities to train Adaboost and Support Vector Machine (ensemble) classifiers directly on a video are compared. Finally, experimental results demonstrate the effectiveness of correcting in-plane face rotation and of the employed self-supervised learning method.

KW - face recognition/clustering

KW - Person indexing

KW - self-supervised learning

UR - http://www.scopus.com/inward/record.url?scp=60349086111&partnerID=8YFLogxK

U2 - 10.1142/S1793351X0700010X

DO - 10.1142/S1793351X0700010X

M3 - Article

AN - SCOPUS:60349086111

VL - 1

SP - 185

EP - 204

JO - International Journal of Semantic Computing

JF - International Journal of Semantic Computing

SN - 1793-351X

IS - 2

ER -