TY - GEN

T1 - Probabilistic Vehicle Reconstruction Using a Multi-Task CNN

AU - Coenen, Max

AU - Rottensteiner, Franz

N1 - Funding information: This work was supported by the German Research Foun- dation (DFG) as a part of the Research Training Group i.c.sens [GRK2159].

PY - 2019/10/30

Y1 - 2019/10/30

N2 - The retrieval of the 3D pose and shape of objects from images is an ill-posed problem. A common way to object reconstruction is to match entities such as keypoints, edges, or contours of a deformable 3D model, used as shape prior, to their corresponding entities inferred from the image. However, such approaches are highly sensitive to model initialisation, imprecise keypoint localisations and/or illumination conditions. In this paper, we present a probabilistic approach for shape-aware 3D vehicle reconstruction from stereo images that leverages the outputs of a novel multi-task CNN. Specifically, we train a CNN that outputs probability distributions for the vehicle's orientation and for both, vehicle keypoints and wireframe edges. Together with 3D stereo information we integrate the predicted distributions into a common probabilistic framework. We believe that the CNN-based detection of wireframe edges reduces the sensitivity to illumination conditions and object contrast and that using the raw probability maps instead of inferring keypoint positions reduces the sensitivity to keypoint localisation errors. We show that our method achieves state-of-the-art results, evaluating our method on the challenging KITTI benchmark and on our own new 'Stereo-Vehicle' dataset.

AB - The retrieval of the 3D pose and shape of objects from images is an ill-posed problem. A common way to object reconstruction is to match entities such as keypoints, edges, or contours of a deformable 3D model, used as shape prior, to their corresponding entities inferred from the image. However, such approaches are highly sensitive to model initialisation, imprecise keypoint localisations and/or illumination conditions. In this paper, we present a probabilistic approach for shape-aware 3D vehicle reconstruction from stereo images that leverages the outputs of a novel multi-task CNN. Specifically, we train a CNN that outputs probability distributions for the vehicle's orientation and for both, vehicle keypoints and wireframe edges. Together with 3D stereo information we integrate the predicted distributions into a common probabilistic framework. We believe that the CNN-based detection of wireframe edges reduces the sensitivity to illumination conditions and object contrast and that using the raw probability maps instead of inferring keypoint positions reduces the sensitivity to keypoint localisation errors. We show that our method achieves state-of-the-art results, evaluating our method on the challenging KITTI benchmark and on our own new 'Stereo-Vehicle' dataset.

KW - 3D scene understanding

KW - Multi task CNN

KW - Pose estimation

KW - Vehicle reconstruction

UR - http://www.scopus.com/inward/record.url?scp=85082443659&partnerID=8YFLogxK

U2 - 10.48550/arXiv.2102.10681

DO - 10.48550/arXiv.2102.10681

M3 - Conference contribution

AN - SCOPUS:85082443659

SN - 978-1-7281-5024-6

T3 - International Conference on Computer Vision Workshops (ICCV)

SP - 822

EP - 831

BT - 2019 International Conference on Computer Vision (ICCVW)

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE/CVF 17th International Conference on Computer Vision Workshop (ICCVW)

Y2 - 27 October 2019 through 28 October 2019

ER -