Self-Supervised Adversarial Shape Completion

Research output: Contribution to journalConference articleResearchpeer review

Authors

  • Torben Peters
  • Konrad Schindler
  • Claus Brenner

External Research Organisations

  • ETH Zurich

Details

Original languageEnglish
Pages (from-to)143-150
Number of pages8
JournalISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Volume5
Issue number2
Publication statusPublished - 17 May 2022
Event2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission II - Nice, France
Duration: 6 Jun 202211 Jun 2022

Abstract

The goal of this paper is 3D shape completion: given an incomplete instance of a known category, hallucinate a complete version of it that is geometrically plausible. We develop an adversarial framework that makes it possible to learn shape completion in a self-supervised fashion, only from incomplete examples. This is enabled by a discriminator network that rejects incomplete shapes, via a loss function that separately assesses local sub-regions of the generated example and accepts only regions with sufficiently high point count. This inductive bias against empty regions forces the generator to output complete shapes. We demonstrate the effectiveness of this approach on synthetic data from ShapeNet and ModelNet, and on a real mobile mapping dataset with nearly 9'000 incomplete cars. Moreover, we apply it to the KITTI autonomous driving dataset without retraining, to highlight its ability to generalise to different data characteristics.

Keywords

    3D Point Clouds, Adversarial Learning, Shape Completion, Unsupervised Learning

ASJC Scopus subject areas

Cite this

Self-Supervised Adversarial Shape Completion. / Peters, Torben; Schindler, Konrad; Brenner, Claus.
In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 5, No. 2, 17.05.2022, p. 143-150.

Research output: Contribution to journalConference articleResearchpeer review

Peters, T, Schindler, K & Brenner, C 2022, 'Self-Supervised Adversarial Shape Completion', ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 5, no. 2, pp. 143-150. https://doi.org/10.5194/isprs-annals-V-2-2022-143-2022
Peters, T., Schindler, K., & Brenner, C. (2022). Self-Supervised Adversarial Shape Completion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 5(2), 143-150. https://doi.org/10.5194/isprs-annals-V-2-2022-143-2022
Peters T, Schindler K, Brenner C. Self-Supervised Adversarial Shape Completion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2022 May 17;5(2):143-150. doi: 10.5194/isprs-annals-V-2-2022-143-2022
Peters, Torben ; Schindler, Konrad ; Brenner, Claus. / Self-Supervised Adversarial Shape Completion. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2022 ; Vol. 5, No. 2. pp. 143-150.
Download
@article{9edc32b73da34bb0a68dd196ef6e4b8a,
title = "Self-Supervised Adversarial Shape Completion",
abstract = "The goal of this paper is 3D shape completion: given an incomplete instance of a known category, hallucinate a complete version of it that is geometrically plausible. We develop an adversarial framework that makes it possible to learn shape completion in a self-supervised fashion, only from incomplete examples. This is enabled by a discriminator network that rejects incomplete shapes, via a loss function that separately assesses local sub-regions of the generated example and accepts only regions with sufficiently high point count. This inductive bias against empty regions forces the generator to output complete shapes. We demonstrate the effectiveness of this approach on synthetic data from ShapeNet and ModelNet, and on a real mobile mapping dataset with nearly 9'000 incomplete cars. Moreover, we apply it to the KITTI autonomous driving dataset without retraining, to highlight its ability to generalise to different data characteristics.",
keywords = "3D Point Clouds, Adversarial Learning, Shape Completion, Unsupervised Learning",
author = "Torben Peters and Konrad Schindler and Claus Brenner",
year = "2022",
month = may,
day = "17",
doi = "10.5194/isprs-annals-V-2-2022-143-2022",
language = "English",
volume = "5",
pages = "143--150",
number = "2",
note = "2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission II ; Conference date: 06-06-2022 Through 11-06-2022",

}

Download

TY - JOUR

T1 - Self-Supervised Adversarial Shape Completion

AU - Peters, Torben

AU - Schindler, Konrad

AU - Brenner, Claus

PY - 2022/5/17

Y1 - 2022/5/17

N2 - The goal of this paper is 3D shape completion: given an incomplete instance of a known category, hallucinate a complete version of it that is geometrically plausible. We develop an adversarial framework that makes it possible to learn shape completion in a self-supervised fashion, only from incomplete examples. This is enabled by a discriminator network that rejects incomplete shapes, via a loss function that separately assesses local sub-regions of the generated example and accepts only regions with sufficiently high point count. This inductive bias against empty regions forces the generator to output complete shapes. We demonstrate the effectiveness of this approach on synthetic data from ShapeNet and ModelNet, and on a real mobile mapping dataset with nearly 9'000 incomplete cars. Moreover, we apply it to the KITTI autonomous driving dataset without retraining, to highlight its ability to generalise to different data characteristics.

AB - The goal of this paper is 3D shape completion: given an incomplete instance of a known category, hallucinate a complete version of it that is geometrically plausible. We develop an adversarial framework that makes it possible to learn shape completion in a self-supervised fashion, only from incomplete examples. This is enabled by a discriminator network that rejects incomplete shapes, via a loss function that separately assesses local sub-regions of the generated example and accepts only regions with sufficiently high point count. This inductive bias against empty regions forces the generator to output complete shapes. We demonstrate the effectiveness of this approach on synthetic data from ShapeNet and ModelNet, and on a real mobile mapping dataset with nearly 9'000 incomplete cars. Moreover, we apply it to the KITTI autonomous driving dataset without retraining, to highlight its ability to generalise to different data characteristics.

KW - 3D Point Clouds

KW - Adversarial Learning

KW - Shape Completion

KW - Unsupervised Learning

UR - http://www.scopus.com/inward/record.url?scp=85132294058&partnerID=8YFLogxK

U2 - 10.5194/isprs-annals-V-2-2022-143-2022

DO - 10.5194/isprs-annals-V-2-2022-143-2022

M3 - Conference article

AN - SCOPUS:85132294058

VL - 5

SP - 143

EP - 150

JO - ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

JF - ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

SN - 2194-9042

IS - 2

T2 - 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission II

Y2 - 6 June 2022 through 11 June 2022

ER -