Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling

Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung

Autoren

  • Lin Chen
  • Daixin Zhao
  • Christian Heipke

Organisationseinheiten

Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Titel des SammelwerksBeiträge
Untertitel39. Wissenschaftlich-Technische Jahrestagung der DGPF e.V.
Herausgeber/-innenThomas P. Kersten
ErscheinungsortMünchen
Seiten375-386
Seitenumfang12
PublikationsstatusVeröffentlicht - 2019
VeranstaltungDreiländertagung OVG-DGPF-SGPF: Photogrammetrie-Fernerkundung-Geoinformation - Universität für Bodenkultur Wien, Wien, Österreich
Dauer: 20 Feb. 2019 → 22 Feb. 2019
https://dgpf.de/con/jt2019.html

Publikationsreihe

NamePublikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.
Herausgeber (Verlag)Deutsche Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.
Band28
ISSN (Print)0942-2870

Abstract

In this paper, we present a complementarity constraint for features computed from different sources of input data before fusion in semantic labelling. A two-branch encoder-decoder architecture with ResNet-50 is proposed and used as classification network. Our proposed complementarity constraint is added to the standard softmax cross-entropy classi-fication loss. The impact of different weights for this constraint in multi-modal data fusion is investigated. The result of the two branch network is also compared to the one obtained with only the spectral information. The constraint is shown to improve the results consistently in our experiments. Different amounts of improvement are achieved when different weighs for the complementarity constraint are used.

Zitieren

Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling. / Chen, Lin; Zhao, Daixin; Heipke, Christian.
Beiträge: 39. Wissenschaftlich-Technische Jahrestagung der DGPF e.V.. Hrsg. / Thomas P. Kersten. München, 2019. S. 375-386 (Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.; Band 28).

Publikation: Beitrag in Buch/Bericht/Sammelwerk/Konferenzband › Aufsatz in Konferenzband › Forschung

Chen, L, Zhao, D & Heipke, C 2019, Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling. in TP Kersten (Hrsg.), Beiträge: 39. Wissenschaftlich-Technische Jahrestagung der DGPF e.V.. Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V., Bd. 28, München, S. 375-386, Dreiländertagung OVG-DGPF-SGPF, Wien, Österreich, 20 Feb. 2019. <https://www.dgpf.de/src/tagung/jt2019/proceedings/proceedings/papers/69_3LT2019_Chen_et_al.pdf>
Chen, L., Zhao, D., & Heipke, C. (2019). Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling. In T. P. Kersten (Hrsg.), Beiträge: 39. Wissenschaftlich-Technische Jahrestagung der DGPF e.V. (S. 375-386). (Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.; Band 28).. https://www.dgpf.de/src/tagung/jt2019/proceedings/proceedings/papers/69_3LT2019_Chen_et_al.pdf
Chen L, Zhao D, Heipke C. Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling. in Kersten TP, Hrsg., Beiträge: 39. Wissenschaftlich-Technische Jahrestagung der DGPF e.V.. München. 2019. S. 375-386. (Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.).
Chen, Lin ; Zhao, Daixin ; Heipke, Christian. / Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling. Beiträge: 39. Wissenschaftlich-Technische Jahrestagung der DGPF e.V.. Hrsg. / Thomas P. Kersten. München, 2019. S. 375-386 (Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.).
Download
@inproceedings{764c653dcdac483886fef83d5841dbae,
title = "Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling",
abstract = "In this paper, we present a complementarity constraint for features computed from different sources of input data before fusion in semantic labelling. A two-branch encoder-decoder architecture with ResNet-50 is proposed and used as classification network. Our proposed complementarity constraint is added to the standard softmax cross-entropy classi-fication loss. The impact of different weights for this constraint in multi-modal data fusion is investigated. The result of the two branch network is also compared to the one obtained with only the spectral information. The constraint is shown to improve the results consistently in our experiments. Different amounts of improvement are achieved when different weighs for the complementarity constraint are used.",
author = "Lin Chen and Daixin Zhao and Christian Heipke",
note = "Funding Information: Part of this work is done by DAIXIN ZHAO during an internship at Institute of Photogrammetry and GeoInformation, Leibniz Universit{\"a}t Hannover. The authors LIN CHEN and CHRISTIAN HEIPKE are grateful to NVIDIA Corp. for the GPU donation. Also, the author LIN CHEN and DAIXIN ZHAO are grateful to CHUN YANG and JUNHUA KANG for valuable discussions.; Dreil{\"a}ndertagung OVG-DGPF-SGPF ; Conference date: 20-02-2019 Through 22-02-2019",
year = "2019",
language = "English",
series = "Publikationen der Deutschen Gesellschaft f{\"u}r Photogrammetrie, Fernerkundung und Geoinformation e.V.",
publisher = "Deutsche Gesellschaft f{\"u}r Photogrammetrie, Fernerkundung und Geoinformation e.V.",
pages = "375--386",
editor = "Kersten, {Thomas P.}",
booktitle = "Beitr{\"a}ge",
url = "https://dgpf.de/con/jt2019.html",

}

Download

TY - GEN

T1 - Complementary Features Learning from RGB and Depth Information for Semantic Image Labelling

AU - Chen, Lin

AU - Zhao, Daixin

AU - Heipke, Christian

N1 - Funding Information: Part of this work is done by DAIXIN ZHAO during an internship at Institute of Photogrammetry and GeoInformation, Leibniz Universität Hannover. The authors LIN CHEN and CHRISTIAN HEIPKE are grateful to NVIDIA Corp. for the GPU donation. Also, the author LIN CHEN and DAIXIN ZHAO are grateful to CHUN YANG and JUNHUA KANG for valuable discussions.

PY - 2019

Y1 - 2019

N2 - In this paper, we present a complementarity constraint for features computed from different sources of input data before fusion in semantic labelling. A two-branch encoder-decoder architecture with ResNet-50 is proposed and used as classification network. Our proposed complementarity constraint is added to the standard softmax cross-entropy classi-fication loss. The impact of different weights for this constraint in multi-modal data fusion is investigated. The result of the two branch network is also compared to the one obtained with only the spectral information. The constraint is shown to improve the results consistently in our experiments. Different amounts of improvement are achieved when different weighs for the complementarity constraint are used.

AB - In this paper, we present a complementarity constraint for features computed from different sources of input data before fusion in semantic labelling. A two-branch encoder-decoder architecture with ResNet-50 is proposed and used as classification network. Our proposed complementarity constraint is added to the standard softmax cross-entropy classi-fication loss. The impact of different weights for this constraint in multi-modal data fusion is investigated. The result of the two branch network is also compared to the one obtained with only the spectral information. The constraint is shown to improve the results consistently in our experiments. Different amounts of improvement are achieved when different weighs for the complementarity constraint are used.

M3 - Conference contribution

T3 - Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e.V.

SP - 375

EP - 386

BT - Beiträge

A2 - Kersten, Thomas P.

CY - München

T2 - Dreiländertagung OVG-DGPF-SGPF

Y2 - 20 February 2019 through 22 February 2019

ER -