Loading [MathJax]/extensions/tex2jax.js

Target-Tailored Source-Transformation for Scene Graph Generation

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Autorschaft

  • Wentong Liao
  • Cuiling Lan
  • Wenjun Zeng
  • Michael Ying Yang
  • Bodo Rosenhahn

Externe Organisationen

  • Microsoft Research
  • University of Twente

Details

OriginalspracheEnglisch
Titel des SammelwerksProceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
Untertitel2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
ErscheinungsortNashville, TN, USA
Seiten1663-1671
Seitenumfang9
ISBN (elektronisch)978-1-6654-4899-4
PublikationsstatusVeröffentlicht - 2021
Veranstaltung2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021 - Virtual, Online, USA / Vereinigte Staaten
Dauer: 19 Juni 202125 Juni 2021

Publikationsreihe

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN (Print)2160-7508
ISSN (elektronisch)2160-7516

Abstract

Scene graph generation aims to provide a semantic and structural description of an image, denoting the objects (with nodes) and their relationships (with edges). The best performing works to date are based on exploiting the context surrounding objects or relations,e.g., by passing information among objects. In these approaches, to transform the representation of source objects is a critical process for extracting information for the use by target objects. In this work, we argue that a source object should give what tar-get object needs and give different objects different information rather than contributing common information to all targets. To achieve this goal, we propose a Target-TailoredSource-Transformation (TTST) method to efficiently propagate information among object proposals and relations. Particularly, for a source object proposal which will contribute information to other target objects, we transform the source object feature to the target object feature domain by simultaneously taking both the source and target into account. We further explore more powerful representations by integrating language prior with the visual context in the transformation for the scene graph generation. By doing so the target object is able to extract target-specific information from the source object and source relation accordingly to refine its representation. Our framework is validated on the Visual Genome bench-mark and demonstrated its state-of-the-art performance for the scene graph generation. The experimental results show that the performance of object detection and visual relation-ship detection are promoted mutually by our method.

ASJC Scopus Sachgebiete

Zitieren

Target-Tailored Source-Transformation for Scene Graph Generation. / Liao, Wentong; Lan, Cuiling; Zeng, Wenjun et al.
Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville, TN, USA , 2021. S. 1663-1671 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).

Publikation: Beitrag in Buch/Bericht/Sammelwerk/KonferenzbandAufsatz in KonferenzbandForschungPeer-Review

Liao, W, Lan, C, Zeng, W, Yang, MY & Rosenhahn, B 2021, Target-Tailored Source-Transformation for Scene Graph Generation. in Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Nashville, TN, USA , S. 1663-1671, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021, Virtual, Online, USA / Vereinigte Staaten, 19 Juni 2021. https://doi.org/10.1109/CVPRW53098.2021.00182
Liao, W., Lan, C., Zeng, W., Yang, M. Y., & Rosenhahn, B. (2021). Target-Tailored Source-Transformation for Scene Graph Generation. In Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (S. 1663-1671). (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).. https://doi.org/10.1109/CVPRW53098.2021.00182
Liao W, Lan C, Zeng W, Yang MY, Rosenhahn B. Target-Tailored Source-Transformation for Scene Graph Generation. in Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville, TN, USA . 2021. S. 1663-1671. (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). doi: 10.1109/CVPRW53098.2021.00182
Liao, Wentong ; Lan, Cuiling ; Zeng, Wenjun et al. / Target-Tailored Source-Transformation for Scene Graph Generation. Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville, TN, USA , 2021. S. 1663-1671 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).
Download
@inproceedings{aa6facdc1486434da53424dcb68d7bd5,
title = "Target-Tailored Source-Transformation for Scene Graph Generation",
abstract = "Scene graph generation aims to provide a semantic and structural description of an image, denoting the objects (with nodes) and their relationships (with edges). The best performing works to date are based on exploiting the context surrounding objects or relations,e.g., by passing information among objects. In these approaches, to transform the representation of source objects is a critical process for extracting information for the use by target objects. In this work, we argue that a source object should give what tar-get object needs and give different objects different information rather than contributing common information to all targets. To achieve this goal, we propose a Target-TailoredSource-Transformation (TTST) method to efficiently propagate information among object proposals and relations. Particularly, for a source object proposal which will contribute information to other target objects, we transform the source object feature to the target object feature domain by simultaneously taking both the source and target into account. We further explore more powerful representations by integrating language prior with the visual context in the transformation for the scene graph generation. By doing so the target object is able to extract target-specific information from the source object and source relation accordingly to refine its representation. Our framework is validated on the Visual Genome bench-mark and demonstrated its state-of-the-art performance for the scene graph generation. The experimental results show that the performance of object detection and visual relation-ship detection are promoted mutually by our method. ",
keywords = "cs.CV",
author = "Wentong Liao and Cuiling Lan and Wenjun Zeng and Yang, {Michael Ying} and Bodo Rosenhahn",
note = "Funding Information: This work was supported by the Center for Digital Innovations (ZDIN), Federal Ministry of Education and Research (BMBF), Germany under the project LeibnizKILa-bor(grant no.01DD20003) and the Deutsche Forschungs-gemeinschaft (DFG) under Germany{\textquoteright}s Excellence Strategy within the Cluster of Excellence PhoenixD (EXC 2122).; 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021 ; Conference date: 19-06-2021 Through 25-06-2021",
year = "2021",
doi = "10.1109/CVPRW53098.2021.00182",
language = "English",
series = "IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops",
pages = "1663--1671",
booktitle = "Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021",

}

Download

TY - GEN

T1 - Target-Tailored Source-Transformation for Scene Graph Generation

AU - Liao, Wentong

AU - Lan, Cuiling

AU - Zeng, Wenjun

AU - Yang, Michael Ying

AU - Rosenhahn, Bodo

N1 - Funding Information: This work was supported by the Center for Digital Innovations (ZDIN), Federal Ministry of Education and Research (BMBF), Germany under the project LeibnizKILa-bor(grant no.01DD20003) and the Deutsche Forschungs-gemeinschaft (DFG) under Germany’s Excellence Strategy within the Cluster of Excellence PhoenixD (EXC 2122).

PY - 2021

Y1 - 2021

N2 - Scene graph generation aims to provide a semantic and structural description of an image, denoting the objects (with nodes) and their relationships (with edges). The best performing works to date are based on exploiting the context surrounding objects or relations,e.g., by passing information among objects. In these approaches, to transform the representation of source objects is a critical process for extracting information for the use by target objects. In this work, we argue that a source object should give what tar-get object needs and give different objects different information rather than contributing common information to all targets. To achieve this goal, we propose a Target-TailoredSource-Transformation (TTST) method to efficiently propagate information among object proposals and relations. Particularly, for a source object proposal which will contribute information to other target objects, we transform the source object feature to the target object feature domain by simultaneously taking both the source and target into account. We further explore more powerful representations by integrating language prior with the visual context in the transformation for the scene graph generation. By doing so the target object is able to extract target-specific information from the source object and source relation accordingly to refine its representation. Our framework is validated on the Visual Genome bench-mark and demonstrated its state-of-the-art performance for the scene graph generation. The experimental results show that the performance of object detection and visual relation-ship detection are promoted mutually by our method.

AB - Scene graph generation aims to provide a semantic and structural description of an image, denoting the objects (with nodes) and their relationships (with edges). The best performing works to date are based on exploiting the context surrounding objects or relations,e.g., by passing information among objects. In these approaches, to transform the representation of source objects is a critical process for extracting information for the use by target objects. In this work, we argue that a source object should give what tar-get object needs and give different objects different information rather than contributing common information to all targets. To achieve this goal, we propose a Target-TailoredSource-Transformation (TTST) method to efficiently propagate information among object proposals and relations. Particularly, for a source object proposal which will contribute information to other target objects, we transform the source object feature to the target object feature domain by simultaneously taking both the source and target into account. We further explore more powerful representations by integrating language prior with the visual context in the transformation for the scene graph generation. By doing so the target object is able to extract target-specific information from the source object and source relation accordingly to refine its representation. Our framework is validated on the Visual Genome bench-mark and demonstrated its state-of-the-art performance for the scene graph generation. The experimental results show that the performance of object detection and visual relation-ship detection are promoted mutually by our method.

KW - cs.CV

UR - http://www.scopus.com/inward/record.url?scp=85115193370&partnerID=8YFLogxK

U2 - 10.1109/CVPRW53098.2021.00182

DO - 10.1109/CVPRW53098.2021.00182

M3 - Conference contribution

T3 - IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

SP - 1663

EP - 1671

BT - Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021

CY - Nashville, TN, USA

T2 - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021

Y2 - 19 June 2021 through 25 June 2021

ER -

Von denselben Autoren