A Comparison of Evaluation Techniques for Building Extraction from Airborne Laser Scanning

Research output: Contribution to journalArticleResearchpeer review

Authors

  • Martin Rutzinger
  • Franz Rottensteiner
  • Norbert Pfeifer

External Research Organisations

  • University of Innsbruck
  • International Institute for Geo-Information Science and Earth Observation - ITC
  • University of Melbourne
  • TU Wien (TUW)
View graph of relations

Details

Original languageEnglish
Pages (from-to)11-20
Number of pages10
JournalIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Volume2
Issue number1
Publication statusPublished - 1 Mar 2009

Abstract

In this paper, different methods for the evaluation of building detection algorithms are compared. Whereas pixel-based evaluation gives estimates of the area that is correctly classified, the results are distorted by errors at the building outlines. These distortions are potentially in an order of 30%. Object-based evaluation techniques are less affected by such errors. However, the performance metrics thus delivered are sometimes considered to be less objective, because the definition of a “correct detection” is not unique. Based on a critical review of existing performance metrics, selected methods for the evaluation of building detection results are presented. These methods are used to evaluate the results of two different building detection algorithms in two test sites. A comparison of the evaluation techniques shows that they highlight different properties of the building detection results. As a consequence, a comprehensive evaluation strategy involving quality metrics derived by different methods is proposed.

Keywords

    Airborne laser scanning (ALS), automatic building extraction, metrics, object detection, object-based error assessment, performance evaluation

ASJC Scopus subject areas

Cite this

A Comparison of Evaluation Techniques for Building Extraction from Airborne Laser Scanning. / Rutzinger, Martin; Rottensteiner, Franz; Pfeifer, Norbert.
In: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 2, No. 1, 01.03.2009, p. 11-20.

Research output: Contribution to journalArticleResearchpeer review

Download
@article{edbfc86d63da43fcb3794a47203264e2,
title = "A Comparison of Evaluation Techniques for Building Extraction from Airborne Laser Scanning",
abstract = "In this paper, different methods for the evaluation of building detection algorithms are compared. Whereas pixel-based evaluation gives estimates of the area that is correctly classified, the results are distorted by errors at the building outlines. These distortions are potentially in an order of 30%. Object-based evaluation techniques are less affected by such errors. However, the performance metrics thus delivered are sometimes considered to be less objective, because the definition of a “correct detection” is not unique. Based on a critical review of existing performance metrics, selected methods for the evaluation of building detection results are presented. These methods are used to evaluate the results of two different building detection algorithms in two test sites. A comparison of the evaluation techniques shows that they highlight different properties of the building detection results. As a consequence, a comprehensive evaluation strategy involving quality metrics derived by different methods is proposed.",
keywords = "Airborne laser scanning (ALS), automatic building extraction, metrics, object detection, object-based error assessment, performance evaluation",
author = "Martin Rutzinger and Franz Rottensteiner and Norbert Pfeifer",
note = "Funding Information: Manuscript received October 30, 2007; revised August 28, 2008. First published March 04, 2009; current version published April 29, 2009. This work supported in part by the research program of the CRC for Spatial Information, established and supported under the Australian Cooperative Research Centres Program.",
year = "2009",
month = mar,
day = "1",
doi = "10.1109/JSTARS.2009.2012488",
language = "English",
volume = "2",
pages = "11--20",
journal = "IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing",
issn = "1939-1404",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "1",

}

Download

TY - JOUR

T1 - A Comparison of Evaluation Techniques for Building Extraction from Airborne Laser Scanning

AU - Rutzinger, Martin

AU - Rottensteiner, Franz

AU - Pfeifer, Norbert

N1 - Funding Information: Manuscript received October 30, 2007; revised August 28, 2008. First published March 04, 2009; current version published April 29, 2009. This work supported in part by the research program of the CRC for Spatial Information, established and supported under the Australian Cooperative Research Centres Program.

PY - 2009/3/1

Y1 - 2009/3/1

N2 - In this paper, different methods for the evaluation of building detection algorithms are compared. Whereas pixel-based evaluation gives estimates of the area that is correctly classified, the results are distorted by errors at the building outlines. These distortions are potentially in an order of 30%. Object-based evaluation techniques are less affected by such errors. However, the performance metrics thus delivered are sometimes considered to be less objective, because the definition of a “correct detection” is not unique. Based on a critical review of existing performance metrics, selected methods for the evaluation of building detection results are presented. These methods are used to evaluate the results of two different building detection algorithms in two test sites. A comparison of the evaluation techniques shows that they highlight different properties of the building detection results. As a consequence, a comprehensive evaluation strategy involving quality metrics derived by different methods is proposed.

AB - In this paper, different methods for the evaluation of building detection algorithms are compared. Whereas pixel-based evaluation gives estimates of the area that is correctly classified, the results are distorted by errors at the building outlines. These distortions are potentially in an order of 30%. Object-based evaluation techniques are less affected by such errors. However, the performance metrics thus delivered are sometimes considered to be less objective, because the definition of a “correct detection” is not unique. Based on a critical review of existing performance metrics, selected methods for the evaluation of building detection results are presented. These methods are used to evaluate the results of two different building detection algorithms in two test sites. A comparison of the evaluation techniques shows that they highlight different properties of the building detection results. As a consequence, a comprehensive evaluation strategy involving quality metrics derived by different methods is proposed.

KW - Airborne laser scanning (ALS)

KW - automatic building extraction

KW - metrics

KW - object detection

KW - object-based error assessment

KW - performance evaluation

UR - http://www.scopus.com/inward/record.url?scp=66149101328&partnerID=8YFLogxK

U2 - 10.1109/JSTARS.2009.2012488

DO - 10.1109/JSTARS.2009.2012488

M3 - Article

AN - SCOPUS:66149101328

VL - 2

SP - 11

EP - 20

JO - IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

JF - IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

SN - 1939-1404

IS - 1

ER -