TY - GEN

T1 - JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING

AU - Sledz, A.

AU - Heipke, C.

N1 - Funding Information: The work is supported by the Arbeitsgemeinschaft industrieller Forschungsvereinigungen (AiF) under IGF-grant no. 19768.

PY - 2022/5/30

Y1 - 2022/5/30

N2 - Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.

AB - Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.

KW - bundle adjustment

KW - multimodal feature matching

KW - phase congruency

KW - Thermal infrared and optical image registration

UR - http://www.scopus.com/inward/record.url?scp=85131954032&partnerID=8YFLogxK

U2 - 10.5194/isprs-archives-XLIII-B1-2022-157-2022

DO - 10.5194/isprs-archives-XLIII-B1-2022-157-2022

M3 - Conference contribution

AN - SCOPUS:85131954032

T3 - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives

SP - 157

EP - 165

BT - XXIV ISPRS Congress “Imaging today, foreseeing tomorrow”

T2 - 2022 24th ISPRS Congress on Imaging Today, Foreseeing Tomorrow, Commission I

Y2 - 6 June 2022 through 11 June 2022

ER -