Details
| Original language | English |
|---|---|
| Pages (from-to) | 531-546 |
| Number of pages | 16 |
| Journal | PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science |
| Volume | 92 |
| Issue number | 5 |
| Early online date | 28 Aug 2024 |
| Publication status | Published - Oct 2024 |
Abstract
Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.
Keywords
- Aerial images, Dense matching, Disparity learning, Disparity refinement, Geometric accuracy
ASJC Scopus subject areas
- Social Sciences(all)
- Geography, Planning and Development
- Physics and Astronomy(all)
- Instrumentation
- Earth and Planetary Sciences(all)
- Earth and Planetary Sciences (miscellaneous)
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science, Vol. 92, No. 5, 10.2024, p. 531-546.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - EnhancedNet, an End-to-End Network for Dense Disparity Estimation and its Application to Aerial Images
AU - Kang, Junhua
AU - Chen, Lin
AU - Heipke, Christian
N1 - Publisher Copyright: © The Author(s) 2024.
PY - 2024/10
Y1 - 2024/10
N2 - Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.
AB - Recent developments in deep learning technology have boosted the performance of dense stereo reconstruction. However, the state-of-the-art deep learning-based stereo matching methods are mainly trained using close-range synthetic images. Consequently, the application of these methods in aerial photogrammetry and remote sensing is currently far from straightforward. In this paper, we propose a new disparity estimation network for stereo matching and investigate its generalization abilities in regard to aerial images. First, we propose an end-to-end deep learning network for stereo matching, regularized by disparity gradients, which includes a residual cost volume and a reconstruction error volume in a refinement module, and multiple losses. In order to investigate the influence of the multiple losses, a comprehensive analysis is presented. Second, based on this network trained with synthetic close-range data, we propose a new pipeline for matching high-resolution aerial imagery. The experimental results show that the proposed network improves the disparity accuracy by up to 40% in terms of errors larger than 1 px compared to results when not including the refinement network, especially in areas containing detailed small objects. In addition, in qualitative and quantitative experiments, we are able to show that our model, pre-trained on a synthetic stereo dataset, achieves very competitive sub-pixel geometric accuracy on aerial images. These results confirm that the domain gap between synthetic close-range and real aerial images can be satisfactorily bridged using the proposed new deep learning method for dense image matching.
KW - Aerial images
KW - Dense matching
KW - Disparity learning
KW - Disparity refinement
KW - Geometric accuracy
UR - http://www.scopus.com/inward/record.url?scp=85202624058&partnerID=8YFLogxK
U2 - 10.1007/s41064-024-00307-w
DO - 10.1007/s41064-024-00307-w
M3 - Article
AN - SCOPUS:85202624058
VL - 92
SP - 531
EP - 546
JO - PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science
JF - PFG - Journal of Photogrammetry, Remote Sensing and Geoinformation Science
SN - 2512-2789
IS - 5
ER -