TY - JOUR

T1 - Structure from motion for ordered and unordered image sets based on random k-d forests and global pose estimation

AU - Wang, Xin

AU - Rottensteiner, Franz

AU - Heipke, Christian

N1 - Funding Information: We thank Zhaopeng Cui for providing the Campus dataset and Kyle Wilson for the discussion about the ground truth of the internet datasets. The author Xin Wang would like to thank the China Scholarship Council (CSC) for financially supporting his PhD study at Leibniz Universität Hannover, Germany.

PY - 2019/1

Y1 - 2019/1

N2 - In this paper, we present a new fast and robust method for structure from motion (SfM) for data sets potentially comprising thousands of ordered or unordered images. Our work focuses on the two most time-consuming procedures: (a) image matching and (b) pose estimation. For image matching, a new method employing a random k-d forest is proposed to quickly obtain pairs of overlapping images from an unordered set. After that, image matching and the estimation of relative orientation parameters are performed only for pairs found to be very likely to overlap. For pose estimation, we use a two-stage global approach, separating the determination of rotation matrices and translation parameters; the latter are computed simultaneously using a new method. In order to cope with outliers in the relative orientations, which global approaches are particularly sensitive to, we present a new constraint based on triplet loop closure errors of rotation and translation. Finally, a robust bundle adjustment is carried out to refine the image orientation parameters. We demonstrate the potential and limitations of our pipeline using various real-world datasets including ordered image data acquired from UAV (unmanned aerial vehicle) and other platforms as well as unordered data from the internet. The experiments show that our work performs better than comparable state-of-the-art SfM systems in terms of run time, while we achieve a similar accuracy and robustness.

AB - In this paper, we present a new fast and robust method for structure from motion (SfM) for data sets potentially comprising thousands of ordered or unordered images. Our work focuses on the two most time-consuming procedures: (a) image matching and (b) pose estimation. For image matching, a new method employing a random k-d forest is proposed to quickly obtain pairs of overlapping images from an unordered set. After that, image matching and the estimation of relative orientation parameters are performed only for pairs found to be very likely to overlap. For pose estimation, we use a two-stage global approach, separating the determination of rotation matrices and translation parameters; the latter are computed simultaneously using a new method. In order to cope with outliers in the relative orientations, which global approaches are particularly sensitive to, we present a new constraint based on triplet loop closure errors of rotation and translation. Finally, a robust bundle adjustment is carried out to refine the image orientation parameters. We demonstrate the potential and limitations of our pipeline using various real-world datasets including ordered image data acquired from UAV (unmanned aerial vehicle) and other platforms as well as unordered data from the internet. The experiments show that our work performs better than comparable state-of-the-art SfM systems in terms of run time, while we achieve a similar accuracy and robustness.

KW - Ordered and unordered images

KW - Random k-d forest

KW - Rotation averaging

KW - Structure-from-motion (SfM)

KW - Translation averaging

UR - http://www.scopus.com/inward/record.url?scp=85056608406&partnerID=8YFLogxK

U2 - 10.1016/j.isprsjprs.2018.11.009

DO - 10.1016/j.isprsjprs.2018.11.009

M3 - Article

AN - SCOPUS:85056608406

VL - 147

SP - 19

EP - 41

JO - ISPRS Journal of Photogrammetry and Remote Sensing

JF - ISPRS Journal of Photogrammetry and Remote Sensing

SN - 0924-2716

ER -