Bundle adjustment (BA) is essential in many robotics and structure-from-motion applications: given a set of images, captured by a single or multiple users (or autonomous platforms), BA provides the maximum a posteriori estimate of camera poses and observed environment (e.g. 3D points). Assuming Gaussian image noise, the problem is equivalent to minimizing the re-projection errors of all image observations in all images. However, as more images are acquired, the involved optimization becomes increasingly computationally expensive: on-line performance over long time periods is therefore and challenging problem.
In this research we introduce incremental light bundle adjustment (iLBA): an efficient optimization framework that substantially reduces computational complexity compared to incremental bundle adjustment. First, the number of variables in the optimization is reduced by algebraic elimination of observed 3D points, leading to a /structureless/ BA. The resulting cost function is formulated in terms of /three-view/ constraints instead of re-projection errors and only the camera poses are optimized. Second, the optimization problem is represented using graphical models and incremental inference is applied, updating the solution using adaptive partial calculations each time a new camera is incorporated into the optimization. Typically, only a small fraction of the camera poses are recalculated in each optimization step. The 3D points, although not explicitly optimized, can be reconstructed based on the optimized camera poses at any time.
Probabilistic aspects of iLBA are analyzed in this paper, that demonstrates that the first two moments of the probability distribution of iLBA and the true distribution, calculated from conventional bundle adjustment, are very similar. In other words, in addition to high-accuracy results, iLBA also produces reliable covariance estimates.
iLBA can also be used in autonomous navigation applications where other sensors, in addition to monocular camera, are available. In our paper we discuss a method to integrate iLBA with high-rate IMU measurements. Using an equivalent IMU factor that summarizes consecutive IMU measurements allows to add variables to the optimization only at camera rate, while the iLBA framework eliminates the necessity in having 3D points as variables to support loop closures. These two elements lead to improved computational complexity compared to state-of-the-art methods.
Code and datasets are publicly available (Software page).
BAFS: Bundle Adjustment With Feature Scale Constraints
We propose to incorporate within bundle adjustment (BA) a new type of constraint that uses feature scale information, leveraging the scale invariance property of typical image feature detectors (e.g., SIFT). While feature scales play an important role in image matching, they have not been utilized thus far for estimation purposes in a BA framework. Our approach exploits the already-available feature scale information and uses it to enhance
the accuracy of BA, especially along the optical axis of the camera in a monocular setup. Importantly, the mentioned feature scale constraints can be formulated on a frame to frame basis and do not require loop closures. We study our approach in synthetic environments
and the real-imagery KITTI dataset, demonstrating significant improvement in positioning error.
RAL-ICRA’18 3-minute spotlight video: