# Incremental Light Bundle Adjustment for Structure from Motion and Autonomous Navigation

Bundle adjustment (BA) is essential in many robotics and structure-from-motion applications: given a set of images, captured by a single or multiple users (or autonomous platforms), BA provides the maximum a posteriori estimate of camera poses and observed environment (e.g. 3D points). Assuming Gaussian image noise, the problem is equivalent to minimizing the re-projection errors of all image observations in all images. However, as more images are acquired, the involved optimization becomes increasingly computationally expensive: on-line performance over long time periods is therefore and challenging problem.

In this research we introduce incremental light bundle adjustment (iLBA): an efficient optimization framework that substantially reduces computational complexity compared to incremental bundle adjustment. First, the number of variables in the optimization is reduced by algebraic elimination of observed 3D points, leading to a /structureless/ BA. The resulting cost function is formulated in terms of /three-view/ constraints instead of re-projection errors and only the camera poses are optimized. Second, the optimization problem is represented using graphical models and incremental inference is applied, updating the solution using adaptive partial calculations each time a new camera is incorporated into the optimization. Typically, only a small fraction of the camera poses are recalculated in each optimization step. The 3D points, although not explicitly optimized, can be reconstructed based on the optimized camera poses at any time.

Probabilistic aspects of iLBA are analyzed in this paper, that demonstrates that the first two moments of the probability distribution of iLBA and the true distribution, calculated from conventional bundle adjustment, are very similar. In other words, in addition to high-accuracy results, iLBA also produces reliable covariance estimates.

iLBA can also be used in autonomous navigation applications where other sensors, in addition to monocular camera, are available. In our paper we discuss a method to integrate iLBA with high-rate IMU measurements. Using an equivalent IMU factor that summarizes consecutive IMU measurements allows to add variables to the optimization only at camera rate, while the iLBA framework eliminates the necessity in having 3D points as variables to support loop closures. These two elements lead to improved computational complexity compared to state-of-the-art methods.

Code and datasets are publicly available (Software page).

## Related Publications:

### Articles

• V. Indelman, R. Roberts, and F. Dellaert. Incremental Light Bundle Adjustment for Structure From Motion and Robotics. Robotics and Autonomous Systems, 70:63-82, 2015.
[BibTeX] [URL] [PDF]
@Article{Indelman15ras,
author = {V. Indelman and R. Roberts and F. Dellaert},
title = {Incremental Light Bundle Adjustment for Structure From Motion and Robotics},
journal = "Robotics and Autonomous Systems",
year = 2015,
volume = 70,
pages = "63-82",
url = "http://www.sciencedirect.com/science/article/pii/S0921889015000810",
pdf = "http://vindelman.technion.ac.il/Publications/Indelman15ras.pdf",
researchtopic = {LBA},
}

### In Collections

• V. Indelman and F. Dellaert. Incremental Light Bundle Adjustment: Probabilistic Analysis and Application to Robotic Navigation. In New Development in Robot Vision, volume 23 of Cognitive Systems Monographs, pages 111-136. Springer Berlin Heidelberg, 2015. doi:10.1007/978-3-662-43859-6_7
[BibTeX] [URL] [PDF]
@incollection{Indelman15chapter,
isbn={978-3-662-43858-9},
booktitle={New Development in Robot Vision},
volume={23},
series={Cognitive Systems Monographs},
doi={10.1007/978-3-662-43859-6_7},
url={http://dx.doi.org/10.1007/978-3-662-43859-6_7},
publisher={Springer Berlin Heidelberg},
author={V. Indelman and F. Dellaert},
pages="111-136",
year={2015},
pdf = "http://vindelman.technion.ac.il/Publications/Indelman15chapter.pdf",
researchtopic = {LBA},
}

### In Proceedings

• V. Indelman, A. Melim, and F. Dellaert. Incremental Light Bundle Adjustment for Robotics Navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2013.
[BibTeX] [PDF] [Slides]
@InProceedings{Indelman13iros,
author = {V. Indelman and A. Melim and F. Dellaert},
booktitle = "IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
year = 2013,
month = "November",
location = {Tokyo, Japan},
pdf = "http://vindelman.technion.ac.il/Publications/Indelman13iros.pdf",
slides = "http://vindelman.technion.ac.il/Publications/Indelman13iros_ppt.pdf",
researchtopic = {LBA, IncrementalSmoothingNav},
}

• V. Indelman, R. Roberts, and F. Dellaert. Probabilistic Analysis of Incremental Light Bundle Adjustment. In IEEE Workshop on Robot Vision (WoRV), *best poster award*, January 2013.
[BibTeX] [PDF] [Slides]
@InProceedings{Indelman13worv,
author = {V. Indelman and R. Roberts and F. Dellaert},
title = {Probabilistic Analysis of Incremental Light Bundle Adjustment},
booktitle = "IEEE Workshop on Robot Vision (WoRV), *best poster award*",
year = 2013,
month = "January",
location = {Clearwater, Florida, USA},
pdf = "http://vindelman.technion.ac.il/Publications/Indelman13worv.pdf",
slides = "http://vindelman.technion.ac.il/Publications/Indelman13worv_ppt.pdf",
researchtopic = {LBA},
}

• V. Indelman, R. Roberts, C. Beall, and F. Dellaert. Incremental Light Bundle Adjustment. In British Machine Vision Conference (BMVC), September 2012.
[BibTeX] [PDF] [Slides] [Video] [Demo]
@InProceedings{Indelman12bmvc,
author = {V. Indelman and R. Roberts and C. Beall and F. Dellaert},
title = {Incremental Light Bundle Adjustment},
booktitle = "British Machine Vision Conference (BMVC)",
year = 2012,
month = "September",
location = {Surrey, UK},
pdf = "http://vindelman.technion.ac.il/Publications/Indelman12bmvc.pdf",
slides = "http://vindelman.technion.ac.il/Publications/Indelman12bmvc_ppt.pdf",
researchtopic = {LBA},
}

• V. Indelman. Bundle Adjustment Without Iterative Structure Estimation and its Application to Navigation. In IEEE/ION Position Location and Navigation System (PLANS) Conference, April 2012.
[BibTeX] [PDF] [Slides]
@InProceedings{Indelman12plans_a,
author = {V. Indelman},
title = {Bundle Adjustment Without Iterative Structure Estimation and its Application to Navigation},
booktitle = "IEEE/ION Position Location and Navigation System (PLANS) Conference",
year = 2012,
month = "April",
location = {South Carolina, USA},
pdf = "http://vindelman.technion.ac.il/Publications/Indelman12plans_a.pdf",
slides = "http://vindelman.technion.ac.il/Publications/Indelman12plans_a_ppt.pdf",
researchtopic = {LBA},
}