Incremental Light Bundle Adjustment for Structure from Motion and Autonomous Navigation

Bundle adjustment (BA) is essential in many robotics and structure-from-motion applications: given a set of images, captured by a single or multiple users (or autonomous platforms), BA provides the maximum a posteriori estimate of camera poses and observed environment (e.g. 3D points). Assuming Gaussian image noise, the problem is equivalent to minimizing the re-projection errors of all image observations in all images. However, as more images are acquired, the involved optimization becomes increasingly computationally expensive: on-line performance over long time periods is therefore and challenging problem.

In this research we introduce incremental light bundle adjustment (iLBA): an efficient optimization framework that substantially reduces computational complexity compared to incremental bundle adjustment. First, the number of variables in the optimization is reduced by algebraic elimination of observed 3D points, leading to a /structureless/ BA. The resulting cost function is formulated in terms of /three-view/ constraints instead of re-projection errors and only the camera poses are optimized. Second, the optimization problem is represented using graphical models and incremental inference is applied, updating the solution using adaptive partial calculations each time a new camera is incorporated into the optimization. Typically, only a small fraction of the camera poses are recalculated in each optimization step. The 3D points, although not explicitly optimized, can be reconstructed based on the optimized camera poses at any time.

Probabilistic aspects of iLBA are analyzed in this paper, that demonstrates that the first two moments of the probability distribution of iLBA and the true distribution, calculated from conventional bundle adjustment, are very similar. In other words, in addition to high-accuracy results, iLBA also produces reliable covariance estimates.

iLBA can also be used in autonomous navigation applications where other sensors, in addition to monocular camera, are available. In our paper we discuss a method to integrate iLBA with high-rate IMU measurements. Using an equivalent IMU factor that summarizes consecutive IMU measurements allows to add variables to the optimization only at camera rate, while the iLBA framework eliminates the necessity in having 3D points as variables to support loop closures. These two elements lead to improved computational complexity compared to state-of-the-art methods.

Code and datasets are publicly available (Software page).

BAFS: Bundle Adjustment With Feature Scale Constraints

We propose to incorporate within bundle adjustment (BA) a new type of constraint that uses feature scale information, leveraging the scale invariance property of typical image feature detectors (e.g., SIFT). While feature scales play an important role in image matching, they have not been utilized thus far for estimation purposes in a BA framework. Our approach exploits the already-available feature scale information and uses it to enhance
the accuracy of BA, especially along the optical axis of the camera in a monocular setup. Importantly, the mentioned feature scale constraints can be formulated on a frame to frame basis and do not require loop closures. We study our approach in synthetic environments
and the real-imagery KITTI dataset, demonstrating significant improvement in positioning error.

RAL-ICRA’18 3-minute spotlight video:

Related Publications:

Articles

  • V. Ovechkin and V. Indelman. BAFS: Bundle Adjustment with Feature Scale Constraints for Enhanced Estimation Accuracy. IEEE Robotics and Automation Letters (RA-L), 3(2):804-810, 2018. doi:10.1109/LRA.2018.2792141
    [BibTeX] [PDF]
    @Article{Ovechkin18ral,
    author = {V. Ovechkin and V. Indelman},
    title = {BAFS: Bundle Adjustment with Feature Scale Constraints for Enhanced Estimation Accuracy},
    journal = {IEEE Robotics and Automation Letters (RA-L)},
    year={2018},
    volume = 3,
    number = 2,
    pages = {804-810},
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Ovechkin18ral.pdf",
    doi = "10.1109/LRA.2018.2792141",
    researchtopic = {LBA},
    }

  • M. Chojnacki and V. Indelman. Vision-based Dynamic Target Trajectory and Ego-motion Estimation Using Incremental Light Bundle Adjustment. International Journal of Micro Air Vehicles, Special Issue on Estimation and Control for MAV Navigation in GPS-denied Cluttered Environments, 10(2):157-170, 2018.
    [BibTeX] [PDF]
    @Article{Chojnacki18mav,
    author = {M. Chojnacki and V. Indelman},
    title = {Vision-based Dynamic Target Trajectory and Ego-motion Estimation Using Incremental Light Bundle Adjustment},
    journal = "International Journal of Micro Air Vehicles, Special Issue on Estimation and Control for MAV Navigation in GPS-denied Cluttered Environments",
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Chojnacki18mav.pdf",
    year = 2018,
    volume = 10,
    number = 2,
    pages = {157-170},
    researchtopic = {LBA},
    }

  • V. Indelman, R. Roberts, and F. Dellaert. Incremental Light Bundle Adjustment for Structure From Motion and Robotics. Robotics and Autonomous Systems, 70:63-82, 2015.
    [BibTeX] [URL] [PDF]
    @Article{Indelman15ras,
    author = {V. Indelman and R. Roberts and F. Dellaert},
    title = {Incremental Light Bundle Adjustment for Structure From Motion and Robotics},
    journal = "Robotics and Autonomous Systems",
    year = 2015,
    volume = 70,
    pages = "63-82",
    url = "http://www.sciencedirect.com/science/article/pii/S0921889015000810",
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Indelman15ras.pdf",
    researchtopic = {LBA},
    }

In Collections

  • V. Indelman and F. Dellaert. Incremental Light Bundle Adjustment: Probabilistic Analysis and Application to Robotic Navigation. In New Development in Robot Vision, volume 23 of Cognitive Systems Monographs, pages 111-136. Springer Berlin Heidelberg, 2015. doi:10.1007/978-3-662-43859-6_7
    [BibTeX] [URL] [PDF]
    @incollection{Indelman15chapter,
    isbn={978-3-662-43858-9},
    booktitle={New Development in Robot Vision},
    volume={23},
    series={Cognitive Systems Monographs},
    doi={10.1007/978-3-662-43859-6_7},
    title={Incremental Light Bundle Adjustment: Probabilistic Analysis and Application to Robotic Navigation},
    url={http://dx.doi.org/10.1007/978-3-662-43859-6_7},
    publisher={Springer Berlin Heidelberg},
    author={V. Indelman and F. Dellaert},
    pages="111-136",
    year={2015},
    url = "http://link.springer.com/chapter/10.1007/978-3-662-43859-6_7",
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Indelman15chapter.pdf",
    researchtopic = {LBA},
    }

In Proceedings

  • V. Ovechkin and V. Indelman. BAFS: Bundle Adjustment with Feature Scale Constraints for Enhanced Estimation Accuracy. In IEEE International Conference on Robotics and Automation (ICRA); submission via IEEE Robotics and Automation Letters (RA-L), May 2018.
    [BibTeX] [PDF] [Poster]
    @InProceedings{Ovechkin18icra,
    author = {V. Ovechkin and V. Indelman},
    title = {BAFS: Bundle Adjustment with Feature Scale Constraints for Enhanced Estimation Accuracy},
    booktitle = "IEEE International Conference on Robotics and Automation (ICRA); submission via IEEE Robotics and Automation Letters (RA-L)",
    year = 2018,
    month = "May",
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Ovechkin18ral.pdf",
    poster = "http://indelman.github.io/ANPL-Website/Publications/Ovechkin18ral_poster.pdf",
    researchtopic = {LBA},
    }

  • V. Indelman, A. Melim, and F. Dellaert. Incremental Light Bundle Adjustment for Robotics Navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2013.
    [BibTeX] [PDF] [Slides]
    @InProceedings{Indelman13iros,
    author = {V. Indelman and A. Melim and F. Dellaert},
    title = {Incremental Light Bundle Adjustment for Robotics Navigation},
    booktitle = "IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
    year = 2013,
    month = "November",
    location = {Tokyo, Japan},
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Indelman13iros.pdf",
    slides = "http://indelman.github.io/ANPL-Website/Publications/Indelman13iros_ppt.pdf",
    researchtopic = {LBA, IncrementalSmoothingNav},
    }

  • V. Indelman, R. Roberts, and F. Dellaert. Probabilistic Analysis of Incremental Light Bundle Adjustment. In IEEE Workshop on Robot Vision (WoRV), *best poster award*, January 2013.
    [BibTeX] [PDF] [Slides]
    @InProceedings{Indelman13worv,
    author = {V. Indelman and R. Roberts and F. Dellaert},
    title = {Probabilistic Analysis of Incremental Light Bundle Adjustment},
    booktitle = "IEEE Workshop on Robot Vision (WoRV), *best poster award*",
    year = 2013,
    month = "January",
    location = {Clearwater, Florida, USA},
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Indelman13worv.pdf",
    slides = "http://indelman.github.io/ANPL-Website/Publications/Indelman13worv_ppt.pdf",
    researchtopic = {LBA},
    }

  • V. Indelman, R. Roberts, C. Beall, and F. Dellaert. Incremental Light Bundle Adjustment. In British Machine Vision Conference (BMVC), September 2012.
    [BibTeX] [PDF] [Slides] [Video] [Demo]
    @InProceedings{Indelman12bmvc,
    author = {V. Indelman and R. Roberts and C. Beall and F. Dellaert},
    title = {Incremental Light Bundle Adjustment},
    booktitle = "British Machine Vision Conference (BMVC)",
    year = 2012,
    month = "September",
    location = {Surrey, UK},
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Indelman12bmvc.pdf",
    slides = "http://indelman.github.io/ANPL-Website/Publications/Indelman12bmvc_ppt.pdf",
    demo = "http://www.youtube.com/watch?v=1k9FEq8sb4o",
    video = "http://videolectures.net/bmvc2012_indelman_bundle_adjustment",
    researchtopic = {LBA},
    }

  • V. Indelman. Bundle Adjustment Without Iterative Structure Estimation and its Application to Navigation. In IEEE/ION Position Location and Navigation System (PLANS) Conference, April 2012.
    [BibTeX] [PDF] [Slides]
    @InProceedings{Indelman12plans_a,
    author = {V. Indelman},
    title = {Bundle Adjustment Without Iterative Structure Estimation and its Application to Navigation},
    booktitle = "IEEE/ION Position Location and Navigation System (PLANS) Conference",
    year = 2012,
    month = "April",
    location = {South Carolina, USA},
    pdf = "http://indelman.github.io/ANPL-Website/Publications/Indelman12plans_a.pdf",
    slides = "http://indelman.github.io/ANPL-Website/Publications/Indelman12plans_a_ppt.pdf",
    researchtopic = {LBA},
    }

wordpress stat