v0.7.0

MAP-Tk v0.7.0 Release Notes

This is a minor release of MAP-Tk that provides both new functionality and
fixes over the previous v0.6.1 release. The highlights of this release are
a new Ceres Solver plugin, support for lens distortion modeling, match
matrix computation and visualization, and a prototype for a new graphical
visualization application. This is the last minor release of MAP-Tk that
does not depend on the Vital project. Starting with MAP-Tk v0.8.0 a large
amount of MAP-Tk code will migrate to Vital to be shared across Kwiver
projects and provide better MAP-Tk integration with Kwiver. MAP-Tk will
then depend on Vital.

Ceres provides a much improved bundle adjustment backend. It is an order
of magnitude faster than the VXL backend while also being more flexible
and configurable. For example, it handles radial distortion parameters
and sharing of camera intrinsics between any subsets of cameras. Ceres
will be the default bundle adjustment backend going forward. The VXL
backend will still be supported, but with a subset of capabilities.

The lens distortion model added to the camera class follows the same
model used by OpenCV with up to eight radial and tangential parameters.
Additional distortion models may be added in the future. Sharing the
OpenCV model means that parameters estimated using OpenCV can be plugged
directly into MAP-Tk without conversion. Note that the OpenCV code for lens
distortion is not actually required in MAP-Tk and lens distortion is supported
even without the OpenCV plugin enabled.

The match matrix provides a useful way to analyze and visualize the statistics
of the feature tracking results. The match matrix contains the number of
features that correspond between any two images. It is useful for guiding
structure from motion algorithms and also provides a good visual summary
of image connectivity and loop closure when rendered as an image. This
release provides tools for writing out the matrix and a Python script for
plotting it. Future release will add visualization directly into the GUI.

The visualization application is graphical user interface (GUI) for
visualizing MAP-Tk output. It renders the 3D pose of cameras and
landmarks and also shows how the landmarks project into the images and
draws the residual vector between landmarks and corresponding feature points.
In this release the GUI is just a preview of what is to come. In future
releases, the GUI will provided capabilities for running the algorithms
directly as well as visualizing the results.

Additional notes about the release are provided below.

Updates since v0.6.1

Main Library

• Added a metric in metrics.h to compute the median reprojection error.
  The median provides a more robust alternative to RMSE and is more useful
  for characterizing error when using robust loss functions.
• Added lens distortion to the camera_intrinsics class. The model uses the
  same conventions that OpenCV uses. The distortion parameter vector
  is compatible between OpenCV and MAP-Tk.
• Added lens undistortion functions to camera_intrinsics which use the
  Gauss-Newton method to iteratively solve for the inverse of distortion.
  Convergence is fast and more accurate than the implementation in OpenCV.
• Added reading and writing of lens distortion parameters to the camera
  stream operators and, as a result, to the KRTD file io.
• Added 3D triangulation algorithms using Eigen to triangulate.h
• Added a constructor from the base class for camera_
• Added a function to camera_ to compute depth of a 3D point to the
  image plane. This can also be used to check if a point is behind
  the camera by checking the sign of the depth.
• Added a new essential matrix data structure, including abstract
  base class, and switched the essential matrix estimation algorithm
  to use this class instead of a 3x3 matrix.
• Added a function in match_matrix.h to compute the symmetric match matrix
  corresponding to a track set for all or a subset of the frames.
• Renamed the algorithm base class clone() method to base_clone() and added
  a new clone() method at the algorithm_impl level. These functions are
  identical except the base_clone() method returns a base class algorithm
  shared pointer and the clone() method returns a shared pointer to the
  specific algorithm intermediate base class.
• Added a Necker reverse transformation to transform.h to help get out
  of a common local minima during bundle adjustment.
• Moved some common camera parameter collection code for camera optimization
  into the base class algorithm and changed the API such that the pure
  virtual function operates on a single camera with vectors of corresponding
  features and landmarks.

Ceres Plugin

• Added a new plugin which uses Google Ceres Solver as a back end for
  optimizations.
• Added an implementation of the bundle_adjust algorithm using Ceres to
  solve the optimization problem. This supports access to Ceres options
  via the configuration block and currently supports selection of various
  different solver algorithms and robust loss functions.
• Added unit tests for the Ceres implementation of bundle_adjust that
  duplicates the same tests used for the VXL back end plus a few additional
  tests focusing on robust estimation.
• Added support to both use and estimate lens distortion parameters in the
  bundle_adjust algorithm using Ceres. Three different distortion models
  can be selected, each adding additional parameters to the previous model.
  Furthermore, many parameters can be locked in the optimization allowing
  for even more fine-grained selection of model parameter subsets.
• Added unit tests for Ceres bundle adjustment covering the use and estimation
  of lens distortion parameters.

Core Plugin

• Added a core plugin implementation of triangulate_landmarks using the
  triangulation functions added to the main library.
• Moved the implementation of initialize_cameras_landmarks from the VXL
  plugin into core and updated it to use the new triangulation and essential
  matrix functionality of the main library to remove dependence on VXL.
• Updated the initialize_cameras_landmarks algorithm with many improvements
  including optional optimization of newly added cameras, optional
  initialization of new cameras from the last camera position, and using
  the match matrix to guide which cameras are added next.

OpenCV Plugin

• Added unit tests to verify that the lens distortion functions in MAP-Tk
  produce equal results to those routines in OpenCV.

VXL Plugin

• Moved the initialize_cameras_landmarks implementation to the core plugin.
• Updated the VXL camera optimization algorithm to use the new abstract API.
  Code for camera parameter collection has moved to the base class to be
  shared.

Visualization Application

• Added a new graphical application that can be used to visualize input data
  and computed results. A 3D "world view" shows the cameras and computed
  landmarks. A "camera view" shows the source imagery with feature points and
  projected landmarks for a selected camera. A "slideshow" mode allows
  automatic switching between cameras at regular intervals and can emulate
  video playback.
• Added a new icon/logo for MAP-Tk to use as the icon for the GUI application.

C Interface

• Updated algorithm related code to use new functions in algorithm C++ base
  class.

Python Interface

• Updated algorithm related code to use new algorithm C-interface functions.

Tools

• Added a new tool to read in a track file and compute the match matrix.
  The match matrix can be written out to a matrix in ASCII form.
• Added an option to the bundle adjust tool to write out an updated track
  file corresponding to the filtered tracks used in bundle adjustment.
• Added an option to the bundle adjust tool to apply a Necker reversal
  to the input cameras and points before continuing to apply bundle
  adjustment. This allows an inverted solution from a previous run to
  be restarted to find the correct solution.
• Change bundle adjustment tool to allow frame subsampling to occur before
  camera initialization. This way one can limited the entire pipeline,
  not just bundle adjustment, to a subset of frames.
• Added an option to the bundle adjust tool to allow initialization of
  additional cameras beyond those that were loaded from disk. If you have
  a solution on a subset of frames you may wish to further optimize just
  that subset or your may wish to use that subset to bootstrap to a
  larger set of frames.

Scripts

• Added a Python script to read the ASCII match matrix file produced by
  the match matrix tool and plot it.
• Added Python modules to read and write KRTD, PLY, and homography files.
• Added a Python script to read an NVM file from VisualSfM and write out
  KRTD camera files
• Added a Python script to read in a PLY file and set of KRTD camera files
  and write out a homography files. This script fits a ground plane to
  the points in the PLY and then uses the cameras to derive homographies
  from each frame back to the first frame as induced by the plane.
• Added a Python script to compute the extents of the output space of
  a sequence of homographies applied to video of fixed size.
• Added a Python script to transform homographies by translating and
  scaling the output space.

Tests

• Moved test code from vxl::triangulate_landmarks to a common header to also
  share with tests for core::triangulate_landmarks and any other future
  implementations of triangulate_landmarks.

Fixes since v0.6.1

OpenCV Plugin

• Fixed error where the size of OCV image memory was incorrectly calculated.

Tests

• Renamed the tests for the main library from "test_core" to "test_maptk"
  to differentiate between tests for the "core" plugin.