v1.0.0

TeleSculptor v1.0.0 Release Notes

This is the first major release of TeleSculptor. TeleSculptor was previously
developed as beta software under the name MAP-Tk. TeleSculptor is a graphical
application for aerial photogrammetry that builds on KWIVER. All algorithms
have been moved into KWIVER and this project now focuses on the user interface.
Some command line utilities remain in this release, but these will move into
KWIVER in the future as well.

This is the first release in which TeleSculptor provides an end-to-end pipeline
for building a dense 3D model from a video in the graphical application without
the need for any command line tools. This release has been well tested on a
particular type of aerial video that uses an orbital flight pattern and
encodes metadata in a KLV stream. In future releases we will generalize the
algorithms to work more reliably on a wider range of aerial video, both with
and without metadata.

There is a very large set of changes since the last beta release. These
changes are documented below in detail. At a high level, these changes are
all focused around transforming MAP-Tk, a developer focused toolkit, into
TeleSculptor, an end-user focused application. In support of this transition,
TeleSculptor can now configure and build itself with a CMake superbuild.
That is, building TeleSculptor can now automatically download and build
Fletch and KWIVER before building itself to provide all dependencies in a
single build. The superbuild will also package up the binaries into an
installer.

Known Issues in this Release

• On MacOS, the color selection buttons function correctly but always show
  the color as white in the button.

• Default colors used during the first run of the application are incorrectly
  set to white or black. However, once the colors are changed these settings
  are cached and reloaded correctly.

• The libraries in the Linux installer package do not properly set the runpath
  so it is required to set LD_LIBRARY_PATH to point to the "lib" directory of
  the package before running the executables.

• A user interface is provided for setting ground control points to constrain
  georegistration of the 3D model, but this release is missing the function
  to apply these constraints to improve the georegistration.

Updates since MAP-Tk v0.10.0

Overall

• TeleSculptor now enforces the use of Python 3 when Python is enabled.

• Added a Dockerfile to aid with generating Linux packages in a clean
  environment.

• Removed all uses of VITAL_FOREACH, which has been removed from KWIVER,
  and replace with standard C++11 range-based for loops.

• Updated everything to use the new feature_track_set class and
  feature_track_state class from KWIVER. The previous data structure,
  track_set is now a base class and is generalized to support other types of
  tracks, like moving object tracks.

• Replaced use of the geo_map vital algorithm with the new geodetic conversion
  functions and data structures provided in KWIVER. The new approach still
  uses plugins (e.g. PROJ4) but only supports one backend at a time that is
  globally accessible throughout KWIVER. There is no more need to explicitly
  pass a geo_map object around. New data structures, like geo_point, know
  how to convert themselves into different coordinate systems.

MAP-Tk Library

• modified extract_feature_colors API to accept a feature_track_set by
  shared_ptr, modify to tracks in-place, and return the same modified
  feature_track_set. It also uses the new track_set API to more quickly
  find track state on a frame and avoids destroying the frame index if
  one is used in the track_set.

• Moved local_geo_cs class into KWIVER and updated TeleSculptor to used the
  KWIVER version.

Plugins

• Moved the Blender and SketchUp plugins out of the scripts directory and
  into a top-level plugins directory, added a README, and configured these
  files to be installed.

• The SketchUp plugin now supports older version of SketchUp
  (back to version 8) and only attempts to load frames which have both an
  image file and camera file available in the project.

TeleSculptor

• Updated the TeleSculptor application to build against VTK 8.0 instead of
  VTK 6.2. This upgrade provides substantial improvements to rendering
  speed of large data sets.

• Changed frame indexing to be one-based rather than zero-based. This change
  was needed since the new streaming video API used in the tracking tools now
  produces frames with one-based indexing. Track files produced with older
  versions that had zero-based indexing are detected and frame numbers are
  shifted by one.

• Video files can now be open directly or via a config file into TeleSculptor
  for viewing and processing.

• Added the concept of a "Project", which is essentially a configuration file
  in a working directory in which algorithms can write results. The user
  must now create or open a project before running an algorithms.

• TeleSculptor now depends on Sprokit from KWIVER and adds a new Sprokit
  powered compute tool to compute feature tracks using the new KLT feature
  tracker in KWIVER. The tool also detects key frames in video and uses
  ORB feature detection and matching on the keyframes with bag of words
  frame matching for loop closure.

• A new tool is available in the compute menu to compute a dense depth
  map on the current frame. This uses the new Super3D algorithms in KWIVER.

• TeleSculptor now supports logging with log4cplus.

• If an input video has KLV metadata (0601 or 0104) TeleSculptor will now
  parse that metadata and use it to construct initial camera models. It will
  also use the metadata to set a geographic origin for the project, which is
  written to the project directory.

• TeleSculptor now has a pair of new tools in the Compute menu to save out
  either keyframes or all frames of video to image files.

• TeleSculptor now uses a common convention for naming all output files
  supporting a frame of video. If the input is an image list it uses the
  source image names, if a video it uses video name plus frame number.

• TeleSculptor now has a new tool for triangulating landmarks. This is
  useful when cameras are provided (e.g. from metadata), but landmarks are
  not.

• TeleSculptor now supports loading and processing videos with increasing
  but non contiguous frame numbers. This may occur when loading a video
  at a reduced frame rate.

• TeleSculptor now has an option to display only keyframes when playing
  back a video.

• The compute menu has been rearranged and now has an "Advanced" submenu
  containing tool that are less stable or less commonly used.

• The loop over video frames to extract metadata for camera initialization now
  occurs in a separate thread. This allows the user to interact with the GUI
  and view the video while the camera initialization occurs.

• The slideshow delay slider has been replaced by a speed slider that controls
  the slideshow playback speed. When the slider is to the far left, slideshow
  speed is 0.25Hz and when to the far right, the slideshow plays without any
  delay between frames.

• Added separate visibility and color options for display of feature tracks
  that have descriptors and those that do not have descriptors. This allows
  for visual distinction between KLT tracks (without descriptors) on each
  frame of video and ORB, or similar, tracks (with descriptors) on keyframes.

• Added a progress bar that appears below the World View to show progress when
  tools are running. It can also display a status message.

• Added CMake superbuild to automatically pull and build a specified version
  of all the necessary libraries from Fletch and KWIVER. This allows
  TeleSculptor to be fully built from a single build, rather than having to
  manually build and connect multiple builds. The superbuild does still allow
  using a manually built KWIVER/Fletch if so desired.

• Updated TeleSculptor to use Qt version 5.11 instead of 4.8. This also
  enables the OpenGL2 backend in VTK 8.0 which provides faster rendering.
  Qt5 now requires Python to build, so we are also requiring Python for
  the superbuild, which enables Python in KWIVER. We are not yet taking
  advantage of KWIVER Python bindings in TeleSculptor, but may in the future.

• Added ability to interactively add and edit a region of interest in the
  world view.

• Added ability to specify image masks. Currently these are supported by
  dense feature tracking.

• Added ability to place and edit ground control points that are synchronized
  across the world and camera views. The points can be exported to a geojson
  file and get imported using the standard import project infrastructure.

• Double-clicking the mouse pointer in the WorldView allows to set the center
  of rotation of the view camera about the nearest picked point in the view.

• Refactored the code to remove the ReconstructionData class that was left
  over from when surface extraction was part of a separate repository.

• Added metrics for RMS error by camera and a check that the rays meeting at a
  point have some minimum angle.

• Switched the iso-surfacing algorithm to
  flying edges.

• Rebranded "MAP-Tk TeleSculptor" to simply "TeleSculptor".
  The name "maptk" is still used in parts of the source code.

• Combined the "Mesh" and "Colored Mesh" export options into a single option
  named "Fused Mesh". This will save the mesh with or without color depending
  on whether color is enabled when saving.

• Added High-DPI icons for display on higher resolution displays.

• Placing ground control points in the view uses a smart guess initial position
  method. When placing points in the world view, a point placer is used which
  snaps the point to a nearby landmark or the ground plane. When placing points
  in the camera view, the depth of the 3D point in the world view is determined
  by picking along the active camera direction vector for landmarks or the
  ground plane.

• We now recompute the geographic origin whenever the landmarks are updated.
  This keeps the ground plane aligned with the data.

• Use robust bounds fit to the landmarks when setting or resetting the ROI.

• Added a new ruler measurement for measuring the distance between two points
  in the view. The ruler measurement displays two synchronized rulers between
  the world and camera view. Placing the end-points of the ruler also uses the
  smart guess initial position approach employed by ground control points.

• The default state of the depth map and volume visibility in the World View
  is now "visible". So you no longer need to turn them on manually to see
  these results.

• Set the default surface threshold to 0.0 (instead of 0.5) as it should be
  and set the spin box increments to 0.1.

• Added an option to the feature tracking tool to provide a maximum number
  of frames to track on. These frames are selected uniformly from the video.

• Added an option to the display to only show frames that have feature
  tracking data.

• Set default filenames in save dialog boxes.

• Added a check box to show only inlier residuals. Previously this was the
  default behavior. Now unchecking the box shows outliers too. Also added
  an option to color outlier residuals in a different color.

Fixes since MAP-Tk v0.10.0

Build System

• The CPack installer packaging was broken on MacOS due to using the variable
  CMAKE_SHARED_LIBRARY_SUFFIX instead of CMAKE_SHARED_MODULE_SUFFIX to find
  KWIVER plugins to include in the package. These variables are the same on
  most platforms but differ on MacOS where libraries are .dylib and modules
  are .so.

Installer

• Fixed an issue where required data files for PROJ were not installed.
  This also requires TeleSculptor and the command line tools to set an
  environment variable to tell PROJ where the data is installed.

TeleSculptor

• Removed an unused signal/slot connection that was generating warnings on
  the command line. Neither the signal nor the slot existed.

• Fixed a visualization bug in which a frame with an invalid camera was still
  rendered in the world view by borrowing the camera from the most recently
  displayed valid frame.

• Fixed a visualization bug in which camera frustums were not rendered when
  built with GCC with compiler optimizations turned on.

• TeleSculptor now catches more algorithm exceptions and turns them into
  warning messages rather than letting the application crash.

• The functionality to color mesh surfaces was broken when TeleSculptor
  switched to using the video_input API to load all imagery. This code
  has now been fixed to work with video_input. It now works as it did
  before. It also no longer requires camera files to loaded again from
  KRTD files and instead uses the in memory cameras.

• Increased the number of significant digits used to 12 when writing the camera
  parameters to a krtd file with the Blender plugin.

• The camera path was not showing since the upgrade to VTK 8, now it is.

• Camera scaling had been broken since it relied on camera path, which was
  broken. Fixing the camera path rendering also fixed this. Also adjusted
  the camera scaling range to make it more reasonable.

• Fixed a crash in the GUI that occurred when the video reader plugin returned
  a nullptr instead of a valid frame.

• Fixed issues with ROI initialization and the use of the ROI in determining
  display scale. If not manually set, the ROI is automatically initialized
  before running tools. When set, the ROI (not landmarks) is used to set the
  scene scale for drawing cameras.

• Switched from calling get_image() in a loop during the depth_to_vtk function
  to outside the loop. This improves processing speed by eliminating
  unnecessary image allocations for certain underlying image classes.

• Fixed the volume surface mesh to use a solid color when not in colorize mode
  previously the color would change with the surface threshold.

• Fixed the colorize checkbox for the volume to return the model to solid color
  when unchecked.

• Fixed a bug in which the active camera would sometimes flicker or disappear
  when updating information without changing the active frame.

• Fixed a bug in which a progress bar would sometimes not go away after a job
  was completed.

• Fixed an issue in which the depth map computation was only using support
  frame coming after the reference frame. The algorithm now uses equal frames
  both before and after the reference frame.

Tests

• All of the unit tests in v0.10.0 were testing functions that had moved
  into KWIVER. Those tests had also been duplicated in KWIVER, so the
  redundant tests in MAP-Tk were removed. This leaves MAP-Tk with no tests.
  New tests, especially for the command line tools, should be added in the
  future.