rgbd_odometry_multi_scale gives inverse transformation

[christian-rauch]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for main branch).

Describe the issue

According to the RGBD odometry documentation at

Open3D/cpp/open3d/t/pipelines/odometry/RGBDOdometry.h

Lines 143 to 144 in 203ac47

	/// \return odometry result, with (4, 4) optimized transformation matrix from
	/// source to target, inlier ratio, and fitness.

, RGBDOdometryMultiScale and hence rgbd_odometry_multi_scale estimate the transformation from source to target, or in other words, it gives the pose of the target data in the source frame.

But it seems to estimate the inverse of this.

Steps to reproduce the bug

#!/usr/bin/python3
import open3d as o3d
from open3d.t.pipelines.odometry import rgbd_odometry_multi_scale, Method
from open3d.t.geometry import Image, RGBDImage
import numpy as np

# fake RGBD images
i1 = np.zeros((120, 160, 3), dtype=np.uint8)
i2 = np.zeros((120, 160, 3), dtype=np.uint8)

d1 = (np.ones((120, 160)) * 1000).astype(np.uint16)
d2 = d1 + 10

d1[:10, :10] += 5
d2[:10, :10] += 5

rgbd1 = RGBDImage(
    Image(np.ascontiguousarray(i1)),
    Image(np.ascontiguousarray(d1)),
)

rgbd2 = RGBDImage(
    Image(np.ascontiguousarray(i2)),
    Image(np.ascontiguousarray(d2)),
)

intrinsics = np.array([
    [80,  0, 80],
    [ 0, 60, 60],
    [ 0,  0,  1],
])

# optimized transformation matrix from source to target, inlier ratio, and fitness
# https://github.com/isl-org/Open3D/blob/203ac47d99b7ead8a1b07535a67a29b71716cb51/cpp/open3d/t/pipelines/odometry/RGBDOdometry.h#L143-L144
result = rgbd_odometry_multi_scale(
    source = rgbd1,
    target = rgbd2,
    intrinsics = intrinsics,
    init_source_to_target = o3d.core.Tensor(np.identity(4)),
    depth_scale = 1000,
    method = Method.PointToPlane,
)

print("transformation from source to target:")
print(result.transformation)

Error message

transformation from source to target:
[[0.9999999999996052 -8.873573103987325e-07 4.515684927591465e-08 -0.008881941186502222],
 [8.873573088095082e-07 0.9999999999996055 3.5326331110456846e-08 -0.008506522379392627],
 [-4.515688062297498e-08 -3.5326291040182566e-08 0.9999999999999979 0.010000016679992096],
 [0 0 0 1]]
Tensor[shape={4, 4}, stride={4, 1}, Float64, CPU:0, 0x160f0e00]

The estimated transformation sets the target (rgbd2) 1cm into the rgbd1 frame.

Expected behavior

The fake data has the RGBD image 1 at 1 meter distance (1000 units) of a plane and RGBD image 2 at 1 meter and 1cm distance (1010 units). I.e. pointing at a plane, the camera would move 1cm backward from image 1 to image 2. Using the OpenCV camera frame convention (which seems to be used by Open3D too www.open3d.org/docs/0.7.0/python_api/open3d.geometry.create_point_cloud_from_depth_image.html), where z is pointing forward, this would mean a motion in negative z direction by 1cm.

With source = rgbd1 and target = rgbd2, the transformation from source to target should be -1cm in z direction, but the estimated transformation is the opposite.

Open3D, Python and System information

- Operating system: Ubuntu 24.04
- Python version: Python 3.12
- Open3D version: output from python: 0.19.0
- System architecture: x86
- How did you install Open3D?: pip

Additional information

The ComputeOdometryResultPointToPlane mentions that the delta transformation is "target_to_source":

Open3D/cpp/open3d/t/pipelines/odometry/RGBDOdometry.cpp

Line 400 in 203ac47

// Delta target_to_source on host.