With Depth, 3D detection not working

[ArghyaChatterjee]

Hello,

I am using a zed camera. I have correctly remapped the topics and able to generate the demos correctly for 2D detections but not able to generate the 3D detection results.

To be specific, I am using ros2 launch yolo_bringup yolo.launch.py model:=yolov8m-seg.pt use_3d:=False and this is working.

This command which should also be showing 3D detection ros2 launch yolo_bringup yolo.launch.py model:=yolov8m-seg.pt is executing but the node seems to hang and the depth and RGB image coming from zed seem to freeze (froze on rviz and also froze when I do topic echo) and I see no detection in 2D image let alone in the 3D one.

The QoS profile zed has for the depth are:

$ ros2 topic info -v /zed/zed_node/left/image_rect_color
Type: sensor_msgs/msg/Image

Publisher count: 1

Node name: zed_node
Node namespace: /zed
Topic type: sensor_msgs/msg/Image
Endpoint type: PUBLISHER
GID: 01.0f.98.94.c7.f4.ac.2e.00.00.00.00.00.00.49.03.00.00.00.00.00.00.00.00
QoS profile:
  Reliability: RELIABLE
  History (Depth): UNKNOWN
  Durability: VOLATILE
  Lifespan: Infinite
  Deadline: Infinite
  Liveliness: AUTOMATIC
  Liveliness lease duration: Infinite

Subscription count: 1

Node name: rviz
Node namespace: /
Topic type: sensor_msgs/msg/Image
Endpoint type: SUBSCRIPTION
GID: 01.0f.98.94.91.7f.d2.b7.00.00.00.00.00.00.34.04.00.00.00.00.00.00.00.00
QoS profile:
  Reliability: RELIABLE
  History (Depth): UNKNOWN
  Durability: VOLATILE
  Lifespan: Infinite
  Deadline: Infinite
  Liveliness: AUTOMATIC
  Liveliness lease duration: Infinite

$ ros2 topic info -v /zed/zed_node/depth/depth_registered
Type: sensor_msgs/msg/Image

Publisher count: 1

Node name: zed_node
Node namespace: /zed
Topic type: sensor_msgs/msg/Image
Endpoint type: PUBLISHER
GID: 01.0f.98.94.c7.f4.ac.2e.00.00.00.00.00.00.79.03.00.00.00.00.00.00.00.00
QoS profile:
  Reliability: RELIABLE
  History (Depth): UNKNOWN
  Durability: VOLATILE
  Lifespan: Infinite
  Deadline: Infinite
  Liveliness: AUTOMATIC
  Liveliness lease duration: Infinite

Subscription count: 1

Node name: rviz
Node namespace: /
Topic type: sensor_msgs/msg/Image
Endpoint type: SUBSCRIPTION
GID: 01.0f.98.94.91.7f.d2.b7.00.00.00.00.00.00.33.04.00.00.00.00.00.00.00.00
QoS profile:
  Reliability: RELIABLE
  History (Depth): UNKNOWN
  Durability: VOLATILE
  Lifespan: Infinite
  Deadline: Infinite
  Liveliness: AUTOMATIC
  Liveliness lease duration: Infinite

This is same for RGB images as well.

The 2 depth publishing mode that zed has:

mono16:

$ ros2 topic echo /zed/zed_node/depth/depth_registered --once
header:
  stamp:
    sec: 1741493413
    nanosec: 735315610
  frame_id: zed_left_camera_optical_frame
height: 376
width: 672
encoding: mono16
is_bigendian: 0
step: 1344
data:
- 32
- 6
- 38
- 6
- 43
- 6
- 48
- 6
- 49
...

32FC1:

$ ros2 topic echo /zed/zed_node/depth/depth_registered --once
header:
  stamp:
    sec: 1741493657
    nanosec: 734350808
  frame_id: zed_left_camera_optical_frame
height: 376
width: 672
encoding: 32FC1
is_bigendian: 0
step: 2688
data:
- 255
- 255
- 255
- 127
..

I tried both but not working. Still have the same issue. Do you have any better idea about why this is happening ?

[mgonzs13]

Hi @ArghyaChatterjee, if you are not getting detections while using 3D and your QoS are correct, you may be having problems with the target frame. By default, the target_frame is base_link.