修复保存深度图至磁盘时出现的重复波纹结构问题

问题背景 #

RViz2中可视化的对齐后深度图显示正常，但按照BundleSDF仓库作者说明，将深度值乘以1000后转换为np.uint16格式保存到磁盘时，深度图出现异常（呈现条纹状失真）。尝试两种不同保存脚本后问题依旧，而对齐后的RGB图像保存格式完全正常。

出错的保存脚本 #

脚本1：固定尺寸重塑 #

from pathlib import Path
from rosbags.highlevel import AnyReader
import numpy as np
from PIL import Image
from datetime import datetime
from matplotlib import image
import cv2
import matplotlib.pyplot as plt

with AnyReader([Path('/home/mona/rosbag2_2023_11_06-15_44_24')]) as reader:
    connections = [x for x in reader.connections if x.topic == '/camera/camera/aligned_depth_to_color/image_raw']

    for connection, timestamp, rawdata in reader.messages(connections=connections):
        msg = reader.deserialize(rawdata, connection.msgtype)
        timestamp_dt = datetime.fromtimestamp(msg.header.stamp.sec + msg.header.stamp.nanosec * 1e-9)
        timestamp_str = timestamp_dt.strftime("%Y-%m-%d %H:%M:%S.%f")

        timestamp_ns = msg.header.stamp.sec * 1e9 + msg.header.stamp.nanosec
        numeric_timestamp = int(timestamp_ns / 1e-9)
        image_data = msg.data.reshape(480, 640,-1)*1000
        
        # 提取单通道灰度图
        grayscale_image = image_data[:, :, 0]
        depth_image_name = 'depth/' + str(numeric_timestamp)[:20] + '.png'
        cv2.imwrite(depth_image_name, grayscale_image.astype(np.uint16))

脚本2：动态尺寸重塑+UMat处理 #

from pathlib import Path
from rosbags.highlevel import AnyReader
import numpy as np
from PIL import Image
from datetime import datetime
from matplotlib import image
import cv2
import matplotlib.pyplot as plt

with AnyReader([Path('/home/mona/rosbag2_2023_11_06-15_44_24')]) as reader:
    connections = [x for x in reader.connections if x.topic == '/camera/camera/aligned_depth_to_color/image_raw']

    for connection, timestamp, rawdata in reader.messages(connections=connections):
        msg = reader.deserialize(rawdata, connection.msgtype)
        timestamp_dt = datetime.fromtimestamp(msg.header.stamp.sec + msg.header.stamp.nanosec * 1e-9)
        timestamp_str = timestamp_dt.strftime("%Y-%m-%d %H:%M:%S.%f")
        timestamp_ns = msg.header.stamp.sec * 1e9 + msg.header.stamp.nanosec
        numeric_timestamp = int(timestamp_ns / 1e-9)        
        w, h = msg.width, msg.height
        image_data = msg.data.reshape(h, w,-1)*1000
        
        # 提取单通道灰度图
        grayscale_image = image_data[:, :, 0]
        depth_image_name = 'depth/' + str(numeric_timestamp)[:20] + '.png'
        depth_data = np.array(grayscale_image, dtype=np.uint16)
        image16 = cv2.UMat(depth_data)
        cv2.imwrite(depth_image_name, image16.get())

系统环境信息 #

(base) mona@ada:~$ ros2 wtf
/opt/ros/humble/lib/python3.10/site-packages/ros2doctor/api/package.py: 112: UserWarning: joy has been updated to a new version. local: 3.1.0 < latest: 3.3.0
/opt/ros/humble/lib/python3.10/site-packages/ros2doctor/api/package.py: 112: UserWarning: sdl2_vendor has been updated to a new version. local: 3.1.0 < latest: 3.3.0

All 5 checks passed

(base) mona@ada:~$ /usr/bin/python3.10
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux

(base) mona@ada:~$ uname -a
Linux ada 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
(base) mona@ada:~$ lsb_release -a
LSB Version:    core-11.1.0ubuntu4-noarch:security-11.1.0ubuntu4-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

修复步骤 #

问题根源在于深度数据的类型转换溢出和OpenCV保存16位图像的格式要求，具体修复如下：

1. 截断溢出值再转换类型
   uint16的最大值为65535，深度值×1000后可能超出该范围，需先将数值截断到0-65535区间，再转换类型：

# 替换原类型转换代码
grayscale_image = image_data[:, :, 0]
grayscale_image = np.clip(grayscale_image, 0, 65535)
depth_data = grayscale_image.astype(np.uint16)

2. 指定OpenCV保存16位PNG的编码格式
   OpenCV默认保存格式可能导致16位图像失真，需明确指定PNG编码参数：

# 替换原cv2.imwrite代码
cv2.imwrite(depth_image_name, depth_data, [cv2.IMWRITE_PNG_COMPRESSION, 0])

3. 确保数据为单通道灰度图
   避免多通道数据干扰，强制提取单通道：

# 确保数据是单通道
if len(grayscale_image.shape) == 3:
    grayscale_image = grayscale_image[:, :, 0]

完整修复后的脚本 #

from pathlib import Path
from rosbags.highlevel import AnyReader
import numpy as np
import cv2
from datetime import datetime

with AnyReader([Path('/home/mona/rosbag2_2023_11_06-15_44_24')]) as reader:
    connections = [x for x in reader.connections if x.topic == '/camera/camera/aligned_depth_to_color/image_raw']

    for connection, timestamp, rawdata in reader.messages(connections=connections):
        msg = reader.deserialize(rawdata, connection.msgtype)
        timestamp_ns = msg.header.stamp.sec * 1e9 + msg.header.stamp.nanosec
        numeric_timestamp = int(timestamp_ns)
        
        # 用消息自带宽高重塑，避免固定尺寸错误
        h, w = msg.height, msg.width
        # 原始深度数据通常为float32类型，先转换为浮点再缩放
        image_data = msg.data.reshape(h, w).astype(np.float32) * 1000
        
        # 截断超出uint16范围的值
        depth_data = np.clip(image_data, 0, 65535).astype(np.uint16)
        
        depth_image_name = f'depth/{numeric_timestamp}.png'
        # 指定PNG编码保存16位图像
        cv2.imwrite(depth_image_name, depth_data, [cv2.IMWRITE_PNG_COMPRESSION, 0])

内容的提问来源于stack exchange，提问作者Mona Jalal