Analyze RealSense D455’s ROS bag file with ROS Noetic on Windows 10

#コンピュータビジョン

#Windows

Yosuke Miyajima

2021.03.18

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

Intel RealSense cameras support recording streams of RGB/depth frames in ROS(Robot Operating System) bag format.

Simple recording or playing a bag file is possible using RealSense Viewer or librealsense API included in RealSense SDK. ROS tools are useful in case you need detailed frame-by-frame analysis on the other hand.

This post will explain how to analyze Intel RealSense D455's ROS bag file with ROS tools. I used ROS Noetic on Windows 10.

(この記事には日本語版があります↓)

Windows 10にROS NoeticをインストールしてRealSense D455のbagファイルを解析する

Hardware/software configuration

Windows10 Home (Ver. 20H2, build 19042.867)
Intel RealSense D455
Intel RealSense SDK 2.0 (v2.42.0)
ROS Noetic Ninjemys
Windows Terminal (Ver. 1.6.10571.0)

Recording and playing bag file using RealSense Viewer

First, record a bag file using RealSense Viewer for analysis. RealSense SDK is already installed on PC. RealSense SDK is assumed to be already installed on your PC. Start RealSense Viewer then connect a RealSense D455 to PC. It's also OK to connect D455 to PC before starting RealSense Viewer.

Recording bag file

After starting RealSense Viewer, set parameters of the sensors to be recorded, then click the on/off button for each sensor to start streaming.

Click the "Record" button to record a bag file after started streaming. Click the "Stop" button to finish recording.

The size of a bag file can be enormous according to the framerate or resolution settings, even if for short recordings. I recommend recording a short bag file first, then check the actual size of the file to ensure you have enough storage for longer recording.

The location of the recorded bag file can be checked and changed in Settings. Click the gear icon at the upper right corner of the RealSense Viewer window to open the settings.

Playing bag file

To play a bag file, click "Add Source" at the upper left of the RealSense Viewer window, then select "Load Recorded Sequence".

Select a bag file to view on File Explorer.

The RealSense Viewer will play the selected bag file automatically. Playback is repeated continuously by default.

Installing ROS Noetic on Windows 10

Install ROS Noetic on Windows 10 following the instruction on the ROS official webpage. ROS needs Visual Studio 2019, Chocolatoly, and Git. The instruction for installing these are described on the webpage.

noetic/Installation/Windows - ROS Wiki

Notes during installation

There should be no traps if you follow the install instructions correctly. However, my installation process froze when I executed the following command described in "5.1 ROS Last Known Good (LKG) Build Installation".

The installation completed smoothly after I hit Ctrl-C and terminated the installation process, then re-tried the process.

choco upgrade ros-noetic-desktop_full -y --execution-timeout=0

Settings for executing ROS setup script automatically

You have to execute the ROS setup script before using ROS commands. A Windows shortcut and a Windows Terminal profile are created when you complete the installation process. This post runs ROS commands from Windows Terminal.

You can select the ROS profile for Windows Terminal by clicking the "⌄" icon next to the "+" icon at the upper side of the Windows Terminal window.

Analyzing RealSense D455 bag file with ROS tools

rosbag info command

Let's analyze RealSense bag files! Select the ROS profile on Windows Terminal then run rosbag info command for your bag file.

The command returns the overview of your bag file includes the following information:

ROS bag format version
ROS bag message recording time(sec)
ROS bag message starting time and ending time
ROS bag file size
Number of messages recorded
Whether the file contents are compressed
Type information of messages in the file
Topic information of messages in the file

RealSense D455 bag file was compliant with ROS bag version 2.0. The start time of the message was not when the recording actually started, but the UNIX epoch.

The following is the entire result of the rosbag info command on my bag file:

C:\bagfile>rosbag info d455-848x480-rgbd-30fps.bag
path:        d455-848x480-rgbd-30fps.bag
version:     2.0
duration:    25.8s
start:       Jan 01 1970 09:00:00.00 (0.00)
end:         Jan 01 1970 09:00:25.76 (25.76)
size:        1.5 GB
messages:    36445
compression: none [1547/1547 chunks]
types:       diagnostic_msgs/KeyValue  [cf57fdc6617a881a88c16e768132149c]
             geometry_msgs/Transform   [ac9eff44abf714214112b05d54a3cf9b]
             realsense_msgs/StreamInfo [311d7e24eac31bb87271d041bf70ff7d]
             sensor_msgs/CameraInfo    [c9a58c1b0b154e0e6da7578cb991d214]
             sensor_msgs/Image         [060021388200f6f0f447d0fcd9c64743]
             std_msgs/Float32          [73fcbf46b49191e672908e50842a83d4]
             std_msgs/String           [992ce8a1687cec8c8bd883ec73ca41d1]
             std_msgs/UInt32           [304a39449588c7f8ce2df6e8001c5fce]
topics:      /device_0/info                                                       13 msgs    : diagnostic_msgs/KeyValue
             /device_0/sensor_0/Depth_0/image/data                               773 msgs    : sensor_msgs/Image
             /device_0/sensor_0/Depth_0/image/metadata                         18552 msgs    : diagnostic_msgs/KeyValue
             /device_0/sensor_0/Depth_0/info                                       1 msg     : realsense_msgs/StreamInfo             /device_0/sensor_0/Depth_0/info/camera_info                           1 msg     : sensor_msgs/CameraInfo
             /device_0/sensor_0/Depth_0/tf/0                                       1 msg     : geometry_msgs/Transform
             /device_0/sensor_0/info                                               2 msgs    : diagnostic_msgs/KeyValue
             /device_0/sensor_0/option/Asic_Temperature/description                1 msg     : std_msgs/String
             /device_0/sensor_0/option/Asic_Temperature/value                      1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Depth_Units/description                     1 msg     : std_msgs/String
             /device_0/sensor_0/option/Depth_Units/value                           1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Emitter_Always_On/description               1 msg     : std_msgs/String
             /device_0/sensor_0/option/Emitter_Always_On/value                     1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Emitter_Enabled/description                 1 msg     : std_msgs/String
             /device_0/sensor_0/option/Emitter_Enabled/value                       1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Emitter_On_Off/description                  1 msg     : std_msgs/String
             /device_0/sensor_0/option/Emitter_On_Off/value                        1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Enable_Auto_Exposure/description            1 msg     : std_msgs/String
             /device_0/sensor_0/option/Enable_Auto_Exposure/value                  1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Error_Polling_Enabled/description           1 msg     : std_msgs/String
             /device_0/sensor_0/option/Error_Polling_Enabled/value                 1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Exposure/description                        1 msg     : std_msgs/String
             /device_0/sensor_0/option/Exposure/value                              1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Frames_Queue_Size/description               1 msg     : std_msgs/String
             /device_0/sensor_0/option/Frames_Queue_Size/value                     1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Gain/description                            1 msg     : std_msgs/String
             /device_0/sensor_0/option/Gain/value                                  1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Global_Time_Enabled/description             1 msg     : std_msgs/String
             /device_0/sensor_0/option/Global_Time_Enabled/value                   1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Hdr_Enabled/description                     1 msg     : std_msgs/String
             /device_0/sensor_0/option/Hdr_Enabled/value                           1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Laser_Power/description                     1 msg     : std_msgs/String
             /device_0/sensor_0/option/Laser_Power/value                           1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Output_Trigger_Enabled/description          1 msg     : std_msgs/String
             /device_0/sensor_0/option/Output_Trigger_Enabled/value                1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Projector_Temperature/description           1 msg     : std_msgs/String
             /device_0/sensor_0/option/Projector_Temperature/value                 1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Sequence_Id/description                     1 msg     : std_msgs/String
             /device_0/sensor_0/option/Sequence_Id/value                           1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Sequence_Name/description                   1 msg     : std_msgs/String
             /device_0/sensor_0/option/Sequence_Name/value                         1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Sequence_Size/description                   1 msg     : std_msgs/String
             /device_0/sensor_0/option/Sequence_Size/value                         1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Stereo_Baseline/description                 1 msg     : std_msgs/String
             /device_0/sensor_0/option/Stereo_Baseline/value                       1 msg     : std_msgs/Float32
             /device_0/sensor_0/option/Visual_Preset/description                   1 msg     : std_msgs/String
             /device_0/sensor_0/option/Visual_Preset/value                         1 msg     : std_msgs/Float32
             /device_0/sensor_0/post_processing                                   10 msgs    : std_msgs/String
             /device_0/sensor_1/Color_0/image/data                               773 msgs    : sensor_msgs/Image
             /device_0/sensor_1/Color_0/image/metadata                         16233 msgs    : diagnostic_msgs/KeyValue
             /device_0/sensor_1/Color_0/info                                       1 msg     : realsense_msgs/StreamInfo             /device_0/sensor_1/Color_0/info/camera_info                           1 msg     : sensor_msgs/CameraInfo
             /device_0/sensor_1/Color_0/tf/0                                       1 msg     : geometry_msgs/Transform
             /device_0/sensor_1/info                                               2 msgs    : diagnostic_msgs/KeyValue
             /device_0/sensor_1/option/Auto_Exposure_Priority/description          1 msg     : std_msgs/String
             /device_0/sensor_1/option/Auto_Exposure_Priority/value                1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Backlight_Compensation/description          1 msg     : std_msgs/String
             /device_0/sensor_1/option/Backlight_Compensation/value                1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Brightness/description                      1 msg     : std_msgs/String
             /device_0/sensor_1/option/Brightness/value                            1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Contrast/description                        1 msg     : std_msgs/String
             /device_0/sensor_1/option/Contrast/value                              1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Enable_Auto_Exposure/description            1 msg     : std_msgs/String
             /device_0/sensor_1/option/Enable_Auto_Exposure/value                  1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Enable_Auto_White_Balance/description       1 msg     : std_msgs/String
             /device_0/sensor_1/option/Enable_Auto_White_Balance/value             1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Exposure/description                        1 msg     : std_msgs/String
             /device_0/sensor_1/option/Exposure/value                              1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Frames_Queue_Size/description               1 msg     : std_msgs/String
             /device_0/sensor_1/option/Frames_Queue_Size/value                     1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Gain/description                            1 msg     : std_msgs/String
             /device_0/sensor_1/option/Gain/value                                  1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Gamma/description                           1 msg     : std_msgs/String
             /device_0/sensor_1/option/Gamma/value                                 1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Global_Time_Enabled/description             1 msg     : std_msgs/String
             /device_0/sensor_1/option/Global_Time_Enabled/value                   1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Hue/description                             1 msg     : std_msgs/String
             /device_0/sensor_1/option/Hue/value                                   1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Power_Line_Frequency/description            1 msg     : std_msgs/String
             /device_0/sensor_1/option/Power_Line_Frequency/value                  1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Saturation/description                      1 msg     : std_msgs/String
             /device_0/sensor_1/option/Saturation/value                            1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/Sharpness/description                       1 msg     : std_msgs/String
             /device_0/sensor_1/option/Sharpness/value                             1 msg     : std_msgs/Float32
             /device_0/sensor_1/option/White_Balance/description                   1 msg     : std_msgs/String
             /device_0/sensor_1/option/White_Balance/value                         1 msg     : std_msgs/Float32
             /device_0/sensor_1/post_processing                                    1 msg     : std_msgs/String
             /device_0/sensor_2/info                                               1 msg     : diagnostic_msgs/KeyValue
             /device_0/sensor_2/option/Enable_Motion_Correction/description        1 msg     : std_msgs/String
             /device_0/sensor_2/option/Enable_Motion_Correction/value              1 msg     : std_msgs/Float32
             /device_0/sensor_2/option/Frames_Queue_Size/description               1 msg     : std_msgs/String
             /device_0/sensor_2/option/Frames_Queue_Size/value                     1 msg     : std_msgs/Float32
             /device_0/sensor_2/option/Global_Time_Enabled/description             1 msg     : std_msgs/String
             /device_0/sensor_2/option/Global_Time_Enabled/value                   1 msg     : std_msgs/Float32
             /file_version                                                         1 msg     : std_msgs/UInt32

rqt_bag command

Next, View the detail of a bag file with rqt_bag command. rqt_bag command is a visualization tool included in ROS. The viewing process has been recorded as a video since many GUI manipulations are needed during the process. You need to run roscore command before running rqt_bag command.

Resolving error during loading bag file on rqt_bag

An error occurred when I tried to load a bag file at first launched rqt_bag as follows.

The error was resolved by modifying bag_timeline.py by referencing this Github issue.

Conclusion

How to analyze Intel RealSense D455's ROS bag file with ROS tools has been explained in this post. ROS bag file and ROS tools are also useful for non-Robotics projects using RealSense cameras.