š¤ Elaborated hand-eye calibration tutorials (ROS-binding)
| Ubuntu 1804 & ROS Melodic Morenia
|
This repo contains a eye-in-hand calibration tool (Cplusplus & ROS) in JDäŗ¬äø GRASPING ROBOT CHALLENGE (News),
and the implements of my paper: Robotic hand-eye calibration with depth camera: A sphere model approach (PDF)
Inside /src
there are 5 ROS packages:
rgbd_srv
used by camera_driver.
camera_driver
drive IntelĀ® RealSenseā¢ RGBD cameras in ROS.
(convert raw stream to ros sensor_msgs::Image
)
camera_transform_publisher
publish the transformation matrix (extrinsics) between camera and a marker( chessboard | aruco tag).
handeye_calib_marker
handeye calib tools that use RGB camera and a marker, modified from handeye_calib_camodocal
----------------------------------------------
with the package above you can calibrate the eye-in-hand transformation on RGB images.
handeye_calib_sphere
fine-tune the eye-in-hand result based on depth.
/usr/local
(both 3.4.0
)$ sudo apt-get install ros-melodic-visp-hand2eye-calibration
0.4.0
)1.14.0
) and instruction at here
1.0.0
)$ sudo apt-get install ros-melodic-cv-bridge
$ sudo apt-get install libpcl-dev ros-melodic-pcl-ros
Download opencv and opencv_contrib (both tested on 3.4.0
).
Install opencv prerequisit:
[compiler] $ sudo apt-get install build-essential
[required] $ sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
[optional] $ sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
Build opencv & opencv_contrib from source: (we turned off the cuda options)
$ mkdir build && cd build
$ cmake \
-D CMAKE_BUILD_TYPE=Release \
-D OPENCV_EXTRA_MODULES_PATH= <path/to/opencv_contrib-3.4.0>/modules/ \
-D BUILD_opencv_cudacodec=OFF \
-D WITH_CUDA=OFF \
-D WITH_CUBLAS=OFF \
-D WITH_CUFFT=OFF \
-D ENABLE_PRECOMPILED_HEADERS=OFF \
-D CMAKE_INSTALL_PREFIX=/usr/local ..
$ make -j5
$ sudo make install
We use SR300/D415 camera and use camera_driver
package to convert raw RGBD images into ROS topic of sensor_msgs::Image
. The librealsense
is required only if you use cameras from RealSenseā¢ family, otherwise make your own camera_driver
.
Following the librealsense installation guide at here.
We installed: librealsense2-dkms
, librealsense2-utils
, librealsense2-dev
, librealsense2-dbg
If you are lucky enough, the ros pre-build ros-melodic-cv-bridge
works well.
However, since the default OpenCV in ros-melodic-cv-bridge
is 3.2
, while we use opencv & opencv_contrib in 3.4.0
, we recommend you to build the cv_bridge from source.
$ git clone https://github.com/ros-perception/vision_opencv.git
$ cd vision_opencv
$ git checkout melodic
$ cd cv_bridge
$ mkdir build && cd build
$ cmake ..
$ make -j5
$ sudo make install
In the meantime, remove the pre-bulid cv_bridge
in ros-melodic path /opt/ros/melodic
:
$ cd /opt/ros/melodic
# or you can archive them to other place
$ sudo rm -rf lib/libcv_bridge.so
$ sudo rm -rf include/cv_bridge
$ sudo rm -rf share/cv_bridge
For more information about cv_bridge
, see here.
$ git clone https://github.com/lixiny/Handeye-Calibration-ROS.git
$ cd Handeye-Calibration-ROS
$ catkin_make
if you only want to calibrate hand-eye transformation on RGB marker,
$ git clone https://github.com/lixiny/Handeye-Calibration-ROS.git
$ cd Handeye-Calibration-ROS
$ git checkout rgb_only
$ catkin_make
Prepare a camera support that mount camera on the robot. We provide the support files [.STL] for UR5 and RealSense D415 & SR300.
Navigate to the Handeye-Calibration-ROS/
folder, and source devel/setup.bash
in each new terminal.
$1 roslaunch camera_driver realsense_driver.launch
The camera intrinsic (later we call it intr
) will appear in current terminal, record it.
The camera_driver
publishes 3 topics to rosmaster:
/camera_link
K
and D
in it.You can also visualize these topics in rviz
.
You can either use a chessboard or an aruco plane at doc/rawArucoPlane.jpg we designed.
After you placed the chessboard inside camera view, run:
$2 roslaunch camera_transform_publisher chessboard_publisher_realsense.launch
# or specify chessboard parameters:
$2 roslaunch camera_transform_publisher chessboard_publisher_realsense.launch \
chessboardWidth:=10 chessboardHeight:=7 squareSize:=0.02
[IMPORTANT] There are 5 user-specified parameters in chessboard_publisher_realsense.launch
:
/realsense/rgb
/realsense/camera_info
)Make sure you have already checked them based on your chessboard.
After you placed our aruco plane inside camera view, run:
$2 roslaunch camera_transform_publisher aruco_publisher_realsense.launch
# or specify aruco plane parameters:
$2 roslaunch camera_transform_publisher aruco_publisher_realsense.launch \
tagSideLen:=0.035 planeSideLen:=0.25
[IMPORTANT] There are 4 user-specified parameters in aruco_publisher_realsense.launch
:
/realsense/rgb
/realsense/camera_info
)Make sure you have already checked them based on the physical length of your aruco plane.
Now in the pop-up window, you will see an AR cube like this:
This camera_transform_publisher
will publish a tf (/camera_link
, /ar_marker
) to ros master. If no marker found, a Identity SE3 tf will be used. You can also visualize the two frame in rviz.
Followed the instrction #3. Bringup UR in doc/install_ur.md.
$3 roslaunch handeye_calib_marker handeye_online.launch
[IMPORTANT] There are 4 user-specified parameters in handeye_online.launch
:
camera_transform_publisher
, /ar_marker
)camera_transform_publisher
, /camera_link
)/ee_link
)/base_link
)Check these 4 tf names before you launch calibraion!
Repeatedly move UR end effector to different configuration. Meanwhile, make sure at each unique configuration, a valid marker can be detected. In current terminal, press s
to record one AX=XB
equation. After sufficient # of equations have been recorded (30+), press q
to perform calibraition. Then in current terminal, you will see some output like:
modify the file: handeye_calib_marker/launch/show_result.launch
,
replace the ${Translation: x y z}
and ${Rotation: qx qy qz qw}
based on your terminal output.
<launch>
<node pkg="tf"
type="static_transform_publisher"
name="realsense_link"
args="${Translation: x y z}
${Rotation: qx qy qz qw}
/ee_link
/camera_link
100"
/>
</launch>
and launch:
$4 roslaunch handeye_calib_marker show_result.launch
or, in a new terminal:
# rosrun tf static_transform_publisher \
# x y z qx qy qz qw \
# frame_id child_frame_id period_in_ms
$4 rosrun tf static_transform_publisher ļ¼¼
0.0263365 0.0389317 0.0792014 -0.495516 0.507599 -0.496076 0.500715 \
/ee_link /camera_link 100
There are several representations/expressions of the HAND-EYE problem:
/ee_link
to /camera_link
/ee_link
, /camera_link
)/ee_link
, child: /camera_link
NOTE: now you should have a ready-to-use handeye transformation.
This results is optimized only based on RGB images. You can still fine-tune the result with depth image (if has) following setp # 6.
// TODO
Yang, Lixin, et al. " Robotic hand-eye calibration with depth camera: A sphere model approach. " 2018 4th International Conference on Control, Automation and Robotics (ICCAR). IEEE, 2018. PDF
@inproceedings{yang2018robotic,
title={Robotic hand-eye calibration with depth camera: A sphere model approach},
author={Yang, Lixin and Cao, Qixin and Lin, Minjie and Zhang, Haoruo and Ma, Zhuoming},
booktitle={2018 4th International Conference on Control, Automation and Robotics (ICCAR)},
pages={104--110},
year={2018},
organization={IEEE}
}
This repo is freely available for free non-commercial use, and may be redistributed under these conditions. Please, see the license for further details