"Kinect Smoothing" helps you to smooth and filter the Kinect depth image as well as trajectory data
git clone https://github.com/intelligent-control-lab/Kinect_Smoothing.git
pip install -r requirements.txt
Running
Please check example.ipynb and kinect_preprocess_example.py.
Data preparation
We saved many frames of depth images in data/sample_img.pkl
and saved corresponding frames of position coordinates in data/sample_pose.pkl
e.g. sample_img = [ [ image_1 ] , [ image_2 ], ... , [ image_t ] ].
each image_i
has a shape of (width, height)
.
In case if anyone wants to use it for multiple files, then the code below should work.
import joblib
import cv2
import glob
X_data = []
file_lists = glob.glob("/*.bmp") # image path
for files in sorted(file_lists):
if files.endswith(".bmp"):
image = cv2.imread(files)
X_data.append(image)
joblib.dump(X_data, 'image_frames.pkl')
1 . Depth Image Smoothing
Hole Filling Filter
In the original Kinect depth image, there are many invalid pixels (they appear black on the image and are registered as 0's). This function helps you to fill the invalid values with based on the valid pixels in the vicinity. The methods for hole filling are as follows:
Denoising Filter
After pixel filling, Denoising filters can be used to improve the resolution of the depth image. The methods for denoising filling are as follows:
2 . Trajectory Smoothing
Crop Filter:
The x, y coordinates of the trajectory were captured by some object detection algorithms (e.g. Openpose). Sometimes the object will be positioned on the background, and the depth coordinates might register as invalid values on the Kinect. The Crop-Filter crops the invalid value and run some interpolation methods to replace it. The methods for invalid value replacement are as follows:
Gradient Crop Filter:
Similar to Crop-Filter, the GradientCrop_Filter crops the large gradient values between near pixels maybe miss-labeled as background. The methods for invalid value replacement are as follows:
Smooth Filter:
Smooth the data with a specific filter, which is effectively reducing the anomaly in the trajectory series. The methods for smoothing filter are as follows:
Motion Sampler:
Motion detection and filtering out the stationary part of the trajectory. (Significant movements are preserved.)
3 . Real World Coordinate Calculation
Coordinate Calculator
Transforming the pixel level coordinate of Kinect image to the real world coordinate.