Demo code for our TIP video stitching paper in 2017
Demo code for the works below:
Video Demo on YouTube:
Including video clips from previous works, see Readme.MD for more details)
*Notes on your own test data: The proposed method is using traditional SIFT + Mesh-based warping to conduct stitching, which cannot handle the case where the parallax is too large, or the light condition is too different. To put it in a simple way, if a single pair of frames from the input video pair can't be stitched by Content-Preserve Warping, then this video stitching method will also fail.
In the folder /case-cuhk_lib
, extract video frames of case17-l.mp4 to folder /left
, and extract video frames of case17-r.mp4 to folder /right
. After the frame extracation, each folder should contain 400 jpg files. The file names should be indexed properly. (e.g. 001.jpg 002.jpg ...). A script video2frames.m
is provided. Modify the filename
You may need to install CVX
if you have not.
Set MATLAB path to /Stitching-1.1.0
, run RunStitching.m
. The generated output frames will be in the auto-created subfolder under /case-cuhk_lib
. (res_demo
if you didn't change the output path). Warning: the motion segmentation code from Shankar Rao is very slow, if your video doesn't have large foreground object, set SKIP_BACKGROUND_SEGMENTATION = true
in RunStitching.m
and treat all trajectories as background. For further details, see the code comments.
Build the OpenCV project in /SeamCut
(You need to set OpenCV's include and library path manually) and copy the executable (e.g. SeamCut.exe
) to the folder containing the output frames (1.jpg, A1.jpg, B1.jpg, 2.jpg, A2.jpg, B2.jpg, ...). This program finds the continuous optimal seam by GraphCut algorithm and use OpenCV's multi-band blending function.
Run ./SeamCut 5 1 400 1 0.2
to see the final result. Blended frames are saved as D1.jpg, D2.jpg, ...
For more details, please read the comments
@article{nie2018dynamic, title={Dynamic Video Stitching via Shakiness Removing}, author={Nie, Yongwei and Su, Tan and Zhang, Zhensong and Sun, Hanqiu and Li, Guiqing}, journal={IEEE Transactions on Image Processing}, volume={27}, number={1}, pages={164--178}, year={2018}, publisher={IEEE} }
@inproceedings{su2016video, title={Video stitching for handheld inputs via combined video stabilization}, author={Su, Tan and Nie, Yongwei and Zhang, Zhensong and Sun, Hanqiu and Li, Guiqing}, booktitle={SIGGRAPH ASIA 2016 Technical Briefs}, pages={25}, year={2016}, organization={ACM} }
Code by Tan Su, Zhensong Zhang and Yongwei Nie. For research purpose ONLY.