1. DefSLAM

Authors: Jose Lamarca, Shaifali Parashar, Adrien Bartoli and J. M. M. Montiel

DefSLAM is a real-time deformable SLAM library for Monocular cameras that computes the camera trajectory and a sparse 3D reconstruction in a deformable environment.

Find more videos of experiments here: https://www.youtube.com/playlist?list=PLKBuKNhAV30SlKGJ9eaMlAExdWRypUy-K

Related Publications:

Lamarca, Jose, et al. "DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences." arXiv preprint arXiv:1908.08918 (2019).

@article{lamarca2019defslam,
  title={DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences},
  author={Lamarca, Jose and Parashar, Shaifali and Bartoli, Adrien and Montiel, JMM},
  journal={arXiv preprint arXiv:1908.08918},
  year={2019}
}

Lamarca J., Montiel J.M.M. (2019) Camera Tracking for SLAM in Deformable Maps. In: Leal-Taixé L., Roth S. (eds) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol 11129. Springer, Cham

@inproceedings{lamarca2018camera,
  title={Camera tracking for SLAM in deformable maps},
  author={Lamarca, Jose and Montiel, JMM},
  booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
  year={2018}
}

2. Prerequisites

We have tested the library in 16.04 and 18.04, but it should be easy to compile in other platforms. A powerful computer (e.g. i7) will ensure real-time performance and provide more stable and accurate results.

Pangolin

We use Pangolin for visualization and user interface. Dowload and install instructions can be found at: https://github.com/stevenlovegrove/Pangolin.

OpenCV

We use OpenCV to manipulate images and features. Dowload and install instructions can be found at: http://opencv.org. Required 4.0.0.

Eigen3

Required by g2o (see below). Download and install instructions can be found at: http://eigen.tuxfamily.org. Required at least 3.1.0.

Ceres library:

We use Ceres to optimize warp and to run the NRSfM.

PCL

It is used just for ground truths. It is not critic for the program. We use PCL mainly for groundtruths.

DBoW2 and g2o and ORBSLAM(Included in Thirdparty folder)

We use modified versions of the DBoW2 library to perform place recognition and g2o library to perform non-linear optimizations. Both modified libraries (which are BSD) are included in the Thirdparty folder. Likewise we include a modified version of ORBSLAM. BOW is only used for ORBSLAM_2.

3. License

DefSLAM is released under a GPLv3 license. For a list of all code/library dependencies (and associated licenses), please see Dependencies.md.

4. Building DefSLAM library and examples

Clone the repository:

git clone https://github.com/JoseLamarca/DefSLAM.git

We provide a script build.sh to build the DefSLAM including. Please make sure you have installed all required dependencies (see section 2). Execute:

cd DefSLAM
chmod +x build.sh
./build.sh

This will create libDefSLAM.so at lib folder and the executables simplestereo simpleCT and simple in Apps folder.

4.1 Windows build

4.1.1 Install vcpkg

Follow the vcpkg install guide

• Set follwoing Environment Variables:
• VCPKG_DEFAULT_TRIPLET = x64-windows
• VCPKG_ROOT = <path to VCPKG> If environment variables not defined then default VCPKG_ROOT="C:\git\vcpkg" set by vSLAM build file

4.1.2 Install dependcies

in VCPKG_ROOT call

vcpkg install OpenCV Pangolin Eigen3 ceres[suitsparse,lapack,eigensparse,tools] pcl

4.1.2 Build

cd DefSLAM
build.bat

5. Datasets

Mandala dataset

Download the sequences with the link: Mandala Dataset

It contains the five sequences presented in the paper. The mandala deformation becomes more challenging with the number of dataset.

The dataset is organized as follows:

-- yalm file
-- Mandala(i)
   -- images
   -- times

Hamlyn dataset

1. Download the dataset from the webpage Hamlyn.

2. To downloaded in the format run in our paper Hamlyn not official

Run Example

Examples of scripts in Apps/rungt.sh

1. Process a sequence.

./DefSLAM <ORBfile> <yaml.file> <ORBfile> <image folder>

If you run it without . It takes the camera in 0.

2. Sequences with depth image for ground truth. (Used for CT phantom dataset)

./DefSLAMCTGT <ORBfile> <yaml.file> <video.avi> <folder with pattern>

3. Sequences with stereo for ground truth.

./DefSLAMGT <ORBfile> <yaml.file> <left_image folder> <right_image folder> <timestamps file>

Script to generate this folders with this format avaiable. Ask me through [email protected]

Code Organization

-- Root folder
  -- Apps. Folder with executables.

  -- Vocabulary. Folder that contains the vocabulary for the BoW.

  -- ThirdParty
    -- BBS. Bspline library adapted for C++ with eigen.
    -- DBoW2. Bag Of word library only used for ORBSLAM_2. Loop closin and relocalization.
    -- g2o. Optimization library for deformation tracking and pose estimation.
    -- ORBSLAM_2. Base clases of the rigid SLAM library modified to work in deforming 
		  environments.

  -- Modules. DefSLAM modules, it includes all the modules needed for the library.
    -- Common. It includes the keyframes, map, map points and Systems.
    -- Mapping.
    -- GroundTruth
    -- Matching
    -- Settings. Check advance settings.
    -- Template. Template tools
    -- ToolsPCL. Tools from PCL library used mainly for the ground truth.
    -- Tracking. 
    -- Viewer.

Advance settings for and ORBSLAM:

To repeat the experiments and save the results you can activate the flag Viewer.SaveResults in the yalm file. The system is running by default in parallel. To change it, you must change the file set_MAC.h and comment the line #define PARALLEL.

To run ORBSLAM, you can uncomment the line 5 in set_MAC.h.

The file CC_MAC.h contain the parameters for the cross correlation matching used in the grundtruth. It is adapted for a better performance in the Mandala dataset, but it should be adapted for each dataset.

To set the number of nodes of the mesh (Initially 10x10), change line 63 and 64 of Template/TriangularMesh.cc

To set the number of nodes for the BBSpline Thirdparty/BBS/bbs_MAC.h