XRMoCap 0.8.0 Release Notes:

Highlights:

• Refactor evaluation for MvP, mvpose_tracking, mvpose and fourdag, sharing the same super-class.
• Add smpl visualization and unit test, based on minimal_pytorch_rasterizer. Multi-person and multi-gender are supported.
• Add mmdeploy for faster human perception.

New Features:

• Add PriorConstraint optimizer for 3D keypoints, filtering out poorly quality bboxes and limbs.
• Add mmdeploy for faster human perception.
• Add mask in smpl_data. The person whose mask is zero will not be plotted.
• Add function auto_load_smpl_data, it chooses a correct class when you forget of which type the npz file is.
• Add smpl visualization and unit test, based on minimal_pytorch_rasterizer. Multi-person and multi-gender are supported.
• Add class Timer for recording average time consumption.

Refactors:

• Refactor evaluation metrics including MPJPE, PA-MPJPE, PCK, PCP, mAP, and recall.
• Refactor evaluation for MvP, mvpose_tracking, mvpose and fourdag, sharing the same super-class.

Improvements:

• Update dockerfiles and docker images.
• Add sample config for SMPL-X estimation.
• Add SMPLifyx support in process_smc tool. Allow process_smc tool not writing images to file system. It runs faster in large RAM machine.
• Clarify difference between param_dict and dict(smpl_data) in SMPLData code.
• Improve visualize_smpl.py by VideoWriter. Now we can visualize a really long video on a machine with poor RAM.
• Improve import dependency, do not require h5py if there's no SMCReader instance.

Documentations:

• Add docs for changes in evaluation results.
• Update benchmark.

CICD:

• Update workflow OS from ubt18 to ubt20, since ubt18 is no longer supported by GitHub.

Bug Fixes:

• Fix the bug when setting expression from param_dict to SMPLXData.
• Remove reference of np.float which is no longer supported after numpy-1.24.0.
• Fix wrong gender type when calling SMPLData.fromfile().
• Fix error when smooth joint loss gets one-frame input.
• Fix wrong hands shape in SMPLXData.to_param_dict()
• Fix the issue that SMPLify was not running correctly with initialized betas.
• Fix wrong displacement shape in SMPLXDData.

New Contributors:

@Yamato-01

Highlights

• Add mview_mperson_end2end_estimator for learning-based method.
• Add SMPLX support and allow smpl_data initiation in mview_sperson_smpl_estimator.
• Add multiple optimizers, detailed joint weights and priors, grad clipping for better SMPLify results.
• Add mediapipe_estimator for human keypoints2d perception.

New Features

• Add mview_mperson_end2end_estimator, performing MvP estimation on customized data.
• Add mediapipe_estimator, another alternative human keypoints2d perception method like mmpose_top_down_estimator.
• Add RemoveDuplicate keypoints3d optimizer to remove duplicate MvP keypoints3d predictions.

Refactors

• Refactor mview_sperson_smpl_estimator, compatible with SMPLX.
• Refactor SMPLify, add grad clipping, joint angle priors, loss-parameter mapping, per-parameter optimizers, and body part weights.
• Refactor evaluation for learning-based methods.

Documentations

• Update download links for aliyun resources.
• Add documents for end2end estimator.
• Update tutorials for Shelf_50 demo.

CICD

• Fix linting error caused by flake8.

Bug Fixes

• Fix joint angle limits for shoulder prior.
• Fix device error for betas initiation.
• Fix file error for saving keypoints3d predicted by multiple GPUs evaluation.

Highlights

• Add 4D Association Graph, the first Python implementation to reproduce this algorithm
• Add Multi-view multi-person top-down smpl estimation
• Add reprojection error point selector

New Features

• Add 4D Association Graph, the first Python implementation to reproduce this algorithm
• Add Multi-view multi-person top-down smpl estimation
• Add structures for mview mperson kps3d/smpl estimator
• Add reprojection error point selector

Refactors

• Refactor Deformable and ProjAttn for MvP

Documentations

• Add readthedocs
• Add shape-aware 3d pose optim doc
• Update docs and tutorials for MvP training and evaluation
• Update docs and benchmark for MVPose and MVPose tracking
• Update docs for single person in getting started
• Add LICENSE note
• Add S-Lab license
• Fix outdata URL, and advices for docs

CICD

• Add some github actions for issue management
• Fix github workflow build job won't fail when pytest fails
• Remove secrets in build CI

Bug Fixes

• Fix SMPL(X/XD)Data
• Fix mistakes for mview sperson
• Fix bugs in MvP training

Highlights

• Support HuMMan Mocap toolchain for multi-view single person SMPL estimation
• Reproduce MvP, a deep-learning-based SOTA for multi-view multi-human 3D pose estimation
• Reproduce MVPose (single frame) and MVPose (temporal tracking and filtering), two optimization-based methods for multi-view multi-human 3D pose estimation
• Support SMPLify, SMPLifyX, SMPLifyD and SMPLifyXD

New Features

• Add peception module based on mmdet, mmpose and mmtrack
• Add Shape-aware 3D Pose Optimization
• Add Keypoints3d optimizer and multi-view single-person api
• Add data_converter and data_visualization for shelf, campus and cmu panoptic datasets
• Add multiple selectors to support more point selection strategies for triangulation
• Add Keypoints and Limbs data structure
• Add multi-way matching registry
• Refactor the pictorial block (c/c++) in python