The system is develped to perform person recognition task on PIPA dataset, the detailed description of the approach of this system can be found here.
|All modality fused||82.31%|
|All modality fused + MRF refining||86.18%|
get features (if you don't wish to extract them)
We use FaceNet for face feature extraction. FaceNet is a CNN trained to directly optimize the embedding itself.
test face feature extractor
train head feature extractor (feel free to experiment with different batch size)
ython pyHumanRecog/head_feature_extractor_train.py --batch_size 32
test head feature extractor
ython pyHumanRecog/head_feature_extractor_test.py --batch_size 32
train body feature extractor (feel free to experiment with different batch size)
ython pyHumanRecog/body_feature_extractor_train.py --batch_size 32
test body feature extractor
ython pyHumanRecog/body_feature_extractor_test.py --batch_size 32
train upper-body feature extractor (feel free to experiment with different batch size)
ython pyHumanRecog/upper_body_feature_extractor_train.py --batch_size 32
test upper-body feature extractor
ython pyHumanRecog/upper_body_feature_extractor_test.py --batch_size 32
We use CPM for pose estimation. The estimated CPM pose will mainly be used for image warping.
CPM pose estimation
ython pyHumanRecog/extract_pose.py <img_dump_folder> <pose_dump_folder>
<image_dump_folder>: folder to dump CPM pose visualization images
<pose_dump_folder>: folder to dump CPM pose positions
For performance evaluation, Please first modify
pyHumanRecog folder) to specify the features you wish to use and their corresponding weights. Then execute the following command.
To perform MRF optimization (which incorporates the photo-level cooccurrence and mutual exclusive pattern into the final prediction), Set
refine_with_photo_level_context = True in