Person Recognition System on PIPA dataset
The system is develped to perform person recognition task on PIPA dataset, the detailed description of the approach of this system can be found here.
Config | Accuracy |
---|---|
Face | 62.18% |
Head | 63.19% |
Upper-body | 67.44% |
Full-body | 58.96% |
All modality fused | 82.31% |
All modality fused + MRF refining | 86.18% |
get dataset
/scripts/get_PIPA.sh
get models
/scripts/get_models.sh
get features (if you don't wish to extract them)
/scripts/get_features.sh
We use FaceNet for face feature extraction. FaceNet is a CNN trained to directly optimize the embedding itself.
test face feature extractor
ython pyHumanRecog/face_feature_extractor_test.py
train head feature extractor (feel free to experiment with different batch size)
ython pyHumanRecog/head_feature_extractor_train.py --batch_size 32
test head feature extractor
ython pyHumanRecog/head_feature_extractor_test.py --batch_size 32
train body feature extractor (feel free to experiment with different batch size)
ython pyHumanRecog/body_feature_extractor_train.py --batch_size 32
test body feature extractor
ython pyHumanRecog/body_feature_extractor_test.py --batch_size 32
train upper-body feature extractor (feel free to experiment with different batch size)
ython pyHumanRecog/upper_body_feature_extractor_train.py --batch_size 32
test upper-body feature extractor
ython pyHumanRecog/upper_body_feature_extractor_test.py --batch_size 32
We use CPM for pose estimation. The estimated CPM pose will mainly be used for image warping.
CPM pose estimation
ython pyHumanRecog/extract_pose.py <img_dump_folder> <pose_dump_folder>
<image_dump_folder>
: folder to dump CPM pose visualization images
<pose_dump_folder>
: folder to dump CPM pose positions
For performance evaluation, Please first modify performance_test_config.py
(within pyHumanRecog
folder) to specify the features you wish to use and their corresponding weights. Then execute the following command.
python pyHumanRecog/performance_test.py
To perform MRF optimization (which incorporates the photo-level cooccurrence and mutual exclusive pattern into the final prediction), Set refine_with_photo_level_context = True
in HumanRecog/performance_test_config.py
.