Fast_Stacked_Hourglass_Network_OpenVino

A fast stacked hourglass network for human pose estimation on OpenVino. Stacked hourglass network proposed by Stacked Hourglass Networks for Human Pose Estimation is a very good network for single-person pose estimation regarding to speed and accuracy. This repo contains a demo to show how to depoly model trained by Keras. It converts a Keras model to IR and shows how to use the generated IR to do inference. Have fun with OpenVino!

Installation

• Python3
• Install OpenVino 2018 R5
• Install python dependencies

keras==2.1.5
scipy==1.2.0
tensorflow==1.12.0
opencv-python==3.4.3.18

[Keras] Convert pre-trained Keras models

Download pre-trained hourglass models

• Download models from Google drive and save them to models. You are going to download two files, one is json file for network configuration while another is weight.
• hg_s2_b1_mobile, inputs: 256x256x3, Channel Number: 256, pckh 78.86% @MPII.
• hg_s2_b1_tiny, inputs:192x192x3, Channel Number: 128, [email protected]%MPII.

Convert keras models to tensorflow forzen pb

• Convert keras models to tf frozen pb

python3 tools/keras_to_tfpb.py --input_model_json ./models/path/to/network/json --input_model_weights
./models/path/to/network/weight/h5 --out_tfpb ./models/hg_s2_b1_tf.pb

Use OpenVino Model Optimizer to convert tf pb to IR.

• For CPU, please use mobile version hg_s2_b1_mobile and FP32

~/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo_tf.py -w ./models/hg_s2_b1_tf.pb --input_shape [1,256,256,3] --data_type FP32 --output_dir ./models/ --model_name hg_s2_mobile

• For NCS2(Myriad), please use tiny version hg_s2_b1_tiny and FP16

~/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo_tf.py -w ./models/hg_s2_b1_tf.pb --input_shape [1,192,128,3] --data_type FP16 --output_dir ./models/ --model_name hg_s2_tiny

• .xml and .bin will be generated.

[PyTorch] Convert pre-trained Onnx models

Download model trained by pytorch

Download the model_best.onnx model from below table to fit your accuracy and speed requirements. hg_s2_b1_mobile_fpd model trained by using the knowledge distillation proposed by paper Fast Human Pose Estimation. Details can be found in Fast_Human_Pose_Estimation_Pytorch.

Convert onnx to IR

Use model optimizer to convert onnx to IR. FP32 for CPU while FP16 for MYRIAD

~/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo.py -w ./models/model_best.onnx --data_type FP32 --output_dir ./models/ --model_name hg_s2_mobile_onnx 

Run demo

• Run single image demo on CPU

cd src
python3 stacked_hourglass.py -i ../models/sample.jpg -m ../models/hg_s2_mobile.xml -d CPU -l /path/to/cpu/extension/library

• Run single image demo on NCS2(MYRIAD)

cd src
python3 stacked_hourglass.py -i ../models/sample.jpg -m ../models/hg_s2_tiny.xml -d MYRIAD

• Run Aysnc demo with camera input on CPU

cd src
python3 stacked_hourglass_camera_async.py -i cam -m ../models/hg_s2_mobile.xml -d CPU -l /path/to/cpu/extension/library

Reference

• OpenVino: https://github.com/opencv/dldt
• OpenCV OpenModelZoo: https://github.com/opencv/open_model_zoo
• Keras implementation for stacked hourglass: https://github.com/yuanyuanli85/Stacked_Hourglass_Network_Keras
• Pytorch-pose: https://github.com/yuanyuanli85/pytorch-pose
• Fast Human Pose Estimation https://github.com/yuanyuanli85/Fast_Human_Pose_Estimation_Pytorch