Python library for analysing faces using PyTorch
Hugging Face Space demo app 🤗
User Guide, Documentation, ChatGPT facetorch guide
Facetorch is a Python library that can detect faces and analyze facial features using deep neural networks. The goal is to gather open sourced face analysis models from the community, optimize them for performance using TorchScript and combine them to create a face analysis tool that one can:
Please, use the library responsibly with caution and follow the ethics guidelines for Trustworthy AI from European Commission. The models are not perfect and may be biased.
pip install facetorch
conda install -c conda-forge facetorch
Docker Compose provides an easy way of building a working facetorch environment with a single command.
docker compose run facetorch python ./scripts/example.py
docker compose run facetorch-gpu python ./scripts/example.py analyzer.device=cuda
Check data/output for resulting images with bounding boxes and facial 3D landmarks.
(Apple Mac M1) Use Rosetta 2 emulator in Docker Desktop to run the CPU version.
The project is configured by files located in conf with the main file: conf/config.yaml. One can easily add or remove modules from the configuration.
FaceAnalyzer is the main class of facetorch as it is the orchestrator responsible for initializing and running the following components:
analyzer
├── reader
├── detector
├── unifier
└── predictor
├── embed
├── verify
├── fer
├── au
├── va
├── deepfake
└── align
└── utilizer
├── align
├── draw
└── save
| model | source | params | license | version |
| ------------- | --------- | --------- | ----------- | ------- |
| RetinaFace | biubug6 | 27.3M | MIT license | 1 |
| model | source | params | license | version |
| ----------------- | ---------- | ------- | ----------- | ------- |
| ResNet-50 VGG 1M | 1adrianb | 28.4M | MIT license | 1 |
include_tensors
needs to be True in order to include the model prediction in Prediction.logits| model | source | params | license | version |
| ---------------- | ----------- | -------- | ------------------ | ------- |
| MagFace+UNPG | Jung-Jun-Uk | 65.2M | Apache License 2.0 | 1 |
| AdaFaceR100W12M | mk-minchul | - | MIT License | 2 |
include_tensors
needs to be True in order to include the model prediction in Prediction.logitsinclude_tensors
needs to be True in order to include the model prediction in Prediction.logits| model | source | params | license | version |
| ----------------- | -------------- | -------- | ------------------ | ------- |
| EfficientNet B0 7 | HSE-asavchenko | 4M | Apache License 2.0 | 1 |
| EfficientNet B2 8 | HSE-asavchenko | 7.7M | Apache License 2.0 | 2 |
| model | source | params | license | version |
| ------------------- | --------- | ------- | ------------------ | ------- |
| OpenGraph Swin Base | CVI-SZU | 94M | MIT License | 1 |
| model | source | params | license | version |
| ----------------- | ---------- | ------- | ----------- | ------- |
| ELIM AL AlexNet | kdhht2334 | 2.3M | MIT license | 1 |
| model | source | params | license | version |
| -------------------- | ---------------- | -------- | ----------- | ------- |
| EfficientNet B7 | selimsef | 66.4M | MIT license | 1 |
| model | source | params | license | version |
| ----------------- | ---------------- | -------- | ----------- | ------- |
| MobileNet v2 | choyingw | 4.1M | MIT license | 1 |
include_tensors
needs to be True in order to include the model prediction in Prediction.logitsModels are downloaded during runtime automatically to the models directory. You can also download the models manually from a public Google Drive folder.
Image test.jpg (4 faces) is analyzed (including drawing boxes and landmarks, but not saving) in about 486ms and test3.jpg (25 faces) in about 1845ms (batch_size=8) on NVIDIA Tesla T4 GPU once the default configuration (conf/config.yaml) of models is initialized and pre heated to the initial image size 1080x1080 by the first run. One can monitor the execution times in logs using the DEBUG level.
Detailed test.jpg execution times:
analyzer
├── reader: 27 ms
├── detector: 193 ms
├── unifier: 1 ms
└── predictor
├── embed: 8 ms
├── verify: 58 ms
├── fer: 28 ms
├── au: 57 ms
├── va: 1 ms
├── deepfake: 117 ms
└── align: 5 ms
└── utilizer
├── align: 8 ms
├── draw_boxes: 22 ms
├── draw_landmarks: 7 ms
└── save: 298 ms
Run the Docker container:
docker compose -f docker-compose.dev.yml run facetorch-dev
docker compose -f docker-compose.dev.yml run facetorch-dev-gpu
Facetorch works with models that were exported from PyTorch to TorchScript. You can apply torch.jit.trace function to compile a PyTorch model as a TorchScript module. Please verify that the output of the traced model equals the output of the original model.
The first models are hosted on my public Google Drive folder. You can either send the new model for upload to me, host the model on your Google Drive or host it somewhere else and add your own downloader object to the codebase.
/conf/analyzer/predictor/
following the FER example in /conf/analyzer/predictor/fer/
/conf/analyzer/predictor/fer/efficientnet_b2_8.yaml
to the new folder
/conf/analyzer/predictor/<predictor_name>/
/conf/analyzer/predictor/<predictor_name>/<model_name>.yaml
/tests/conftest.py
file./tests/test_<predictor_name>.py
black facetorch
CPU:
environment.yml
fileconda lock -p linux-64 -f environment.yml --lockfile conda-lock.yml
docker compose -f docker-compose.dev.yml run facetorch-lock
conda-lock install --name env conda-lock.yml
GPU:
gpu.environment.yml
fileconda lock -p linux-64 -f gpu.environment.yml --lockfile gpu.conda-lock.yml
docker compose -f docker-compose.dev.yml run facetorch-lock-gpu
conda-lock install --name env gpu.conda-lock.yml
pytest tests --verbose --cov-report html:coverage --cov facetorch
pdoc --html facetorch --output-dir docs --force --template-dir pdoc/templates/
python -m cProfile -o profiling/example.prof scripts/example.py
snakeviz profiling/example.prof
I want to thank the open source code community and the researchers who have published the models. This project would not be possible without their work.
Logo was generated using DeepAI Text To Image API