A lightweight 3D Morphable Face Model library in modern C++
This is a release with quite a few nice changes and improvements. Please read through the release notes carefully.
This release was in the devel branch for a few months, and never actually left the devel branch. The MorphableModel format version was increased to 3, from adding landmark_definitions. Since there are quite a few model files with this version in circulation, it was deemed best to make it an "official" model version and release it as v0.17.0.
Major changes & improvements:
"eye.right.outer_corner" <=> 177 from a generic name to vertex ids - and then a second mappings file can be used, that maps from 2D landmark identifiers (like the number 37 from ibug) to these generic names ("eye.right.outer_corner"). This separates the vertex definition from a particular set of 2D landmarks.
Alternatively, users can still use direct mappings, like in the file ibug_to_sfm.txt, which map directly from 2D landmark identifiers to 3D vertex indices.MorphableModel for landmark_definitions (e995b07821038567284bf02202feb6021289d1e0)vertex_idx in the fitting depending on whether the model contains landmark definitions (f123e518c3a85bb5deaf9d17e7c11931ede9225a)Image class to a better and more future-proof design (14b7f6f6d998bb30de3f2047fa7f6af374786f3c)
Note: The storage order is now row-major, as opposed to col-major before. In the future, we may add means to choose the storage order.eos::core::Image (be436f4d99406cfb62addabcc45303a7aa780a66)Fixes:
mpark::variant compilation error on VS >=15.8; Updated pybind11 submodule to fix errors on VS 15.8 (a45a888bbf29b251f2bd472d13a06208387e6a8b, ecc626946da2cbea42623759ff09dec6ef9eba34)Minor improvements, potentially breaking changes:
get_all_mappings() to get_mappings() (9e0f539652e28a8b461bde9a53a202c471b04677)eos/core/image/ and moved Image_opencv_interop.hpp there (ad06db1b9e85319e8bddbca67b52f248b4dda331)triangle_list by const reference (a7ede7bfca5cc1929b43d629dc90db7c5c3ae14d)__repr__ methods.initial_rendering_params parameter from fit_shape_and_pose (ba46285497485c1ab8f0f80722e8c09eb8b26e43)Other minor changes/improvements:
read_obj a bit to handle a few more cases. Still, don't rely on it to parse every .obj file, it's not meant to do that. Use your own or assimp or something like that.And as always, various small improvements to the code style and to the documentation.
A few minor updates and fixes:
Fixes:
draw_sample (a6900a2)R instead of 3x3 in the Python bindings of ScaledOrthoProjectionParameters (5863f3b)Updates:
render::Rect to core namespaceeos::core::Rect and eos::core::Landmark<Eigen::Vector2f>. The Python bindings are now using Landmark and LandmarkCollection natively, without having to keep a separate list of landmark identifiers.long_description for PyPI (link)This is a major release, bringing full support for expression PCA models. MorphableModels can now either contain no expression model, a set of blendshapes, or a PCA expression model.
The fit_shape_and_pose supports both as well, and calls either the _linear or _nnls solver, depending on whether a model contains blendshapes or an expression PCA model.
The main changes that go along with that:
MorphableModel file format to version to 2 to include an optional expression model. Version 1 models can still be read.ExpressionModelType to MorphableModel to easily query the type of the expression model.Note: The behaviour of MorphableModel::get_mean() has been changed: It now returns the overall shape and expression mean. (see #188)
Other small additions:
As always, when updating from the GitHub repository, make sure to run git submodule update --init --recursive.
The binary files are provided for convenience only - it is recommended to build the examples from source, and use the library by cloning the GitHub repository.
Windows binaries are compiled with Windows 10 64bit, Visual Studio 2017. A .whl with python bindings for python 3.6 is available on PyPI.
Linux binaries are compiled with Linux Mint 18.3, based on Ubuntu 16.04, gcc-7. Make sure to follow the instructions for how to use gcc-7 with pip if using the Python bindings on Linux.
This release improves a few crucial things over the last release.
Support for pre-C++17 compilers should be a lot better again - eos compiles again in C++14 mode on VS 2017, and, although not officially tested in CI, likely also compiles again with the likes of g++-5:
akrzemi1::optional on all pre-C++17 compilers, not only on Apple platforms.optional also works correctly on all platforms (pre-C++17 and C++17 compilers); and updated cereal to the latest develop branch commit.optional now also works within the python bindings for pre-C++17 and C++17 compilers.eos::cpp17::clamp, which uses std::clamp on C++17 compilers, or a fallback otherwise.Matlab bindings:
eos-matlab-fitting bindings! They now work again in the latest version to use the fitting from Matlab.eos-matlab-render bindings for now, as they would be more work to update. Having the fitting bindings back should suffice for most use cases.Other small things:
consts, a couple documentation updates.eos is now fully working on Apple Xcode again and CI-tested with Xcode 9.2.
Main changes:
eos::cpp17::optional throughout the code base, to support platforms that do not have <optional> yet (currently only macOS - #ifdef __APPLE__). It is a namespace alias, and will be resolved to std::optional on compilers that have <optional>.eigen-git-mirror as submodule at tag 3.3.4, instead of the patrikhuber/eigen forkKeyframe-related code to work again with the latest eos versionThis is an update release with various small improvements throughout and two highlights:
One major highlight is proper (and tested) support for the BFM2009 (and partially the BFM2017).
Another highlight is that the multi_image_fit_devel branch is finally merged into master, and multi-frame fitting is thus now available in the main version.
The changes include, alongside other minor changes:
multi_image_fit_devel branch. Thank you very much @PhilippKopp for the contribution!sample_to_mesh
find_if result for std::end in get_nearest_contour_correspondences to avoid continuing in Release builds without any error messageconst qualifiers where possibleMesh and Blendshape)This release contains some major changes, mainly to the software-side of the library, and not to the algorithms.
All in all, these changes make using the library much easier, and they should especially enable installation of the python bindings via pip install eos-py on any system, given >=cmake-3.8.2 and a recent compiler.
The changes, alongside other minor changes, include:
Eigen::Vector
cv::Mat (eos/core/Image_opencv_interop.hpp))This is a minor release, mainly adding functionality to build PCA models (include/eos/pca/pca.hpp).
This will be the last version that supports gcc-5, clang-3.x, and VS2015. The next release will require >=gcc-7, >=clang-5 and >=VS2017U3.
Other minor changes:
const, improved documentation, updates of submodulesThis update brings a major speed-up of the shape and blendshape fitting. Depending on the setup (model resolution used, number of coefficients fitted), the speed-up is around 3x to 50x - typically around 10x. The shape and blendshape fitting now take only around or below 1ms per iteration (previously ~10ms). This was achieved by changing all linear algebra operations from OpenCV to Eigen.
Please see also the release notes of v0.12.0 below for the latest changes and updates, some of them breaking changes. See also the release notes of v0.12.0 for information on the PyPi package (eos-py) and the provided binaries.
This is mainly a maintenance release but contains quite a number of improvements. There are some breaking changes in this release and models have to be re-downloaded in the new format.
Major changes:
PcaModel, MorphableModel and blendshapes to use Eigen for matrices and vectors instead of OpenCV's cv::Mat. This change was motivated by the fact that we need Eigen already anyway for solving linear systems. And since Eigen is a header-only library, the core of eos becomes more independent of OpenCV and easier to compile in the near future.
This change means that the model format has changed as well, so users will have to re-download the models and blendshapes in the new model format.
PcaModel constructor to take an orthonormal basis (4662c59c4d9ae233047d1b5130470db1ed6bf894)
This introduces a breaking change: If you're currently using the PcaModel constructor and are passing it a PCA basis matrix, your code will (probably silently) break: The function now expects an orthonormal basis matrix, whereas before, it expected a rescaled basis matrix. However it makes more sense for the function to take an orthonormal basis matrix, as this is usually the result after computing a PCA on data.Minor changes:
get_*_pca_basis(int) to get_*_pca_basis_at_point(int) (354deb3e962dce0e7dab7b20f1d4c5886a06e103)pip install ./eos, and packages can be created with python setup.py sdist and python setup.py bdist_wheel. A setup.cfg.template exists to set library paths to OpenCV and Boost.sdist package. An external Eigen can still be used by setting EIGEN3_INCLUDE_DIR.fit-model-ceres
eos python bindings are now available on PyPI! There's a wheel for python 3.5 on Windows, and a source dist. The source dist needs CMake's find_package to be able to find OpenCV and Boost automatically for it to work (so install it beforehand from a package manager).
If there are issues, you can always clone the repo and do pip install ./eos locally, after adjusting the paths in setup.cfg.
As always, make sure to clone the repository with git clone --recursive, or, if you're updating, run git submodule update --init --recursive.
Windows binaries are compiled with Windows 8.1 64bit, Visual Studio 2015. A .whl with python bindings for python 3.5 is available on PyPI. The Matlab bindings are created for Matlab R2016b. Linux binaries are compiled with Linux Mint 18, based on Ubuntu 16.04, gcc-5.4.