POT : Python Optimal Transport
ot.da.BaseTransport
fit methods. This fix addresses the original issue introduced PR #587 (PR #593)Full Changelog: https://github.com/PythonOT/POT/compare/0.9.2...0.9.3
This new release contains several new features and bug fixes. Among the new features we have a new solver for estimation of nearest Brenier potentials (SSNB) that can be used for OT mapping estimation (on small problems), new Bregman Alternated Projected Gradient solvers for GW and FGW, and new solvers for Bures-Wasserstein barycenters. We also provide a first solver for Low Rank Sinkhorn that will be ussed to provide low rak OT extensions in the next releases. Finally we have a new exact line-search for (F)GW solvers with KL loss that can be used to improve the convergence of the solvers.
We also have a new LazyTensor
class that can be used to model OT plans and low rank tensors in large scale OT. This class is used to return the plan for the new wrapper for geomloss
Sinkhorn solver on empirical samples that can lead to x10/x100 speedups on CPU or GPU and have a lazy implementation that allows solving very large problems of a few millions samples.
We also have a new API for solving OT problems from empirical samples with ot.solve_sample
Finally we have a new API for Gromov-Wasserstein solvers with ot.solve_gromov
function that centralizes most of the (F)GW methods with unified notation. Some example of how to use the new API below:
# Generate random data
xs, xt = np.random.randn(100, 2), np.random.randn(50, 2)
# Solve OT problem with empirical samples
sol = ot.solve_sample(xs, xt) # Exact OT betwen smaples with uniform weights
sol = ot.solve_sample(xs, xt, wa, wb) # Exact OT with weights given by user
sol = ot.solve_sample(xs, xt, reg= 1, metric='euclidean') # sinkhorn with euclidean metric
sol = ot.solve_sample(xs, xt, reg= 1, method='geomloss') # faster sinkhorn solver on CPU/GPU
sol = ot.solve_sample(x,x2, method='factored', rank=10) # compute factored OT
sol = ot.solve_sample(x,x2, method='lowrank', rank=10) # compute lowrank sinkhorn OT
value_bw = ot.solve_sample(xs, xt, method='gaussian').value # Bures-Wasserstein distance
# Solve GW problem
Cs, Ct = ot.dist(xs, xs), ot.dist(xt, xt) # compute cost matrices
sol = ot.solve_gromov(Cs,Ct) # Exact GW between samples with uniform weights
# Solve FGW problem
M = ot.dist(xs, xt) # compute cost matrix
# Exact FGW between samples with uniform weights
sol = ot.solve_gromov(Cs, Ct, M, loss='KL', alpha=0.7) # FGW with KL data fitting
# recover solutions objects
P = sol.plan # OT plan
u, v = sol.potentials # dual variables
value = sol.value # OT value
# for GW and FGW
value_linear = sol.value_linear # linear part of the loss
value_quad = sol.value_quad # quadratic part of the loss
Users are encouraged to use the new API (it is much simpler) but it might still be subjects to small changes before the release of POT 1.0.
We also fixed a number of issues, the most pressing being a problem of GPU memory allocation when pytorch is installed that will not happen now thanks to Lazy initialization of the backends. We now also have the possibility to deactivate some backends using environment which prevents POT from importing them and can lead to large import speedup.
get_backend
to ignore None
inputs (PR #525)ot.da.sinkhorn_l1l2_gl
are now vectorized to improve performance (PR #507)linspace
method of the backends now has the type_as
argument to convert to the same dtype and device. (PR #533)convolutional_barycenter2d
and convolutional_barycenter2d_debiased
functions now work with different devices.. (PR #533)ot.solve_gromov
function (PR #536)gromov_wasserstein
and fused_gromov_wasserstein
with KL loss (PR #556)ot.solve_sample
for solving OT problems from empirical samples (PR #563)stop_criterion
feature to (un)regularized (f)gw barycenter solvers (PR #578)fixed_structure
and fixed_features
to entropic fgw barycenter solver (PR #578)ot.gaussian
and example (PR #582, PR #584)SinkhornL1l2Transport
now supports JAX backend (PR #587)fgw_barycenters
to init_C
and init_X
(Issue #547, PR #566)ot/bregman/
repository (Issue #567, PR #569)entropic_fused_gromov_barycenters
(Issue #574, PR #573)kl_loss
(PR #576)Full Changelog: https://github.com/PythonOT/POT/compare/0.9.1...0.9.2
This new release contains several new features and bug fixes.
New features include a new submodule ot.gnn
that contains two new Graph neural network layers (compatible with Pytorch Geometric) for template-based pooling of graphs with an example on graph classification. Related to this, we also now provide FGW and semi relaxed FGW solvers for which the resulting loss is differentiable w.r.t. the parameter alpha
. Other contributions on the (F)GW front include a new solver for the Proximal Point algorithm that can be used to solve entropic GW problems (using the parameter solver="PPA"
), new solvers for entropic FGW barycenters, novels Sinkhorn-based solvers for entropic semi-relaxed (F)GW, the possibility to provide a warm-start to the solvers, and optional marginal weights of the samples (uniform weights ar used by default). Finally we added in the submodule ot.gaussian
and ot.da
new loss and mapping estimators for the Gaussian Gromov-Wasserstein that can be used as a fast alternative to GW and estimates linear mappings between unregistered spaces that can potentially have different size (See the update linear mapping example for an illustration).
We also provide a new solver for the Entropic Wasserstein Component Analysis that is a generalization of the celebrated PCA taking into account the local neighborhood of the samples. We also now have a new solver in ot.smooth
for the sparsity-constrained OT (last plot) that can be used to find regularized OT plans with sparsity constraints. Finally we have a first multi-marginal solver for regular 1D distributions with a Monge loss (see here).
The documentation and testings have also been updated. We now have nearly 95% code coverage with the tests. The documentation has been updated and some examples have been streamlined to build more quickly and avoid timeout problems with CircleCI. We also added an optional CI on GPU for the master branch and approved PRs that can be used when a GPU runner is online.
Many other bugs and issues have been fixed and we want to thank all the contributors, old and new, who made this release possible. More details below.
ot.gnn
(PR #488)ot.smooth
and added projection_sparse_simplex
to ot.utils
(PR #459)median
method to all inherited classes of backend.Backend
(PR #472)solver="PPA"
in ot.gromov.entropic_gromov_wasserstein
+ examples (PR #455)warmstart
and kwargs
in ot.gromov.entropic_gromov_wasserstein
to respectively perform warmstart on dual potentials and pass parameters to ot.sinkhorn
(PR #455)ot.gromov.entropic_fused_gromov_wasserstein
and entropic FGW barycenters + examples (PR #455)warmstartT
and kwargs
to all CG and entropic (F)GW barycenter solvers (PR #455)ot.gromov
+ examples (PR #455)._gw
, ._bregman
and ._semirelaxed
(PR #455)cdist
(PR #487)utils.cost_normalization
function issue to work with multiple backends (PR #472)Full Changelog: https://github.com/PythonOT/POT/compare/0.9.0...0.9.1
This new release contains so many new features and bug fixes since 0.8.2 that we decided to make it a new minor release at 0.9.0.
The release contains many new features. First we did a major update of all Gromov-Wasserstein solvers that brings up to 30% gain in computation time (see PR #431) and allows the GW solvers to work on non symmetricmatrices. It also brings novel solvers for the veryefficient semi-relaxed GW problem that can be used to find the best re-weighting for one of the distributions. We also now have fast and differentiable solvers for Wasserstein on the circle and sliced Wasserstein on the sphere. We are also very happy to provide new OT barycenter solvers such as the Free support Sinkhorn barycenter and the Generalized Wasserstein barycenter. A new differentiable solver for OT across spaces that provides OT plans between samples and features simultaneously and called Co-Optimal Transport has also been implemented. Finally we began working on OT between Gaussian distributions and now provide differentiable estimation for the Bures-Wasserstein divergence and mappings.
Another important first step toward POT 1.0 is the implementation of a unified API for OT solvers with introduction of the ot.solve
function that can solve (depending on parameters) exact, regularized and unbalanced OT and return a new OTResult
object. The idea behind this new API is to facilitate exploring different solvers with just a change of parameter and get a more unified API for them. We will keep the old solvers API for power users but it will be the preferred way to solve problems starting from release 1.0.0. We provide below some examples of use for the new function and how to recover different aspects of the solution (OT plan, full loss, linear part of the loss, dual variables) :
#Solve exact ot
sol = ot.solve(M)
# get the results
G = sol.plan # OT plan
ot_loss = sol.value # OT value (full loss for regularized and unbalanced)
ot_loss_linear = sol.value_linear # OT value for linear term np.sum(sol.plan*M)
alpha, beta = sol.potentials # dual potentials
# direct plan and loss computation
G = ot.solve(M).plan
ot_loss = ot.solve(M).value
# OT exact with marginals a/b
sol2 = ot.solve(M, a, b)
# regularized and unbalanced OT
sol_rkl = ot.solve(M, a, b, reg=1) # KL regularization
sol_rl2 = ot.solve(M, a, b, reg=1, reg_type='L2')
sol_ul2 = ot.solve(M, a, b, unbalanced=10, unbalanced_type='L2')
sol_rkl_ukl = ot.solve(M, a, b, reg=10, unbalanced=10) # KL + KL
The function is fully compatible with backends and will be implemented for different types of distribution support (empirical distributions, grids) and OT problems (Gromov-Wasserstein) in the new releases. This new API is not yet presented in the kickstart part of the documentation as there is a small change that it might change when implementing new solvers but we encourage users to play with it.
Finally, in addition to those many new this release fixes 20 issues (some long standing) and we want to thank all the contributors who made this release so big. More details below.
ot.optim
to support relative and absolute loss variations as stopping criterions (PR #431)ot.gromov
+ examples (PR #431)ot.sliced.sliced_wasserstein_sphere
and ot.sliced.sliced_wasserstein_sphere_unif
+ examples (PR #434)ot.lp.solver_1d.wasserstein_circle
(PR #434)ot.lp.solver_1d.binary_search_circle
+ examples (PR #434)ot.lp.solver_1d.semidiscrete_wasserstein2_unif_circle
(PR #434)ot.gaussian
(PR ##428)ot.solve
(PR #388)ot.partial
and ot.smooth
(PR #388 and #449)ot.bregman
(PR #437)ot.da.SinkhornTransport
(PR #440)ot.dr
now uses the new Pymanopt API and POT is compatible with current
Pymanopt (PR #443)nx.abs()
at the end of wasserstein_1d()
(PR #448)log=True
for SinkhornLpl1Transport
(Issue #412,
PR #413)warn
parameter in sinkhorn2
(PR #417)stopThr
in empirical_sinkhorn_divergence
was rendered useless by subcalls
that explicitly specified stopThr=1e-9
(Issue #421, PR #422).line_search_armijo
and entropic_gromov_wasserstein
(Issue #445, #PR 446)Full Changelog: https://github.com/PythonOT/POT/compare/0.8.2...0.9.0
This releases introduces several new notable features. The less important but most exiting one being that we now have a logo for the toolbox (color and dark background) :
This logo is generated using with matplotlib and using the solution of an OT problem provided by POT (with ot.emd
). Generating the logo can be done with a simple python script also provided in the documentation gallery.
New OT solvers include Weak OT and OT with factored coupling that can be used on large datasets. The Majorization Minimization solvers for non-regularized Unbalanced OT are now also available. We also now provide an implementation of GW and FGW unmixing and dictionary learning. It is now possible to use autodiff to solve entropic an quadratic regularized OT in the dual for full or stochastic optimization thanks to the new functions to compute the dual loss for entropic and quadratic regularized OT and reconstruct the OT plan on part or all of the data. They can be used for instance to solve OT problems with stochastic gradient or for estimating the dual potentials as neural networks.
On the backend front, we now have backend compatible functions and classes in the domain adaptation ot.da
and unbalanced OT ot.unbalanced
modules. This means that the DA classes can be used on tensors from all compatible backends. The free support Wasserstein barycenter solver is now also backend compatible.
Finally we have worked on the documentation to provide an update of existing examples in the gallery and and several new examples including GW dictionary learning and weak Optimal Transport.
ot.gpu
submodule (PR #361)minigallery
(PR #334)ot.lp.free_support_barycenter
(PR #340)ot.emd2
and ot.gromov_wasserstein2
so that they are
centered (Issue #364, PR #363)autograd
function ValFunction
(Issue #337,
PR #338)This release fixes several bugs and introduces two new backends: Cupy and Tensorflow. Note that the tensorflow backend will work only when tensorflow has enabled the Numpy behavior (for transpose that is not by default in tensorflow). We also introduce a simple benchmark on CPU GPU for the sinkhorn solver that will be provided in the backend documentation.
This release also brings a few changes in dependencies and compatibility. First we removed tests for Python 3.6 that will not be updated in the future. Also note that POT now depends on Numpy (>= 1.20) because a recent change in ABI is making the wheels non-compatible with older numpy versions. If you really need an older numpy POT will work with no problems but you will need to build it from source.
As always we want to that the contributors who helped make POT better (and bug free).
ot.dist
function when non euclidean distance (Issue #305, PR #306)nx.set_gradients
(Issue #309, PR #310)gromov_barycenters
(Issue #317, PR #3018)This new stable release introduces several important features.
First we now have an OpenMP compatible exact ot solver in ot.emd
. The OpenMP version is used when the parameter numThreads
is greater than one and can lead to nice speedups on multi-core machines.
Second we have introduced a backend mechanism that allows to use standard POT function seamlessly on Numpy, Pytorch and Jax arrays. Other backends are coming but right now POT can be used seamlessly for training neural networks in Pytorch. Notably we propose the first differentiable computation of the exact OT loss with ot.emd2
(can be differentiated w.r.t. both cost matrix and sample weights), but also for the classical Sinkhorn loss with ot.sinkhorn2
, the Wasserstein distance in 1D with ot.wasserstein_1d
, sliced Wasserstein with ot.sliced_wasserstein_distance
and Gromov-Wasserstein with ot.gromov_wasserstein2
. Examples of how this new feature can be used are now available in the documentation where the Pytorch backend is used to estimate a minimal Wasserstein estimator, a Generative Network (GAN), for a sliced Wasserstein gradient flow and optimizing the Gromov-Wassersein distance. Note that the Jax backend is still in early development and quite slow at the moment, we strongly recommend for Jax users to use the OTT toolbox when possible. As a result of this new feature, the old ot.gpu
submodule is now deprecated since GPU implementations can be done using GPU arrays on the torch backends.
Other novel features include implementation for Sampled Gromov Wasserstein and Pointwise Gromov Wasserstein, Sinkhorn in log space with method='sinkhorn_log'
, Projection Robust Wasserstein, and deviased Sinkorn barycenters.
This release will also simplify the installation process. We have now a pyproject.toml
that defines the build dependency and POT should now build even when cython is not installed yet. Also we now provide pe-compiled wheels for linux aarch64
that is used on Raspberry PI and android phones and for MacOS on ARM processors.
Finally POT was accepted for publication in the Journal of Machine Learning Research (JMLR) open source software track and we ask the POT users to cite this paper from now on. The documentation has been improved in particular by adding a "Why OT?" section to the quick start guide and several new examples illustrating the new features. The documentation now has two version : the stable version https://pythonot.github.io/ corresponding to the last release and the master version https://pythonot.github.io/master that corresponds to the current master branch on GitHub.
As usual, we want to thank all the POT contributors (now 37 people have contributed to the toolbox). But for this release we thank in particular Nathan Cassereau and Kamel Guerda from the AI support team at IDRIS for their support to the development of the backend and OpenMP implementations.
ot.bregman
(PR #280)ot.optim
(PR #282)ot.gromov
(PR #294, PR #302)method='sinkhorn_log'
(PR #290)pyproject.toml
and build POT without installing cython first (PR #293)emd_1d
, non respected bounds (Issue #169, PR #170)flake8
(PR #176)setup.py
(PR #174)This is the new stable release for POT. We made a lot of changes in the documentation and added several new features such as Partial OT, Unbalanced and Multi Sources OT Domain Adaptation and several bug fixes. One important change is that we have created the GitHub organization PythonOT that now owns the main POT repository https://github.com/PythonOT/POT and the repository for the new documentation is now hosted at https://PythonOT.github.io/.
This is the first release where the Python 2.7 tests have been removed. Most of the toolbox should still work but we do not offer support for Python 2.7 and will close related Issues.
A lot of changes have been done to the documentation that is now hosted on https://PythonOT.github.io/ instead of readthedocs. It was a hard choice but readthedocs did not allow us to run sphinx-gallery to update our beautiful examples and it was a huge amount of work to maintain. The documentation is now automatically compiled and updated on merge. We also removed the notebooks from the repository for space reason and also because they are all available in the example gallery. Note that now the output of the documentation build for each commit in the PR is available to check that the doc builds correctly before merging which was not possible with readthedocs.
The CI framework has also been changed with a move from Travis to Github Action which allows to get faster tests on Windows, MacOS and Linux. We also now report our coverage on Codecov.io and we have a reasonable 92% coverage. We also now generate wheels for a number of OS and Python versions at each merge in the master branch. They are available as outputs of this action. This will allow simpler multi-platform releases from now on.
In terms of new features we now have OTDA Classes for unbalanced OT, a new Domain adaptation class form multi domain problems (JCPOT), and several solvers to solve the Partial Optimal Transport problems.
This release is also the moment to thank all the POT contributors (old and new) for helping making POT such a nice toolbox. A lot of changes (also in the API) are comming for the next versions.
This is a beta test pre release for the new version of POT.
Do not use in production.