Pyro Versions Save

This patch release merely

• Pins to requirements to torch<1.8 to avoid breaking changes in torch 1.8.0 (introduced in pytorch/pytorch#50547 pytorch/pytorch#50581).
• Fixes an fft bug #2731

New features

• #2693 A GumbelSoftmaxReparam for relaxed categorical distributions

Bug fixes

• #2683 Support PyTorch 1.7
• #2682 Fix help(MyDistribution)
• #2679 Fix TraceEnum_ELBO.compute_marginals()
• #2677 Warn if infer_discrete() finds no discrete sites

New features

• New Tutorials:
  • Deep Generative Modeling with Single Cell Data
  • Conditional Variational Autoencoder
  • Forecasting with Dynamic Linear Models
• New distributions and transforms:
  • OrderedLogistic distribution and OrderedTransform
  • ConditionalMatrixExponential normalizing flow
  • ConditionalSplineAutoregressive normalizing flow
• contrib.forecast now supports HaarReparam, DiscreteCosineReparam, and poutine.trace to record posterior samples of latent variables.
• CompartmentalModel now supports a .finalize() method to add likelihoods that couple states across time.
• Integration with Funsor, an experimental intermediate language for probabilistic programming
  • pyro.contrib.funsor is a new backend for Pyro that aims to simplify the implementations of Pyro's most powerful inference engines. For details, see: tutorial 1, tutorial 2, example usage with pyroapi
  • poutine.collapse and pyro.barrier provide experimental support for collapsing conjugate fragments within existing inference algorithms, using Funsor under the hood.

Breaking changes

• Require PyTorch 1.6.
• Drop support for Python 3.5; require Python 3.6+.
• Zero inflated distributions changed interface. #2643

Bug fixes & performance tweaks

• pyro.factor statements are now allowed in guides without warning. #2664
• Fix model-directed subsampling in autoguides. #2638
• Fix sample shape bug in LKJCorrCholesky distribution. #2617
• Speed up log-matmul-exp operations in discrete enumeration and DiscreteHMM. #2640
• Fix potential_fn issues in MCMC. #2591

New features

• A new pyro.contrib.epidemiology module for discrete-state discrete-time stochastic compartmental models. #2426
• New tutorials on:
  • Normalizing flows
  • Epidemiology
  • PyroModule
  • MAP/MLE inference
• New transforms and normalizing flows:
  • Monotonic rational quadratic Spline flow.
  • A SplineCoupling flow.
  • A SplineAutogregressive flow.
  • A MatrixExponential flow.
  • Conditional versions of AffineAutoregressive, Householder, NeuralAutogregressive, Spline, and GeneralizedChannelPermute flows.
  • A Haar wavelet transform and reparameterizer.
  • Permute and AffineCoupling can operate on a specific dimension with dim keyword argument #2472
• New distributions:
  • CoalescentTimes, CoalescentTimesWithRate, and CoalescentRateLikelihood for coalescent processes in phylogenetics.
  • TruncatedPolyGamma.
  • ExtendedBinomial and ExtendedBetaBinomial with relaxed support.
  • ImproperUniform for expressing factor graphs.
• Improvements to MCMC:
  • structured mass matrix adaptation #2425 #2473
  • arrowhead precision matrix adaptation #2465
  • support for randomized init strategies like init_to_generated #2417
• Improvements to SMC:
  • systematic resampling #2488
  • resampling based on effective sample size (ESS) #2486
• A new SplitReparam allows different inference strategies to apply to different parts of a tensor random variable.
• A new initialization strategy init_to_generated.
• A RandomVariable container class to support method chaining syntax for transforming distributions #2448

Bug fixes

• Support sequential plates in RenyiELBO #2541
• Fixes to AffineAutogregressive #2504
• Fixes to BatchNorm TransformModule #2459
• Fixes to how some transforms handle parameters #2544
• Fixes to reraising logic that clean up error reporting during inference #2494
• many other fixes to documentation and code

New features

• A new Spline transform which implements element-wise rational spline bijections of linear order.
• A new ConditionalAffineCoupling transform which implements the affine coupling layer of RealNVP that conditions on an additional context variable.

Enhancements to the pyro.contrib.forecast module

• Support drawing samples in batches.
• Add walltime to backtest to measure performance of model training and forecasting.
• Support more likelihood distributions: Geometric, NegativeBinomial, ZeroInflatedNegativeBinomial.

Bug fixes

• #2399 raises an error when HMC/NUTS is used for a model with subsampling.
• #2390 makes PyroModule compatible with torch.nn.RNN.
• #2388 allows unused params in CSIS inference.
• #2384 fixes some caching issues in calculation of log_abs_det_jacobian of TransformModules
• #2365 fixes a naming bug in LocScaleReparam whereby all loc-scale reparameterized sites shared a single centeredness parameter.
• #2355 makes jit_compile=True flag in HMC/NUTS work for models with pyro.param statements.

New features

• A new AutoNormal guide that supports data subsampling, thanks to @patrickeganfoley.
• Data subsampling support for the AutoDelta guide.
• A new pyro.subsample primitive to aide in subsampling.
• An AutoNormalizingFlow autoguide.
• A new pyro.contrib.forecast module for multivariate hierarchical heavy-tailed forecasting, together with three tutorials: uivariate, heavy-tailed, state space models, and hierarchical models.
• A tutorial on Boosting Black Box Variational Inference, thanks to @lorenzkuhn, @gideonite, @sharrison5, and @TNU-yaoy.
• A NeuTraReparam example.
• PyroModule's interface is now stable, no longer EXPERIMENTAL.
• Better validation for GaussianHMM and LinearHMM distributions.
• Miscellaneous new GaussianHMM method to handle conjugacy.

Bug fixes

• #2345 remove pillow-simd dependency
• #2327 Make pyro.deterministic not warn when called outside of inference
• #2321 Support plates in RenyiELBO
• #2266 Fixes to transform handling in MCMC api

Patches 1.2.0 with the following bug fixes:

• Fix for MCMC with parallel chains using multiprocessing, where transforms to the latent sites' support was not being correctly stored.
• Other minor rendering related fixes for tutorials.

Misc changes

• Updated to PyTorch 1.4.0 and torchvision 0.5.0.
• Changed license from MIT to Apache 2.0 and removed Uber CLA as part of Pyro's move to the Linux foundation.

Reparameterization

This release adds a new effect handler and a collection of strategies that reparameterize models to improve geometry. These tools are largely orthogonal to other inference tools in Pyro, and can be used with SVI, MCMC, and other inference algorithms.

• poutine.reparam() is a new effect handler that transforms models into other models for which inference may be easier (Gorinova et al. 2019).
• pyro.infer.reparam is a collection of reparameterization strategies following a standard Reparam interface:
  • Decentering transforms for location-scale families (Gorinova et al. 2019).
  • Transform unwrapping to deconstruct TransformedDistributions.
  • Discrete Cosine transforms for frequency-domain parameterizations (useful for inference in time series).
  • Auxiliary variable methods for Levy Stable and StudentT distributions.
  • Linear Hidden Markov Model reparameterization, allowing a range of non-Gaussian HMMs to be treated as conditional Gaussian processes.
  • Neural Transport uses SVI to learn the geometry of a model before drawing samples using HMC (Hoffman et al. 2019).
• A tutorial on inference with Levy Stable distrubutions, demonstrating StableReparam, DiscreteCosineReparam, and EnergyDistance.

Other new features

• A tutorial on Dirichlet process mixture modeling, contributed by @m-k-S
• Added a LinearHMM distribution with an .rsample() method. This supports non-Gaussian noise such as Levy Stable and StudentT, but requires reparameterization for inference.
• Implemented a GaussianHMM.rsample() method for drawing joint samples from a linear-Gaussian HMM.
• Added a LowerCholeskyAffine transform.
• #2264 improves speed and numerical stability of MultivariateNormal conversion from scale_tril to precision.

Bug fixes

• #2263 fixes MCMC api to allow implementations other than HMC and NUTS.
• #2244 fixes an event_dim issue in ConditionedPlanar flow.
• #2243 fixes a bug in AffineCoupling.
• #2227 fixes device placement of the MultivariateStudentT.df param.
• #2226 fixes an edge case bug in discrete enumeration.

New Features

• pyro.infer.ReweightedWakeSleep implements the Reweighted Wake Sleep algorithm (Le et al. 2019). Contributed by Siddharth Narayanaswamy and Tuan Anh Le.

• pyro.infer.TraceTMC_ELBO implements the Tensor Monte Carlo marginal likelihood estimator (Aitchinson 2019), a generalization of the importance-weighted autoencoder objective.

• pyro.infer.EnergyDistance implements a likelihood-free inference algorithm based on Szekely's energy statistics, a multidimensional generalization of CRPS (Gneiting & Raftery 2007).

• pyro.contrib.cevae implements the Causal Inference VAE of (Louizos et al. 2017). See examples/contrib/cevae/synthetic.py for an end-to-end usage example.

• pyro.deterministic primitive to record deterministic values in the trace.

• pyro.nn.to_pyro_module_() recursively converts an regular nn.Module to a PyroModule in-place.

• A default implementation for Distribution.expand() that is available to all Pyro distributions that subclass from TorchDistribution, making it easier to create custom distributions.

New distributions and transforms

• MultivariateStudentT is a heavy-tailed multivariate distribution.
• Stable implements a Lévy α-stable distribution with reparametrized .rsample() method but no .log_prob(). This can be fit using EnergyDistance inference.
• ZeroInflatedNegativeBinomial is a distribution for count data.
• LowerCholeskyAffine is a multivariate affine transform.

Other Changes / Bug Fixes

• pyro.util.save_visualization has been deprecated, and dependency on graphviz is removed.
• #2197 fixed a naming bug in PyroModule that affected mutliple sub-PyroModules with conflicting names.
• #2192 Bug fix in Planar normalizing flow implementation
• #2188 Make error messages for incorrect arguments to effect handlers more informative

The objective of this release is to stabilize Pyro's interface and thereby make it safer to build high level components on top of Pyro.

Stability statement

• Behavior of documented APIs will remain stable across minor releases, except for bug fixes and features marked EXPERIMENTAL or DEPRECATED.
• Serialization formats will remain stable across patch releases, but may change across minor releases (e.g. if you save a model in 1.0.0, it will be safe to load it in 1.0.1, but not in 1.1.0).
• Undocumented APIs, features marked EXPERIMENTAL or DEPRECATED, and anything inpyro.contrib may change at any time (though we aim for stability).
• All deprecated features throw a FutureWarning and specify possible work-arounds. Features marked as deprecated will not be maintained, and are likely to be removed in a future release.
• If you want more stability for a particular feature, contribute a unit test.

New features

• pyro.infer.Predictive is a new utility for serving models, supporting jit tracing and serialization.
• pyro.distributions.transforms has many new transforms, and includes helper functions to easily create a variety of normalizing flows. The transforms library has also been reorganized.
• pyro.contrib.timeseries is an experimental new module with fast Gaussian Process inference for univariate and multivariate time series and state space models.
• pyro.nn.PyroModule is an experimental new interface that adds Pyro effects to an nn.Module. PyroModule is already used internally by AutoGuide, EasyGuide pyro.contrib.gp, pyro.contrib.timeseries, and elsewhere.
• FoldedDistribution is a new distribution factory, essentially equivalent to TransformedDistribution(-, AbsTransform()) but providing a .log_prob() method.
• A new tutorial illustrates the usage of pyro.contrib.oed in the context of adaptive election polling.

Breaking changes

• Autoguides have slightly changed interfaces:
  • AutoGuide and EasyGuide are now nn.Modules and can be serialized separately from the param store. This enables serving via torch.jit.trace_module.
  • The Auto*Normal family of autoguides now have init_scale arguments, and init_loc_fn has better support. Autoguides no longer support initialization by writing directly to the param store.
• Many transforms have been renamed to enforce a consistent interface, such as the renaming of InverseAutoregressiveFlow to AffineAutoregressive.
• pyro.generic has been moved to a separate project pyroapi.
• poutine.do has slightly changed semantics to follow Single World Intervention Graph semantics.
• pyro.contrib.glmm has been moved to pyro.contrib.oed.glmm and will eventually be replaced by BRMP.
• Existing DeprecationWarnings have been promoted to FutureWarnings.

Deprecated features

• pyro.random_module: The pyro.random_module primitive has been deprecated in favor of PyroModule which can be used to create Bayesian modules from torch.nn.Module instances.
• SVI.run: The SVI.run method is deprecated and users are encouraged to use the .step method directly to run inference. For drawing samples from the posterior distribution, we recommend using the Predictive utility class, or directly by using the trace and replay effect handlers.
• TracePredictive: The TracePredictive class is deprecated in favor of Predictive, that can be used to gather samples from the posterior and predictive distributions in SVI and MCMC.
• mcmc.predictive: This utility function has been absorbed into the more general Predictive class.