Tensor2tensor Versions Save

• Added a MultiProblem class for Multitask Learning. Thanks @urvashik !

• Added decoding option to pass through the features dictionary to predictions. Thanks @rsepassi !

• Enabled MLEngine path to use Cloud TPUs. Thanks @rsepassi !

• Added a simple One-Hot Symbol modality. Thanks @mbz !

• Added Cleverhans integration. Thanks @aidangomez !

• Problem definitions added for:

  • Allen Brain Atlas problems. Thanks @cwbeitel !
  • LSUN Bedrooms dataset.
  • Added various NLP datasets. Thanks @urvashik !
    • MSR Paraphrase Corpus,
    • Quora Question Pairs,
    • Stanford Sentiment Treebank,
    • Question Answering NLI classification problems,
    • Recognizing Textual Entailment,
    • Corpus of Linguistic Acceptability,
    • Winograd NLI.
  • Added a data generator for WSJ parsing.

• Model additions:

  • Implemented Targeted Dropout for Posthoc Pruning. Thanks @aidangomez !
  • Added self attention to VQA attention model.
  • Added fast block parallel transformer model
  • Implemented auxiliary losses from Stochastic Activation Pruning for Robust Adversarial Defense. Thanks @alexyku !
  • Added probability based scheduled sampling for SV2P problem. Thanks @mbz !
  • Reimplementated Autoencoder and Eval. Thanks @piotrmilos !
  • Relative memory efficient unmasked self-attention.

• Notable bug fixes:

  • bug with data_gen in style transfer problem Thanks @tlatkowski !
  • wmt_enfr dataset should not use vocabulary based on "small" dataset. Thanks @nshazeer !

• Many more fixes, tests and work on Model based RL, Transfomer, Video and other models and problems.

• added Mozilla common voice as Problem and style transfer one others!
• improvements to ASR data preprocessing (thanks to jarfo)
• decoding works for Transformer on TPUs and for timeseries problems
• corrections and refactoring of the RL part
• Removed deprecated Experiment API code, and support SessionRunHooks on TPU.
• many other corrections and work on video problems, latent variables and other

Great thanks to everyone!

• registry.hparams now returns an HParams object instead of a function that returns an HParams object
• New MultistepAdamOptimizer thanks to @fstahlberg
• New video models and problems and improvements to VideoProblem
• Added pylintrc and lint tests to Travis CI
• Various fixes, improvements, and additions

• --random_seed is unset by default now. Set it to an integer value to get reproducible results.
• bAbI text understanding tasks added
• Have the ML Engine and TPU codepaths use TF 1.8
• Various cloud-related bug fixes
• WikisumWeb data generation fixes
• Various other fixes

• Lambada and wikitext103 datasets.
• ASR model with Transformer and iPython notebook.
• Many other improvements including RL code, autoencoders, the latent transformer (transformer_vae) and more.

• Release scripts to generate the data for Generating Wikipedia by Summarizing Long Sequences
• New RTransformer model, a recurrent Transformer
• New English-Estonian translation dataset thanks to @stefan-it
• New ROC_AUC metric thanks to @jjtan
• Various fixes, improvements, additions, etc.

• --problems command-line flag renamed to --problem
• hparams.problems renamed to hparams.problem_hparams and hparams.problem_instances renamed to hparams.problem (and neither are lists now)
• Dropped support for TensorFlow 1.4
• Various additions, fixes, etc.

• Distillation codepath added
• Improved support for serving language models
• New TransformerScorer model which return log prob of targets on infer
• Support for bfloat16 weights and activations on TPU
• SRU gate added to common_layers
• --checkpoint_path supported in interactive decoding
• Improved support for multiple outputs
• VideoProblem base class
• Various fixes, additions, etc.

• Scalar summary support on TPUs
• New Squad and SquadConcat problem for question answering (and relevant base class)
• New video problems
• bfloat16 support for Transformer on TPUs
• New SigmoidClassLabelModality for binary classification
• Support batch prediction with Cloud ML Engine
• Various fixes, improvements, additions

• Updates to experimental RL codebase
• ImageTransformer on TPU
• Various updates, fixes, additions, etc.