Keras Adamw Versions Save

FEATURES:

• Automatic weight decay inference and zeroing; set as usual, e.g. kernel_regularizer=l2(1e-4)
• L1, L1-L2 decay now supported
• PyPi-indexed: pip install keras-adamw

BREAKING:

• Moved some utils to new utils_common.py

BUGFIXES:

• fill_dict_in_order: _list_of_vals -> values_list
• example.py: from keras_adamw.optimizers -> from keras_adamw

MISC:

• Update docstrings
• Bump TF version to 2.1 in testing
• Changed README example
• Changed some tests

New features:

• Simplified imports; just use from keras_adamw import for both the optimizers and utils, only requiring import os; os.environ["TF_KERAS"]='1' if using tensorflow.keras imports.

Breaking changes:

• Environment flags are now set via '1' and '0', instead of 'True' and 'False'

Bug fixes:

• get_configs() incorrectly cast eta_min, eta_max, and eta_t via int - changed to float

New features:

• Full compatibility with TensorFlow 2.0.0 and Keras 2.3.0 (keras + tf.keras)
• Full compatibility with TensorFlow 1.14.0 and Keras 2.2.5 (keras + tf.keras)^
• Run-based weight decay normalization scheme, normalizing over arbitrary # of iterations independent of LR scheduler (e.g. over all epochs) (total_iterations_wd)
• utils now contain all common optimizer ops, e.g. _apply_weight_decays()
• Reference implementation compatible with tf.python.keras, TF2 + Keras 2.3.0; see docstring
• Cleaner code

^ - also compatible w/ TensorFlow 1.13.0 & 1.15.0, Keras 2.2.3-2.2.4

---

BUGFIXES:

• lr_multipliers were not being applied for AdamW
• SGDW was missing K.symbolic wrapper
• and total_iterations != 0 was missing for NadamW and SGDW, yielding NaN optimizer weights for total_iterations = 0
• TF reset seed method was incompatible w/ TF 2.0.0; added support for both

---

Misc changes:

• Updated README.md w/ info on total_iterations_wd
• Added README.md to keras_adamw on module version selection
• Added unit tests for respective optimizers
• Revamped test.sh; each test has a dedicated folder to ease coverage isolation
• Revamped .travis.yml:
  • Added versioning builds w/ TF_KERAS, TF_EAGER, TF_VERSION, and KERAS_VERSION flags
  • Removed boolean KERAS flag; repo is based on Keras
  • Reorder buggy setuptools & conda update -all commands
• Moved testing module requirements from requirements.txt to requirements-test.txt

Initial release - includes:

• AdamW, AdamWR
• NadamW, NadamWR
• SGDW, SGDWR
• Warm restarts
• Cosine annealing learning rate schedule
• Layer-wise learning rate multipliers