Cleverhans Versions Save

We now support 3 frameworks: JAX, PyTorch, and TF2; we no longer support TF1.

If necessary, you can still access the TF1 implementations under the cleverhans_v3.1.0/ directory.

This is the final release that supports TF1 and Python 2; future versions will only support TF2, PyTorch and JAX on Python 3.

Compared to v3.0.1, this release provides various bug fixes and code refactoring (the main difference is that attacks are now stored in separate files for better modularity), as well as beta implementations of attacks in TF2, PyTorch and JAX in preparation for the upcoming version 4 release.

Compared to v.3.0.0, this release fixes bugs related to the:

• Early stopping threshold in SPSA
• Eps argument of PGD

Relative to v2.1.0, this release adds:

• Consistent use of logits and probabilities in attack interfaces
• A plotting module
• Pylint
• Certification code by Raghunathan et al.
• Semantic adversarial examples
• Bundled confidence reports
• Dataset module
• Model zoo
• Fast Spatial Attack
• PGD-based attacks consolidation
• Loss module
• New training and eval modules with multi-device support
• Partial PyTorch support
• Partial support for defenses (through modules like loss and training)
• SPSA
• Various bug fixes, code maintainability improvements, and better support for various platforms

Support for TensorFlow prior to 1.8 is now deprecated.

CleverHans version numbers are based on the semantic versioning system. This release increments the major version number because it makes API changes that are incompatible with the previous release.

In particular, many attacks are now stronger in this release. Benchmark results generated using 2.x should not be compared to benchmark results using 3.x.

The following features were added:

• Added support for float64 to prevent numerical instabilities
• Added SPSA gradient-free attack
• Added optimizer and decision rule options to EAD
• dtype of data can now be passed as input to Attack classes

The following additions were made to the examples/ folder:

• Added adversarial patch notebook
• Added code for the NIPS adversarial competition

This release also provides various bug fixes and code maintainability improvements, as well as improvements to the documentation.

CleverHans version numbers are based on the semantic versioning system. This release increments the minor version number because it adds functionality in a backwards-compatible manner.

Relative to v1.0.0, this release adds:

• Model and Attack base classes providing a uniform interface for all library functionality and the ability to extend the library with 3rd party subclasses
• Independence from Keras (it remains possible to use Keras, but Keras is no longer a requirement for any library functionality)
• Ability to reload saved models
• The Carlini-Wagner attack
• Black box attacks
• Virtual Adversarial Training
• Visualizations of adversarial examples
• Starter files for the NIPS 2017 Adversarial Attack and Defense competition
• An interface to the Madry Lab challenge
• A CIFAR-10 example
• Several improvements to the tutorials, including using the Adam optimizer
• A configurable logging system
• Python 3 compatibility
• PEP8 compliance
• Various bug fixes, code maintainability improvements, and better support for various platforms like Mac OS X

This release also deprecates some features:

• Theano support

CleverHans version numbers are based on the semantic versioning system. This release increments the major version number because it makes API changes that are incompatible with the previous release. Most of these API changes are related to the standardization of the attacks to all implement the Attack interface and access the model via the Model interface.

New features and improvements since v0.1:

• Added jsma, the saliency map attack
• Improved the MNIST model in the tutorial, for greater speed and accuracy
• The major version number has incremented due to bug fixes that affect the reproducibility of benchmarks:
  • Fixed a bug in tf_model_loss that reduced the effectiveness of both model training and adversarial example construction.
  • Fixed a bug in tf_model_eval that could result in overestimating the accuracy.

This initial public release includes:

• The fast gradient sign method (tensorflow)
• A tutorial using keras on MNIST showing how to:
  • Evaluate vulnerability to the fast gradient sign method
  • Use adversarial training with the fast gradient sign method