Spark Deep Learning Versions Save

Deep Learning Pipelines for Apache Spark


3 years ago
  • Deprecated some submodules and classes
  • Updated dependencies


4 years ago
  • HorovodRunner's main function can now have a return value.
  • Better support for multiple GPU machines (Databricks only version).


4 years ago
  • Upgrade to TensorFlow 1.12.0.


4 years ago
  • Added HorovovodRunner API.
  • Simplified test and doc build w/ Docker and conda.
  • Updated public Python API docs.
  • Removed persistence from DeepImageFeaturizer.


4 years ago
  • ignore nullable in DeepImageFeaturizer.validateSchema
  • upgrade TensorFrames version to 0.5.0
  • upgrade Tensorflow version to 1.10.0 and Keras version to 2.2.2


4 years ago
  • keras_image_file_estimator support both sparse and dense vectors
  • upgrade TensorFrames version to 0.4.0
  • add style checks to Travis CI
  • doc fixes


4 years ago

This is the 1.0.0 release. It brings compatibility with newer versions of Spark (2.3) and Tensorflow (1.6+). The custom image schema formerly defined in this package has been replaced with Spark's ImageSchema so there may be some breaking changes when updating to this version.

Notable changes:

  • (breaking change) Using the definition of images from Spark 2.3.0. The new definition uses the BGR channel ordering for 3-channel images instead of the RGB ordering used in this project before the change.
  • Persistence for DeepImageFeaturizer (both Python and Scala).


5 years ago

This is the final release of dl-pipelines prior to migrating to new ImageSchema.

Notable changes:

  • Added vgg16, vgg19 models to DeepImageFeaturizer/DeepImagePredictor (Python).
  • Added a Scala API for DeepImageFeaturizer (for transfer learning for images).
  • Added TFTransformer and KerasTransformer for applying TensorFlow graphs or TensorFlow-backed Keras models to a column of arrays in a Spark DataFrame.


5 years ago

This is the final release for Deep Learning Pipelines 0.2.0

Notable additions since 0.1.0:

  • KerasImageFileEstimator API (train a Keras model on image files)
  • SQL UDF support for Keras models
  • Added Xception, Resnet50 models to DeepImageFeaturizer/DeepImagePredictor.