JPMML-SparkML-LightGBM

JPMML-SparkML plugin for converting LightGBM-Spark models to PMML.

Prerequisites

• Apache Spark 2.4.x
• LightGBM-Spark 0.18.1.

Installation

Enter the project root directory and build using Apache Maven:

mvn clean install

The build installs JPMML-SparkML-LightGBM library into local repository using coordinates org.jpmml:jpmml-sparkml-lightgbm:1.0-SNAPSHOT.

Usage

The JPMML-SparkML-LightGBM library extends the JPMML-SparkML library with support for com.microsoft.ml.spark.lightgbm.LightGBMClassificationModel prediction model classes.

add org.jpmml:jpmml-sparkml-lightgbm:1.0-SNAPSHOT.jar to CLASSPATH

import  mmlspark
import mmlspark.train
from pyspark.ml import PipelineModel

df = spark.sql("select * from algo_dc_ml_split_data")
model = PipelineModel.load("/user/turing/lightgbm_spark")

from pyspark2pmml import PMMLBuilder

pmmlBuilder = PMMLBuilder(spark.sparkContext, df, model)

pmmlBuilder.buildFile("algo_dc_ml_2c_lightgbm_spark.xml")

hdfs_client.upload("algo_dc_ml_2c_lightgbm_spark.xml","${hdfs_path}")

License

JPMML-SparkML-LightGBM is licensed under the Apache License 2.0. Other licenses are available on request.