AutoDeploy is a single configuration deployment library
For the rest : So you're a data scientist and have the greatest model on planet earth to classify dogs and cats! :). What next? It's a steeplearning cusrve from building your model to getting it to production. MLOps, Docker, Kubernetes, asynchronous, prometheus, logging, monitoring, versioning etc. Much more to do right before you The immediate next thoughts and tasks are
What if you could only configure a single file and get up and running with a single command. That is what AutoDeploy is!
Read our documentation to know how to get setup and get to serving your models.
and many more...
Install docker
Install docker-compose
wget https://github.com/kartik4949/AutoDeploy/files/7134516/model_dependencies.zip
bash build.sh -r path/to/model/requirements.txt -c path/to/model/config.yaml
bash start.sh -f path/to/config/file/in/autodeploy
bash build.sh -r path/to/model/requirements.txt -c path/to/model/config.yaml
kubectl -f k8s apply
kubectl get pod
kubectl port-forward autodeploy-pod-name 8000:8000
kubectl port-forward grafana-pod-name 3000:3000
Clone repo.
Dump your iris sklearn model via pickle, lets say custom_model.pkl
.
Make a dir model_dependencies inside AutoDeploy.
Move custom_model.pkl
to model_dependencies.
Create or import a reference iris_reference.npy
file for data drift monitoring.
iris_reference.npy
is numpy reference array used to find drift in incomming data.(n, *shape_of_input)
e.g for iris data : np.zeros((100, 4))Move iris_reference.npy
to model_dependencies folder.
Refer below config file and make changes in configs/iris/config.yaml and save it.
Lastly make an empty reqs.txt file inside model_dependencies folder.
model:
model_type: 'sklearn'
model_path: 'custom_model.pkl' # Our model pickle file.
model_file_type: 'pickle'
version: '1.0.0'
model_name: 'sklearn iris detection model.'
endpoint: 'predict'
protected: 0
input_type: 'structured'
server:
name: 'autodeploy'
port: 8000
dependency:
path: '/app/model_dependencies'
input_schema:
petal_length: 'float'
petal_width: 'float'
sepal_length: 'float'
sepal_width: 'float'
out_schema:
out: 'int'
probablity: 'float'
status: 'int'
monitor:
server:
name: 'rabbitmq'
port: 5672
data_drift:
name: 'KSDrift'
reference_data: 'iris_reference.npy'
type: 'info'
metrics:
average_per_day:
type: 'info'
bash build.sh -r model_dependencies/reqs.txt -c configs/iris/config.yaml
bash start.sh -f configs/iris/config.yaml
Tada!! your model is deployed.
model.onnx
.model.onnx
to model_dependencies.classification_reference.npy
file for data drift monitoring.classification_reference.npy
to model_dependencies folder.model:
model_type: 'onnx'
model_path: 'horse_zebra.onnx'
model_file_type: 'onnx'
version: '1.0.0'
model_name: 'computer vision classification model.'
endpoint: 'predict'
protected: 0
input_type: 'serialized'
input_shape: [224, 224, 3]
server:
name: 'autodeploy'
port: 8000
preprocess: 'custom_preprocess_classification'
input_schema:
input: 'string'
out_schema:
out: 'int'
probablity: 'float'
status: 'int'
dependency:
path: '/app/model_dependencies'
monitor:
server:
name: 'rabbitmq'
port: 5672
data_drift:
name: 'KSDrift'
reference_data: 'structured_ref.npy'
type: 'info'
custom_metrics: 'image_brightness'
metrics:
average_per_day:
type: 'info'
pillow
import cv2
import numpy as np
from register import PREPROCESS
@PREPROCESS.register_module(name='custom_preprocess')
def iris_pre_processing(input):
return input
@PREPROCESS.register_module(name='custom_preprocess_classification')
def custom_preprocess_fxn(input):
_channels = 3
_input_shape = (224, 224)
_channels_first = 1
input = cv2.resize(
input[0], dsize=_input_shape, interpolation=cv2.INTER_CUBIC)
if _channels_first:
input = np.reshape(input, (_channels, *_input_shape))
else:
input = np.reshape(input, (*_input_shape, _channels))
return np.asarray(input, np.float32)
from register import POSTPROCESS
@POSTPROCESS.register_module(name='custom_postprocess')
def custom_postprocess_fxn(output):
out_class, out_prob = output[0], output[1]
output = {'out': output[0],
'probablity': output[1],
'status': 200}
return output
import numpy as np
from PIL import Image
from register import METRICS
@METRICS.register_module(name='image_brightness')
def calculate_brightness(image):
image = Image.fromarray(np.asarray(image[0][0], dtype='uint8'))
greyscale_image = image.convert('L')
histogram = greyscale_image.histogram()
pixels = sum(histogram)
brightness = scale = len(histogram)
for index in range(0, scale):
ratio = histogram[index] / pixels
brightness += ratio * (-scale + index)
return 1.0 if brightness == 255 else brightness / scale
bash build.sh -r model_dependencies/reqs.txt -c configs/classification/config.yaml
bash start.sh -f configs/classification/config.yaml
image_brightness
: goto grafana and add panel to the dashboard with image_brightness as metric.admin
.from register import PREPROCESS
@PREPROCESS.register_module(name='custom_preprocess')
@PREPROCESS.register_module(name='custom_preprocess')
def function(input):
# process input
input = process(input)
return input
custom_preprocess
name in our config file, add this in your config file.preprocess: custom_preprocess
out_schema:
out: 'int'
probablity: 'float'
status: 'int'
import numpy as np
from PIL import Image
from register import METRICS
@METRICS.register_module(name='image_brightness')
def calculate_brightness(image):
image = Image.fromarray(np.asarray(image[0][0], dtype='uint8'))
greyscale_image = image.convert('L')
histogram = greyscale_image.histogram()
pixels = sum(histogram)
brightness = scale = len(histogram)
for index in range(0, scale):
ratio = histogram[index] / pixels
brightness += ratio * (-scale + index)
return 1.0 if brightness == 255 else brightness / scale
image_brightness
in config file to expose this metric function.monitor:
server:
name: 'rabbitmq'
port: 5672
data_drift:
name: 'KSDrift'
reference_data: 'structured_ref.npy'
type: 'info'
custom_metrics: ['metric1', 'metric2']
metrics:
average_per_day:
type: 'info'
import numpy as np
from PIL import Image
from register import METRICS
@METRICS.register_module(name='metric1')
def calculate_brightness(image):
return 1
@METRICS.register_module(name='metric2')
def metric2(image):
return 2
monitor:
server:
name: 'rabbitmq'
port: 5672
data_drift:
name: 'KSDrift'
reference_data: 'structured_ref.npy'
type: 'info'
custom_metrics: ['metric1', 'metric2']
metrics:
average_per_day:
type: 'info'
Distributed under the MIT License. See LICENSE
for more information.
Kartik Sharma - [email protected] Nilav Ghosh - [email protected]