a simplified machine learning container platform that helps teams get started with an automated workflow
A simplified machine learning container platform that helps teams get started with an automated workflow.
Server default configurations in config.yml
These configurations are loaded by default if not overridden in config.yml
.
expire: "1300"
deployexpire: "86400"
uploadsize: 2000000000
envsize: 20
https:
listen: "0.0.0.0"
port: "8443"
debug: false
verifytls: false
db:
db: 0
maxretries: 20
docker:
endpoint: "unix:///var/run/docker.sock"
cpushares: 512
memory: 1073741824
gotty:
tls: false
These configurations need to be set in your environment variables.
# Server admin username and password
DamaUser # example: DamaUser="tim"
DamaPassword # example: DamaPassword="9e9692478ca848a19feb8e24e5506ec89"
# Redis database password if applicable
DBPassword # example: DBPassword="9e9692478ca848a19feb8e24e5506ec89"
All configurations types
images: ["perlogix:minimal"] # required / string array
expire: "1300" # string
deployexpire: "86400" # string
uploadsize: 2000000000 # int
envsize: 20 # int
https:
listen: "0.0.0.0" # string
port: "8443" # string
pem: "/opt/dama.pem" # required / string
key: "/opt/dama.key" # required / string
debug: false # bool
verifytls: false # bool
db:
network: "unix" # required / string
address: "./tmp/redis.sock" # required / string
db: 0 # int
maxretries: 20 # int
docker:
endpoint: "unix:///var/run/docker.sock" # string
cpushares: 512 # int
memory: 1073741824 # int
gotty:
tls: false # bool
These environment variables need to be exported in order to use dama-cli.
DAMA_SERVER # example: export DAMA_SERVER="https://localhost:8443/"
DAMA_USER # example: export DAMA_USER="tim"
DAMA_KEY # example: export DAMA_KEY="9e9692478ca848a19feb8e24e5506ec89"
Usage: dama [options] <args>
-new Create a new environment from scratch and delete the old one
-run Create environment and run with dama.yml
-file Run with dama.yml in different directory
-env Create an environment variable or secret for runtime
-img Specify a docker image to be used instead of the default image
-dl Download file from workspace in your environment to your local computer
-up Upload files from your local computer to workspace in your environment
-deploy Deploy API and get your unique URI
-show-api Show API details: URL, Health and Type
-show-images Show images available to use
dama -new
dama -run
dama -run -file ../dama.yml
dama -env "AWS_ACCESS_KEY_ID=123,AWS_SECRET_ACCESS_KEY=234"
dama -deploy
dama -run -img tensorflow:lite
dama -show-images
dama -show-api
dama -up data.csv
dama -dl model.pkl
This a simple dama.yml
to setup your environment and run a Flask API.
image: "perlogix:minimal"
port: "5000"
pip: |
Flask==0.12.2
scikit-learn==0.19.1
numpy==1.14.2
scipy==1.0.0
python: |
from flask import Flask, request, jsonify
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.externals import joblib
X, y = datasets.load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
model = RandomForestClassifier(random_state=101)
model.fit(X_train, y_train)
print("Score on the training set is: {:2}".format(model.score(X_train, y_train)))
print("Score on the test set is: {:.2}".format(model.score(X_test, y_test)))
model_filename = 'iris-rf-v1.0.pkl'
print("Saving model to {}...".format(model_filename))
joblib.dump(model, model_filename)
app = Flask(__name__)
MODEL = joblib.load('iris-rf-v1.0.pkl')
MODEL_LABELS = ['setosa', 'versicolor', 'virginica']
@app.route('/predict')
def predict():
sepal_length = request.args.get('sepal_length')
sepal_width = request.args.get('sepal_width')
petal_length = request.args.get('petal_length')
petal_width = request.args.get('petal_width')
features = [[sepal_length, sepal_width, petal_length, petal_width]]
label_index = MODEL.predict(features)
label = MODEL_LABELS[label_index[0]]
return jsonify(status='complete', label=label)
if __name__ == '__main__':
app.run(debug=False, host="0.0.0.0", threaded=True)
cURL API in sandbox or deploy
curl -ks https://localhost:8443/api/<insert sandbox key>/predict?sepal_length=5&sepal_width=3.1&petal_length=2.5&petal_width=1.2
Even simpler environment setup with model training.
image: "perlogix:tensorflow"
checkout: "https://github.com/aymericdamien/TensorFlow-Examples.git"
cmd: |
cd TensorFlow-Examples/examples/3_NeuralNetworks
python neural_network.py
All YAML configuration option types.
project # string - proejct name
env # string array - env variables
checkout # string - git checkout master branch
time_format # string - python time format used in container as env variable TIMESTAMP
setup_cmd # string - run setup /initial command before cmd or python
cmd # string - run BASH Linux command
python # string - run inline Python
pip # string - install pip packages
image # string - define container image for environment
port # string - port to expose for web service
git:
url # string - git URL
branch # string - git branch
sha # string - git SHA
aws_s3:
file # string - file to push or pull
dir # string - directory to push or pull
bucket_push # string - push file or dir to S3
bucket_pull # string - pull file or dir from S3
Add these lines to your Dockerfiles for your CLI to connect via websockets
RUN cd /usr/bin && curl -L https://github.com/yudai/gotty/releases/download/v1.0.1/gotty_linux_amd64.tar.gz | tar -xz
CMD ["/usr/bin/gotty", "--reconnect", "-w", "/bin/bash"]
make build