RTP-LLM: Alibaba's high-performance LLM inference engine for diverse app...
A multi-functional library for full-stack Deep Learning. Simplifies Mode...
Code samples for the Lightbend tutorial on writing microservices with Ak...
Common library for serving TensorFlow, XGBoost and scikit-learn models i...
JetStream is a throughput and memory optimized engine for LLM inference ...
A scalable, high-performance serving system for federated learning models
BentoML Example Projects ?
Serving PyTorch models with TorchServe :fire:
flink-jpmml is a fresh-made library for dynamic real time machine learni...
Deploy DL/ ML inference pipelines with minimal extra code.
MONAI Deploy App SDK offers a framework and associated tools to design, ...
A collection of model deployment library and technique.
Code and presentation for Strata Model Serving tutorial
fastText model serving service
An umbrella project for multiple implementations of model serving