Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max). A PyTorch LLM library that seamlessly integrates with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, DeepSpeed, vLLM, FastChat, ModelScope, etc
Note: BigDL v2.3.0 has been updated to include functional and security updates. Users should update to the latest version.
Nano
trace
and quantization
process (for PyTorch and TensorFlow model optimizations)Orca:
Chronos
bigdl.chronos.aiops
module for AIOps use case on top of Chronos algorithms.Friesian:
PPML
Note: BigDL v2.2.0 has been updated to include functional and security updates. Users should update to the latest version.
Note: BigDL v2.1.0 has been updated to include functional and security updates. Users should update to the latest version.
Continue RNN optimization. We support both LSTM and GRU integration with MKL-DNN which acheives ~3x performance
ONNX support. We support loading third party framework models via ONNX
Richer data preprocssing support and segmentation inference pipeline support