Multimodal instruction-following model for text generation that runs on your CPU. Less than 14 GB of RAM required.
Models hallucinates - this is meant for fun. We do not recommend using the model in production - unless you
hallucinate for a living know what you are doing.
The implementation may contain bugs and int4 quantization performed is not optimal – This might lead to worse performance than the original model.
git clone https://github.com/nolanoOrg/smol-gpt
pip install -r requirements.txt
cd cpp && make
python3 app.py(May take a few minutes to download and load the model)
http://127.0.0.1:4241/in your browser.`
Contributions are welcome. Please open an issue or a PR. New features will be community driven. Following features can be easily added for the model:
The model used are Clip and Bert following Blip-2 and Flan-T5 for instruction following.