LLM chatbot server with ChatGPT plugins
Important: The war in Ukraine is still ongoing. Every day, Russian soldiers rape, murder, torture and deport Ukrainian civilians. Visit this page to see how you can support Ukraine today.
Horace lets you implement a custom chatbot with your choice of an LLM and a set of ChatGPT plugins. It runs as a WebSocket server, allowing you to easily add an LLM-powered chatbot to your web or mobile app.
Features:
none
, user_http
, service_http
Horace builds upon and extends GRACE, my original LLM-powered chatbot. Here is Horace running in debug mode with the Klarna ChatGPT plugin, accessed via a web client:
cd horace
python3 -m venv ./venv
source ./venv/bin/activate
pip3 install -r requirements.txt
app
directory: cd app
OPENAI_API_KEY=openai-api-key python3 main.py
(replace openai-api-key
with your OpenAI API key - get it here)cd horace
source ./venv/bin/activate
python3 app/horace-cli.py
client-demo/index.html
in Explorer/Finder etc. to open it in your browser.cd horace
docker build -t horace:latest .
docker run --rm \
-e OPENAI_API_KEY=openai-api-key \
-p 8001:8001 \
--name horace \
horace:latest
docker exec -it horace python3 /app/horace-cli.py
client-demo/index.html
in Explorer/Finder etc. to open it in your browser.main.py
and horace-cli.py
support a few command-line arguments:
python3 main.py --help
usage: main.py [-h] [--host HOST] [--port PORT] [--debug]
optional arguments:
-h, --help show this help message and exit
--host HOST bind host name
--port PORT bind port number
--debug enable debug mode
python3 app/horace-cli.py --help
usage: horace-cli.py [-h] [--host HOST] [--port PORT]
optional arguments:
-h, --help show this help message and exit
--host HOST server host name
--port PORT server port number
At the moment, only the OpenAI API backend is supported. (I am not aware of non-OpenAI LLMs with a level of instruction-following sufficient for plugin invocation right now.) However, adding a custom LLM backend with Horace is quite straightforward:
Backend
base located in backends/backend.py
and implement the complete()
methodbackends
directorymain.py
BACKENDS
mapping in main.py
to include your new backend's alias and class nameconfig.yaml
:backend:
name: my_llm_backend
Refer to the OpenAI API backend (backends/openai_backend.py
) as an example.
The OpenAI API backend lets you switch between the following models:
See config.yaml
for switching between the OpenAI API models.
Horace works with ChatGPT plugins out of the box. To enable a plugin, add its hostname with an optional port number to the plugins
section in config.yaml
:
router:
plugins:
# For https://github.com/artmatsak/chatgpt-todo-plugin/
# - localhost:5002
- www.klarna.com
Upon starting, the server requests http://[hostname]/.well-known/ai-plugin.json
for each plugin and takes it from there to load them.
Horace currently supports the none
, user_http
and server_http
auth methods for ChatGPT plugins. If an auth token is required for a plugin, Horace asks you for one during server startup. At the moment, auth tokens are saved unencrypted in .plugin_auth.json
.
The default LLM prompt for Horace is designed to make the bot neutral. The bot is neither limited to plugin-facilitated user requests (like a restaurant booking bot would be, for example), nor does it proactively push the plugin-enabled functionality onto the user. In other words, you can chat with the bot like you normally would with ChatGPT; if the bot feels that invoking a plugin method is needed, it will do so.
In real-world scenarios, you may want to limit the bot to a particular scope like booking a table (see GRACE's prompt for inspiration), or perhaps provide it with a unique voice/personality. To do this, you can add instructions to the LLM prompt using the extra_instructions
property in config.yaml
:
horace:
# Any extra instructions to prepend the prompt with
extra_instructions: >-
You are a helpful and polite AI assistant.
# extra_instructions: >-
# In your speech, you impersonate Jedi Master Yoda and crack jokes in response
# to mentions of other Star Wars characters.
Try uncommenting the Yoda block above to see how the voice of the chatbot changes accordingly. (Don't forget to restart the server after making any changes to the config.)
Tests have not yet been updated since forking from GRACE. To be fixed.