ChatdollKit enables you to make your 3D model into a chatbot
🖼️ Support vision
Set image to request message as payload for multimodal conversation.
🚀 Configuration-free demo
Just start without any configurations. Set API key (and others if you want) at runtime.
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.7.1...v0.7.2
Support changing avatar on runtime. Use ModelController.SetAvatar to change the avatar to another on runtime.
If you want to try it on editor, set another avatar on the scene to the inspector of ModelController and push Change Avatar button(appears runtime only).
Mute microphone when not listening for WakeWord or VoiceRequest.
Also added IsMuted property to DialogController so that you can control mute/unmute manually. Default is false.
You can change the value manually on inspector or in script.
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.7.0...v0.7.1
✅Multiple LLMs: ChatGPT / Azure OpenAI Service, Anthropic Claude, Google Gemini Pro and others ✅Agents: Function Calling (ChatGPT / Gemini) or your prompt engineering ✅Multimodal: GPT-4V and Gemini-Pro-Vision are suppored ✅Emotions: Autonomous face expression and animation
We've developed a versatile framework that standardizes the processes of Routing, Chatting (including facial expressions and motions), and executing Tools using various Large Language Models (LLMs). This framework allows for easy customization by simply swapping out LLM-specific components.
Additionally, you can support any LLM by creating your own components that implement the ILLMService interface.
To use this new LLM-based dialog, attach LLMRouter, LLMContentSkill and ILLMService component like ChatGPTService in ChatdollKit.LLM package. Also attach function skills that extends LLMFunctionSkillBase if you build an AI agent. See DemoChatGPT that works right out-of-the-box🎁.
NOTE: ChatdollKit.Dialog.Processor.ChatGPT* components are deprecated.
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6.6...v0.7.0
Previously, ChatGPT (OpenAI API) sometimes does not return a response, which prevents the character from continuing the conversation.😔 However, in this update, the system will automatically retry in the absence of a response, allowing the conversation to continue.
Set animations before starting conversation. (e.g. at Start())
chatGPTContentSkill.RegisterAnimation("waving_arm", new Model.Animation("BaseParam", 10, 3.0f));
chatGPTContentSkill.RegisterAnimation("look_away", new Model.Animation("BaseParam", 6, 3.0f, "AGIA_Layer_look_away_01", "Additive Layer"));
List them in the prompt and provide a guide on how to use them.
* You can express your emotions through the following animations:
- waving_arm
- look_away
* If you want to express emotions with gestures, insert the animation into the response message like [anim:waving_arm].
You can have multiple idling mode. The default is normal. Add animations and face expressions if you want to add another mode like below:
modelController.AddIdleAnimation(new Model.Animation("BaseParam", 101, 5f), mode: "sleep");
modelController.AddIdleFace("sleep", "Blink");
Switch using ChangeIdlingModeAsync.
modelController.ChangeIdlingModeAsync("sleep");
The animation for sleep starts immediately.
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6.5...v0.6.6
To use Azure OpenAI Service set following info on inspector of ChatGPTService component:
Chat Completion Url
format: https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id}/chat/completions?api-version={api-version}
API Key to Api Key
Set true to Is Azure
NOTE: Model on inspector is ignored. Engine in url is used.
Support multiple languages and allow you to switch between them without changing configurations.
NOTE: Text-to-Speech doesn't support WebGL.
Use AzureStreamVoiceRequestProvider to perform real-time speech recognition.
This component depends on Azure Speech SDK.
NOTE: Microphone volume slider doesn't support controlling this component.
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6.4...v0.6.5
By applying eye makeup, I enhanced the attractiveness of her face.🪄
You can create RAG-based AI agent just by adding ChatGPT function skill. See RetrievalQASkill example that can answer based on OpenAI terms of use.
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6.3...v0.6.4
Minor updates but they are necessary for creating apps.
See Unagirl in dress at vroid hub https://hub.vroid.com/characters/7204218679890800631/models/6961078537805052952
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6.2...v0.6.3
ChatGPT is no longer just one of the various skills but is integrated as a conversation processing system of ChatdollKit.
You can add skills much smarter: All you have to do is making the definitions of the interface of ChatGPT function calling and to implement the logic to handle the result from the function.
See also the example. https://github.com/uezo/ChatdollKit/blob/master/Examples/ChatGPT/FunctionSkills/WeatherSkill.cs
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6.1...v0.6.2
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.6...v0.6.1
Add ChatGPTEmotionSkill as an out-of-the-box example that express the emotion as face expressions while conversation.
Also add sample motions while thinking (waiting response from ChatGPT) and support multiple idle animations (change pose periodically).
You can build rich 3D AI virtual assistant lightning fast⚡️
Support one-shot prompt: system message content and a pair of user and assistant message content as an example turn. You can configure it on inspector, and even runtime for they are the public fields.
Support configuration at runtime. This enables you to build apps that uses user own VRM.
Improve FPS about 30% on my machine and stability of controlling animation gets better. They are basically internal refactoring but that includes some destructive changes.
// ~ v0.5.3
response.AddAnimation("AnimationName");
// v0.6
response.AddAnimation("BaseParam", 6);
PreGap and PostGap no longer available for the argument of Animation and FaceExpression. Use duration to adjust timing.Update() by @uezo in https://github.com/uezo/ChatdollKit/pull/211
Full Changelog: https://github.com/uezo/ChatdollKit/compare/v0.5.3...v0.6