1. Support multi agent v2.
2. Updated data serializer.
3. Bug fixes.

1. Updated audio capturing.
2. Updated auto reconnection.
3. Simplified APIs for the incoming multiagent features.
4. Bug fixes.
5. Support Sentis Lipsync Model (Github release only)

How to use Sentis Lipsync:

1. Fetch com.unity.sentis in Unity's package manager.
2. Go to Assets/Inworld/Inworld.Native/Lipsync~, rename the folder to Lipsync

You will see the sample scene there. NOTE: The first time running would be laggy, that's because the sentis package's shader error warnings.

3.3.1

1. Implemented feedback
2. Added character selecting in global panel.
3. Added config canvas in global panel.
4. Bug fixes.

Main Features:

1. Updated auto reconnection. Optimized connection time.
2. Use tarball to replace github ref.

1. Reorg package structures.
2. Update animations and character assets.

1. Add WebGL Microphone Input.
2. Support Saving/Loading Session State
3. Fixed a bug of occasionally unable to load lipsync.
4. Fixed a bug of occasionally unable to reset animations

3.1.0

New Features:

1. Procedually loading packages for avoiding import errors.
2. Multiple audio input method support including turn based, push to talk, and aec.
3. Pausing interactions (By default it's space key).
4. Skipping interactions (By default it's left ctrl).
5. Save/load sessions for websocket.

Changelog

First release of version 3.

IMPORTANT NOTICE: This is a pre-release/beta version of the InworldAI Unity runtime SDK and is not intended for production use. This version is released for testing and development purposes only. We provide this version 'as-is' and make no guarantees or warranties of any kind, express or implied, about its completeness, accuracy, reliability, suitability or availability. Please make sure to backup your data regularly and expect bugs and issues. We encourage users to provide feedback and report any bugs or issues they encounter to help us improve the final version. Use this at your own risk.

Packages Overview

InworldAI.unitypackage

The default package using NDK and Protobuf as transport layer. Contains samples of InworldAI.Sample.RPM. The samples are all from the version 2, not only showcasing all the different features, but also ensuring a smooth upgrade path. Works in Windows only.

InworldAI.Lite.unitypackage

A basic 2D implementation for Inworld using websocket as transport layer. Supports all platforms.

Package Structure

Inworld Unity SDK package comprises the following modules:

Inworld.AI

This is the core module that encompasses all fundamental Inworld logic, WebSocket-based Inworld interactions, and a 2D sample. All other modules reference it.

Inworld.Assets

This module encompasses various Inworld functionalities implemented within Unity, such as 2D and 3D animations, fonts, logos, UI icons, Lipsync data, and more.

Inworld.NDK

This module constitutes Inworld communication implementation based on NDK (C++ based). It employs Protobuf for data transmission. It offers faster performance compared to WebSocket-based implementation, with lower bandwidth consumption. Additionally, it features an echo cancellation module. Currently limited to Windows support.

Inworld.Samples.RPM

This is a rewrite based on Inworld Unity SDK 3.0, built upon all the features and samples of the 2.0 version. It encompasses all functionalities of 2.0 and leverages Ready Player Me for character model implementation. It is not supported on WebGL.

Inworld.Samples.Innequin

This is a sample built upon Inworld's next-generation model, Innequin, and follows a distinct approach from Ready Player Me. It is based on .fbx models, making it suitable for all platforms.

Features

Each Unity package contains different modules. For specific details, please refer to the table below:

Packages	Inworld.AI	Inworld.Assets	Inworld.NDK	Inworld.Samples.RPM	Inworld.Samples.Innequin
Inworld.AI.Full	√	√	√	√	√
InworldAI.Lite	√

Compatibility

Platform	Inworld.AI	InworldAI.Lite	InworldAI.Sample.RPM	InworldAI.Sample.Innequin	Inworld v1	Inworld v2
Unity Minimum Version	2020.1	2020.1	2020.1	2020.1	2019.4	2021.3
Windows	√	√	√	√	√	√
Mac Intel 64-bit	×	√	√	√	Only in Editor	Only in Editor
Mac Apple Silicon	×	√	√	√	Only in Editor	Only in Editor
Android	×	√	√	√	√	√
Oculus	×	√	√	√	√	√
iOS	×	√	√	√	√	√
Linux	×	√	√	√	×	×
WebGL	×	√	×	√	×	×

How to use

1. Prepare your assets.

To integrate your own characters into your Unity project, you will need to ensure that you can access:

1. At least one Workspace.
2. At least one API Key in your workspace.
3. At least one Character in your workspace.
4. At least one Inworld Scene in your workspace.
5. Ensure that your Inworld Scene contains all the Characters you need to interact with.

If you do not know how to create them, please check our Prerequisities page.

2. Fill in the full name of your scene in InworldController.

3. Fill in the API key/secret of your scene in InworldClient.

4. Fill in the full name of your character in InworldCharacter's Data.

What's Changed

Our Unity Integration has had severals significant changes between v2 and v3. Chief among them being a modular transport layer for client-server communication that allows you to switch between using Unity Websockets as well as a NDK implementation that uses protobufs and grpc

Callback Changes

Our callbacks have also changed and been extended. Any scripts that are setup to add listeners to these callbacks will need to be refactored to account for updated names such as:

• OnStateChanged -> OnStatusChanged
• RuntimeEvent -> OnStatusChanged
• OnBeginSpeaking -> onBeginSpeaking
• OnFinishedSpeaking -> onEndSpeaking
• OnCharacterSpeaks -> onCharacterSpeaks
• OnGoalCompleted -> onGoalCompleted
• CustomEvent -> ClientTrigger The Inworld Character script has also been extended to have events for onPacketReceived and onEmotionChanged

The onPacketReceived event now exists on a per character basis in the InworldCharacter script as well. This event is invoked through the onPacketReceived event in the InworldClient script any time a packet is received from the server (the chain of events is represented in red in the above screenshot going from the Client script to the Interaction script and finally to the corresponding Character script).

The OnInteractionChanged method in the InworldCharacter script is added as a listener to the Action with the same name in the InworldInteraction script.

Migration Process

Note: Before proceeding with any changes, please make a copy or backup of your project to ensure you do not lose anything and to use as reference

Steps:

• Remove v2 of the Inworld AI integration from your project being careful not to remove any necessary assets, models, textures etc such as for ReadyPlayerMe characters.

• Add v3 of the Inworld AI integration to your project

• At this point you will have various missing script references

• Replace the obsolete PlayerController object in the relevant scenes and nested prefabs with one of the updated PlayerController prefabs which most suits your purpose

• Replace the obsolete Inworld Controller missing script reference with the one provided in v3 and ensure a valid client reference and scene Full name are set

• Ensure the Client script has the correct API key and secret corresponding to the Scene

• Replace the obsolete InworldCharacter missing script reference with the updated v3 InworldCharacter script or one of it's derived scripts which best suits your use case

• Ensure the Data field has a valid corresponding brain name for the chosen character which can be obtained from the url of your character on the Inworld Studio.

• Add Inworld Body Animation, and Facial animation scripts as necessary

• Add any missing Unity Event scene references again

• Resolve compile errors where necessary by refactoring any scripts using obsolete callbacks to utilize the updated and new Actions & Events detailed above accordingly

Changelog

1. Updated the interaction feature, make receiving data faster.
2. Add comments for all public properties and functions.

IMPORTANT NOTICE: This is a pre-release/beta version of the InworldAI Unity runtime SDK and is not intended for production use. This version is released for testing and development purposes only. We provide this version 'as-is' and make no guarantees or warranties of any kind, express or implied, about its completeness, accuracy, reliability, suitability or availability. Please make sure to backup your data regularly and expect bugs and issues. We encourage users to provide feedback and report any bugs or issues they encounter to help us improve the final version. Use this at your own risk.

Packages Overview

InworldAI.unitypackage

The default package using NDK and Protobuf as transport layer. Contains samples of InworldAI.Sample.RPM. The samples are all from the version 2, not only showcasing all the different features, but also ensuring a smooth upgrade path. Works in Windows only.

InworldAI.Lite.unitypackage

A basic 2D implementation for Inworld using websocket as transport layer. Supports all platforms.

Package Structure

Inworld Unity SDK package comprises the following modules:

Inworld.AI

This is the core module that encompasses all fundamental Inworld logic, WebSocket-based Inworld interactions, and a 2D sample. All other modules reference it.

Inworld.Assets

This module encompasses various Inworld functionalities implemented within Unity, such as 2D and 3D animations, fonts, logos, UI icons, Lipsync data, and more.

Inworld.NDK

This module constitutes Inworld communication implementation based on NDK (C++ based). It employs Protobuf for data transmission. It offers faster performance compared to WebSocket-based implementation, with lower bandwidth consumption. Additionally, it features an echo cancellation module. Currently limited to Windows support.

Inworld.Samples.RPM

This is a rewrite based on Inworld Unity SDK 3.0, built upon all the features and samples of the 2.0 version. It encompasses all functionalities of 2.0 and leverages Ready Player Me for character model implementation. It is not supported on WebGL.

Inworld.Samples.Innequin

This is a sample built upon Inworld's next-generation model, Innequin, and follows a distinct approach from Ready Player Me. It is based on .fbx models, making it suitable for all platforms.

Features

Each Unity package contains different modules. For specific details, please refer to the table below:

Packages	Inworld.AI	Inworld.Assets	Inworld.NDK	Inworld.Samples.RPM	Inworld.Samples.Innequin
Inworld.AI.Full	√	√	√	√	√
InworldAI.Lite	√

Compatibility

Platform	Inworld.AI	InworldAI.Lite	InworldAI.Sample.RPM	InworldAI.Sample.Innequin	Inworld v1	Inworld v2
Unity Minimum Version	2020.1	2020.1	2020.1	2020.1	2019.4	2021.3
Windows	√	√	√	√	√	√
Mac Intel 64-bit	×	√	√	√	Only in Editor	Only in Editor
Mac Apple Silicon	×	√	√	√	Only in Editor	Only in Editor
Android	×	√	√	√	√	√
Oculus	×	√	√	√	√	√
iOS	×	√	√	√	√	√
Linux	×	√	√	√	×	×
WebGL	×	√	×	√	×	×

How to use

1. Prepare your assets.

To integrate your own characters into your Unity project, you will need to ensure that you can access:

1. At least one Workspace.
2. At least one API Key in your workspace.
3. At least one Character in your workspace.
4. At least one Inworld Scene in your workspace.
5. Ensure that your Inworld Scene contains all the Characters you need to interact with.

If you do not know how to create them, please check our Prerequisities page.

2. Fill in the full name of your scene in InworldController.

3. Fill in the API key/secret of your scene in InworldClient.

4. Fill in the full name of your character in InworldCharacter's Data.

What's Changed

Our Unity Integration has had severals significant changes between v2 and v3. Chief among them being a modular transport layer for client-server communication that allows you to switch between using Unity Websockets as well as a NDK implementation that uses protobufs and grpc

Callback Changes

Our callbacks have also changed and been extended. Any scripts that are setup to add listeners to these callbacks will need to be refactored to account for updated names such as:

• OnStateChanged -> OnStatusChanged
• RuntimeEvent -> OnStatusChanged
• OnBeginSpeaking -> onBeginSpeaking
• OnFinishedSpeaking -> onEndSpeaking
• OnCharacterSpeaks -> onCharacterSpeaks
• OnGoalCompleted -> onGoalCompleted
• CustomEvent -> ClientTrigger The Inworld Character script has also been extended to have events for onPacketReceived and onEmotionChanged

The onPacketReceived event now exists on a per character basis in the InworldCharacter script as well. This event is invoked through the onPacketReceived event in the InworldClient script any time a packet is received from the server (the chain of events is represented in red in the above screenshot going from the Client script to the Interaction script and finally to the corresponding Character script).

The OnInteractionChanged method in the InworldCharacter script is added as a listener to the Action with the same name in the InworldInteraction script.

Migration Process

Note: Before proceeding with any changes, please make a copy or backup of your project to ensure you do not lose anything and to use as reference

Steps:

• Remove v2 of the Inworld AI integration from your project being careful not to remove any necessary assets, models, textures etc such as for ReadyPlayerMe characters.

• Add v3 of the Inworld AI integration to your project

• At this point you will have various missing script references

• Replace the obsolete PlayerController object in the relevant scenes and nested prefabs with one of the updated PlayerController prefabs which most suits your purpose

• Replace the obsolete Inworld Controller missing script reference with the one provided in v3 and ensure a valid client reference and scene Full name are set

• Ensure the Client script has the correct API key and secret corresponding to the Scene

• Replace the obsolete InworldCharacter missing script reference with the updated v3 InworldCharacter script or one of it's derived scripts which best suits your use case

• Ensure the Data field has a valid corresponding brain name for the chosen character which can be obtained from the url of your character on the Inworld Studio.

• Add Inworld Body Animation, and Facial animation scripts as necessary

• Add any missing Unity Event scene references again

• Resolve compile errors where necessary by refactoring any scripts using obsolete callbacks to utilize the updated and new Actions & Events detailed above accordingly

Changelog

1. Add NDK Support for Mac, Android (v8a only), iOS
2. Updated Audio samplings, with Audio test scenes.
3. UI Improves.
4. Bug fixes.

IMPORTANT NOTICE: This is a pre-release/beta version of the InworldAI Unity runtime SDK and is not intended for production use. This version is released for testing and development purposes only. We provide this version 'as-is' and make no guarantees or warranties of any kind, express or implied, about its completeness, accuracy, reliability, suitability or availability. Please make sure to backup your data regularly and expect bugs and issues. We encourage users to provide feedback and report any bugs or issues they encounter to help us improve the final version. Use this at your own risk.

Packages Overview

InworldAI.unitypackage

The default package using NDK and Protobuf as transport layer. Contains samples of InworldAI.Sample.RPM. The samples are all from the version 2, not only showcasing all the different features, but also ensuring a smooth upgrade path. Works in Windows only.

InworldAI.Lite.unitypackage

A basic 2D implementation for Inworld using websocket as transport layer. Supports all platforms.

Package Structure

Inworld Unity SDK package comprises the following modules:

Inworld.AI

This is the core module that encompasses all fundamental Inworld logic, WebSocket-based Inworld interactions, and a 2D sample. All other modules reference it.

Inworld.Assets

This module encompasses various Inworld functionalities implemented within Unity, such as 2D and 3D animations, fonts, logos, UI icons, Lipsync data, and more.

Inworld.NDK

This module constitutes Inworld communication implementation based on NDK (C++ based). It employs Protobuf for data transmission. It offers faster performance compared to WebSocket-based implementation, with lower bandwidth consumption. Additionally, it features an echo cancellation module. Currently limited to Windows support.

Inworld.Samples.RPM

This is a rewrite based on Inworld Unity SDK 3.0, built upon all the features and samples of the 2.0 version. It encompasses all functionalities of 2.0 and leverages Ready Player Me for character model implementation. It is not supported on WebGL.

Inworld.Samples.Innequin

This is a sample built upon Inworld's next-generation model, Innequin, and follows a distinct approach from Ready Player Me. It is based on .fbx models, making it suitable for all platforms.

Features

Each Unity package contains different modules. For specific details, please refer to the table below:

Packages	Inworld.AI	Inworld.Assets	Inworld.NDK	Inworld.Samples.RPM	Inworld.Samples.Innequin
Inworld.AI.Full	√	√	√	√	√
InworldAI.Lite	√

Compatibility

Platform	Inworld.AI	InworldAI.Lite	InworldAI.Sample.RPM	InworldAI.Sample.Innequin	Inworld v1	Inworld v2
Unity Minimum Version	2020.1	2020.1	2020.1	2020.1	2019.4	2021.3
Windows	√	√	√	√	√	√
Mac Intel 64-bit	×	√	√	√	Only in Editor	Only in Editor
Mac Apple Silicon	×	√	√	√	Only in Editor	Only in Editor
Android	×	√	√	√	√	√
Oculus	×	√	√	√	√	√
iOS	×	√	√	√	√	√
Linux	×	√	√	√	×	×
WebGL	×	√	×	√	×	×

How to use

1. Prepare your assets.

To integrate your own characters into your Unity project, you will need to ensure that you can access:

1. At least one Workspace.
2. At least one API Key in your workspace.
3. At least one Character in your workspace.
4. At least one Inworld Scene in your workspace.
5. Ensure that your Inworld Scene contains all the Characters you need to interact with.

If you do not know how to create them, please check our Prerequisities page.

2. Fill in the full name of your scene in InworldController.

3. Fill in the API key/secret of your scene in InworldClient.

4. Fill in the full name of your character in InworldCharacter's Data.

What's Changed

Our Unity Integration has had severals significant changes between v2 and v3. Chief among them being a modular transport layer for client-server communication that allows you to switch between using Unity Websockets as well as a NDK implementation that uses protobufs and grpc

Callback Changes

Our callbacks have also changed and been extended. Any scripts that are setup to add listeners to these callbacks will need to be refactored to account for updated names such as:

• OnStateChanged -> OnStatusChanged
• RuntimeEvent -> OnStatusChanged
• OnBeginSpeaking -> onBeginSpeaking
• OnFinishedSpeaking -> onEndSpeaking
• OnCharacterSpeaks -> onCharacterSpeaks
• OnGoalCompleted -> onGoalCompleted
• CustomEvent -> ClientTrigger The Inworld Character script has also been extended to have events for onPacketReceived and onEmotionChanged

The onPacketReceived event now exists on a per character basis in the InworldCharacter script as well. This event is invoked through the onPacketReceived event in the InworldClient script any time a packet is received from the server (the chain of events is represented in red in the above screenshot going from the Client script to the Interaction script and finally to the corresponding Character script).

The OnInteractionChanged method in the InworldCharacter script is added as a listener to the Action with the same name in the InworldInteraction script.

Migration Process

Note: Before proceeding with any changes, please make a copy or backup of your project to ensure you do not lose anything and to use as reference

Steps:

• Remove v2 of the Inworld AI integration from your project being careful not to remove any necessary assets, models, textures etc such as for ReadyPlayerMe characters.

• Add v3 of the Inworld AI integration to your project

• At this point you will have various missing script references

• Replace the obsolete PlayerController object in the relevant scenes and nested prefabs with one of the updated PlayerController prefabs which most suits your purpose

• Replace the obsolete Inworld Controller missing script reference with the one provided in v3 and ensure a valid client reference and scene Full name are set

• Ensure the Client script has the correct API key and secret corresponding to the Scene

• Replace the obsolete InworldCharacter missing script reference with the updated v3 InworldCharacter script or one of it's derived scripts which best suits your use case

• Ensure the Data field has a valid corresponding brain name for the chosen character which can be obtained from the url of your character on the Inworld Studio.

• Add Inworld Body Animation, and Facial animation scripts as necessary

• Add any missing Unity Event scene references again

• Resolve compile errors where necessary by refactoring any scripts using obsolete callbacks to utilize the updated and new Actions & Events detailed above accordingly