A fast and minimal framework for building agent-integrated systems
Includes Parameterization of Max Workers in ResourceManager #185 from @migueldl96 🎉
Updates AMQPSpace to use a topic exchange for better observability. Thanks @wwj718! 🙏
I'm proud to announce Agency v1.6.0. Technically, it represents a big step forward and I believe is a milestone in terms of clarity for the project.
It took some iterating and lots of thoughtful feedback, but I think the learnings from the last few versions have come together to create a clearer picture for the API and architecture. I believe there is a good foundation now to focus on the next steps: js client, code execution, and multimodal support.
Special thanks to @wwj718 and @hidaris for their reviews and feedback that helped shape this release!
As with any major release, please do not hesitate to open issues or discussions if you have any questions or feedback. 🙇♂️
Space types now support mixed threading/multiprocessing for agentsVersion 1.4 introduced multiprocessing but did not allow threading and multiprocessing based agents to be used together. Additionally the API prevented some use cases that were previously possible, due to not allowing direct access to agents from the main thread.
In 1.6.0, all agents are now add'ed using multiprocessing based subprocesses by default. Agents added this way may not be directly accessed, similar to 1.4.
With 1.6.0 you now have the option to run agents in the "foreground" using Space.add_foreground. When adding an agent this way, an instance of the agent is returned in the current thread. For example:
my_agent = my_space.add_foreground(MyAgent, "my_agent")
The tradeoff for adding an agent in the foreground is performance. Due to the python GIL, agents running in the foreground may not be fully parallel with other agents, and may block the main thread.
It's recommended to use Space.add_foreground only when needed.
MultiprocessSpace and ThreadSpace classes replaced with LocalSpaceSince all Space types now support mixed threading/multiprocessing, LocalSpace combines these older types into a single class for in-application communication.
Space.destroy method addedThese two additions allow for improved resource management. Creating and cleaning up a Space can now be done using the context manager syntax:
with AMQPSpace() as space:
space.add(Host, "Host")
...
This form will automatically clean up Space related resources upon exit of the with block.
Direct control of cleanup is also possible using the Space.destroy method.
space = AMQPSpace()
space.add(Host, "Host")
...
space.destroy()
Agent.respond_with and Agent.raise_with methods addedThese methods are intended for use with Agent.request and the response callbacks: Agent.handle_action_value and Agent.handle_action_error.
Automatic response behavior has changed from v1.5. Actions will no longer automatically return their return value to the sender.
To explicitly return a value to the sender associated with the current message, use Agent.respond_with. For example:
@action
def ping(self):
self.respond_with("pong")
The above will invoke the handle_action_value callback on the sender with the value "pong", or return the value "pong" directly when used with the request method.
Similarly you may return an error with:
@action
def ping(self):
self.raise_with(Exception("oops"))
Actions that raise exceptions will automatically call the raise_with method.
Note that respond_with and raise_with may be called multiple times in a single action. Each call will invoke the appropriate callback on the sender.
When using request the first call to respond_with will return the value to the calling thread. Subsequent responses will invoke the handle_action_value callback.
All busy waiting has been replaced with event based processing. The test suite completes about 80% faster than in 1.5 and CPU overhead should now be minimized.
Agent.__init__ signature updatedThe Agent constructor has been reverted to its older signature which does not include a Queue parameter. Make sure to update your classes.
def __init__(self, id: str, receive_own_broadcasts: bool = True):
This release addresses the request related issues brought up here regarding the use of multiple "meta" id's and the semantics of automatic responses to messages. Shoutout to @wwj718 for the great feedback for these updates!
Since this includes breaking changes, the version has been bumped to 1.5.0.
All messages now populate the meta.id field with a UUID by default. Agents may set their own custom meta.id upon sending to overwrite the generated ID.
In addition, Agent.send() now returns the meta.id of the sent message. This way you may store the generated ID for your own purposes.
Removed the use of meta.response_id and meta.request_id. These have been replaced by meta.parent_id. I think the term "parent" is clearer. This choice is inspired by the Jupyter messaging docs.
To be clear, meta.parent_id if set, refers to the meta.id of the original message which the current message is responding to.
To illustrate, the following would be typical meta fields in a send/response pair:
# send
{
"meta": {
"id": "123", # the message ID here
},
...
}
# response
{
"meta": {
"id": "456", # response has its own unique meta.id
"parent_id": "123", # refers to the prior message
},
...
}
All messages will now result in a [response] message like the above, returning the return value or error result from the message's action. These messages (unless using the request() method) will invoke the handle_action_value() and handle_action_error() methods appropriately.
One might want to filter these messages from view in a user interface. The demo Gradio app has been updated to demonstrate this. See the _should_render() and get_chatbot_message() methods for an example.
Agent.original_message() has been renamed to Agent.parent_message(). Additionally parent_message() takes a message as a parameter, allowing you to traverse a chain of messages in reverse.
Internally renamed Agent._pending_responses to Agent._pending_requests which I think is clearer.
The logger has been updated to provide colorization of output for easier reading. You may change the default "monokai" pygments style used for object output, by setting the LOG_PYGMENTS_STYLE environment variable.
Log "warning" messages for unimplemented handle_action_value and handle_action_error methods have been updated to only print once per session, in order to reduce logging noise in the default setting.
Tests and documentation updated.
Fixes a regression in the 1.4.0 release related to the AMQP exchange parameter
This release brings multiprocessing support and a few nice API improvements. Notably, I've added an implementation of synchronous messaging with the new request() method, and I've introduced logging support for development and debugging.
There are a number of breaking API changes in this release. Please read through the following summary to update your codebase.
Since this update required a rather large internal change, please beware of bugs and report any issues that you find. ❤️
Space Class ChangesSpace.add()Space.add() no longer accepts an Agent instance as an argument. Spaces will now create the Agent instance for you when adding them to the space.
An example of the updated syntax:
space.add(MyAgent, "my_agent_id")
Note how it accepts the Agent type. Any remaining parameters starting with the id are passed to the type's constructor when creating the agent.
MultiprocessSpace addedThe new MultiprocessSpace class allows for better parallelism on a single host. Adding this option was the initial focus of this update. It uses the multiprocessing library for both concurrency and communication.
NativeSpace renamed to ThreadSpaceThe NativeSpace class has been renamed to ThreadSpace for clarity.
AMQPSpace updated to use multiprocessing for local concurrencySince multiprocessing is now supported, it made sense to use it rather than threading with the AMQPSpace class for local concurrency. This means that you do not need to define separate applications to achieve parallelism on a single host when using AMQP.
Agent Class ChangesAgent.request()To allow calling actions synchronously, the Agent class now includes a request() method.
For example:
try:
return_value = self.request({
"to": "ExampleAgent",
"action": {
"name": "example_action",
"args": {
"content": "hello"
}
}
}, timeout=5)
except ActionError as e:
print(e.message)
The Agent.request() method is a synchronous version of the send() method that allows you to call an action and receive its return value or exception synchronously.
response() and error() renamed to handle_action_value() and handle_action_error()Additionally, these methods are no longer implemented as actions. You should remove the @action decorator from these methods when you update them.
To clarify moving forward, an action (a method decorated by @action) should be considered a public method available to other agents. Callbacks are called upon certain events, rather than being directly invoked by an agent.
In the cases of handle_action_value() and handle_action_error(), these callbacks are invoked when a previously sent action returns a value or raises an exception.
Also note that their signatures have removed the original_message_id argument and simplified to:
def handle_action_value(self, value):
and
def handle_action_error(self, error: ActionError):
See the original_message() method for how the original message may now be accessed.
Agent.current_message() is now a methodAgent.current_message() may be called during any action or action related callback to inspect the full message being processed.
Agent.original_message() addedSimilar to current_message(), original_message() may be called during the handle_action_value() and handle_action_error() callbacks to inspect the original message that the current message is responding to.
The original message must have defined the meta.id field for this method to return it. Otherwise, this method will return None.
id field to meta.idThe id field has been moved within the meta field to be treated similarly as any other metadata.
If set, the id field may be used to correlate response messages (internal messages that return a value or exception) with the original message they correspond to. See the original_message() method for more details.
action.args is now optionalaction.args can be omitted when calling an action that takes no arguments. For example, given the following action:
@action
def example_action(self):
...
The following message is valid:
self.send({
"to": "ExampleAgent",
"action": {
"name": "example_action"
}
})
Beware that this may cause errors if your code currently inspects messages and expects the action.args field to be present.
Some basic logging has been added. Set the LOGLEVEL environment variable to see the logs.
The default setting is warning. info will show messages as they are being sent. debug shows more fine grained handling of messages as they are received and processed.
If you'd like to add more logging or make changes, please open a PR or issue.
A multiprocess example has been added.
Run demo run multiprocess to try it out
The native service has been renamed to threaded
The request_permission callback on Host class has been bypassed temporarily.
Asking for terminal input in a multiprocess application doesn't work well yet. A future update should address this. So for the time being, all actions on the host will be allowed.
Unfortunately gradio live reloading also had to be disabled. You'll have to restart the app for code changes to take effect. If either of the above changes cause problems for you, please open an issue.
Fixes an issue that caused high CPU usage when using the AMQPSpace class. Thanks @wwj718!
This release brings many API and internal improvements. I've incorporated a lot of the great feedback I've been getting and improved multiple areas in terms of clarity, organization, and implementation.
What I'm most excited about is that the API feels like it's beginning to reach a relatively clean and extensible form, enabling more possibilities to follow.
A number of syntactic and semantic changes have been made. I've summarized the changes here, but for full details also refer to the updated docs/ or readme.
As always please let me know what you think, especially if you find any issues!
Introduced a new @action decorator based syntax for defining actions and their metadata, replacing the @access_policy decorator. For example:
@action
def my_action(self):
...
An example showing how the access_policy is now declared:
@action(access_policy=ACCESS_PERMITTED)
def my_action(self):
...
Additionally, the _action__ prefix is no longer needed on action method names.
Agent id's are now enforced to be unique.
Allowing duplicate id's creates more problems than it solves. If one needs to duplicate or distribute messages to multiple consumers, that can be achieved by simply sending multiple messages from the agent.
Leading underscores removed from several agent methods and callbacks.
Moving forward, methods with leading underscores should be treated as internal API methods and should generally not be referenced in except when overriding methods or extending a class. To follow this convention, the following methods have been renamed:
Agent class methods:
_send is now send
_before_add is now before_add
_after_add is now after_add
_before_remove is now before_remove
_after_remove is now after_remove
_request_permission is now request_permission
Additionally, the special action return is now renamed to response.
The signatures for response(formerly return) and error have changed to the following:
@action
def response(self, data, original_message_id: str):
...
@action
def error(self, error: str, original_message_id: str):
...
Note that the original_message argument is replaced with the original_message_id parameter. This parameter is discussed with the message schema changes below.
Updated help functionality
The help action now returns a new schema for describing actions. The data structure returned is now a dictionary similar to the following example:
{
"say": {
"description": "Say something to this agent",
"args": {
"content": {
"type": "string"
"description": "The content to say"
}
},
"returns": {
"type": "string"
"description": "A response"
}
},
...
}
The help structure above is automatically generated from the method's docstring and signature. Additionally, you can provide a custom help schema using the @action decorator.
See the "Defining Actions" section for more details on the help schema.
Space.remove_all() method added for removing all agents who were added through the receiving space instance. In other words, in an AMQP distributed system, only the locally added agents would be removed.
Added an optional id field. This field is automatically provided with response or error messages for you to correlate with the original message.
Added an optional meta field. This field is an optional dictionary of key-value pairs for you to attach metadata to a message. For example, you may use this to store timestamps, or a "thoughts" field for recording reasoning.
The to field is now required. To send a broadcast use the special address *.
The action field is now an object containing the name and args fields.
The full schema is summarized with this example:
{
"id": "optional string id",
"meta": {
"an": "optional",
"field": ["for", "metadata"]
},
"to": "an agent id", # or '*' to broadcast
"from": "sender's id",
"action": {
"name": "my_action",
"args": {
"the": "args"
}
}
}
See the docs "Schema" section for more details.
Broadcasts are now self received by default. This is the normal semantics for broadcasts.
This behavior is now an option on the Agent class: receive_own_broadcasts: bool = True.
Setting it to False will restore the previous behavior.
Messages sent to a non-existent agent will now be silently ignored. Previously this would result in an error message being returned. Note that calling a non-existent action on an existent agent will still result in an error message.
This change brings us more in line with the Actor model. Reliability options may be added at the Agent level in the future.
Removed colorama dependency
Removed use of id_queue in AMQPSpace for checking agent existence.
Space._route() no longer sets the from field on messages. The entire
message must be populated within the Agent instance before calling _route().
Threading improvements
Threading code has been greatly cleaned up and improved. The updated implementation should make it easy to support additional forms of multi-processing like green threads and the multiprocessing module. I'm really happy about this change in particular.
Space classes (NativeSpace and AMQPSpace) moved to the agency.spaces namespace.
Tests have been better organized into separate files.
The Space class abstract methods have been changed. Space subclasses now implement the following methods:
_connect()
_disconnect()
_deliver()
_consume()
This is a change from the previous interface. See the Space class for more details.
OpenAIFunctionAgent now supplies argument descriptions to the OpenAI function calling API, enabled by the new help information schema.
Agent mixins have been refactored to better separate behavior.
Updated slash syntax in the UI. Slash commands now follow this example:
/agent_id.action_name arg1:value1 arg2:value2 ...
The agent_id above defines the to field of the message and is required.
To broadcast an action, use the syntax:
/*.action_name arg1:value1 arg2:value2 ...
Note that unformatted text is still considered a broadcasted say action.
That's all for now!
WebApp class to ReactApp../examples/demo/apps directory.Host and OpenAIFunctionAgent by
default for a better initial experience.HelpMethods, PromptMethods) to a ./mixins subdirectoryI'm excited to add a Gradio UI!
To implement the chat interface I used Gradio's Chatbot component. It supports some limited markdown formatting, making the output look a lot nicer especially when there are code samples. The downside was that I had to use a form of polling to get the UI to update on new messages, and write some custom css to make it look nicer. But hopefully we can improve this more along the way.
Streaming output is not yet implemented but is possible. I'll be adding a separate issue for that.
As far as being a base for multimodal features - The chatbot already supports display of some multimedia content, though this is not yet implemented in the demo. So I think it's a good foundation to start and we can experiment with using components alongside the chat component for more complex interactions when multimodal features are added.
As always I'm eager to hear your thoughts if you have any. Thanks!
Fixes a bug in the Agent._help method. https://github.com/operand/agency/pull/86