A fast and minimal framework for building agent-integrated systems
Includes Parameterization of Max Workers in ResourceManager #185 from @migueldl96 🎉
Updates AMQPSpace
to use a topic exchange for better observability. Thanks @wwj718! 🙏
I'm proud to announce Agency v1.6.0. Technically, it represents a big step forward and I believe is a milestone in terms of clarity for the project.
It took some iterating and lots of thoughtful feedback, but I think the learnings from the last few versions have come together to create a clearer picture for the API and architecture. I believe there is a good foundation now to focus on the next steps: js client, code execution, and multimodal support.
Special thanks to @wwj718 and @hidaris for their reviews and feedback that helped shape this release!
As with any major release, please do not hesitate to open issues or discussions if you have any questions or feedback. 🙇♂️
Space
types now support mixed threading/multiprocessing for agentsVersion 1.4 introduced multiprocessing but did not allow threading and multiprocessing based agents to be used together. Additionally the API prevented some use cases that were previously possible, due to not allowing direct access to agents from the main thread.
In 1.6.0, all agents are now add
'ed using multiprocessing
based subprocesses by default. Agents added this way may not be directly accessed, similar to 1.4.
With 1.6.0 you now have the option to run agents in the "foreground" using Space.add_foreground
. When adding an agent this way, an instance of the agent is returned in the current thread. For example:
my_agent = my_space.add_foreground(MyAgent, "my_agent")
The tradeoff for adding an agent in the foreground is performance. Due to the python GIL, agents running in the foreground may not be fully parallel with other agents, and may block the main thread.
It's recommended to use Space.add_foreground
only when needed.
MultiprocessSpace
and ThreadSpace
classes replaced with LocalSpace
Since all Space
types now support mixed threading/multiprocessing, LocalSpace
combines these older types into a single class for in-application communication.
Space.destroy
method addedThese two additions allow for improved resource management. Creating and cleaning up a Space
can now be done using the context manager syntax:
with AMQPSpace() as space:
space.add(Host, "Host")
...
This form will automatically clean up Space related resources upon exit of the with
block.
Direct control of cleanup is also possible using the Space.destroy
method.
space = AMQPSpace()
space.add(Host, "Host")
...
space.destroy()
Agent.respond_with
and Agent.raise_with
methods addedThese methods are intended for use with Agent.request
and the response callbacks: Agent.handle_action_value
and Agent.handle_action_error
.
Automatic response behavior has changed from v1.5. Actions will no longer automatically return their return value to the sender.
To explicitly return a value to the sender associated with the current message, use Agent.respond_with
. For example:
@action
def ping(self):
self.respond_with("pong")
The above will invoke the handle_action_value
callback on the sender with the value "pong"
, or return the value "pong"
directly when used with the request
method.
Similarly you may return an error with:
@action
def ping(self):
self.raise_with(Exception("oops"))
Actions that raise exceptions will automatically call the raise_with
method.
Note that respond_with
and raise_with
may be called multiple times in a single action. Each call will invoke the appropriate callback on the sender.
When using request
the first call to respond_with
will return the value to the calling thread. Subsequent responses will invoke the handle_action_value
callback.
All busy waiting has been replaced with event based processing. The test suite completes about 80% faster than in 1.5 and CPU overhead should now be minimized.
Agent.__init__
signature updatedThe Agent constructor has been reverted to its older signature which does not include a Queue
parameter. Make sure to update your classes.
def __init__(self, id: str, receive_own_broadcasts: bool = True):
This release addresses the request related issues brought up here regarding the use of multiple "meta" id's and the semantics of automatic responses to messages. Shoutout to @wwj718 for the great feedback for these updates!
Since this includes breaking changes, the version has been bumped to 1.5.0.
All messages now populate the meta.id
field with a UUID by default. Agents may set their own custom meta.id
upon sending to overwrite the generated ID.
In addition, Agent.send()
now returns the meta.id
of the sent message. This way you may store the generated ID for your own purposes.
Removed the use of meta.response_id
and meta.request_id
. These have been replaced by meta.parent_id
. I think the term "parent" is clearer. This choice is inspired by the Jupyter messaging docs.
To be clear, meta.parent_id
if set, refers to the meta.id
of the original message which the current message is responding to.
To illustrate, the following would be typical meta fields in a send/response pair:
# send
{
"meta": {
"id": "123", # the message ID here
},
...
}
# response
{
"meta": {
"id": "456", # response has its own unique meta.id
"parent_id": "123", # refers to the prior message
},
...
}
All messages will now result in a [response]
message like the above, returning the return value or error result from the message's action. These messages (unless using the request()
method) will invoke the handle_action_value()
and handle_action_error()
methods appropriately.
One might want to filter these messages from view in a user interface. The demo Gradio app has been updated to demonstrate this. See the _should_render()
and get_chatbot_message()
methods for an example.
Agent.original_message()
has been renamed to Agent.parent_message()
. Additionally parent_message()
takes a message as a parameter, allowing you to traverse a chain of messages in reverse.
Internally renamed Agent._pending_responses
to Agent._pending_requests
which I think is clearer.
The logger has been updated to provide colorization of output for easier reading. You may change the default "monokai" pygments style used for object output, by setting the LOG_PYGMENTS_STYLE
environment variable.
Log "warning" messages for unimplemented handle_action_value
and handle_action_error
methods have been updated to only print once per session, in order to reduce logging noise in the default setting.
Tests and documentation updated.
Fixes a regression in the 1.4.0 release related to the AMQP exchange parameter
This release brings multiprocessing support and a few nice API improvements. Notably, I've added an implementation of synchronous messaging with the new request()
method, and I've introduced logging support for development and debugging.
There are a number of breaking API changes in this release. Please read through the following summary to update your codebase.
Since this update required a rather large internal change, please beware of bugs and report any issues that you find. ❤️
Space
Class ChangesSpace.add()
Space.add()
no longer accepts an Agent
instance as an argument. Spaces will now create the Agent
instance for you when adding them to the space.
An example of the updated syntax:
space.add(MyAgent, "my_agent_id")
Note how it accepts the Agent
type. Any remaining parameters starting with the id
are passed to the type's constructor when creating the agent.
MultiprocessSpace
addedThe new MultiprocessSpace
class allows for better parallelism on a single host. Adding this option was the initial focus of this update. It uses the multiprocessing library for both concurrency and communication.
NativeSpace
renamed to ThreadSpace
The NativeSpace
class has been renamed to ThreadSpace
for clarity.
AMQPSpace
updated to use multiprocessing for local concurrencySince multiprocessing is now supported, it made sense to use it rather than threading with the AMQPSpace
class for local concurrency. This means that you do not need to define separate applications to achieve parallelism on a single host when using AMQP.
Agent
Class ChangesAgent.request()
To allow calling actions synchronously, the Agent
class now includes a request()
method.
For example:
try:
return_value = self.request({
"to": "ExampleAgent",
"action": {
"name": "example_action",
"args": {
"content": "hello"
}
}
}, timeout=5)
except ActionError as e:
print(e.message)
The Agent.request()
method is a synchronous version of the send()
method that allows you to call an action and receive its return value or exception synchronously.
response()
and error()
renamed to handle_action_value()
and handle_action_error()
Additionally, these methods are no longer implemented as actions. You should remove the @action
decorator from these methods when you update them.
To clarify moving forward, an action (a method decorated by @action
) should be considered a public method available to other agents. Callbacks are called upon certain events, rather than being directly invoked by an agent.
In the cases of handle_action_value()
and handle_action_error()
, these callbacks are invoked when a previously sent action returns a value or raises an exception.
Also note that their signatures have removed the original_message_id
argument and simplified to:
def handle_action_value(self, value):
and
def handle_action_error(self, error: ActionError):
See the original_message()
method for how the original message may now be accessed.
Agent.current_message()
is now a methodAgent.current_message()
may be called during any action or action related callback to inspect the full message being processed.
Agent.original_message()
addedSimilar to current_message()
, original_message()
may be called during the handle_action_value()
and handle_action_error()
callbacks to inspect the original message that the current message is responding to.
The original message must have defined the meta.id
field for this method to return it. Otherwise, this method will return None.
id
field to meta.id
The id
field has been moved within the meta
field to be treated similarly as any other metadata.
If set, the id
field may be used to correlate response messages (internal messages that return a value or exception) with the original message they correspond to. See the original_message()
method for more details.
action.args
is now optionalaction.args
can be omitted when calling an action that takes no arguments. For example, given the following action:
@action
def example_action(self):
...
The following message is valid:
self.send({
"to": "ExampleAgent",
"action": {
"name": "example_action"
}
})
Beware that this may cause errors if your code currently inspects messages and expects the action.args
field to be present.
Some basic logging has been added. Set the LOGLEVEL
environment variable to see the logs.
The default setting is warning
. info
will show messages as they are being sent. debug
shows more fine grained handling of messages as they are received and processed.
If you'd like to add more logging or make changes, please open a PR or issue.
A multiprocess
example has been added.
Run demo run multiprocess
to try it out
The native
service has been renamed to threaded
The request_permission
callback on Host
class has been bypassed temporarily.
Asking for terminal input in a multiprocess application doesn't work well yet. A future update should address this. So for the time being, all actions on the host will be allowed.
Unfortunately gradio live reloading also had to be disabled. You'll have to restart the app for code changes to take effect. If either of the above changes cause problems for you, please open an issue.
Fixes an issue that caused high CPU usage when using the AMQPSpace class. Thanks @wwj718!
This release brings many API and internal improvements. I've incorporated a lot of the great feedback I've been getting and improved multiple areas in terms of clarity, organization, and implementation.
What I'm most excited about is that the API feels like it's beginning to reach a relatively clean and extensible form, enabling more possibilities to follow.
A number of syntactic and semantic changes have been made. I've summarized the changes here, but for full details also refer to the updated docs/ or readme.
As always please let me know what you think, especially if you find any issues!
Introduced a new @action
decorator based syntax for defining actions and their metadata, replacing the @access_policy
decorator. For example:
@action
def my_action(self):
...
An example showing how the access_policy
is now declared:
@action(access_policy=ACCESS_PERMITTED)
def my_action(self):
...
Additionally, the _action__
prefix is no longer needed on action method names.
Agent id
's are now enforced to be unique.
Allowing duplicate id
's creates more problems than it solves. If one needs to duplicate or distribute messages to multiple consumers, that can be achieved by simply sending multiple messages from the agent.
Leading underscores removed from several agent methods and callbacks.
Moving forward, methods with leading underscores should be treated as internal API methods and should generally not be referenced in except when overriding methods or extending a class. To follow this convention, the following methods have been renamed:
Agent
class methods:
_send
is now send
_before_add
is now before_add
_after_add
is now after_add
_before_remove
is now before_remove
_after_remove
is now after_remove
_request_permission
is now request_permission
Additionally, the special action return
is now renamed to response
.
The signatures for response
(formerly return
) and error
have changed to the following:
@action
def response(self, data, original_message_id: str):
...
@action
def error(self, error: str, original_message_id: str):
...
Note that the original_message
argument is replaced with the original_message_id
parameter. This parameter is discussed with the message schema changes below.
Updated help
functionality
The help
action now returns a new schema for describing actions. The data structure returned is now a dictionary similar to the following example:
{
"say": {
"description": "Say something to this agent",
"args": {
"content": {
"type": "string"
"description": "The content to say"
}
},
"returns": {
"type": "string"
"description": "A response"
}
},
...
}
The help structure above is automatically generated from the method's docstring and signature. Additionally, you can provide a custom help schema using the @action
decorator.
See the "Defining Actions" section for more details on the help schema.
Space.remove_all()
method added for removing all agents who were added through the receiving space instance. In other words, in an AMQP distributed system, only the locally added agents would be removed.
Added an optional id
field. This field is automatically provided with response
or error
messages for you to correlate with the original message.
Added an optional meta
field. This field is an optional dictionary of key-value pairs for you to attach metadata to a message. For example, you may use this to store timestamps, or a "thoughts" field for recording reasoning.
The to
field is now required. To send a broadcast use the special address *
.
The action
field is now an object containing the name
and args
fields.
The full schema is summarized with this example:
{
"id": "optional string id",
"meta": {
"an": "optional",
"field": ["for", "metadata"]
},
"to": "an agent id", # or '*' to broadcast
"from": "sender's id",
"action": {
"name": "my_action",
"args": {
"the": "args"
}
}
}
See the docs "Schema" section for more details.
Broadcasts are now self received by default. This is the normal semantics for broadcasts.
This behavior is now an option on the Agent
class: receive_own_broadcasts: bool = True
.
Setting it to False
will restore the previous behavior.
Messages sent to a non-existent agent will now be silently ignored. Previously this would result in an error message being returned. Note that calling a non-existent action on an existent agent will still result in an error message.
This change brings us more in line with the Actor model. Reliability options may be added at the Agent level in the future.
Removed colorama
dependency
Removed use of id_queue
in AMQPSpace
for checking agent existence.
Space._route()
no longer sets the from
field on messages. The entire
message must be populated within the Agent instance before calling _route()
.
Threading improvements
Threading code has been greatly cleaned up and improved. The updated implementation should make it easy to support additional forms of multi-processing like green threads and the multiprocessing module. I'm really happy about this change in particular.
Space classes (NativeSpace
and AMQPSpace
) moved to the agency.spaces
namespace.
Tests have been better organized into separate files.
The Space
class abstract methods have been changed. Space
subclasses now implement the following methods:
_connect()
_disconnect()
_deliver()
_consume()
This is a change from the previous interface. See the Space
class for more details.
OpenAIFunctionAgent
now supplies argument descriptions to the OpenAI function calling API, enabled by the new help
information schema.
Agent mixins have been refactored to better separate behavior.
Updated slash syntax in the UI. Slash commands now follow this example:
/agent_id.action_name arg1:value1 arg2:value2 ...
The agent_id
above defines the to
field of the message and is required.
To broadcast an action, use the syntax:
/*.action_name arg1:value1 arg2:value2 ...
Note that unformatted text is still considered a broadcasted say
action.
That's all for now!
WebApp
class to ReactApp
../examples/demo/apps
directory.Host
and OpenAIFunctionAgent
by
default for a better initial experience.HelpMethods
, PromptMethods
) to a ./mixins
subdirectoryI'm excited to add a Gradio UI!
To implement the chat interface I used Gradio's Chatbot
component. It supports some limited markdown formatting, making the output look a lot nicer especially when there are code samples. The downside was that I had to use a form of polling to get the UI to update on new messages, and write some custom css to make it look nicer. But hopefully we can improve this more along the way.
Streaming output is not yet implemented but is possible. I'll be adding a separate issue for that.
As far as being a base for multimodal features - The chatbot already supports display of some multimedia content, though this is not yet implemented in the demo. So I think it's a good foundation to start and we can experiment with using components alongside the chat component for more complex interactions when multimodal features are added.
As always I'm eager to hear your thoughts if you have any. Thanks!
Fixes a bug in the Agent._help
method. https://github.com/operand/agency/pull/86