βοΈ Build multimodal AI applications with cloud-native stack
3.22.4)Release time: 2023-10-31 14:33:21
This release contains 1 bug fix.
In order to allow complex nested documents to pass through streaming endpoints, the GET endpoint can now get the data from the body or the parameters of the HTTP request.
We would like to thank all contributors to this release:
3.22.3)Release time: 2023-10-25 12:03:58
This release contains 1 bug fix.
HTTP streaming when using a GET endpoint only allows Documents whose schemas contain only str, int, and float fields.
By adding a POST endpoint and modifying the Jina Client to use it, it is now possible to send Documents with complex nested document schemas over streaming HTTP.
We would like to thank all contributors to this release:
3.22.2)Release time: 2023-10-20 14:42:15
This release contains 1 bug fix and 5 documentation improvements.
A bug was found that prevented Flows from starting when they had Executors with different but compatible schemas. This is now fixed.
We would like to thank all contributors to this release:
3.22.1)Release time: 2023-10-06 13:52:45
This release contains 1 bug fix.
When using dynamic batching (either with the @dynamic_batching decorator on an Executor endpoint or by setting the uses_dynamic_batching parameter on a Deployment, timeouts were not being properly taken into account because the timer might not be triggered until a previous batch finished. This has been fixed.
We would like to thank all contributors to this release:
3.22.0)Release time: 2023-10-04 12:16:53
This release contains 1 new feature and 2 documentation improvements.
This release implements some changes in the way we handle dynamic batching:
preferred_batch_size documents.We would like to thank all contributors to this release:
3.21.1)Release time: 2023-09-29 10:45:55
This release contains 1 bug fix.
When multiple Executors inside a Flow had the same model name and they were used as docker containers or had set the environment variable JINA_MP_START_METHOD=spawn, there was a problem serving them via HTTP. This has been fixed by making sure that only one model instance is served for each model name.
We would like to thank all contributors to this release:
3.21.0)Release time: 2023-09-26 15:54:28
This release contains 1 new feature, 4 bug fixes, and 1 documentation improvement.
return_type parameter to gateway streamer methods (#6027)By default, GatewayStreamer will fetch executor input and output schemas and reconstruct docarray models dynamically. The output schemas will be used to cast the responses at the gateway level.
Although the cast responses are nearly identical to the original schemas defined at the executor level, they might fail some checks. For example, if the gateway receives a document doc from an executor with output MyDoc, the following check will fail:
assert isinstance(doc, MyDoc)
Similarly, adding doc to a DocList[MyDoc] will fail the type checks.
To prevent this, the user of a GatewayStreamer can now use the return_type parameter to explicitly specify the type that will be used to cast the response. Output responses received from gateway streamer methods will always match the specified return type.
Previously, if the endpoint model schemas of different Executors in a Flow did not match exactly, the Flow would fail to start, Even if the difference was as small as a different default value, it would still give rise to this error.
We have fixed this bug by relaxing the model schema checking to only verify that the types of properties match.
In the consensus golang module, some allocated strings were not being properly released. We have repaired this.
This bug is related to #6027. We now use the return_type parameter in GatewayStreamer to ensure that Documents received at the gateway level are properly cast to the correct schema. This prevents validation and serialization errors that previously occurred.
Remove support for deploying Executors in the Jina Cloud sandbox, since the sandbox has been deprecated.
We would like to thank all contributors to this release:
3.20.3)Release time: 2023-09-07 08:46:43
This release contains 1 bug fix.
When deploying an Executor inside a Flow with a BaseDoc model that has any attribute with a ClassVar value, the service would fail to initialize because the Gateway could not properly create the schemas. We have fixed this by securing access to __fields__ when dynamically creating these pydantic models.
We would like to thank all contributors to this release:
3.20.2)Release time: 2023-09-06 07:37:25
This release contains 1 bug fix and 1 refactoring.
Fix an installation issue that appeared because a new release of opentelemetry-sdk prevented pip from finding compatible libraries.
Some hard-coded strings were replaced with constants to improve code readability.
We would like to thank all contributors to this release:
3.20.1)Release time: 2023-08-10 15:49:12
This release contains 2 bug fixes and 2 documentation improvements.
Streaming endpoints in Executors can be deployed behind a Gateway (when using include_gateway=True in Deployment).
In this case, the Gateway acts as a load balancer. However, prior to this release, when the HTTP protocol is used, the Gateway would wait until all chunks of the responses had been streamed from the Executor.
Only when all the chunks were received would it send them back to the client. This resulted in delays and suppressed the desired behavior of a streaming endpoint (namely, displaying tokens streamed from an LLM with a typewriter effect). This release fixes this issue by making the Gateway stream chunks of responses from the forwarded request as soon as they are received from the Executor.
No matter whether you are setting include_gateway to True or False in Deployment, streaming endpoints should give the same desired behavior.
When a deeply nested schema (DocArray schema with 2+ levels of nesting) was specified as an input or output of an Executor endpoint, and the Executor was deployed in a Flow, the Gateway would fail to fetch information about the endpoints and their input/output schemas:
from typing import List, Optional
from docarray import BaseDoc, DocList
from jina import Executor, Flow, requests
class Nested2Doc(BaseDoc):
value: str
class Nested1Doc(BaseDoc):
nested: Nested2Doc
class RootDoc(BaseDoc):
nested: Optional[Nested1Doc]
class NestedSchemaExecutor(Executor):
@requests(on='/endpoint')
async def endpoint(self, docs: DocList[RootDoc], **kwargs) -> DocList[RootDoc]:
rets = DocList[RootDoc]()
rets.append(
RootDoc(
text='hello world', nested=Nested1Doc(nested=Nested2Doc(value='test'))
)
)
return rets
flow = Flow().add(uses=NestedSchemaExecutor)
with flow:
res = flow.post(
on='/endpoint', inputs=RootDoc(text='hello'), return_type=DocList[RootDoc]
)
...
2023-08-07 02:49:32,529 topology_graph.py[608] WARNING Getting endpoints failed: 'definitions'. Waiting for another trial
This was due to an internal utility function failing to convert such deeply nested JSON schemas to DocArray models. This release fixes the issue by propagating global schema definitions when generating models for nested schemas.
create-app.md (#6023)We would like to thank all contributors to this release: