Temporal service
This release mitigates a problem where invalid UTF-8 data could be supplied to the history service, causing a denial of service
Temporal Docs Server Docker Compose Helm Chart
1.22.7)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
Full Changelog: https://github.com/temporalio/temporal/compare/v1.22.6...v1.22.7
This release mitigates a problem where invalid UTF-8 data could be supplied to the history service, causing a denial of service
Temporal Docs Server Docker Compose Helm Chart
1.21.6)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
Full Changelog: https://github.com/temporalio/temporal/compare/v1.21.5...v1.21.6
This release mitigates a problem where invalid UTF-8 data could be supplied to the history service, causing a denial of service
Temporal Docs Server Docker Compose Helm Chart
1.20.5)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
Full Changelog: https://github.com/temporalio/temporal/compare/v1.20.4...v1.20.5
github.com/gogo/protobuf has been replaced with google.golang.org/protobuf
We've fully replaced the use of gogo/protobuf with the official google protobuf runtime. This has both developmental and operational impacts as prior to Server version v1.23.0 our protobuf code generator allowed invalid UTF-8 data to be stored as proto strings. This isn't allowed by the proto3 spec, so if you're running a custom-built temporal server and think some tenant may store arbitrary binary data in our strings you should set -tags protolegacy when compiling the server. If you use our Makefile this is already done.
If you don't and see an error like grpc: error unmarshalling request: string field contains invalid UTF-8 then you will need to enable this when building the server. If you're unsure then you should specify it anyways as there's no harm in doing so unless you relied on the protobuf compiler to ensure all strings were valid UTF-8.
Developers using our protobuf-generated code will notice that:
time.Time in proto structs will now be [timestamppb.Timestamp](https://pkg.go.dev/google.golang.org/[email protected]/types/known/timestamppb#section-documentation)
time.Duration will now be [durationpb.Duration](https://pkg.go.dev/google.golang.org/protobuf/types/known/durationpb)
[go.temporal.io/api/temporalproto](https://pkg.go.dev/go.temporal.io/api/temporalproto)
reflect.DeepEqual or anything that uses it. This includes testify and mock equality testers!
reflect.DeepEqual for any reason you can use go.temporal.io/api/temporalproto.DeepEqual insteadtestify require/assert compatible checkers you can use the go.temporal.io/server/common/testing/protorequire, go.temporal.io/server/common/testing/protoassert packagesgo.temporal.io/server/common/testing/protomock
New System Search Attributes
We added two new system search attributes: ParentWorkflowId and ParentRunId. If you have previously created custom search attributes with one of these names, attempts to set them will start to fail. We suggest updating your workflows to not set those search attributes, delete those search attributes and then upgrade Temporal to this version.
Alternatively, you can also set the dynamic config system.supressErrorSetSystemSearchAttribute to true. When this dynamic config is set values for system search attributes will be ignored instead of causing your workflow to fail. Please use this as temporary workaround, because it could hide real issue in users workflows.
Before upgrading your Temporal Cluster to v1.23.0, you must upgrade your core and visibility schemas to the following:
Please see our upgrade documentation for the necessary steps to upgrade your schemas.
complete_workflow_command, continue_as_new_command etc.) with a single metric called command which has a tag “commandType” describing the specific command type (see https://github.com/temporalio/temporal/pull/4995)LIKE operator will no longer be supported in v1.24.0. It never did what it was meant to do, and only added confusing behavior when used with Elasticsearch.For situations where an operator wants to handle a bad deployment using workflow reset, the batch reset operation can now reset to before the first workflow task processed by a specific build id. This is based on reset points that are created when build id changes between workflow tasks. Note that this also applies across continue-as-new.
This operation is not currently supported by a released version of the CLI, but you can use it through the gRPC API directly, e.g. using the Go SDK:
client.WorkflowService().StartBatchOperationRequest(ctx, &workflowservice.StartBatchOperationRequest{
JobId: uuid.New(),
Namespace: "my-namespace",
// Select workflows that were touched by a specific build id:
VisibilityQuery: fmt.Sprintf(`BuildIds = "unversioned:bad-build"`),
Reason: "reset bad build",
Operation: &workflowservice.StartBatchOperationRequest_ResetOperation{
ResetOperation: &batch.BatchOperationReset{
Identity: "bad build resetter",
Options: &commonpb.ResetOptions{
Target: &commonpb.ResetOptions_BuildId{
BuildId: "bad-build",
},
ResetReapplyType: enumspb.RESET_REAPPLY_TYPE_SIGNAL,
},
},
},
})
We've added a DLQ to history service to handle poison pills in transfer / timer queues and other history task queues including visibility and replication queues. You can see our operators guide for more details.
If you want tasks experiencing unexpected errors to go to the DLQ after a certain number of failures you can set the history.TaskDLQUnexpectedErrorAttempts dynamic config.
Once this feature is enabled, our task queues will be roughly FIFO.
This is disabled by default in 1.23, as we continue testing it but expect that it’ll be enabled by default in 1.24. To enable it the following config should be set to a short duration (e.g. 5sec) from its current default value (10yrs): "matching.backlogNegligibleAge"
We've added the following metrics as part of this effort:
poll_latency - this is a per-task-queue histogram of the duration between worker poll request and response (with or without task) calculated from the Matching server’s perspectivetask_dispatch_latency - this is a histogram of schedule_to_start time from Matching's perspective, broken down by task queue and task source (backlog vs history)We've added the ability to specify global (cluster level) rate limiting value for the persistence layer. You can configure by specifying the following dynamic config values:
frontend.persistenceGlobalMaxQPS
history.persistenceGlobalMaxQPS
matching.persistenceGlobalMaxQPS
worker.persistenceGlobalMaxQPS
You can also specify this on the per-namespace level using
frontend.persistenceGlobalNamespaceMaxQPS
history.persistenceGlobalNamespaceMaxQPS
matching.persistenceGlobalNamespaceMaxQPS
worker.persistenceGlobalNamespaceMaxQPS
Please be aware that this functionality is experimental. This global rate limiting isn’t workload aware but shard-aware; we currently allocate this QPS to each pod based on the number of shards they own rather than the demands of the workload, so pods with many low-workload shards will have a higher allocation of this limit than pods with fewer but more active workloads. If you plan to use this you will want to set the QPS value with some headroom (like 25%) to account for this.
The metrics exported by the deadlock detector were renamed to use a dd_ prefix to avoid confusion with other lock latency metrics. Affected metrics: dd_cluster_metadata_lock_latency, dd_cluster_metadata_callback_lock_latency, dd_shard_controller_lock_latency, dd_shard_lock_latency, dd_namespace_registry_lock_latency.
Visibility API now supports prefix search by using the keyword STARTS_WITH. Eg: WorkflowType STARTS_WITH 'hello_world'. Check the Visibility documentation for additional information on supported operators.
Temporal Docs Server Docker Compose Helm Chart
v1.23.0)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
Full Changelog: https://github.com/temporalio/temporal/compare/v1.22.6...v1.23.0
This release mitigates a rollback problem introduced into one of our v1.23.0 release candidates. This has no impact on OSS users using official releases.
2024-02-29 - 2899920e9 - Bump Server version to 1.22.6 2024-02-29 - 1eba091c8 - Update Go SDK to handle SDKPriorityUpdateHandling flag (#5468)
Temporal Docs Server Docker Compose Helm Chart
1.22.6)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
2024-02-22 - 2787da350 - Bump Server version to 1.22.5 2024-02-22 - 760ea9c09 - Bump Server version to 1.22.5-rc2 2024-02-21 - 2ea05b30d - Ensure PollActivityTaskQueueResponse.ScheduleToCloseTimeout is not nil (#5444) 2024-02-02 - d4f38c207 - Bump Server version to 1.22.5-rc1 2024-02-01 - 64fe53cb9 - Backport code to drop internal errors encountered during task processing (#5385) 2024-01-31 - 2647b3675 - Fix scheduled task rescheduling on failover (#5377)
Temporal Docs Server Docker Compose Helm Chart
1.22.5)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
2024-01-12 - fb617040c - Bump Server version to 1.22.4 2024-01-12 - f2659e725 - Change auth order (#5294) 2024-01-12 - 4489f174a - Fix buildkite cassandra setup (#5263) 2024-01-08 - 6dcab7349 - Update GHA setup-go to pull version from go.mod file (#5207)
Temporal Docs Server Docker Compose Helm Chart
1.22.4)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
The dependency golang.org/x/net was updated to address CVE-2023-39325, CVE-2023-3978, CVE-2023-44487.
2023-12-07 - 4eca060f9 - Bump Server version to 1.22.3 2023-11-30 - 407ff8a1e - Fix nil panic (#5153) 2023-11-30 - db10079e0 - Activate changes in #4990 and #5034 (#5035) 2023-11-30 - 20a482943 - Always set RPS on dynamic rate limiter refresh (#5180) 2023-11-30 - 79bacb94a - Fix default persistence request priority (#5177) 2023-11-30 - b4b6581c5 - Update force replication with low priority context (#5010) 2023-11-27 - 294e8a333 - 1.22.3 build fix for cherry pick build error (#5163) 2023-11-22 - 38f0eb298 - Fix set up search attributes in secondary SQL visibility (#5143) 2023-11-20 - 1f04629c2 - Add more logging message to monitor branch token update (#5058) 2023-11-20 - 6ba388cb3 - Handle branch token update with long poll API (#4943) 2023-11-20 - 98b5700e7 - golang.org/x/net: update to address CVE-2023-39325, CVE-2023-3978, CV… (#5011)
Temporal Docs Server Docker Compose Helm Chart
1.22.3)Server Server With Auto Setup (what is Auto-Setup?) Admin-Tools
2023-11-13 - db23e8671 - Add action metrics for schedule and updates (#5101) 2023-11-03 - 37da275af - Allow list of keywords in dual visibility setting (#5065) 2023-11-01 - 26114d774 - Hopefully avoid panic in DescribeWorkflowExecution (#5057) 2023-10-31 - ddf979916 - Bump server version to 1.22.2 (#5056) 2023-10-31 - ef3b92b86 - Fix schedule workflow unit test (#5052) 2023-10-30 - 2320fc172 - Add force-continue-as-new signal to schedule workflow (#5034) 2023-10-30 - b8163ceb6 - Use ContinueAsNewSuggested in scheduler workflow (#4990) 2023-10-30 - 88f8f11da - Update Go SDK to v1.25.1 (#4955) 2023-10-30 - cba2ce7ec - Improve scheduler workflow unit test (#4962) 2023-10-30 - 53a4979f4 - Expose buffer size and # dropped actions in ScheduleInfo (#4839) 2023-10-30 - 0fc50f827 - Adjust action metrics (#5043) 2023-10-30 - 91425ba5d - Emit action metrics for batch of markers (#4905) 2023-10-30 - f442eb703 - Validate search attributes values in queries to ES (#5036) 2023-10-30 - 19ba0cec9 - Disable eager activities for incompatible versioned activities (#5030) 2023-10-30 - cd14e159f - Add refresh task for close workflow (#4999) 2023-10-30 - e022877a8 - Refresh upsert visibility task only for running workflows (#4909) 2023-10-30 - f99bd882d - Lock get current execution for API requests (#4970) 2023-10-30 - 8dc833c0a - Fix GenerateMigrationTasks behavior (#4987) 2023-10-30 - 1a3083326 - Return InvalidArgument error when we get HTTP 400 from elasticsearch (#4900) 2023-10-30 - b6c9868a6 - Reuse cached RemoteAdminClient in VerifyReplicationTasks (#4997) (#5050)
There was a security issue in the Temporal Docker image that could expose your DB password in the shell logs. The issue is fixed in https://github.com/temporalio/docker-builds/pull/147, and new Docker image builds packing this version or later will contain the fix. Note that this issue is not related in any way to the Temporal Server itself. It only affects a script in the Docker image to launch your Temporal Server.
2023-10-13 - 717352261 - Ensure latest Go version in GH workflows (#4974) 2023-10-12 - b154bcda8 - Bump base-ci-builder image (#4969) 2023-10-06 - 759b60896 - Refactor TaskPoller.PollAndProcessWorkflowTask (#4903) 2023-10-06 - 4a70d6293 - Don't track overlapping scheduled workflows (#4911) 2023-10-06 - 1f01c72a3 - Use RateBurst in RateLimitInterceptorProvider (#4933) 2023-10-06 - 9abaafd16 - Take history length into account when rate limiting generate replication task (#4929) 2023-10-06 - ba6391831 - Improve XDC logging (#4932) 2023-10-06 - 8cadb9400 - Handle XDC replication task retry error / not found error correctly (#4924) 2023-10-06 - 8d1e43aa9 - Make rate multi min and max dynamic (#4920) 2023-10-06 - b4df68298 - Add metric for dynamic rate limit multiplier (#4921) 2023-10-06 - 68604955a - Improve XDC Logging (#4917) 2023-10-06 - b5f73191d - Simplify DescribeSchedule (#4810) 2023-10-06 - 1af21c404 - Build id scavenger shouldn't look at recent build ids (#4846) 2023-10-06 - 9547e8249 - Exit build id scavenger on rate limiter error (#4841) 2023-10-06 - e9985714d - Fix QueryWorkflow forwarding on versioned queue (#4847) 2023-10-06 - 277e97d49 - Make versioning data limits configurable by namespace (#4843)