## What changed?
Unify Visibility query converters.
The new query converter is the struct `query.QueryConverter[ExprT]`. The
generic parameter specify the output type of the converter (in ES is
`elastic.Query`, in SQL is `sqlparser.Expr`).
The query converter has a `query.StoreQueryConverter[ExprT]`. This is
the Visibility store specific implementation for building the output
query.
For example, the store needs to implement `BuildAndExpr(exprs)` to tell
the query converter how to build the `AND` expression. In SQL, it's
`*sqlparser.AndExpr{exprs}`. In Elasticsearch, it's
`elastic.NewBoolQuery().Filter(exprs)`.
The store query converter for SQL needs a plugin query converter called
`sqlplugin.VisibilityQueryConverter` which is implemented by each plugin
(MySQL, PostgreSQL, SQLite). This interface is needed to build DB
specific syntax for `KeywordList` and `Text` searches as well as to
build the final `SELECT` statements.
Added dynamic config `system.VisibilityEnableUnifiedQueryConverter` that
acts as switch between the legacy and the unified query converters.
## Why?
There are two query converters: one for Elasticsearch, and another for
SQL.
They are somewhat similar, and have quite a few code duplications for
validation, modifying the query, etc.
Unifying the query converter ensures that we provide the exact same
behavior across different Visibility stores.
## How did you test it?
- [x] built
- [x] run locally and tested manually
- [x] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
I've added a lot of unit tests for the new code with coverage close to
100%.
Note that there has been basically no changes to functional tests, which
suggests that the change works well.
## Potential risks
There might be some edge cases that either query converters that this
unified query converter might deal differently.
## What changed?
- This PR adds revision number mechanics to handle task dispatch
inconsistencies that could arise since our versioning API's are becoming
eventually-consistent.
## Why?
- Making our versioning API's eventually consistent.
## How did you test it?
- [ ] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
- There are risks but they are gated by a dynamic config.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Adds revision-number semantics to task dispatch/versioning and
introduces a new per-deployment TQ user-data schema, updating APIs,
persistence, and server logic (DC-gated) with comprehensive tests.
>
> - **APIs (proto changes)**:
> - Add `revision_number` to `taskqueue.v1.TaskVersionDirective` and
propagate `task_dispatch_revision_number` in History
`Record*TaskStarted` requests.
> - Matching `SyncDeploymentUserDataRequest`: new fields
`deployment_name`, `update_routing_config`, `upsert_versions_data`,
`forget_versions`; response includes `routing_config_changed`. Deprecate
old `update_version_data`/`forget_version` oneof.
> - Deployment API: add `WorkerDeploymentVersionData`; mark
`DeploymentVersionData` deprecated where applicable.
> - **Persistence (task queue user data)**:
> - Extend `DeploymentData` with `deployments_data` (map of deployment →
`WorkerDeploymentData` holding `RoutingConfig` and per-build version
data). Deprecate legacy `versions` and `unversioned_ramp_data` fields.
> - **Server logic**:
> - Matching/History: compute target version using revision numbers;
start workflow deployment transitions using revision-aware decisions;
support mixed old/new schemas when calculating current/ramping.
> - Matching: new helpers to migrate/clean old-format entries, apply
routing-config updates atomically, and prevent query blackholes using
status.
> - Task dispatch carries and records revision number; internal task
struct/plumbing updated.
> - **Dynamic config**:
> - Add `system.useRevisionNumberForWorkerVersioning` (plumbed through
History/Matching) to gate new behavior.
> - **Tests**:
> - Add/expand unit and functional tests covering new schema,
routing-config updates, revision-number behavior, and propagation.
> - **Misc**:
> - Describe/Stats paths updated to read both schemas; minor build/deps
update (api-go replace).
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
e84f484506. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: ShahabT <shahab.tajik@temporal.io>
## What changed?
- Gating stamp increment on activity retries from
https://github.com/temporalio/temporal/pull/8536/files behind a feature
flag
- Returning err when dropping stale activities on passive so we have
metrics emitted
## Why?
- We need stamp increments to only occur after the change from the PR
has fully rolled out to all clusters, else we risk compatibility issues
and dropped tasks.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
- CHASM: Best effort pure task deletion
## Why?
- Performance improvement, prevent invalid physical pure tasks from
firing.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Drop repeated workflow task failure reports
## Why?
To avoid busy loop of repeated workflow task failures
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
No
## What changed?
Add a new search attribute `TemporalReportedProblems` when a workflow
task fails or timeouts N consecutive times
## Why?
Enables users to easily discover workflows that are not making progress.
After a workflow task fails or times out N consecutive times a Search
Attribute, `TemporalReportedProblems` a `KeywordList`, will be added
with two entries, a `cause` and a `category`. These search attributes
will be queryable by users with queries like:
* `TemporalReportedProblems IS NOT NULL`
* `TemporalReportedProblems IN ('category=WorkflowTaskFailed')` or
`TemporalReportedProblems IN ('category=WorkflowTaskTimedout')`
* `'TemporalReportedProblems IN ("cause=UnhandledApplicationFailure")'
OR 'TemporalReportedProblems IN ("cause=ScheduleToCloseTimeout")'`
## How did you test it?
- [ ] built
- [X] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [X] added new functional test(s)
- [x] added new xdc test(s)
## Potential risks
Flapping _could_ happen with this change, but is unlikely. There's a
possibility of overloading the visibility system, but these changes are
protected with a dynamic config.
---------
Co-authored-by: Roey Berman <roey@temporal.io>
## What changed?
Optimize workflow task timeouts by completing them when the workflow
task finishes.
This optimization is enabled when the dynamic config
system.enableDeleteHistoryTasksOnUpdate is turned on.
It is disabled by default, as it can cause performance degradation in
cassandra based persistence.
## Why?
A large fraction of all timer tasks processed is workflow task timeouts.
In most of the cases, these are no-ops.
We can complete these tasks when the corresponding workflow task
finishes. This will reduce the number of timer tasks processes.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
- Introduced a new "move (task) group" action replacing the "incorrect"
slice-predicate action. See inline comment for how it works.
- Improve task Predicate AND/OR calculation to reduce predicate size.
- Skip empty slices when adding queue slices to readers.
## Why?
- Better namespace isolation.
## How did you test it?
- [x] built
- [x] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
- The new action can be disabled via dc.
## What changed?
Make replicate stream liveness check dynamic configurable
## Why?
Easy to adjust the liveness check internal without restart servers.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Moving StreamSender and ExecutableTask in our replication stack to use
dynamic config.
## Why?
Gives us the ability to dynamically modify the retry config of the
replication stream on source cluster and retry config during application
on the target side.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Added 2 new namespace specific dynamic configs that allows use to
control the rollout of the new CHASM Scheduler.
## Why?
Once CHASM Scheduler is feature complete we need the ability to control
whether or not it's enabled.
---------
Co-authored-by: alex stanfield <chaptersix@users.noreply.github.com>
## What changed?
Add metrics for dataloss events that containe workflowID, runID and
namespaceID.
This metric will only be emitted if config
`system.enableDataLossMetrics` is enabled.
## Why?
To find affected workflow runs if there are any dataloss errors.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Remove Eager Refresh Namespace Dynamic Config
## Why?
The feature is enabled for long time and we can enable it by default and
no dc is required.
## How did you test it?
- [x] built
- [x] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
no risk.
## What changed?
This adds support to the lru cache to actively expire entries older than
the TTL, by spawning a background goroutine that periodically deletes
old entries. A dynamic config, off by default, is added that can enable
this feature for the workflow cache.
## Why?
This can reduce the memory usage, and associated Go GC resource usage,
for workflow entries that won't be utilized since they are past their
TTL.
## How did you test it?
- [x] built
- [x] run locally and tested manually
- [x] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
Also running this in test setups to verify expected memory reduction.
## Potential risks
## What changed?
Deprecate ndc_history_resender code
## Why?
It is replaced by Resend Handler which support resend local generated
events.
## How did you test it?
- [x] built
- [x] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
no risk. The dynamic config is enabled long time in prod.
## What changed?
Follow-up to https://github.com/temporalio/temporal/pull/7921 which
introduced a server-side retry for an UwS that was aborted due to a
closing workflow. This changed the status code so it's retryable by the
client as well.
## Why?
If an UwS didn't perform a start and was aborted due to a closing
workflow, we want to retry the UwS.
## How did you test it?
- [ ] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [x] added new functional test(s)
## Potential risks
Added a dynamic config to turn off the new behavior if unforeseen issues
arise.
## What changed?
Added a server-side retry for when the Update of Update-with-Start was
aborted due to a closing workflow.
## Why?
Users expect it to retry.
## How did you test it?
- [ ] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [x] added new functional test(s)
## Potential risks
In case the change goes badly (for technical or user reasons), there's a
dynamic config to disable it again.
## What changed?
Add config flag for replication rate limiter
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Add monitor in stream sender on sync status
## Why?
Receiver send replication status per seconds.
Sender should monitor on this signal and assume the receiver side is
stop if it loses this sync status.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Add replication heartbeat to sync replication state when no task
## Why?
We want to have this to monitor stream liveness.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
This adds an interceptor to the handler service that logs latency and
errors from handlers, and can be reused in other services.
Adds the interceptor to the history service, that takes a health
aggregator provider. This provider is also added to the handler such
that we can read these values in the DeepHealthCheck handler within the
history service.
A future PR will add this check to the other services.
## What changed?
- Remove shard level workflow cache logic and always use host level
workflow cache.
## Why?
- Simplify code. Host level workflow cache has been enabled by default
for a long time.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Added a log for every MultiOperation error in history.
## Why?
Track down a race condition.
## How did you test it?
- [ ] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
Since it's only logging errors, I expect the log volume to be
reasonable/low.
## What changed?
1. Remove max retry times for replication stream
2. Improve log msg
## Why?
We have separate the ReplicationService error and ReplicationStream
error. It is not necessary to disconnect the stream if it is service
error.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
No risk. We have proper metrics and alert on it.
## What changed?
Removed config option `FollowReusePolicyAfterConflictPolicyTerminate`.
## Why?
It was [put in place](https://github.com/temporalio/temporal/pull/7099)
to have the ability to roll back a behavior change. It's been 4+ months
and it's okay to remove it now.
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
Resend parent for standby child completion verification.
## Why?
In the case that standby fails to verify child completion when parent
workflow is not found or not ready, standby should request resend parent
from active so standby can finish verification earlier.
## How did you test it?
- [x] built
- [x] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
New calls will be made to active. But this will only happen after the
new config maxLocalParentWorkflowVerificationDuration. So impacts to
active will be low.
## What changed?
- Implement chasm engine Update/Read Component method
## Why?
- CHASM work stream
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## What changed?
<!-- Describe what has changed in this PR -->
Dynamic config to enable generating request id reference links
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
Remove license header from every file. Because it is really hard to
follow in this PR here is the summary:
1. License header is removed from all `.go` and `.proto` files
:fireworks::fireworks:🎆.
2. `LICENSE` file in the root directory has only Temporal and Uber
copyrights.
3. 5 other `LICENSE` files added to the packages which have copyrights
different from Temporal and Uber: Datadog, Xargin, "Mat Ryer, Tyler
Bunnell and contributors".
4. `license_file` flag is removed from all code generation tools.
5. `copyright_file` flag is removed from `go:generate mockgen`
directive.
6. All copyright related targets are removed from `Makefile`.
7. Updated Temporal copyright year to 2025 everywhere.
## Why?
<!-- Tell your future self why have you made these changes -->
I double checked with legal department that it is not needed to have
license header in every file. One file per repo is enough. I put all
copyrights to the root `LICENSE` file and removed header from all other
files. Also updated tools and `Makefile`.
## What changed?
<!-- Describe what has changed in this PR -->
- Support update closed workflow by mutation
## Why?
<!-- Tell your future self why have you made these changes -->
- Today, any updates (e.g. update callbacks after workflows & update
branch token. With CHASM, there will be more cases) for closed workflows
require persisting the entire mutable snapshot using the
`SetWorkflowExecution` persistence API in order to skip the current
record validation.
- Application logic is complex and have to call different APIs based on
workflow state
- Performance is also bad as we are writing unnecessary data when
persisting a snapshot, even if they haven't been mutated.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Unit & persistence integration tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
- No.
## What changed?
Change to send raw history blobs from history service to frontend
service. History service returns a new proto message that has a repeated
bytes history field.
This response is wire compatible with the original response which has
temporal.api.history.v1.History type for this field. This allows history
service to not deserialize events from this data blob. This considerably
reduces CPU usage.
History service still needs event_id and version decoded from history
events. For this we use a new proto message StrippedHistoryEvent which
has these two fields only. It takes considerably less CPU to decode
events to this struct.
## Why?
We have seen incidents of high history CPU usage when large number of
GetWorkflowExecutionHistory calls are made to workflows which has large
history. With this change we can reduce the CPU burden on history
service during this API call.
## How did you test it?
Existing unit and functional tests.
## Potential risks
## Documentation
## Is hotfix candidate?
## What changed?
<!-- Describe what has changed in this PR -->
Added support for Update Registry to suggest CAN.
## Why?
<!-- Tell your future self why have you made these changes -->
To allow Workflow to CAN before it reaches the Update limit.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Added unit tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
Reverts https://github.com/temporalio/temporal/pull/7224 but keeps the
bug fix.
## Why?
<!-- Tell your future self why have you made these changes -->
Change was unnecessary.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Modified existing tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
Using the new dynamic config subscribers in the Update registry.
## Why?
<!-- Tell your future self why have you made these changes -->
To keep the Update registry's settings up-to-date.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Updated existing tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
Add `OnConflictOptions` to `StartWorkflowExecution`.
- if the workflow id conflict policy is not `USE_EXISTING`, then no-op;
- if the field is nil, then no-op;
- otherwise, then it will add a `WorkflowExecutionOptionsUpdatedEvent`
to the existing running workflow.
## Why?
<!-- Tell your future self why have you made these changes -->
Ability to attach completion callbacks to existing running workflows.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
WIP: writing tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
Add `history.alignMembershipChange` to allow history to use aligned
membership changes also.
## Why?
Better behavior during restarts/deployments.
## How did you test it?
not yet
## What changed?
<!-- Describe what has changed in this PR -->
Make WorkflowIdConflictPolicy TerminateExisting follow the
WorkflowIdReusePolicy after an _unsuccessful_ termination.
NOTE: The frontend change was made in
https://github.com/temporalio/temporal/pull/7099
## Why?
<!-- Tell your future self why have you made these changes -->
When the termination from TerminateExisting fails, the user would expect
the WorkflowIdReusePolicy to be applied as the Workflow is not running.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Well ... there's no way to test this well right now. There are no unit
tests; and the functional test cannot be written without the use of
[testhooks](https://github.com/temporalio/temporal/pull/6938) since
there is no other way to simulate the race condition here.
I manually added a `sync.Once` into the code that terminates the
workflow and can confirm the expected behavior.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
Adding log for when update registry size limit is reached.
## Why?
<!-- Tell your future self why have you made these changes -->
To identify a update registry size limit; and then enforce it in the
next change.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Added test. And ran functional test with limit.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
Visibility Queries slower than 1s (configurable with dynamic config) are
logged for analysis.
## Why?
To better understand the profile of the slowest queries.
## How did you test it?
Run a query which exceeds the dynamicConfig threshold.
## Potential risks
Threshold is too low, causing excessive logging. Existing metrics for
Query Latency were evaluated in largest cells to check f or rate. Expect
>1 log per hour, per cell.
## Documentation
N/A
## Is hotfix candidate?
No
## What changed?
Removed the experimental HSM scheduler code.
## Why?
Now that we've started the productionized implementation, we'd like to
avoid naming conflicts between the two. Experimental HSM scheduler isn't
in use.
## How did you test it?
- `make && make lint && make unit-test`
## Potential risks
- None
## Documentation
## Is hotfix candidate?
## What changed?
Added a dynamic config (default `false`) to gate the resetting with
pending children feture.
## Why?
The feature is still in development. So we don't want to expose it yet.
## How did you test it?
Existing tests since it's just wiring the config.
## Potential risks
N/A
## Documentation
N/A
## Is hotfix candidate?
No
## What changed?
<!-- Describe what has changed in this PR -->
Drop speculative WFT even if it had events.
## Why?
<!-- Tell your future self why have you made these changes -->
Special compatibility flag was
[added](https://github.com/temporalio/api/pull/467) to
`RespondWorkflowTaskCompletedRequest`. If this flag is set to true, then
server can drop speculative WFT with update rejections, even if it
shipped events to the worker. SDK support will be added later.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Added new unit tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
No risks. Compatibility flag is used to support backward compatibility
and don't drop speculative WFT if worker doesn't support it (old go
SDKs).
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
Yes, updated.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No.
## What changed?
Clean inactive namespace channels from
InterleavedWeightedRoundRobinScheduler after a specified delay.
This delay can be specified through the dynamic config option
history.taskSchedulerInactiveNamespaceChannelDeletionDelay.
## Why?
Older namespace stay in the flattened list forever. This is making task
scheduler slow.
## How did you test it?
Unit test
## Potential risks
## Documentation
## Is hotfix candidate?
No
---------
Co-authored-by: Yichao Yang <yichao@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
Added two dynamic configs:
- min blob size of memo or search attributes to be removed from mutable
state after the close visibility task is done.
- disabling fetch those attributes from visibility in the describe
workflow API.
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Unit tests, and verified locally.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
- Treat serviceerror.Internal as non-retryable
- Deprecate `history.TaskDLQInternalErrors` and always DLQ history tasks
when internal error is returned.
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
- Track sub state machine tombstones in mutable state
## Why?
<!-- Tell your future self why have you made these changes -->
- For new state-based replication stack
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Unit test
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
- N/A, feature flag not enabled.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
Add dynamic config to limit multi-cursor predicate size.
There are two separate configs:
- `history.queueMaxPredicateSize` for all history queues except for the
outbound queue - set to unlimited by default to avoid changing behavior
- `history.outboundQueueMaxPredicateSize` for the outbound queue - set
to 10K by default
## Why?
Limit the size of the shard info record. In stress test scenarios we've
seen this record grow to over 10MB.
## How did you test it?
Added unit tests.
## Is hotfix candidate?
This should be applied to any server deployments enabling Nexus
workloads.
## What changed?
Add a host level replication progress cache. The cache will be updated whenever a SyncVersionedTransition replication task is sent or SyncState API is called.
## Why?
To reduce the duplication of replication tasks.
## How did you test it?
unittest.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Will Duan <xinw.duan@gmail.com>
Co-authored-by: Roey Berman <roey@temporal.io>
Co-authored-by: Yichao Yang <yichao@temporal.io>
