## What changed?
Check the child's FirstExecutionRunId when
RecordChildExecutionCompleted.
## Why?
When there is conflict resolution and parent's branch changes, there may
be a new child run2. Old child run1 and new run2 have same initiated
event id, but different started events on different branches. If the old
child run1 reports execution completion to parent, parent will
incorrectly complete new run2.
## How did you test it?
unit test.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
Change to send raw history blobs from history service to frontend
service. History service returns a new proto message that has a repeated
bytes history field.
This response is wire compatible with the original response which has
temporal.api.history.v1.History type for this field. This allows history
service to not deserialize events from this data blob. This considerably
reduces CPU usage.
History service still needs event_id and version decoded from history
events. For this we use a new proto message StrippedHistoryEvent which
has these two fields only. It takes considerably less CPU to decode
events to this struct.
## Why?
We have seen incidents of high history CPU usage when large number of
GetWorkflowExecutionHistory calls are made to workflows which has large
history. With this change we can reduce the CPU burden on history
service during this API call.
## How did you test it?
Existing unit and functional tests.
## Potential risks
## Documentation
## Is hotfix candidate?
## What changed?
<!-- Describe what has changed in this PR -->
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
- had forgotten to update + add the client who last modifies the
worker-deployment workflows via the write API's
## Why?
<!-- Tell your future self why have you made these changes -->
- versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- altered existing suite of tests to now account for checking the client
when current/ramp changes
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
- **Revert "Fix history batch validations (#7281)"**
- **Revert "Send raw history blobs from history service to frontend
(#7179)"**
## What changed?
Revert the changes for returning raw history event from history to
frontend.
## Why?
Some logic is moved from history to frontend handler. This might cause
issues during upgrade.
I will push a change to first copy these operations to frontend. Once
that change is released, I will reapply this change.
## How did you test it?
## Potential risks
## Documentation
## Is hotfix candidate?
## What changed?
<!-- Describe what has changed in this PR -->
- Eliminate the DescribeWorkerDeployment fan-out to Versions for getting
the summaries.
- Sync drainage status with the deployment workflow when it changes.
- Drainage WF now can CaN!
- Move drainage args to it's own message to be future proof.
- Made WorkerDeploymentLocalState.versions a map for easier access to
versions.
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
- Skip drainage
## Why?
<!-- Tell your future self why have you made these changes -->
- Versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Added a new test which passes when poller_history config is changed
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: ShahabT <shahab.tajik@temporal.io>
Co-authored-by: Carly de Frondeville <carly.defrondeville@temporal.io>
## What changed?
Make updates and signals use args
Try to delete oldest to newest version if we hit limit
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
New functional test
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Shivam <57200924+shivs11@users.noreply.github.com>
Co-authored-by: Shahab Tajik <shahab@temporal.io>
Co-authored-by: Shivam Saraf <shivam.saraf@temporal.io>
Co-authored-by: ShahabT <shahab.tajik@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
Syncing TQ User Data for all types of a single TQ name at once to reduce
the SetCurrent and SetRamping latency and flakiness of tests.
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Carly de Frondeville <carly.defrondeville@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
Child workflows start in parent's pinned Worker Deployment Version.
## Why?
<!-- Tell your future self why have you made these changes -->
So user do not have to worry about interface compatibility between
pinned parents and children.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Added new tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
- The initial current version of a worker deployment is now
`"__unversioned__"` instead of `""`
- When the ramping version of a worker deployment becomes unversioned,
the deployment tells all the task queues in the current version that
they now have an unversioned ramp.
- If the unversioned ramp is promoted to current version, or if the
unversioned ramping version is unset, the deployment tells all the task
queues of the previous current version that they no longer have an
unversioned ramp.
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
- Made sure all existing WorkerDeploymentSuite and
DeploymentVersionSuite tests pass
- Wrote new tests in WorkerDeploymentSuite for the specific edge cases
that this code handles:
- `TestSetCurrentVersion_Unversioned_NoRamp`: Test that when current
version changes from versioned -> unversioned, the task queues become
unversioned
- `TestSetCurrentVersion_Unversioned_PromoteUnversionedRamp`: Test that
when the current version changes from versioned -> unversioned and
unversioned was previously ramping, the task queues become unversioned
with no more unversioned ramp
- `TestSetRampingVersion_Unversioned_UnversionedCurrent`: Test that this
fails with "Ramping version __unversioned__ is already current" error
- `TestSetRampingVersion_Unversioned_VersionedCurrent`: Test that the
ramping version of the current version's task queues becomes unversioned
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Shivam <57200924+shivs11@users.noreply.github.com>
Co-authored-by: Shahab Tajik <shahab@temporal.io>
Co-authored-by: Shivam Saraf <shivam.saraf@temporal.io>
Co-authored-by: ShahabT <shahab.tajik@temporal.io>
## What changed?
1. Pass Task Queue Types to `CheckIfTaskQueuesHavePollers` activity so
that we check pollers for only the task queue types that are registered
in that version.
2. Remove `DeploymentVersionData` from the per-type map in
`TaskQueueFamilyData`. `DeploymentVersionData` stores the same info as
`VersionLocalState`, so repeating it num_task_queue times inside the
VersionLocalState was a waste of space, and kind of confusing. I spoke
with Shahab yesterday about how `TaskQueueFamilyData` should contain
information that is specific to each task queue type within that
version. So far, the only data that is specific to the version + task
queue tuple is `first_poller_time`, so I put that in a new
`TaskQueueVersionData` struct and put it in `TaskQueueFamilyData`.
## Why?
See explanation above.
## How did you test it?
Tested that the existing Versioning 3.1 functional tests still work, including DeleteVersion with poller presence.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
- added poller presence (called MissingTaskQueue in code) checks when a
version starts to ramp or wants to be set as current
- happy path tests for delete version have been added
- happy path tests for poller presence checks have been added
## Why?
<!-- Tell your future self why have you made these changes -->
- versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- NO TESTS have been added - just an initial prototype for review.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Carly de Frondeville <carly.defrondeville@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Shivam <57200924+Shivs11@users.noreply.github.com>
Co-authored-by: Shahab Tajik <shahab@temporal.io>
Co-authored-by: Shivam Saraf <shivam.saraf@temporal.io>
Co-authored-by: ShahabT <shahab.tajik@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
Updated code to use latest API changes in which Deployment Versions are
represented by string fields rather than structs.
`WorkerDeploymentVersion` proto message still exists but only for the
internal APIs.
## Why?
<!-- Tell your future self why have you made these changes -->
See https://github.com/temporalio/api/pull/547.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Carly de Frondeville <carly.defrondeville@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Shivam <57200924+Shivs11@users.noreply.github.com>
## What changed?
<!-- Describe what has changed in this PR -->
- Deployment BuildId's are no longer unique across a namespace. The
combination of <DeploymentName, BuildID> will be.
- Constraints for not allowing "/" and "__" in deploymentName and
buildID respectively
- We want the APIs to leave open the possibility of using a custom
version id instead of deployment_name/build_id to set current or
ramping. Also we want to accept the "__unversioned__" string without
having to accep an "unversioned version" . To support this, refactored
APIs to accept version strings. Refactored internal code to use build id
string for build id only, and version string for fully-qualified version
string only.
## Why?
<!-- Tell your future self why have you made these changes -->
- Versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Existing suite of tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Carly de Frondeville <carly.defrondeville@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
- Ramping PR for versioning-3.1
- now, if a ramping version becomes current, the ramp will be unset.
## Why?
<!-- Tell your future self why have you made these changes -->
- Versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Added more functional tests
- Existing suite
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
- None, going to feature.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No
## What changed?
<!-- Describe what has changed in this PR -->
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
Add DrainageStatus child workflow to worker deployment system
## Why?
To periodically update the version workflow with results from
visibility.
## How did you test it?
Functional tests. Currently broken (see todo comment in code)
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Shivam Saraf <shivam.saraf@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
- ListWorkerDeployments API
- Also removed presence of locks when adding a version to a
worker-deployment workflow. Causes deadlocks and we don't need them.
## Why?
<!-- Tell your future self why have you made these changes -->
- Versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Added functional tests (happy-path)
- Existing suite
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
- None, feature
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
- No
## What changed?
<!-- Describe what has changed in this PR -->
- title
## Why?
<!-- Tell your future self why have you made these changes -->
- versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- existing suite of tests + added functional tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
- None, feature branch
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
- No
## What changed?
<!-- Describe what has changed in this PR -->
Matching to Worker Deployment Ramping Version and its percentage when
routing tasks. User-facing APIs are not implemented yet.
The implementation also supports ramping from a Deployment Version to
unversioned workers.
## Why?
<!-- Tell your future self why have you made these changes -->
Part of Worker Versioning features.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Added unit and functional tests.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
None.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
None.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No
---------
Co-authored-by: Shivam Saraf <shivam.saraf@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
- DescribeWorkerDeployment API for versioning-3.1
For this API to work, I realized that the worker deployment workflow has
to have information about the various deployment versions it
encapsulates. Now that version deployment workflows won't have a
`deploymentName` present in their workflowID (for now), partial matching
workflow ID's with the received deployment name to target version
workflows was not feasible.
Another option was to store the mapping from "versions" to "the local
state of a version workflow". However, this would have made the
deployment workflow arguments large in size since in theory, we could
have many deployment versions each with large sized local states.
The option I ended up going with was storing the version in a deployment
workflow during task-queue registration. During a
`DescribeWorkerDeployment` call, we would call `Describe` on each stored
version and return the aggregated info to the user. A version will be
removed from a worker deployment when it becomes "scavenged" (to be
implemented)
Potential future improvements include adding a caching mechanism.
## Why?
<!-- Tell your future self why have you made these changes -->
- Versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Added a functional test + more tests to follow
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
- None, going inside a feature.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
- No
## What changed?
<!-- Describe what has changed in this PR -->
Using new Deployment Options fields sent by SDK in Versioning 3
functionality. Old fields are still used when new fields are absent.
Implementation did not change, both new and old fields sent in polls and
task responses are still converted to old `Deployment` object and used
as before. Later, code will be refactored to change the `Deployment`
usages to `DeploymentVersion`.
Also added new fields to replace `Deployment` with `DeploymentVersion`
fields in internal protos where needed. Matching<->History communication
happens via these new fields, only new internal fields are written but
both new and old fields are read.
## Why?
<!-- Tell your future self why have you made these changes -->
Incorporating latest renames in Versioning APIs.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Existing tests changed to use new fields (or both old and new depending
on the test).
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
None.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
None yet.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No.
## What changed?
<!-- Describe what has changed in this PR -->
- Updated deployment entity wf's API's (naming) for versioning-3.1.
Removed those which will not be required.
- Added functionality for RegisterTaskQueue + DescribeVersion
- Added functional tests for verifying the above two work.
- UserData stuff has been removed right now since there are pending
discussions
## Why?
<!-- Tell your future self why have you made these changes -->
- Versioning-3.1
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
- Functional tests have been added. Note, the tests added are exact
replicas (in terms of their core functionality) with the existing tests
under `deployment_test.go`
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
- None, feature branch.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
- No
## What changed?
- Added `is_reset_run` in ParentExecutionInfo
- Overriding `startRequest.WorkflowIdReusePolicy` &
`startRequest.WorkflowIdConflictPolicy` for children that were recorded
in the `executionInfo.ChildrenInitializedPostResetPoint`
- Once a child is started with the overridden policies, we set
`executionInfo.ChildrenInitializedPostResetPoint[childID] = false` so
that we don't do this again if the parent starts another instance of the
same child.
## Why?
This is needed so that the parent can terminate any running children
that were started in the previous run. Without this after reset, the
parent may not make progress.
## How did you test it?
Manual testing. Will add functional tests in followup PRs once the
feature is ready.
## Potential risks
N/A
## Documentation
No. Documentation change is pending.
## Is hotfix candidate?
No
## What changed?
use transition history if enabled when long poll history
## Why?
For state-based replication, we do allow event version version to go
backwards as long as the overall versioned transition is keep
increasing. So we need to change token to use versioned transition.
## How did you test it?
unit test.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
- Rename operation ID to token in the code and protos
- Upgrade the Nexus Go SDK to latest main (to be tagged 0.2.0)
- Upgrade the api dependency to get rename changes from ID to token
- Fixed some lint warnings around usage of `nexus.HandlerErrorf`
## What changed?
Added a new field `children_initialized_post_reset_point` to collect a
set of children that need to be restarted after the parent is reset.
## Why?
We need to remember the set of children that need to be restarted after
resetting a parent. So adding this field which will be populated in
`workflowResetterImpl` (in the followup PR)
## How did you test it?
Existing tests. This is just the protochange.
## Potential risks
N/A
## Documentation
N/A
## Is hotfix candidate?
No
## What changed?
<!-- Describe what has changed in this PR -->
- Proto definition for CHASM nodes
## Why?
<!-- Tell your future self why have you made these changes -->
- CHASM project
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
Change to send raw history blobs from history service to frontend
service. History service returns a new proto message that has a repeated
bytes history field.
This response is wire compatible with the original response which has
temporal.api.history.v1.History type for this field. This allows history
service to not deserialize events from this data blob. This considerably
reduces CPU usage.
History service still needs event_id and version decoded from history
events. For this we use a new proto message StrippedHistoryEvent which
has these two fields only. It takes considerably less CPU to decode
events to this struct.
## Why?
We have seen incidents of high history CPU usage when large number of
GetWorkflowExecutionHistory calls are made to workflows which has large
history. With this change we can reduce the CPU burden on history
service during this API call.
## How did you test it?
Existing unit tests and manual test to run workflows.
## Potential risks
## Documentation
## Is hotfix candidate?
No
## What changed?
<!-- Describe what has changed in this PR -->
Add `OnConflictOptions` to `StartWorkflowExecution`.
- if the workflow id conflict policy is not `USE_EXISTING`, then no-op;
- if the field is nil, then no-op;
- otherwise, then it will add a `WorkflowExecutionOptionsUpdatedEvent`
to the existing running workflow.
## Why?
<!-- Tell your future self why have you made these changes -->
Ability to attach completion callbacks to existing running workflows.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
WIP: writing tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
<!-- Describe what has changed in this PR -->
1. Add activity batch unpause API implementation.
2. Add implementation for commulative changes for a single activity API:
* add 'jitter' fields to Reset and Unpause
* 'Flatten' unpause. Previous approach was marked as overcomplicated
* Add 'keep_unpause' flag to Reset. By default reset will also unpause
the activity.
* Remove no-wait flag. By default, if activity is in retry, they will be
scheduled immediately (*jitter)
* Add "activity type" as a routing parameter to every activity
operation. If activity type is provided - every pending activity of this
type will be paused/reset/updated/unpaused
* fix/update many tests
TODO: separate func tests for batch unpause.
## Why?
<!-- Tell your future self why have you made these changes -->
Part of the activity API work.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
fixing and running unit/func tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
N/A
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
Not for now.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No
## What changed?
<!-- Describe what has changed in this PR -->
Do not send backfill task when event is empty
## Why?
<!-- Tell your future self why have you made these changes -->
Backfill tasks should always have associated events. If there is no
event(i.e. for a state only transition), then no backfill to perform and
we should skip the task
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
n/a
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
n/a
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
n/a
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
no
## What changed?
Set delete_on_completion to reserved
## Why?
delete_on_completion is no longer needed
## How did you test it?
make
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
No
## What changed?
<!-- Describe what has changed in this PR -->
Peopagate "ActivityPaused" to the workers via "heartbeat" response, in
both cases.
## Why?
<!-- Tell your future self why have you made these changes -->
To let long-running activity workers know that activity was paused.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
unit tests
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
No
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No
## What changed?
- initial protobufs for the V2 (CHASM) scheduler
## Why?
- initial pass at internal protobufs for CHASM scheduler
## How did you test it?
- No tests for the protobufs themselves
## Potential risks
- New fields in new structs, **except** for `RequestId` on
`BufferedStart`. As a completely new field the V1 scheduler won't look
at, I don't think there's a risk.
## What changed?
- Cloned service/worker/deployment -> service/worker/worker-deployment
- Cloned proto/internal/.../deployment -> a section at the top of the
document with `WorkerDeployment` and `WorkerDeploymentVersion`-related
names
- Renamed `Deployment Series` -> `Worker Deployment`
- Renamed `Deployment` -> `Worker Deployment Version`
The contents of the functions and structs have not changed except to use
the new terminology and new constant names. As we implement the new
APIs, I expect the contents of the structs and functions to change.
I have tried to remove irrelevant fields from the proto messages in
preparation for the new APIs, but more might need to be removed in the
future. I think it's worth keeping the old ones around so that we can
edit the code from the previous Deployments implementation instead of
starting from scratch.
## Why?
To lay a foundation with agreed-upon naming for versioning crew to work
on the new Worker Deployment APIs
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
---------
Co-authored-by: Shivam Saraf <shivam.saraf@temporal.io>
## What changed?
<!-- Describe what has changed in this PR -->
Now, a pinned workflow can start activities in task queues belonging to
different deployments. Such activities are versioned independently from
the workflow and will start on their own task queue's current
deployment.
## Why?
<!-- Tell your future self why have you made these changes -->
Previously, independent activities for pinned workflows were not
supported and the workflow would get stuck if attempting to start an
activity on a task queue not belonging to the same deployment and the
workflow task queue.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Added test.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
The independence check happens when the task is scheduled and then when
the spooled task is being dispatched. If the independence check have a
different result for the spooled task, we re-spool it in the new
backlog.
Re-spooling tasks in a different queue is a new behavior, but since it
should happen at most once for a task (from default queue to pinned
queue) and only in rare edge cases dealing with late arriving pollers,
it seems safe.
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
None so far.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No.
## What changed?
<!-- Describe what has changed in this PR -->
Add the following activity retry policy properties to the activity
replication logic:
* InitialInterval
* MaximumInterval
* MaximumAttempts
* BackoffCoefficient
## Why?
<!-- Tell your future self why have you made these changes -->
Before there was assumption that activity retry policy can't change.
Because of that it was not replicated.
Now we can change activity retry policy via UpdateActivity API. Those
changes should be replicated.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Add func tests that check few updated properties.
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
Increasing replication size.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No
## What changed?
<!-- Describe what has changed in this PR -->
Populate WorkflowExtendedInfo as a part of DescribeWorfklow response.
## Why?
<!-- Tell your future self why have you made these changes -->
Feature request. Users want to see workflow expiration time as a part of
DescribeWorfklow response.
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
Update unit tests.
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
No
## What changed?
<!-- Describe what has changed in this PR -->
Added attempt count to `nexusoperations.InvocationTask` and
`nexusoperations.CancelationTask`
Exported `outbound_queue_factory.StateMachineTask` to make it accessible
to executor wrappers
## Why?
<!-- Tell your future self why have you made these changes -->
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
## What changed?
The deployment workflow waits for user data to propagate to all task
queue partitions before updating its state.
## Why?
We should ensure that the desired dispatch semantics will be in effect
on all task queue partitions.
## How did you test it?
existing tests, new unit test
## What changed?
<!-- Describe what has changed in this PR -->
Handle state transition enable->disable->re-enable case
## Why?
<!-- Tell your future self why have you made these changes -->
for safely roll out/back
## How did you test it?
<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
unit test
## Potential risks
<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
no risk
## Documentation
<!-- Have you made sure this change doesn't falsify anything currently
stated in `docs/`? If significant
new behavior is added, have you described that in `docs/`? -->
n/a
## Is hotfix candidate?
<!-- Is this PR a hotfix candidate or does it require a notification to
be sent to the broader community? (Yes/No) -->
no
## What changed?
Removed the experimental HSM scheduler code.
## Why?
Now that we've started the productionized implementation, we'd like to
avoid naming conflicts between the two. Experimental HSM scheduler isn't
in use.
## How did you test it?
- `make && make lint && make unit-test`
## Potential risks
- None
## Documentation
## Is hotfix candidate?
