mirror of
https://github.com/temporalio/temporal.git
synced 2026-08-31 02:51:51 -07:00
main
817 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Alan Wu
|
00d09082cc |
Add data race summary to CI report (#11211)
## What changed? Add data race summary to CI report ## Why? Notify when CI detects data race issues in main. ## How did you test it? - [X] built - [X] run locally and tested manually - [X] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s) |
||
|
Stephan Behnke
|
5c210f4c73 |
Replace errors.As with errors.AsType (#11674)
Go 1.27 prerequisite that applies the `errorsastype` Go fixer and its required error-interface updates. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Stephan Behnke
|
24434b11e9 |
Use slices.Backward for reverse iteration (#11676)
Go 1.27 prerequisite that applies the `slicesbackward` Go fixer for reverse iteration. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
David Porter
|
462de7718b |
tdbg: add per-schedule V1/V2/sentinel status check to schedule migrate status (#11636)
## What
Adds a `--schedule-id` flag to `tdbg schedule migrate status`. When set,
it looks up one specific schedule instead of the namespace-wide V1-vs-V2
counts, and reports whether that schedule is currently V1
(workflow-backed), V2 (CHASM), or caught in a migration sentinel state.
## Why
During schedule migration triage, an operator often needs to know the
status of *one specific* schedule, not aggregate counts. There was no
way to answer that without manually poking `execution describe` and
reasoning about CHASM node internals by hand.
## How it works
It always probes **both** sides — the V1 workflow ID
(`temporal-sys-scheduler:<id>`) and the V2/CHASM business ID (`<id>`) —
via the same `DescribeMutableState` RPC that `tdbg execution describe`
uses, regardless of which ID form was passed in. It deliberately does
not short-circuit based on the input's shape (e.g. its prefix) without
confirming against the server — an unconfirmed inference is
confusing/untrustworthy in a diagnostic tool.
This lets it flag both kinds of migration sentinel:
- a **CHASM-side sentinel** `Scheduler` component reserving the ID
during a V1→V2 migration
- a **V1-side `DummyWorkflow`** reserving the workflow ID during a V2→V1
rollback
Output leads with a plain-language headline describing the schedule's
current, authoritative form (written for someone with no prior knowledge
of the V1/V2 migration internals), followed by a note about any sentinel
found on the other side, a details table of both sides' raw status, and
the exact `execution describe` invocations to inspect each side further.
### Example output (V2→V1 rollback in flight)
```
Schedule "foo" is a V2 (CHASM) schedule.
Additionally, a placeholder ("sentinel") V1 workflow exists at workflow ID "temporal-sys-scheduler:foo",
reserving that ID while a V2→V1 rollback is in progress. This is expected during rollback and requires no action.
Details:
V1 (workflow-backed) [workflow ID temporal-sys-scheduler:foo]: sentinel (V2→V1 rollback placeholder)
V2 (CHASM) [business ID foo]: genuine
Inspect further:
V1: tdbg execution describe --workflow-id temporal-sys-scheduler:foo -n <namespace>
V2: tdbg execution describe --workflow-id foo --archetype scheduler.scheduler -n <namespace>
```
## Testing
- `go test ./tools/tdbg/... -run TestScheduleStatus -v` — new tests
cover: prefixed input, genuine V1, genuine V2, V1 genuine + CHASM
sentinel, V1 sentinel + genuine V2, not-found on both sides, and an
unexpected/inconsistent-state fallback. Existing aggregate-count tests
are unaffected.
- `go build ./tools/...`, `go vet ./tools/tdbg/...`
- Manually checked `tdbg schedule migrate status --help` output.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Tihomir Surdilovic
|
7168377484 |
Register archival task category in tdbg (#11587)
# What changed? `tdbg` now registers the **archival** task category (`5`) in its task-category registry (`tools/tdbg/app.go`), so `tdbg dlq list/merge/purge --dlq-type 5` works. Previously `merge`/`purge` rejected it with `unknown dlq category 5`. Because the `--dlq-type` help text is generated from the registry, archival now appears there too. Adds a regression test to `TestDLQCommand_V2`. ## Why? `tdbg` built its registry with `tasks.NewDefaultTaskCategoryRegistry()`, which omits `CategoryArchival`. The server registers archival conditionally in `TaskCategoryRegistryProvider` (`temporal/fx.go`) when archival is enabled, but `tdbg` is a client tool with no access to the cluster's archival config, so it rejected `--dlq-type 5` client-side — even though archival tasks are written to the history task DLQ and the server's `AdminService.MergeDLQTasks` accepts the category. That left archival DLQ tasks unrecoverable via the standard tool (operators had to call the RPC directly). Registering the category unconditionally in `tdbg` is safe: with archival disabled there is simply no archival DLQ to operate on. Fixes #11586. ## How did you test it? - [x] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) --------- Signed-off-by: Tihomir Surdilovic <tihomir@temporal.io> Co-authored-by: Prathyush PV <prathyush.pv@temporal.io> |
||
|
Alex Stanfield
|
c431d1a43b |
Improve flaky report presentation (#11528)
## What changed? - Shows Bayesian commit suspects directly below the overall stats. - Separates synthetic test-runner timeouts from test failures and reports affected artifacts. - Shows final-retry test failures as affected-artifact counts rather than workflow-run rates. ## Why? The synthetic `testrunner.TotalTimeout` event was reported both as a flaky test and as a CI breaker, while the former CI-breaker percentage mixed artifact and workflow-run units. ## How did you test it? - `go test -tags test_dep ./tools/flakereport ./tools/common/github` - `make lint-code` ## Potential risks This changes report categorization and presentation only; raw `failures.json` retains synthetic timeout records as timeout events. |
||
|
Alex Stanfield
|
abcb948730 |
Index GitHub Actions runs for flake bisecting (#11524)
## What changed? - Build one index from parsed test attempts keyed by normalized test name and GitHub Actions run commit. - Reuse that index when selecting and bisecting tests instead of rescanning every parsed attempt for each candidate. - Use `githubActions...` names in the bisect path to distinguish GitHub Actions runs from Temporal workflows. ## Why? A report can contain millions of parsed attempts. The previous bisect loop rescanned all of them for every candidate test. ## How did you test it? - [x] covered by existing tests - [x] checked the diff for whitespace errors - [ ] built - [ ] run locally and tested manually - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks This is stacked on #11523. The index preserves full test counts for qualification while excluding attempts without an in-range GitHub Actions commit from commit observations, matching the previous behavior. |
||
|
Alex Stanfield
|
b4238b3319 |
Use Go client for flaky report GitHub API calls (#11523)
## What changed? - Replace the `gh api` subprocesses used by `flakereport` with a shared Go HTTP client. - Rate-limit GitHub API requests to 10 RPS by default; allow an explicit `--rps` override. - Stream artifact downloads to temporary files while preserving the existing 60-second download deadline and artifact worker pool. ## Why? The report performs a GitHub API request for every artifact. Avoiding a new `gh` process for each request removes that per-request overhead without adding a more complex processing pipeline. ## How did you test it? - [x] added new unit test(s) - [x] checked the diff for whitespace errors - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new functional test(s) ## Potential risks The client uses the existing `GH_TOKEN`/`GITHUB_TOKEN` environment variables, falling back to one `gh auth token` call. Transient API errors are surfaced to the existing per-artifact error handling rather than retried. |
||
|
Stephan Behnke
|
32e366289f |
Limit flakereport size (#11552)
## What changed? Limit every dynamic Markdown report table to its top 100 rows. ## Why? Large flake reports can exceed GitHub Actions’ 1 MiB step-summary limit and abort summary publication. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Stephan Behnke
|
02075b937b |
Extract shared JUnit XML handling (#11480)
## What changed? Extract generic JUnit XML file reading and writing into `tools/common/junit`. ## Why? Centralizing format-level JUnit handling. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sean Kane <spkane31@gmail.com> |
||
|
Sean Kane
|
b9f7c9edad |
refactor(tools): share Slack utilities (#11319)
## What changed? * Extracted shared Slack Block Kit message construction, Markdown rendering, text-limit truncation, and webhook delivery into `tools/common/slack`. * Migrated `ci-notify` and `flakereport` to use the shared package while preserving their report-specific content and ordering. ## Why? Both tools maintained duplicate Slack payload types and webhook logic. ## How did you test it? - [x] covered by existing tests - [x] added new unit test(s) ## Potential risks Slack block text is now centrally truncated to Slack’s documented per-block limits. This affects only oversized content and prevents rejected webhook payloads. |
||
|
Stephan Behnke
|
c3750a9c29 |
Report final failures in CI notifications (#11019)
## What changed?
`ci-notify` reports failure details.
<details><summary>Example</summary>
<p>
```
ci-notify dry run for latest origin/main
generated_at: 2026-07-10T23:18:44Z
commit:
|
||
|
Stephan Behnke
|
4c3bd8329a |
Extract reusable GitHub helpers (#11018)
## What changed? Extract various github helpers into `temporal/tools/common/github`. No behavioral change expected/intended. ## Why? Make it easier to re-use github-related code. |
||
|
David Porter
|
40d5f2017c |
Added status command (#10904)
## What changed? Adds a status command for migrations ``` ./tdbg -n default schedule status Namespace: default V1 (workflow-backed): 1 V2 (CHASM): 0 Total: 1 ``` ## Why? to make it easier to understand what is going on during migrations via a visibility count ## How did you test it? - [ ] built - [X] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Very low risk, cli change only. |
||
|
Stephan Behnke
|
d91f3af3e3 |
Apply testifylint fixes (#10792)
## What changed? - Applied the server-owned subset of testifylint auto-fixes. - Exact commands that were run: ```sh .bin/golangci-lint-v2.9.0 run --allow-parallel-runners --concurrency 4 --fix --enable-only testifylint --build-tags disable_grpc_modules,test_dep --timeout 20m --config=.github/.golangci.yml make goimports git diff --check ``` - No manual or AI changes were made; unless commented as such. - Some fixes caused lint errors; those were reverted again. - Changes were all reviewed by me. |
||
|
Yu Xia
|
862f9d9d7c |
Add unit tests for replication logic (#10815)
## What changed? Add unit tests for replication logic ## Why? do a house cleanup to prepare a min test coverage check. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks No. Only unit test changes. |
||
|
Stephan Behnke
|
4c33955f06 |
Apply staticcheck fixes (#10793)
## What changed? - Applied the server-owned subset of staticcheck auto-fixes. - Exact commands that were run: ```sh .bin/golangci-lint-v2.9.0 run --allow-parallel-runners --concurrency 4 --fix --enable-only staticcheck --build-tags disable_grpc_modules,test_dep --timeout 20m --config=.github/.golangci.yml make goimports git diff --check ``` - No manual or AI changes were made. - Some fixes caused lint errors; those were reverted again. - Changes were all reviewed by me. |
||
|
Stephan Behnke
|
558aef251e |
Add gofix fixes (#10794)
## What changed? - Applied the server-owned subset of small gofix analyzer fixes. - Exact commands that were run: ```sh make fmt-gofix make goimports make fmt git diff --check ``` - No manual or AI changes were made. - Some fixes caused lint errors; those were reverted again. - Changes were all reviewed by me. |
||
|
Alex Stanfield
|
313fc9386b |
tdbg: decode CHASM callback and activity state (#10602)
## What changed `tdbg execution describe` now decodes CHASM callback and standalone activity component state instead of falling back to raw blob output. - Registers the CHASM callback component in the `tdbg` CHASM registry. - Registers the CHASM activity component via a nil activity library so SAA executions can be decoded without task handlers. - Imports the callback and activity protobuf packages for descriptor lookup during decode. This makes the JSON output show fields like callback `requestId`, callback `status`, activity type, task queue, and activity execution `status`. |
||
|
David Porter
|
de9f407e52 |
feat: Adds tdbg migration script to power migrations via visibility (#10495)
## What changed?
Adds a cli set of flags to the tdbg schedule migrate subcommand, to
fetch from visibility (or stdin) and perform a migration.
Using Visibility
```
tdbg schedule migrate --target chasm --from-visibility
[dry-run] would migrate default/verify-sa-1780095016 -> chasm
Dry-run: 1 schedule(s) would be migrated. Re-run with --execute to perform.
```
```
tdbg --namespace default schedule migrate --target workflow --from-visibility --execute
migrated default/verify-sa-1780095016 -> workflow
Done: 1 migrated, 0 failed (of 1).
```
Using stdin:
```
./tdbg schedule migrate --target chasm --from-visibility --output-log target.log
[dry-run] would migrate default/schedule-6 -> chasm
[dry-run] would migrate default/schedule-5 -> chasm
[dry-run] would migrate default/schedule-4 -> chasm
...
[dry-run] would migrate default/schedule-8 -> chasm
[dry-run] would migrate default/schedule-7 -> chasm
[dry-run] would migrate default/YourScheduleId-5 -> chasm
Dry-run: 20 schedule(s) would be migrated. Re-run with --execute to perform.
➜ temporal git:(bugfix/handling-list-with-migration-problems) ✗ head target.log
{"timestamp":"2026-06-05T06:41:27Z","namespace":"default","schedule_id":"schedule-6","target":"chasm","status":"dry-run"}
{"timestamp":"2026-06-05T06:41:27Z","namespace":"default","schedule_id":"schedule-5","target":"chasm","status":"dry-run"}
{"timestamp":"2026-06-05T06:41:27Z","namespace":"default","schedule_id":"schedule-
....
8","target":"chasm","status":"dry-run"}
{"timestamp":"2026-06-05T06:41:27Z","namespace":"default","schedule_id":"YourScheduleId-7","target":"chasm","status":"dry-run"}
➜ temporal git:(bugfix/handling-list-with-migration-problems) ✗ cat target.log | ./tdbg schedule migrate --target chasm
[dry-run] would migrate default/schedule-6 -> chasm
[dry-run] would migrate default/schedule-5 -> chasm
[dry-run] would migrate default/schedule-4 -> chasm
[dry-run] would migrate default/schedule-3 -> chasm
[dry-run] would migrate default/schedule-2 -> chasm
...
[dry-run] would migrate default/schedule-8 -> chasm
[dry-run] would migrate default/schedule-7 -> chasm
[dry-run] would migrate default/YourScheduleId-5 -> chasm
Dry-run: 20 schedule(s) would be migrated. Re-run with --execute to perform.
```
## Why?
Temporal is moving to a V2/Chasm based scheduler. Migrating schedules
manually is easier when done via visibility, so this is a hopefully more
ergonomic set of commands for such a migration:
## How did you test it?
- [ ] built
- [x] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
## Potential risks
Not likely to be a lot relating to the tool itself, the migration path
probably needs a bit more testing through.
|
||
|
Stephan Behnke
|
7c6368ecb4 |
UUID for test databases (#10680)
## What changed? Use UUID for database names. ## Why? From https://github.com/temporalio/temporal/actions/runs/27384406387 ``` panic: FATAL: LoadSchema [recovered, repanicked] goroutine 928163 [running]: testing.tRunner.func1.2({0x680fe20, 0xc06cafdb30}) /opt/hostedtoolcache/go/1.26.4/arm64/src/testing/testing.go:1974 +0x2b8 testing.tRunner.func1() /opt/hostedtoolcache/go/1.26.4/arm64/src/testing/testing.go:1977 +0x460 panic({0x680fe20?, 0xc06cafdb30?}) /opt/hostedtoolcache/go/1.26.4/arm64/src/runtime/panic.go:860 +0x12c go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x2, 0xc073c7d110, {0x7e042e0?, 0x33?, 0xc035698000?}) /home/runner/go/pkg/mod/go.uber.org/zap@v1.27.1/zapcore/entry.go:196 +0xa8 go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc073c7d110, {0xc07be44680, 0x2, 0x2}) /home/runner/go/pkg/mod/go.uber.org/zap@v1.27.1/zapcore/entry.go:272 +0x278 go.uber.org/zap.(*Logger).Panic(0xc03360c400, {0xc075e3bda0, 0x11}, {0xc07be44680, 0x2, 0x2}) /home/runner/go/pkg/mod/go.uber.org/zap@v1.27.1/logger.go:285 +0x60 go.temporal.io/server/common/log.(*zapLogger).Panic(0xc0361fe750, {0xc075e3bda0, 0x11}, {0xc06cafdac0, 0x1, 0x1}) /home/runner/work/temporal/temporal/common/log/zap_logger.go:174 +0x1b4 go.temporal.io/server/common/testing/testlogger.(*TestLogger).Fatal(0xc0361fe5d0, {0x755ae1f, 0xa}, {0xc06cafdac0, 0x1, 0x1}) /home/runner/work/temporal/temporal/common/testing/testlogger/testlogger.go:506 +0x2c8 go.temporal.io/server/common/persistence/sql.(*TestCluster).LoadSchema(0xc0542a5ce0, {0xc074d4b680, 0x47}) /home/runner/work/temporal/temporal/common/persistence/sql/test_sql_persistence.go:206 +0x804 go.temporal.io/server/common/persistence/sql.(*TestCluster).SetupTestDatabase(0xc0542a5ce0) /home/runner/work/temporal/temporal/common/persistence/sql/test_sql_persistence.go:84 +0x21c go.temporal.io/server/common/persistence/persistence-tests.(*TestBase).Setup(0xc05af8d1e0, 0xc08d0b6040) /home/runner/work/temporal/temporal/common/persistence/persistence-tests/persistence_test_base.go:206 +0x2b4 go.temporal.io/server/tests/testcore.newClusterWithPersistenceTestBaseFactory(0xc05dfe3208, 0xc05af8d040, {0x7 ``` _might_ indicate a potential database name conflict. It's not 100% clear but this doesn't hurt IMO. |
||
|
Stephan Behnke
|
4f8de868c4 |
Improve testrunner timeout behavior (#10713)
## What changed? Old behavior was ``` 10m GitHub step timeout 5m --total-timeout (derived from -timeout) 5m go test -timeout ``` New behavior is ``` 10m GitHub step timeout 9m TEST_RUNNER_TIMEOUT / --total-timeout 5m go test -timeout ``` ie attach the testrunner timeout to the ceiling not the floor. Example now with logs: https://github.com/temporalio/temporal/actions/runs/27570791993/job/81507067577?pr=10713#step:9:46 ## Why? (1) We don't see any logs when the testrunner out. (2) testrunner has no time anymore to do a retry when `go test` times out. |
||
|
Fred Tzeng
|
dd7e5463e3 |
testrunner: exit non-zero when total-timeout fires (#10584)
## What changed? Make testrunner propagate test timeout as non-zero exit code. Also skip legitimate test failures for now to make tests pass again. ## Why? Otherwise silently hides stuck tests. --------- Co-authored-by: Stephan Behnke <stephanos@users.noreply.github.com> |
||
|
Stephan Behnke
|
a131913091 |
Flakereport trendlines (#10508)
## What changed? Add trendline to flakereport. ## Why? Help to quickly see if a flake is still relevant, increasing, decreasing etc. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) Example: https://github.com/temporalio/temporal/actions/runs/26913808641 |
||
|
Stephan Behnke
|
3d14d91a7d |
Add slowest runs to CI digest (#10493)
## What changed? Adds a "Slowest Runs" section to the ci-notify success report for the 3 longest completed workflow runs. ## Why? Better insights into slow runs. |
||
|
Stephan Behnke
|
9a33ab1b5a |
Generate JSON test summary (#10381)
## What changed? (1) Generate JSON test summary, too (2) Tweak summary detail capture esp for `Await` ## Why? Allow AI agents to easily/quickly/cheaply inspect CI results. <img width="1582" height="707" alt="Screenshot 2026-05-25 at 6 46 13 PM" src="https://github.com/user-attachments/assets/ca232f52-4f6f-45cb-9b52-450d32999d0c" /> <img width="2158" height="254" alt="Screenshot 2026-05-25 at 6 47 03 PM" src="https://github.com/user-attachments/assets/447ccf28-bec7-424b-a961-630c44a02cb0" /> |
||
|
Rodrigo Zhou
|
824f28dc23 |
Add flag to skip setting up ES cluster settings (#10152)
## What changed? Add flag to skip setting up ES cluster settings. Eg: `temporal-elasticsearch-tool setup-schema --skip-cluster-settings` ## Why? Allow users to not use our provided cluster settings (eg: avoid overwriting their cluster settings). https://github.com/temporalio/temporal/issues/9857 ## How did you test it? - [x] built - [x] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks |
||
|
Stephan Behnke
|
e4294817dc |
JUnit summary with failures (#9912)
## What changed? Add failure details per test to GitHub summary view. ## Why? Avoid digging through logs. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) **Example**: <img width="1193" height="644" alt="Screenshot 2026-04-13 at 1 58 43 PM" src="https://github.com/user-attachments/assets/9743dba8-7e66-42c9-a877-240e36d7b9b9" /> https://github.com/temporalio/temporal/actions/runs/24366106128?pr=9912 |
||
|
Veeral Patel
|
5bd15b0dd6 |
feat: add tdbg CLI command to get user data for a task queue partition (#9935)
## What changed? This PR adds a `tdbg taskqueue get-user-data` CLI command to `tdbg` to get the `TaskQueueUserData` for a particular task queue partition. This PR depends on #9934. See #9934 for more context. ## Files changed | File | Change | |---|---| | `tools/tdbg/tdbg_commands.go` | Registered `get-user-data` subcommand under `taskqueue` with `--namespace`, `--task-queue`,`--task-queue-type`, and `--partition-id` flags | | `tools/tdbg/task_queue_commands.go` | Implemented `AdminGetTaskQueueUserData`: reads flags, calls `AdminService.GetTaskQueueUserData`, pretty-prints response | | `tools/tdbg/task_queue_commands_test.go` | Added unit tests | ## How did you test it? - [x] built - [x] run locally and tested manually - [x] added new unit test(s) - [x] added new integration test(s) — not applicable - [x] added new functional test(s) — not applicable (CLI change) ### Unit tests | Test case | Input | Expected | |---|---|---| | Missing namespace | `--task-queue my-queue` only | Error before RPC is called | | Missing task queue | `--namespace default` only | Error before RPC is called | | Invalid task queue type | `--task-queue-type INVALID` | `StringToEnum` returns error before RPC is called | | Unspecified task queue type | `--task-queue-type TASK_QUEUE_TYPE_UNSPECIFIED` | Defaults to `TASK_QUEUE_TYPE_WORKFLOW`, succeeds | | Root partition (default) | Valid flags, no `--partition-id` | Calls RPC with `partition_id=0`, prints response | | Non-root partition | `--partition-id 1` | Calls RPC with `partition_id=1`, prints response | ### Manual tests **Setup** ``` make start-sqlite temporal operator namespace create -n default temporal task-queue versioning insert-assignment-rule --namespace default --task-queue my-queue --build-id "test-build-1" --rule-index 0 --yes temporal task-queue config set --namespace default --task-queue my-queue --task-queue-type activity --queue-rps-limit 50 --queue-rps-limit-reason "manual test" ``` **Root partition, workflow type** ``` $ ./tdbg taskqueue get-user-data --namespace default --task-queue my-queue --task-queue-type TASK_QUEUE_TYPE_WORKFLOW { "version": "2" } ``` `version=2` reflects writes from the assignment rule; `user_data` absent as expected (versioning rules live in `versioning_data`, not `per_type`). **Root partition, activity type** ``` $ ./tdbg taskqueue get-user-data --namespace default --task-queue my-queue --task-queue-type TASK_QUEUE_TYPE_ACTIVITY { "version": "2", "user_data": { "config": { "queueRateLimit": { "rateLimit": { "requestsPerSecond": 50 }, "metadata": { "reason": "manual test", "updateTime": "..." } } } } } ``` `user_data` populated with the rate limit config set via `UpdateTaskQueueConfig`. **Non-root partition, workflow type** ``` $ ./tdbg taskqueue get-user-data --namespace default --task-queue my-queue --task-queue-type TASK_QUEUE_TYPE_WORKFLOW --partition-id 1 { "version": "2" } ``` Same version as root — replication is working. **Non-root partition, activity type** ``` $ ./tdbg taskqueue get-user-data --namespace default --task-queue my-queue --task-queue-type TASK_QUEUE_TYPE_ACTIVITY --partition-id 1 { "userData": { "config": { "queueRateLimit": { "rateLimit": { "requestsPerSecond": 50 }, "metadata": { "reason": "manual test", "updateTime": "2026-04-13T21:40:39.888Z" } } } }, "version": "2" } ``` **Unspecified type defaults to workflow** ``` $ ./tdbg taskqueue get-user-data --namespace default --task-queue my-queue --task-queue-type TASK_QUEUE_TYPE_UNSPECIFIED { "version": "2" } ``` Passing `TASK_QUEUE_TYPE_UNSPECIFIED` defaults to workflow type instead of erroring — output matches the workflow-type test above. --------- Co-authored-by: Veeral Patel <veeralpatel@Veerals-MacBook-Pro.local> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Chetan Gowda
|
e5c977db86 |
[Nexus-CHASM] Add workflow methods to handle nexus events (#9546)
## What changed?
This PR adds methods in workflow component to handle nexus events.
## Why?
Migrating Nexus from HSM to CHASM
## How did you test it?
- [x] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [ ] added new unit test(s)
- [ ] added new functional test(s)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Touches the workflow-task completion path and CHASM workflow wiring
(registry injection and history-event application), so misregistration
or missing context could cause runtime command failures despite largely
being additive/refactor changes.
>
> **Overview**
> Adds CHASM `Workflow` helpers to *emit and apply* Nexus operation
lifecycle events (started/completed/failed/canceled/timed-out),
including consistent failure wrapping via
`NexusOperationExecutionFailure`.
>
> Refactors CHASM workflow command/event registration by moving
`workflowregistry` into `chasm/lib/workflow` as `Registry`, injecting it
into CHASM context, and updating Nexus workflow command handlers to use
`AddAndApplyHistoryEvent` so command-emitted history events immediately
run their registered event definitions. Updates wiring/tests/callers
across services to construct `NewLibrary(NewRegistry())` and use the new
types/errors.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
|
||
|
Sean Kane
|
971ce3fd75 |
fix: only report likely problematic commits that increased flakes (#9867)
## What changed? 1. The report was flagging commits that fixed a flake as culprits despite showing fewer failures, this change fixes false positives by making sure the commit caused an increase in flakes 2. Sort the table by bayesian probability instead of overall failure rate. New run is here: https://github.com/temporalio/temporal/actions/runs/24151267149 ## Why? The Bayesian model in runBisect is direction-agnostic: it identifies statistical transition points where the failure rate changes significantly, but treats a 10%→0% transition identically to a 0%→10% one. Commits that fixed a flake therefore appeared as high-confidence "suspects", generating misleading and unactionable report entries. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Minimal, still getting confidence in this reporting method. |
||
|
Sean Kane
|
5c1e6e8369 |
Add bayesian git bisection to identify problematic commits (#9788)
## What changed? Added a Bayesian commit bisect analysis to the `flakereport` tool that automatically identifies which commits most likely introduced test flakiness. Inspired by this Python tool: https://github.com/hauntsaninja/git_bayesect Sample run of the report can be found [here](https://github.com/temporalio/temporal/actions/runs/23914956307). This change uses the set of test runs which are mapped to commit history to infer with Bayesian statistics the probability that a commit introduced a flaky test or tests. There is a fuller explanation in `doc.go`. A tl;dr: The algorithm ranks each commit (as a transition) in the history by assigning equal p_before and p_after and using a Beta-Binomial marginal likelihood to determine the posterior probability. There is a heuristic to lower the prior probability for commits that only touch documentation or ci configuration. ## Why? This can help us identify the actual commit that introduced a problem and also help us get commit authors to validate their work instead of engineers without all of the context for a change. ## How did you test it? - [ ] built - [X] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Minimal, reporting only |
||
|
Sean Kane
|
77a23c02a9 |
Write test results reports in between runs to preserve on crash (#9692)
## What changed? testrunner writes intermediate JUnit XML reports after each completed test attempt and enforces its own total deadline derived from the go test -timeout flag. ## Why? When a test run exceeds the GitHub Actions job timeout (40 min), the entire process group is killed before the testrunner can write its final merged XML, producing a synthetic junit.crash.xml showing `functional-test (crash)`, with no visibility into actual test failures. [Sample run](https://github.com/temporalio/temporal/actions/runs/23567876357) that actually had 108 failures on manual inspection There are two distinct failure modes this addresses: 1. Killed between attempts — attempt 1 completes with failures, the process is killed during attempt 2. Previously: no XML on disk. Now: attempt 1's results survive in the intermediate file. 2. Go test timeout doesn't fire — the test binary hangs past its own `-timeout=35m` (e.g. deadlocked goroutines). Previously: GitHub kills at 40 min with no XML. Now: the testrunner enforces the same 35m deadline itself, kills `gotestsum`, extracts any `--- FAIL:` lines captured from stdout, writes a report, and exits cleanly within the 5-minute buffer before GitHub's kill. ## How did you test it? - [X] built - [ ] run locally and tested manually - [ ] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s) ## Potential risks The testrunner's context deadline fires at the same time as go test's own -timeout. If both fire simultaneously, parseTestTimeouts (existing path) and the new `ctx.Err()` check (new path) can both trigger. The `ctx.Err()` check is guarded to only activate if the normal timeout-panic path didn't already break the loop, so there is no double-processing. |
||
|
Yichao Yang
|
0463b59dd9 |
CHASM: support archetypeID in admin handler (#9309)
## What changed? - Accept archetypeID in admin API requests ## Why? - With this we no longer need to register chasm components to worker service, which currently performs an archetypeID to name conversion before calling admin apis. - For backward compatibility, we can only stop registering chasm components to worker service starting from next cloud release. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) |
||
|
Alex Stanfield
|
ef9f3f4020 |
Add NewNilLibrary to scheduler for registration-only contexts (#9592)
## Summary - Adds `NewNilLibrary()` constructor to the scheduler package that returns a `Library` with all nil executors - Updates tdbg to use `NewNilLibrary()` instead of `NewLibrary(nil, nil, ...)`, avoiding breakage when new task executors are added ## Test plan - [x] `go build ./tools/tdbg/ ./chasm/lib/scheduler/` passes |
||
|
Lina Jodoin
|
e535cb5d01 |
[Scheduled Actions] Attach Nexus callbacks to schedules migrated from V1/Legacy (#9560)
## What changed? - Nexus callbacks are attached to running workflows after migration from a V1 workflow-backed schedule. This is done through a new side-effect task. ## Why? - Drives workflow completion events. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [x] added new functional test(s) |
||
|
Lina Jodoin
|
c52d2b7ebd |
Add tdbg command for 'schedule migrate' (#9502)
## What changed? - Added the 'tdbg schedule migrate' command ## Why? - Allows us an operator to manually trigger a migration (roll forward or backward) to a schedule via tdbg CLI ## How did you test it? - Tested manually with V1->CHASM direction. Verified (via `temporal` and `tdbg`) CHASM scheduler was created, V1 scheduler workflow closed successfully. |
||
|
Stephan Behnke
|
608da62841 |
Fix Crash and CI Breakers reporting (#9478)
## What changed? Fixed Crash and CI Breakers reporting in flakereport. ## Why? Misleading aggregation. ## How did you test it? ``` go run ./cmd/tools/flakereport generate --days 7 --output-dir out ``` <details><summary>GitHub (abridged)</summary> <p> ### Crashes | Test | Crash Rate | Last Failure | Links | |------|------------|-------------|-------| | `unit-test` | 4.0% (1/25) | 3h ago | [1](https://github.com/temporalio/temporal/actions/runs/22966733319) [2](https://github.com/temporalio/temporal/actions/runs/22923165266) | | `functional-test` | 3.5% (35/1003) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22970016748/job/66684243853) [2](https://github.com/temporalio/temporal/actions/runs/22970016748/job/66684243820) [3](https://github.com/temporalio/temporal/actions/runs/22969997600/job/66684218138) | ### CI Breakers (Failed All Retries) | Test | CI Break Rate | Last Failure | Links | |------|------------|-------------|-------| | `TestNexusStateReplicationTestSuite/EnableTransitionHistory/TestNexusCallbackReplicated` | **50.0% (13/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674252859) [2](https://github.com/temporalio/temporal/actions/runs/22931602170/job/66576965326) [3](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593587) | | `TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ClosedWorkflow` | **30.8% (8/26)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923524988/job/66528148344) [2](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396416) [3](https://github.com/temporalio/temporal/actions/runs/22929738731/job/66548562937) | | `TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ResetWorkflow` | **19.2% (5/26)** | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22928356811/job/66544280229) [2](https://github.com/temporalio/temporal/actions/runs/22923524988/job/66528148344) [3](https://github.com/temporalio/temporal/actions/runs/22929738731/job/66548562947) | | `TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ClosedWorkflow` | **19.2% (5/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674252859) [2](https://github.com/temporalio/temporal/actions/runs/22931602170/job/66576965326) [3](https://github.com/temporalio/temporal/actions/runs/22929738731/job/66548563003) | | `TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ResetWorkflow` | **15.4% (4/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22928356811/job/66544280299) [2](https://github.com/temporalio/temporal/actions/runs/22941944027/job/66585445737) [3](https://github.com/temporalio/temporal/actions/runs/22926291741/job/66537603391) | | `TestDeploymentVersionSuiteV2/TestReactivationSignalCache_Deduplication_Reset` | **11.5% (3/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) [2](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) [3](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestScheduleFunctionalSuite/TestListSchedulesReturnsWorkflowStatus` | **7.7% (2/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) [2](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Batching` | **7.7% (2/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) [2](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestClientDataConverterTestSuite` | **7.7% (2/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) [2](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal` | **7.7% (2/26)** | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967653083/job/66676176237) [2](https://github.com/temporalio/temporal/actions/runs/22967653083/job/66676176283) | | `TestWorkerDeploymentSuiteV2/TestResourceExhaustedErrors_Converted_To_ReadableMessage` | **7.7% (2/26)** | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) [2](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) | | `TestDeploymentVersionSuiteV0/TestVersionMissingTaskQueues_ValidSetCurrentVersion` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_PinnedOverride_WithSuggest/ForceTaskForwardNoPollForwardAllowSync` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_NormalWFT_WithSuggest/ForceTaskForwardNoPollForwardForceAsync` | 3.8% (1/26) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) | | `TestDeploymentVersionSuiteV2/TestVersionScavenger_DeleteOnAdd` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithSignal` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestVersioning3FunctionalSuiteV2/TestVersionedQueueUnload` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967653083/job/66676176237) | | `TestScheduleFunctionalSuite/TestNextTimeCache` | 3.8% (1/26) | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) | | `TestVersioningFunctionalSuite/TestDispatchActivityFail/NoTaskForwardForcePollForwardForceAsync` | 3.8% (1/26) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) | | `TestPrioritySuite/TestSubqueue_Migration` | 3.8% (1/26) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) | | `TestAdvancedVisibilitySuite/TestWorkerTaskReachability_Unversioned_InTaskQueue` | 3.8% (1/26) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) | | `TestDeploymentVersionSuiteV2/TestDeleteVersion_DrainingVersion` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned_WithConflictPolicy` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestTaskQueueStats_Classic_Suite/TestCurrentAbsorbsUnversionedBacklog_WhenRampingToUnversioned/ForceTaskForwardForcePollForwardAllowSync` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967653083/job/66676176237) | | `TestWorkflowUpdateSuite/SpeculativeWorkflowTask_QueryFailureClearsWFContext` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardForcePollForwardForceAsync` | 3.8% (1/26) | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) | | `TestTaskQueueStats_Pri_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/ForceTaskForwardNoPollForwardForceAsync` | 3.8% (1/26) | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) | | `TestVersioningFunctionalSuite/TestDispatchActivityTimeoutCrossTq/NoTaskForwardForcePollForwardAllowSync` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestWorkerDeploymentSuiteV0/TestDrainRollbackedVersion` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestWorkerDeploymentSuiteV0/TestSetWorkerDeploymentRampingVersion_SetRampingAsCurrent` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestStandaloneActivityTestSuite/TestHeartbeat/ActivityRetriesOnHeartbeatTimeout` | 3.8% (1/26) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) | | `TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/NoTaskForwardForcePollForwardForceAsync` | 3.8% (1/26) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) | | `TestTaskQueueStats_Pri_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardForcePollForwardAllowSync` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967653083/job/66676176237) | | `TestWorkerDeploymentSuiteV0/TestSetRampingVersion_Concurrent_DifferentVersions_NoUnexpectedErrors` | 3.8% (1/26) | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) | | `TestDeploymentVersionSuiteV0/TestVersionMissingTaskQueues_ValidSetRampingVersion` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestDeploymentVersionSuiteV2/TestVersionIgnoresDrainageSignalWhenCurrentOrRamping` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_TransientWFT_WithSuggest/ForceTaskForwardForcePollForwardAllowSync` | 3.8% (1/26) | 19h ago | [1](https://github.com/temporalio/temporal/actions/runs/22931281792/job/66553396454) | | `TestStandaloneActivityTestSuite/TestDescribeActivityExecution_DeadlineExceeded/NoCallerDeadline` | 3.8% (1/26) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22970016748/job/66684243820) | | `TestAdvancedVisibilitySuiteLegacy/TestUpsertWorkflowExecutionSearchAttributes` | 3.8% (1/26) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) | | `TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_NoSuggest/ForceTaskForwardForcePollForwardForceAsync` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestDeploymentVersionSuiteV0/TestVersionIgnoresDrainageSignalWhenCurrentOrRamping` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestVersioning3FunctionalSuiteV2/TestAutoUpgradeCaN_UpgradeOnCaN/ForceTaskForwardForcePollForwardAllowSync` | 3.8% (1/26) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22929090805/job/66546593717) | | `TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_SetImpliedPinnedSuccess` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_StartWorkflow` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestDeploymentVersionSuiteV2/TestVersionMissingTaskQueues_ValidSetRampingVersion` | 3.8% (1/26) | 28h ago | [1](https://github.com/temporalio/temporal/actions/runs/22912627598/job/66489344605) | | `TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_UpdateOptions` | 3.8% (1/26) | 122h ago | [1](https://github.com/temporalio/temporal/actions/runs/22775553239/job/66068056068) | | `TestNexusApiTestSuiteWithTemporalFailures/TestNexusCancelOperation_Outcomes/handler_timeout/ByNamespaceAndTaskQueue` | 3.8% (1/26) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) | | `TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967233794/job/66674253006) | | `TestCronTestSuite/TestCronWorkflow` | 3.8% (1/26) | 2h ago | [1](https://github.com/temporalio/temporal/actions/runs/22967653083/job/66676176283) | | `TestWorkerDeploymentSuiteV0/TestDeploymentVersionLimits` | 3.8% (1/26) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22923165266/job/66526932600) | </p> </details> <details><summary>Slack</summary> <p> Flaky Tests Report - Last 7 Days *CI Success Rate:* 9/26 (34.62%) *Total Test Runs:* 1134261 *Total Failures:* 3774 *Failure Rate:* 3.33 per 1000 tests *CI Breakers:* 50 *Crashes:* 2 *Flaky Tests:* 416 *Timeouts:* 0 *CI Breakers (top 10):* • 50.0% (13/26 runs): `TestNexusStateReplicationTestSuite/EnableTransitionHistory/TestNexusCallbackReplicated` • 30.8% (8/26 runs): `TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ClosedWorkflow` • 19.2% (5/26 runs): `TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ResetWorkflow` • 19.2% (5/26 runs): `TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ClosedWorkflow` • 15.4% (4/26 runs): `TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ResetWorkflow` • 11.5% (3/26 runs): `TestDeploymentVersionSuiteV2/TestReactivationSignalCache_Deduplication_Reset` • 7.7% (2/26 runs): `TestWorkerDeploymentSuiteV2/TestResourceExhaustedErrors_Converted_To_ReadableMessage` • 7.7% (2/26 runs): `TestClientDataConverterTestSuite` • 7.7% (2/26 runs): `TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Batching` • 7.7% (2/26 runs): `TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal` *Flaky Tests (top 10):* • 100.0% (4 failures): `DATA RACE: Data race detected` • 100.0% (24 failures): `TestClientDataConverterTestSuite` • 100.0% (4 failures): `PANIC: Fail in goroutine after TestWorkerDeploymentSuiteV2/TestConcurrentPollers_DifferentTaskQueues_SameVersion_SetCurrentVersion has completed — in TestDeploymentVersionSuiteV2/TestBatchUpdateWorkflowExecutionOptions_SetPinnedThenUnset` • 100.0% (36 failures): `PANIC: Fail in goroutine after TestWorkerDeploymentSuiteV2/TestConcurrentPollers_DifferentTaskQueues_SameVersion_SetCurrentVersion has completed — in TestWorkerDeploymentSuiteV2/TestConcurrentPollers_DifferentTaskQueues_SameVersion_SetCurrentVersion` • 100.0% (4 failures): `DATA RACE: Data race detected — in go.uber.org/zap/zaptest.TestingWriter.Write` • 49.3% (187 failures): `TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithoutSignal` • 48.9% (183 failures): `TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithSignal` • 47.5% (161 failures): `TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal` • 47.4% (172 failures): `TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithoutSignal` • 27.2% (71 failures): `TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ClosedWorkflow` </p> </details> |
||
|
Stephan Behnke
|
b9c650d841 |
Fix flakereport (#9399)
## What changed? Fix issues with flakereport; see comments for details. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) <details><summary>Example</summary> <p> ## Flaky Tests Report - 2026-02-25 11:25:50 ### Overall Statistics * **CI Success Rate**: 4/8 (50.00%) * **Total Test Runs**: 261830 * **Total Failures**: 404 * **Overall Failure Rate**: 1.5 per 1000 tests ### Failure Categories Summary | Category | Unique Tests | |----------|--------------| | CI Breakers | 1 | | Crashes | 1 | | Timeouts | 0 | | Flaky Tests | 129 | ### CI Breakers (Failed All Retries) | Test | CI Runs Broken | Total Failures | Last Failure | Links | |------|---------------|----------------|-------------|-------| | `TestVersionWorkflowSuite/v2/Test_DeleteVersion_QueryAfterDeletion` | 1 | 1 | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/1) | ### Crashes | Test | Flake Rate | Last Failure | Links | |------|------------|-------------|-------| | `unit-test` | **100.0% (2/2)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/1) [2](https://github.com/temporalio/temporal/actions/runs/22296780641/job/1) | ### Flaky Tests | Test | Flake Rate | Last Failure | Links | |------|------------|-------------|-------| | `TestClientDataConverterTestSuite` | **100.0% (1/1)** | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560060) | | `PANIC: runtime error: index out of range [0] with length 0 [recovered, repanicked] — in go.temporal.io/server/tests.TestWorkflowUpdateSuite.func21.2` | **100.0% (1/1)** | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560060) | | `PANIC: Fail in goroutine after TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky has completed — in TestVersioning3FunctionalSuiteV0/TestActivityRetryAutoUpgradeDuringBackoff` | **100.0% (1/1)** | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `PANIC: Fail in goroutine after TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/ForceTaskForwardNoPollForwardForceAsync has completed — in TestVersioning3FunctionalSuiteV2/TestActivityRetryAutoUpgradeDuringBackoff` | **100.0% (1/1)** | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `DATA RACE: Data race detected` | **100.0% (1/1)** | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `PANIC: runtime error: invalid memory address or nil pointer dereference — in go.temporal.io/server/tests.(*DeploymentVersionSuite).TestForceCAN_WithOverrideState.func1` | **100.0% (1/1)** | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal` | **48.9% (45/92)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456075) [2](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456229) [3](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456270) | | `TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithoutSignal` | **48.9% (45/92)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456093) [2](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456143) [3](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456167) | | `TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithSignal` | **48.9% (45/92)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456136) [2](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) [3](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456112) | | `TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithoutSignal` | **43.4% (36/83)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456143) [2](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456236) [3](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456250) | | `TestVersionWorkflowSuite/v2/Test_DeleteVersion_QueryAfterDeletion` | **42.9% (3/7)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/1) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/1) [3](https://github.com/temporalio/temporal/actions/runs/22373551837/job/1) | | `TestUserData_FetchesUpTree` | **16.7% (1/6)** | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/1) | | `TestNewServerWithOTEL/with_OTEL_Collector_running` | **16.7% (1/6)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/1) | | `TestInterleavedWeightedRoundRobinSchedulerSuite/TestInactiveChannelDeletionRace` | **16.7% (1/6)** | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/1) | | `TestUserData_FetchesActivityToWorkflow` | **16.7% (1/6)** | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/1) | | `TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_StartWorkflow` | **16.1% (9/56)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456136) [2](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) [3](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068532) | | `TestNewServer` | **15.4% (2/13)** | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/1) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/1) | | `TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_SignalWithStart` | **11.3% (6/53)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512764) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) [3](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198192) | | `TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned_WithConflictPolicy` | **9.6% (5/52)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512848) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198189) [3](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068462) | | `TestDeploymentVersionSuiteV2/TestVersionScavenger_DeleteOnAdd` | **9.6% (5/52)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512818) [3](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198207) | | `TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned` | **9.3% (5/54)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512766) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198202) [3](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068483) | | `TestDeploymentVersionSuiteV0/TestSignalWithStartWorkflowExecution_ReactivateVersionOnPinned` | **7.8% (4/51)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512865) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512840) [3](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512870) | | `TestFairnessAutoEnableSuite/Test_Activity_Basic` | **7.7% (4/52)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456112) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512799) [3](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068532) | | `TestDeploymentVersionSuiteV2/TestReactivationSignalCache_Deduplication_UpdateOptions` | **7.7% (4/52)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) [3](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestActivityClientTestSuite/TestActivityScheduleToClose_FiredDuringBackoff` | **7.5% (4/53)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [3](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198228) | | `TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_Reset` | **6.0% (3/50)** | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068487) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560087) [3](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114101) | | `TestDeploymentVersionSuiteV0/TestResetWorkflowExecution_ReactivateVersionOnPinned` | **6.0% (3/50)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512828) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) [3](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068520) | | `TestDeploymentVersionSuiteV2/TestVersionMissingTaskQueues_ValidSetRampingVersion` | **6.0% (3/50)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512818) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198207) [3](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114123) | | `TestTaskQueueStats_Pri_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/ForceTaskForwardForcePollForwardAllowSync` | **5.9% (3/51)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512818) [2](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068532) [3](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114123) | | `TestWorkerDeploymentSuiteV0/TestSetCurrentVersion_Concurrent_DifferentVersions_NoUnexpectedErrors` | **5.9% (3/51)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198193) [3](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_ReactivateVersionOnPinned` | **5.9% (3/51)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456393) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512824) [3](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestCronTestClientSuite/TestCronWorkflowCompletionStates` | **5.8% (3/52)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512824) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512861) [3](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestCallbacksSuiteCHASM/TestWorkflowNexusCallbacks_CarriedOver/ContinueAsNew` | **5.8% (3/52)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456103) [2](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068523) [3](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestStandaloneActivityTestSuite/TestHeartbeat/HeartbeatKeepsActivityAlive` | **5.8% (3/52)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512824) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198210) [3](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198202) | | `TestCallbacksSuiteCHASM/TestWorkflowNexusCallbacks_CarriedOver/WorkflowRunTimeout` | **5.8% (3/52)** | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456103) [2](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068523) [3](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestWorkerDeploymentSuiteV0/TestDescribeWorkerDeployment_SetCurrentVersion` | **5.8% (3/52)** | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [2](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [3](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198228) | | `TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_UpdateOptions` | 4.1% (2/49) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456093) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560043) | | `TestTaskQueueStats_Classic_Suite/TestAddMultipleTasks_ValidateStats_Cached` | 4.0% (2/50) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestTaskQueueStats_Pri_Suite/TestInactiveVersionDoesNotAbsorbUnversionedBacklog/NoTaskForwardNoPollForwardAllowSync` | 4.0% (2/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) [2](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198189) | | `TestTaskQueueStats_Pri_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardForcePollForwardAllowSync` | 4.0% (2/50) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512821) [2](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestWorkerDeploymentSuiteV0/TestSetRampingVersion_AfterDrained` | 4.0% (2/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068420) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestWorkerDeploymentSuiteV2/TestDrainRollbackedVersion` | 4.0% (2/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068525) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ResetWorkflow` | 4.0% (2/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456146) [2](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068440) | | `TestDeploymentVersionSuiteV0/TestVersionScavenger_DeleteOnAdd` | 4.0% (2/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068551) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Concurrent_DifferentVersions_NoUnexpectedErrors` | 4.0% (2/50) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560090) | | `TestPrioritySuite/TestSubqueue_Migration` | 3.9% (2/51) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) [2](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestCallbacksSuiteCHASM/TestWorkflowNexusCallbacks_CarriedOver/WorkflowFailureRetry` | 3.9% (2/51) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) [2](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560001) | | `TestWorkerDeploymentSuiteV0/TestDrainRollbackedVersion` | 3.9% (2/51) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198228) [2](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068512) | | `TestVersioning3FunctionalSuiteV0/TestPinnedTask_NoProperPoller/NoTaskForwardNoPollForwardForceAsync` | 2.1% (1/48) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068483) | | `TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_WithSuggest/ForceTaskForwardNoPollForwardForceAsync` | 2.1% (1/48) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114123) | | `TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_SetImpliedPinnedSuccess` | 2.1% (1/48) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512818) | | `TestGetHistoryFunctionalSuite/DisableTransitionHistory/TestGetWorkflowExecutionHistory_All` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestVersioning3FunctionalSuiteV2/TestUnpinnedCaN` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryNonFirstTask/NoTaskForwardForcePollForwardAllowSync` | 2.1% (1/48) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456112) | | `TestWorkerDeploymentSuiteV0/TestListWorkerDeployments_TwoVersions_SameDeployment_OneCurrent_OneRamping` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardNoPollForwardForceAsync` | 2.1% (1/48) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456212) | | `TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky/ForceTaskForwardForcePollForwardForceAsync` | 2.1% (1/48) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestAdvancedVisibilitySuiteLegacy/TestListWorkflow_StringQuery` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestVersioning3FunctionalSuiteV0/TestChildWorkflowInheritance_ParentPinnedByOverride` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198284) | | `TestStandaloneActivityTestSuite/TestRequestCancel/MismatchedTokenComponentRef` | 2.1% (1/48) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) | | `TestDeploymentVersionSuiteV2/TestVersionMissingTaskQueues_ValidSetCurrentVersion` | 2.1% (1/48) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) | | `TestVersioning3FunctionalSuiteV0/TestPinnedQuery_DrainedVersion_PollersAbsent/ForceTaskForwardNoPollForwardAllowSync` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestVersioning3FunctionalSuiteV0/TestPinnedWorkflowWithLateActivityPoller/ForceTaskForwardNoPollForwardAllowSync` | 2.1% (1/48) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114104) | | `TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardForcePollForwardAllowSync` | 2.1% (1/48) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512839) | | `TestDeploymentVersionSuiteV2/TestStartWorkflowExecution_ReactivateVersionOnPinned_WithConflictPolicy` | 2.1% (1/48) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) | | `TestDeploymentVersionSuiteV2/TestSignalWithStartWorkflowExecution_ReactivateVersionOnPinned` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestVersioningFunctionalSuite/TestDispatchQueryOld/ForceTaskForwardNoPollForwardAllowSync` | 2.1% (1/48) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198239) | | `TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryFirstTask/ForceTaskForwardForcePollForwardForceAsync` | 2.1% (1/48) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestTaskQueueStats_Pri_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardForcePollForwardForceAsync` | 2.1% (1/48) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068525) | | `TestWorkerDeploymentSuiteV0/TestDescribeWorkerDeployment_MultipleVersions_Sorted` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestDeploymentVersionSuiteV2/TestDrainageStatus_SetCurrentVersion_YesOpenWFs` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestVersioning3FunctionalSuiteV0/TestAutoUpgradeCaN_UpgradeOnCaN/ForceTaskForwardNoPollForwardAllowSync` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Batching` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068420) | | `TestTaskQueueStats_Classic_Suite/TestCurrentAbsorbsUnversionedBacklog_WhenRampingToUnversioned/NoTaskForwardForcePollForwardForceAsync` | 2.0% (1/49) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560082) | | `TestTaskQueueSuite/TestTaskQueueRateLimit_UpdateFromWorkerConfigAndAPI` | 2.0% (1/49) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456111) | | `TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/ForceTaskForwardNoPollForwardForceAsync` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestWorkflowUpdateSuite/StickySpeculativeWorkflowTask_AcceptComplete_StickyWorkerUnavailable` | 2.0% (1/49) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560060) | | `TestWorkerDeploymentSuiteV2/TestResourceExhaustedErrors_Converted_To_ReadableMessage` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114116) | | `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_TransientWFT_WithSuggest/ForceTaskForwardForcePollForwardForceAsync` | 2.0% (1/49) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) | | `TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_PinnedOverride_WithSuggest/NoTaskForwardForcePollForwardAllowSync` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestTaskQueueStats_Pri_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardNoPollForwardForceAsync` | 2.0% (1/49) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestChildWorkflowSuite/TestCronChildWorkflowExecution` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114114) | | `TestNexusStateReplicationTestSuite/DisableTransitionHistory/TestNexusCallbackReplicated` | 2.0% (1/49) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456192) | | `TestWorkerDeploymentSuiteV0/TestSetWorkerDeploymentRampingVersion_WithCurrent_Unset_Ramp` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ClosedWorkflow` | 2.0% (1/49) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198205) | | `TestTaskQueueStats_Classic_Suite/TestCurrentVersionAbsorbsUnversionedBacklog_NoRamping/ForceTaskForwardNoPollForwardForceAsync` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114074) | | `TestWorkflowUpdateSuite/SpeculativeWorkflowTask_ScheduleToStartTimeoutOnNormalTaskQueue` | 2.0% (1/49) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560060) | | `TestStandaloneActivityTestSuite/TestRequestCancel/StaleAttemptToken` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestTaskQueueStats_Classic_Suite/TestCurrentVersionAbsorbsUnversionedBacklog_NoRamping/NoTaskForwardForcePollForwardAllowSync` | 2.0% (1/49) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198284) | | `TestVersioning3FunctionalSuiteV2/TestUnpinnedQuery_NoSticky/ForceTaskForwardForcePollForwardForceAsync` | 2.0% (1/49) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512861) | | `TestTaskQueueStats_Classic_Suite/TestRampingAbsorbsUnversionedBacklog_WhenCurrentIsUnversioned/NoTaskForwardForcePollForwardForceAsync` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestTaskQueueStats_Pri_Suite/TestCurrentVersionAbsorbsUnversionedBacklog_NoRamping/NoTaskForwardNoPollForwardAllowSync` | 2.0% (1/49) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456212) | | `TestTaskQueueStats_Pri_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/NoTaskForwardNoPollForwardAllowSync` | 2.0% (1/49) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560089) | | `TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ClosedWorkflow` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114115) | | `TestWorkerDeploymentSuiteV0/TestSetRampingVersion_Concurrent_DifferentVersions_NoUnexpectedErrors` | 2.0% (1/49) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) | | `TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/NoTaskForwardNoPollForwardAllowSync` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/NoTaskForwardForcePollForwardAllowSync` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestVersioning3FunctionalSuiteV2/TestUnpinnedQuery_NoSticky/NoTaskForwardForcePollForwardAllowSync` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068523) | | `TestDeploymentVersionSuiteV0/TestForceCAN_WithOverrideState` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestVersioningFunctionalSuite/TestDispatchActivityFailCrossTq/NoTaskForwardForcePollForwardAllowSync` | 2.0% (1/49) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068496) | | `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_TransientWFT_WithSuggest/ForceTaskForwardNoPollForwardForceAsync` | 2.0% (1/49) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestWorkerDeploymentSuiteV0/TestSetWorkerDeploymentRampingVersion_Batching` | 2.0% (1/49) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestDeploymentVersionSuiteV0/TestDrainageStatus_SetCurrentVersion_NoOpenWFs` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068523) | | `TestScheduleFunctionalSuite/TestNextTimeCache` | 2.0% (1/50) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) | | `TestVersioning3FunctionalSuiteV2/TestUnpinnedQuery_NoSticky/ForceTaskForwardNoPollForwardAllowSync` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestPollerScalingFunctionalSuite/TestPollerScalingDecisionsAreSeenProbabilistically` | 2.0% (1/50) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestFairnessSuite/TestMigration_FromFair` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114116) | | `TestFairnessAutoEnableSuite/TestMigration_FromClassic` | 2.0% (1/50) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560075) | | `TestTaskQueueStats_Pri_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardNoPollForwardAllowSync` | 2.0% (1/50) | 23h ago | [1](https://github.com/temporalio/temporal/actions/runs/22367698230/job/64738198193) | | `TestWorkflowTaskTestSuite/TestWorkflowTaskHeartbeatingWithEmptyResult` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114142) | | `TestAdvancedVisibilitySuite/TestWorkerTaskReachability_Unversioned_InTaskQueue` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestDeploymentVersionSuiteV2/TestDeleteVersion_DeleteRampedVersion` | 2.0% (1/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) | | `TestWorkerDeploymentSuiteV0/TestDeploymentVersionLimits` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114116) | | `TestVersioning3FunctionalSuiteV2/TestPinnedQuery_RollbackDrainedVersion/ForceTaskForwardForcePollForwardForceAsync` | 2.0% (1/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) | | `TestVersioning3FunctionalSuiteV2/TestUnpinnedQuery_NoSticky/NoTaskForwardNoPollForwardAllowSync` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068512) | | `TestDeploymentVersionSuiteV2/TestDeleteVersion_Drained_But_Pollers_Exist` | 2.0% (1/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) | | `TestVersioning3FunctionalSuiteV2/TestQueryWithPinnedOverride_Sticky/ForceTaskForwardForcePollForwardAllowSync` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068512) | | `TestFairnessAutoEnableSuite/TestMigration_FromFair` | 2.0% (1/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) | | `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_SpeculativeWFT_NoSuggest/NoTaskForwardNoPollForwardAllowSync` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068525) | | `TestWorkerDeploymentSuiteV0/TestDescribeWorkerDeployment_TwoVersions_Sorted` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068512) | | `TestDeploymentVersionSuiteV2/TestDrainageStatus_SetCurrentVersion_NoOpenWFs` | 2.0% (1/50) | 1h ago | [1](https://github.com/temporalio/temporal/actions/runs/22410112796/job/64881456152) | | `TestDeploymentVersionSuiteV2/TestDeleteVersion_ValidDelete` | 2.0% (1/50) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512833) | | `TestScheduleFunctionalSuite/TestListSchedulesReturnsWorkflowStatus` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068523) | | `TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryFirstTask/ForceTaskForwardForcePollForwardAllowSync` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestCronTestSuite/TestCronWorkflow` | 2.0% (1/50) | 44h ago | [1](https://github.com/temporalio/temporal/actions/runs/22329840058/job/64609560092) | | `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_NormalWFT_WithSuggest/ForceTaskForwardNoPollForwardForceAsync` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068420) | | `TestVersioning3FunctionalSuiteV2/TestAutoUpgradeCaN_UpgradeOnCaN/ForceTaskForwardForcePollForwardAllowSync` | 2.0% (1/50) | 38h ago | [1](https://github.com/temporalio/temporal/actions/runs/22337683305/job/64634068472) | | `TestDeploymentVersionSuiteV2/TestReactivationSignalCache_Deduplication_SignalWithStart` | 2.0% (1/50) | 60h ago | [1](https://github.com/temporalio/temporal/actions/runs/22296780641/job/64495114191) | | `TestVersioning3FunctionalSuiteV2/TestPinnedQuery_DrainedVersion_PollersAbsent/ForceTaskForwardNoPollForwardAllowSync` | 2.0% (1/50) | 21h ago | [1](https://github.com/temporalio/temporal/actions/runs/22373551837/job/64758512837) | ### Flaky Suites | Suite | Flake Rate | Last Failure | |-------|------------|-------------| | `TestAcquireShard_DeadlineExceededErrorSuite` | 2.1% (1/47) | 38h ago | | `TestAcquireShard_OwnershipLostErrorSuite` | 2.1% (1/48) | 44h ago | | `TestActivityClientTestSuite` | **8.3% (4/48)** | 1h ago | | `TestActivityTestSuite` | 2.1% (1/48) | 44h ago | | `TestAdminTestSuite` | 2.1% (1/48) | 44h ago | | `TestAdvancedVisibilitySuite` | 4.2% (2/48) | 44h ago | | `TestAdvancedVisibilitySuiteLegacy` | 2.1% (1/48) | 23h ago | | `TestCallbacksSuiteCHASM` | **8.3% (4/48)** | 1h ago | | `TestChasmTestSuiteLegacy` | 2.1% (1/48) | 44h ago | | `TestChildWorkflowSuite` | 2.1% (1/48) | 60h ago | | `TestClientDataConverterTestSuite` | **100.0% (1/1)** | 44h ago | | `TestClientMiscTestSuite` | 2.1% (1/48) | 44h ago | | `TestCronTestClientSuite` | **6.2% (3/48)** | 21h ago | | `TestCronTestSuite` | 2.1% (1/48) | 44h ago | | `TestDLQSuite` | 4.2% (2/48) | 44h ago | | `TestDeploymentVersionSuiteV0` | **62.5% (30/48)** | 1h ago | | `TestDeploymentVersionSuiteV2` | **22.9% (11/48)** | 1h ago | | `TestFairnessAutoEnableSuite` | **8.3% (4/48)** | 1h ago | | `TestFairnessSuite` | 2.1% (1/48) | 60h ago | | `TestFuncClustersTestSuite` | **8.3% (4/48)** | 1h ago | | `TestGetHistoryFunctionalSuite` | 2.1% (1/48) | 23h ago | | `TestNamespaceInterceptorTestSuite` | **6.2% (3/48)** | 38h ago | | `TestNamespaceSuite` | 1.9% (1/53) | 44h ago | | `TestNexusApiTestSuiteWithTemporalFailures` | 2.1% (1/48) | 44h ago | | `TestNexusStateReplicationTestSuite` | 2.1% (1/48) | 1h ago | | `TestNexusWorkflowTestSuite` | 2.1% (1/48) | 44h ago | | `TestPollerScalingFunctionalSuite` | 2.1% (1/48) | 44h ago | | `TestPrioritySuite` | 4.2% (2/48) | 21h ago | | `TestPurgeDLQTasksSuite` | 2.1% (1/48) | 44h ago | | `TestQueryWorkflowSuite` | 2.1% (1/48) | 38h ago | | `TestRawHistoryClientSuite` | 2.1% (1/48) | 44h ago | | `TestRelayTaskTestSuite` | 2.1% (1/47) | 44h ago | | `TestResetWorkflowTestSuite` | 2.1% (1/48) | 44h ago | | `TestScheduleFunctionalSuite` | **6.2% (3/48)** | 21h ago | | `TestStandaloneActivityTestSuite` | **10.4% (5/48)** | 1h ago | | `TestTLSFunctionalSuite` | 2.1% (1/48) | 44h ago | | `TestTaskQueueStats_Classic_Suite` | **18.8% (9/48)** | 1h ago | | `TestTaskQueueStats_Pri_Suite` | **25.0% (12/48)** | 1h ago | | `TestTaskQueueSuite` | 4.2% (2/48) | 1h ago | | `TestTransientTaskSuite` | 2.1% (1/48) | 44h ago | | `TestUpdateWorkflowSdkSuite` | 2.1% (1/48) | 44h ago | | `TestUserMetadataSuite` | 2.1% (1/47) | 44h ago | | `TestUserTimersTestSuite` | 2.1% (1/47) | 44h ago | | `TestVersionWorkflowSuite` | **20.0% (1/5)** | 21h ago | | `TestVersioning3FunctionalSuiteV0` | **100.0% (48/48)** | 1h ago | | `TestVersioning3FunctionalSuiteV2` | **100.0% (48/48)** | 1h ago | | `TestVersioningFunctionalSuite` | **10.4% (5/48)** | 1h ago | | `TestWorkerDeploymentSuiteV0` | **16.7% (8/48)** | 21h ago | | `TestWorkerDeploymentSuiteV2` | **12.5% (6/48)** | 21h ago | | `TestWorkflowMemoTestSuite` | 2.1% (1/48) | 44h ago | | `TestWorkflowTaskTestSuite` | 4.2% (2/48) | 38h ago | | `TestWorkflowUpdateSuite` | 2.1% (1/48) | 44h ago | </p> </details> |
||
|
David Reiss
|
f9529d0481 |
Add matching fairness simulator (#8158)
## What changed? Add a new cli tool to run the matching fairness simulator, and some tests for basic fairness behavior. ## Why? So users can tell how it will behave for their workloads. |
||
|
Stephan Behnke
|
a5bc02b0fe |
Extend flakereport (#9379)
## What changed?
- Consolidate report rendering: tables (.md) for GitHub, plain-text
lists for Slack
- Emitting slack and github report artifacts instead of individual
report files
- Add per-test flake rate (percentage) and last-failure timestamp to
report tables
- Add per-suite breakdown table to GitHub summary that reports _all_
failures
- Use new `(final)` marker to label and identify CI breakers
- Report (and sort by) failure rate instead of absolute failures
## Why?
More detailed reporting helps with measuring progress and assigning
ownership.
## How did you test it?
- [ ] built
- [ ] run locally and tested manually
- [x] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
<details><summary>failures.json</summary>
<p>
```
[
{
"suite_name": "TestVersioning3FunctionalSuiteV0",
"test_name": "TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal",
"failure_date": "2026-02-21T02:49:39Z",
"link": "https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594600",
"failure_type": "flaky"
},
{
"suite_name": "TestDeploymentVersionSuiteV0",
"test_name": "TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned",
"failure_date": "2026-02-21T02:49:39Z",
"link": "https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594578",
"failure_type": "flaky"
},
{
"suite_name": "TestVersioning3FunctionalSuiteV0",
"test_name": "TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal",
"failure_date": "2026-02-21T02:49:39Z",
"link": "https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594578",
"failure_type": "flaky"
},
...
]
```
</p>
</details>
<details><summary>github-report.md</summary>
<p>
## Flaky Tests Report - 2026-02-22 09:57:39
### Overall Statistics
* **CI Success Rate**: 11/12 (91.67%)
* **Total Test Runs**: 540855
* **Total Failures**: 690
* **Overall Failure Rate**: 1.3 per 1000 tests
### Failure Categories Summary
| Category | Unique Tests |
|----------|--------------|
| CI Breakers | 0 |
| Crashes | 0 |
| Timeouts | 0 |
| Flaky Tests | 154 |
### Flaky Tests
| Test | Flake Rate | Last Failure | Links |
|------|------------|-------------|-------|
| `unit-test` | 100.0% (1/1) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/1)
|
| `DATA RACE: Data race detected` | 100.0% (1/1) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044027)
|
| `PANIC: Fail in goroutine after
TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky
has completed` | 100.0% (2/2) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
[2](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463408)
|
| `PANIC: Fail in goroutine after
TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky
has completed — in
TestVersioning3FunctionalSuiteV0/TestActivityRetryAutoUpgradeDuringBackoff`
| 100.0% (1/1) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795670)
|
|
`TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithoutSignal`
| 48.1% (89/185) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594471)
[2](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594526)
[3](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594553)
|
|
`TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithSignal`
| 47.8% (88/184) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594541)
[2](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594541)
[3](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594580)
|
|
`TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal`
| 47.8% (86/180) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594600)
[2](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594578)
[3](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594490)
|
|
`TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithoutSignal`
| 44.2% (76/172) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594471)
[2](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594526)
[3](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594504)
|
|
`TestMatchingEngine_Classic_Suite/TestMultipleWorkersLesserNumberOfPollersThanTasksNoDBErrors`
| 20.0% (3/15) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/1)
[2](https://github.com/temporalio/temporal/actions/runs/22202643938/job/1)
[3](https://github.com/temporalio/temporal/actions/runs/22192966846/job/1)
|
|
`TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned`
| 17.2% (20/116) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594578)
[2](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594490)
[3](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594455)
|
|
`TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_StartWorkflow`
| 15.0% (17/113) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594511)
[2](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512733)
[3](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512722)
|
| `TestNewServer` | 14.8% (4/27) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/1)
[2](https://github.com/temporalio/temporal/actions/runs/22202643938/job/1)
[3](https://github.com/temporalio/temporal/actions/runs/22202643938/job/1)
|
|
`TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_ReactivateVersionOnPinned`
| 14.3% (16/112) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512701)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677594)
[3](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677594)
|
|
`TestDeploymentVersionSuiteV0/TestSignalWithStartWorkflowExecution_ReactivateVersionOnPinned`
| 13.5% (15/111) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986322)
[2](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377523)
[3](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377509)
|
| `TestFairnessAutoEnableSuite/Test_Activity_Basic` | 12.7% (14/110) | 1
day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594557)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677620)
[3](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677659)
|
| `TestDeploymentVersionSuiteV2/TestVersionScavenger_DeleteOnAdd` |
11.1% (12/108) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594542)
[2](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512713)
[3](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677629)
|
|
`TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_SignalWithStart`
| 11.1% (12/108) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594504)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677635)
[3](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211836)
|
|
`TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_UpdateOptions`
| 8.6% (9/105) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677604)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677686)
[3](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211899)
|
| `TestProcessInvocationTask/invocation_timeout_by_request_timeout` |
7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/1)
|
| `TestRegistryWatchSuite/TestWatchNamespaceDeletedDuringRefresh` | 7.7%
(1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/1)
|
|
`TestCalculateTaskQueueVersioningInfo/new_format:_unversioned_ramping_with_newer_timestamp_with_another_ramping_version_in_a_different_deployment_->_ramping_is_still_versioned`
| 7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/1)
|
| `TestUserData_FetchesOnInit` | 7.7% (1/13) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192840306/job/1)
|
|
`TestProcessInvocationTask/invocation_timeout_by_ScheduleToCloseTimeout`
| 7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/1)
|
| `TestRegistryWatchSuite/TestWatchChannelCloses` | 7.7% (1/13) | 2 days
ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/1)
|
|
`TestMatchingEngine_Classic_Suite/TestConcurrentAdd_PollWorkflowTasksNoDBErrors`
| 7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/1)
|
| `TestVersionWorkflowSuite/v0/Test_DeleteVersion_QueryAfterDeletion` |
7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/1)
|
| `TestSQLiteTransactionContextCancellation` | 7.7% (1/13) | 2 days ago
|
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/1)
|
| `TestRegistryWatchSuite/TestWatchProcessEventError` | 7.7% (1/13) | 2
days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/1)
|
|
`TestMatchingEngine_Pri_Suite/TestMultipleWorkersLesserNumberOfPollersThanTasksNoDBErrors`
| 7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/1)
|
|
`TestCalculateTaskQueueVersioningInfo/new_format:_unversioned_current_with_newer_timestamp_with_another_current_version_in_a_different_deployment_->_current_is_still_versioned`
| 7.7% (1/13) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/1)
|
|
`TestDeploymentVersionSuiteV0/TestResetWorkflowExecution_ReactivateVersionOnPinned`
| 6.8% (7/103) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677654)
[2](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211855)
[3](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044027)
|
| `TestScheduleFunctionalSuite/TestListSchedulesReturnsWorkflowStatus` |
6.8% (7/103) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594458)
[2](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512677)
[3](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211905)
|
|
`TestWorkflowTaskTestSuite/TestWorkflowTaskHeartbeatingWithEmptyResult`
| 6.8% (7/103) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677624)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044048)
[3](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044038)
|
|
`TestDeploymentVersionSuiteV2/TestVersionMissingTaskQueues_ValidSetRampingVersion`
| 5.9% (6/102) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594542)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677629)
[3](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211915)
|
|
`TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned_WithConflictPolicy`
| 5.9% (6/102) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044077)
[2](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986296)
[3](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202983)
|
| `TestCronTestSuite/TestCronWorkflow` | 5.9% (6/102) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512677)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677630)
[3](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986281)
|
|
`TestFuncClustersTestSuite/DisableTransitionHistory/TestForceMigration_ClosedWorkflow`
| 4.0% (4/100) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044035)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044065)
[3](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202982)
|
|
`TestWorkflowTestSuite/TestWorkflowTaskAndActivityTaskTimeoutsWorkflow`
| 4.0% (4/100) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677594)
[2](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986281)
[3](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202904)
|
| `TestFairnessSuite/TestMigration_FromClassic` | 4.0% (4/100) | 1 day
ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594552)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677659)
[3](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377543)
|
| `TestVersioning3FunctionalSuiteV0/TestPinnedCaN_SameTQ_PinnedOverride`
| 4.0% (4/100) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211825)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044048)
[3](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377550)
|
| `TestWorkerDeploymentSuiteV0/TestDrainRollbackedVersion` | 4.0%
(4/100) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594458)
[2](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677640)
[3](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677630)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedWorkflowWithLateActivityPoller/NoTaskForwardNoPollForwardAllowSync`
| 3.1% (3/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677640)
[2](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986331)
[3](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066924)
|
|
`TestWorkerDeploymentSuiteV0/TestSetCurrentVersion_Concurrent_DifferentVersions_NoUnexpectedErrors`
| 3.0% (3/99) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986331)
[2](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377534)
[3](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795670)
|
| `TestFairnessSuite/TestMigration_FromPri` | 3.0% (3/99) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594511)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044089)
[3](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066839)
|
|
`TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ResetWorkflow`
| 3.0% (3/99) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211888)
[2](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795652)
[3](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227310)
|
| `TestCronTestClientSuite/TestCronWorkflowCompletionStates` | 3.0%
(3/99) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677594)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044048)
[3](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203021)
|
| `TestDeploymentVersionSuiteV2/TestDeleteVersion_DrainingVersion` |
3.0% (3/99) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677711)
[2](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986362)
[3](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066808)
|
| `TestWorkerDeploymentSuiteV2/TestDrainRollbackedVersion` | 3.0% (3/99)
| 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044075)
[2](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202851)
[3](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463304)
|
|
`TestWorkerDeploymentSuiteV0/TestConcurrentPollers_DifferentTaskQueues_SameVersion_SetCurrentVersion`
| 3.0% (3/99) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044048)
[2](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
[3](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066765)
|
|
`TestWorkerDeploymentSuiteV0/TestSetRampingVersion_Concurrent_DifferentVersions_NoUnexpectedErrors`
| 2.1% (2/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
[2](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463408)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/ForceTaskForwardNoPollForwardForceAsync`
| 2.1% (2/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677630)
[2](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795564)
|
|
`TestWorkerDeploymentSuiteV0/TestSetManagerIdentity_WithSetRampSetCurrent`
| 2.1% (2/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202851)
[2](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463408)
|
|
`TestSizeLimitFunctionalSuite/TestTerminateWorkflowCausedByHistoryCountLimit`
| 2.0% (2/98) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512700)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044036)
|
|
`TestWorkflowUpdateSuite/SpeculativeWorkflowTask_QueryFailureClearsWFContext`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211923)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044066)
|
|
`TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_Reset`
| 2.0% (2/98) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512722)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044066)
|
|
`TestTaskQueueStats_Classic_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/ForceTaskForwardForcePollForwardAllowSync`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377504)
[2](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227241)
|
|
`TestTaskQueueStats_Classic_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/NoTaskForwardForcePollForwardAllowSync`
| 2.0% (2/98) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512684)
[2](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066999)
|
|
`TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/NoTaskForwardForcePollForwardAllowSync`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377517)
[2](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066925)
|
|
`TestDeploymentVersionSuiteV2/TestReactivationSignalCache_Deduplication_SignalWithStart`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
[2](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377550)
|
|
`TestTaskQueueSuite/TestTaskQueueRateLimit_UpdateFromWorkerConfigAndAPI`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211915)
[2](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986291)
|
|
`TestWorkerDeploymentSuiteV2/TestResourceExhaustedErrors_Converted_To_ReadableMessage`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044075)
[2](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202851)
|
| `TestFairnessAutoEnableSuite/TestMigration_FromFair` | 2.0% (2/98) | 2
days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
[2](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795564)
|
|
`TestWorkerDeploymentSuiteV0/TestDescribeWorkerDeployment_SetCurrentVersion`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044048)
[2](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203029)
|
| `TestFairnessAutoEnableSuite/TestMigration_FromClassic` | 2.0% (2/98)
| 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
[2](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066765)
|
|
`TestActivityClientTestSuite/TestActivityScheduleToClose_FiredDuringBackoff`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
[2](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202997)
|
|
`TestVersioning3FunctionalSuiteV0/TestChildWorkflowInheritance_ParentPinnedByOverride`
| 2.0% (2/98) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594542)
[2](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512684)
|
|
`TestQueryWorkflowSuite/TestQueryWorkflow_WithRawHistoryBytesToMatchingService`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986281)
[2](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795543)
|
| `TestUpdateWorkflowSdkSuite/TestTimeoutWithRetryAfterUpdateAdmitted` |
2.0% (2/98) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512681)
[2](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044064)
|
|
`TestFuncClustersTestSuite/EnableTransitionHistory/TestForceMigration_ClosedWorkflow`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211871)
[2](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227310)
|
|
`TestTaskQueueStats_Classic_Suite/TestRampingAbsorbsUnversionedBacklog_WhenCurrentIsUnversioned/NoTaskForwardForcePollForwardAllowSync`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202992)
[2](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203029)
|
| `TestDeploymentVersionSuiteV0/TestVersionScavenger_DeleteOnAdd` | 2.0%
(2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795638)
[2](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066924)
|
| `TestVersioning3FunctionalSuiteV2/TestPinnedCaN_SameTQ_PinnedOverride`
| 2.0% (2/98) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594477)
[2](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512671)
|
|
`TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky/ForceTaskForwardForcePollForwardAllowSync`
| 2.0% (2/98) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
[2](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795670)
|
|
`TestStandaloneActivityTestSuite/TestHeartbeat/HeartbeatWithNoTimeoutDoesNotKillActivity`
| 1.1% (1/89) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377534)
|
|
`TestWorkerDeploymentSuiteV0/TestSetWorkerDeploymentRampingVersion_WithCurrent_Unset_Ramp`
| 1.1% (1/94) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594455)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/96) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594527)
|
|
`TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryFirstTask/ForceTaskForwardForcePollForwardForceAsync`
| 1.0% (1/96) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/NoTaskForwardNoPollForwardAllowSync`
| 1.0% (1/96) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795648)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedTask_NoProperPoller/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/96) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463139)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/96) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795564)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedWorkflowWithLateActivityPoller/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/96) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463408)
|
| `TestWorkerDeploymentSuiteV0/TestSetRampingVersion_AfterDrained` |
1.0% (1/96) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594490)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_TransientWFT_WithSuggest/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/96) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066904)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedQuery_RollbackDrainedVersion/NoTaskForwardForcePollForwardForceAsync`
| 1.0% (1/96) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677630)
|
|
`TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Concurrent_DifferentVersions_NoUnexpectedErrors`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227188)
|
|
`TestWorkerDeploymentSuiteV2/TestDeleteVersion_ServerDeleteMaxVersionsReached`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203029)
|
|
`TestNexusApiTestSuiteWithLegacyErrorPaths/TestNexusStartOperation_Outcomes/handler_timeout/ByEndpoint`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986331)
|
|
`TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryFirstTask/NoTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795670)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedQuery_DrainedVersion_PollersAbsent/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
|
|
`TestStandaloneActivityTestSuite/TestHeartbeat/HeartbeatKeepsActivityAlive`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512677)
|
|
`TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
|
|
`TestCancelWorkflowSuite/TestRequestCancelWorkflowCommandExecution_TargetRunning`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594527)
|
| `TestPrioritySuite/TestSubqueue_Migration` | 1.0% (1/97) | 2 days ago
|
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795638)
|
|
`TestVersioning3FunctionalSuiteV2/TestUnpinnedQuery_NoSticky/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203021)
|
|
`TestStandaloneActivityTestSuite/TestDescribeActivityExecution_Completed/termination`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
|
|
`TestTaskQueueStats_Pri_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044048)
|
| `TestClientMiscTestSuite/TestTooManyPendingActivities` | 1.0% (1/97) |
2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066765)
|
|
`TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryNonFirstTask/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377519)
|
| `TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Batching` | 1.0%
(1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986331)
|
|
`TestWorkerDeploymentSuiteV0/TestDeleteVersion_ServerDeleteMaxVersionsReached`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
|
|
`TestDeploymentVersionSuiteV2/TestVersionIgnoresDrainageSignalWhenCurrentOrRamping`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202978)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedQuery_DrainedVersion_PollersAbsent/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377550)
|
| `TestVersioning3FunctionalSuiteV2/TestVersionedQueueUnload` | 1.0%
(1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066924)
|
|
`TestVersioning3FunctionalSuiteV2/TestUnpinnedQuery_NoSticky/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044038)
|
|
`TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_SetPinnedThenUnset`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795666)
|
| `TestActivityTestSuite/TestTryActivityCancellationFromWorkflow` | 1.0%
(1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594542)
|
|
`TestWorkerDeploymentSuiteV2/TestSetWorkerDeploymentRampingVersion_DuplicateRamp`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795564)
|
|
`TestPollerScalingFunctionalSuite/TestPollerScalingDecisionsAreSeenProbabilistically`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066832)
|
|
`TestDeploymentVersionSuiteV2/TestReactivationSignalCache_Deduplication_Reset`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377550)
|
| `TestWorkflowBufferedEventsTestSuite/TestBufferedEvents` | 1.0% (1/97)
| 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227199)
|
|
`TestCallbacksSuiteHSM/TestWorkflowNexusCallbacks_CarriedOver/WorkflowRunTimeout`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677630)
|
|
`TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryNonFirstTask/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044036)
|
|
`TestFuncClustersTestSuite/EnableTransitionHistory/TestLocalNamespaceMigration`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377741)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedQuery_DrainedVersion_PollersAbsent/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594527)
|
|
`TestDeploymentVersionSuiteV2/TestResetWorkflowExecution_ReactivateVersionOnPinned`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463717)
|
|
`TestTaskQueueStats_Pri_Suite/TestInactiveVersionDoesNotAbsorbUnversionedBacklog/ForceTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594511)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedQuery_DrainedVersion_PollersAbsent/NoTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594527)
|
|
`TestTaskQueueStats_Pri_Suite/TestRampingAbsorbsUnversionedBacklog_WhenCurrentIsUnversioned/NoTaskForwardNoPollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066999)
|
|
`TestVersioning3FunctionalSuiteV2/TestChildWorkflowInheritance_CrossTQ_NoInherit`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795638)
|
|
`TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_SetImpliedPinnedSuccess`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677629)
|
|
`TestWorkerDeploymentSuiteV2/TestConflictToken_Describe_SetCurrent_SetRamping`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066791)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedQuery_RollbackDrainedVersion/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377550)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_PinnedOverride_NoSuggest/NoTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044027)
|
| `TestActivityTestSuite/TestActivityHeartBeatWorkflow_Timeout` | 1.0%
(1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677620)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_SpeculativeWFT_NoSuggest/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203033)
|
|
`TestWorkerDeploymentSuiteV0/TestConflictToken_Describe_SetCurrent_SetRamping`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
|
|
`TestCallbacksSuiteHSM/TestWorkflowNexusCallbacks_CarriedOver/ContinueAsNew`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986295)
|
|
`TestTaskQueueStats_Pri_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/NoTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202810693/job/64219986362)
|
|
`TestDeploymentVersionSuiteV2/TestDrainageStatus_SetCurrentVersion_NoOpenWFs`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227199)
|
|
`TestTaskQueueStats_Pri_Suite/TestInactiveVersionDoesNotAbsorbUnversionedBacklog/NoTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066939)
|
|
`TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_SetPinnedSetUnpinned`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377517)
|
|
`TestDeploymentVersionSuiteV2/TestDeleteVersion_Drained_But_Pollers_Exist`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22193844135/job/64189066924)
|
|
`TestTaskQueueStats_Classic_Suite/TestMultipleTasks_WithMatchingBehavior_ValidateStats/NoTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512684)
|
|
`TestStandaloneActivityTestSuite/TestRequestCancel/MismatchedTokenComponentRef`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22232694520/job/64316677629)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_WithSuggest/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202978)
|
|
`TestVersioningFunctionalSuite/TestWorkflowTaskRedirectInRetryFirstTask/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22246779209/job/64362512723)
|
|
`TestTaskQueueStats_Classic_Suite/TestRampingAndCurrentAbsorbUnversionedBacklog/ForceTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795712)
|
|
`TestVersioning3FunctionalSuiteV2/TestChildWorkflowInheritance_ParentPinnedByOverride`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22210016119/job/64242211866)
|
|
`TestStandaloneActivityTestSuite/Test_ScheduleToCloseTimeout_WithRetry`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227199)
|
|
`TestVersioning3FunctionalSuiteV2/TestWorkflowWithPinnedOverride_NoSticky/NoTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463408)
|
|
`TestDeploymentVersionSuiteV2/TestUpdateWorkflowExecutionOptions_SetImpliedPinnedSuccess`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
|
|
`TestVersioning3FunctionalSuiteV2/TestChildWorkflowInheritance_CrossTQ_Inherit`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192840306/job/64185463045)
|
|
`TestTaskQueueStats_Classic_Suite/TestInactiveVersionDoesNotAbsorbUnversionedBacklog/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22194310573/job/64192795725)
|
|
`TestTaskQueueStats_Classic_Suite/TestRampingAbsorbsUnversionedBacklog_WhenCurrentIsUnversioned/ForceTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227188)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedCaN_UpgradeOnCaN_NormalWFT_WithSuggest/NoTaskForwardNoPollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210202978)
|
|
`TestVersioning3FunctionalSuiteV0/TestUnpinnedQuery_NoSticky/NoTaskForwardForcePollForwardForceAsync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044027)
|
|
`TestNexusStateReplicationTestSuite/DisableTransitionHistory/TestNexusCallbackReplicated`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377525)
|
|
`TestVersioning3FunctionalSuiteV0/TestPinnedCaN_UpgradeOnCaN_TransientWFT_WithSuggest/ForceTaskForwardNoPollForwardAllowSync`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377457)
|
| `TestUpdateWorkflowSdkSuite/TestTerminateWorkflowAfterUpdateAdmitted`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377527)
|
|
`TestVersioning3FunctionalSuiteV2/TestPinnedQuery_RollbackDrainedVersion/NoTaskForwardForcePollForwardAllowSync`
| 1.0% (1/97) | 3 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22192966846/job/64186227181)
|
|
`TestWorkerDeploymentSuiteV2/TestSetCurrentVersion_Unversioned_PromoteUnversionedRamp`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22208626141/job/64238044038)
|
| `TestActivityClientTestSuite/Test_ActivityTimeouts` | 1.0% (1/97) | 2
days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22199962060/job/64210203021)
|
|
`TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_EmptyFields`
| 1.0% (1/97) | 2 days ago |
[1](https://github.com/temporalio/temporal/actions/runs/22202643938/job/64219377543)
|
|
`TestDeploymentVersionSuiteV2/TestUpdateWorkflowExecutionOptions_ReactivateVersionOnPinned`
| 1.0% (1/97) | 1 day ago |
[1](https://github.com/temporalio/temporal/actions/runs/22248930374/job/64368594552)
|
### Flaky Suites
| Suite | Flake Rate | Last Failure |
|-------|------------|-------------|
| `TestActivityClientTestSuite` | 16.7% (2/12) | 2 days ago |
| `TestActivityTestSuite` | 16.7% (2/12) | 1 day ago |
| `TestCallbacksSuiteHSM` | 16.7% (2/12) | 2 days ago |
| `TestCancelWorkflowSuite` | 8.3% (1/12) | 1 day ago |
| `TestClientMiscTestSuite` | 8.3% (1/12) | 2 days ago |
| `TestCronTestClientSuite` | 25.0% (3/12) | 2 days ago |
| `TestCronTestSuite` | 50.0% (6/12) | 1 day ago |
| `TestDeploymentVersionSuiteV0` | 100.0% (12/12) | 1 day ago |
| `TestDeploymentVersionSuiteV2` | 100.0% (12/12) | 1 day ago |
| `TestFairnessAutoEnableSuite` | 75.0% (9/12) | 1 day ago |
| `TestFairnessSuite` | 50.0% (6/12) | 1 day ago |
| `TestFuncClustersTestSuite` | 58.3% (7/12) | 2 days ago |
| `TestMatchingEngine_Classic_Suite` | 33.3% (4/12) | 2 days ago |
| `TestMatchingEngine_Pri_Suite` | 8.3% (1/12) | 2 days ago |
| `TestNexusApiTestSuiteWithLegacyErrorPaths` | 8.3% (1/12) | 2 days ago
|
| `TestNexusStateReplicationTestSuite` | 8.3% (1/12) | 2 days ago |
| `TestPollerScalingFunctionalSuite` | 8.3% (1/12) | 2 days ago |
| `TestPrioritySuite` | 8.3% (1/12) | 2 days ago |
| `TestQueryWorkflowSuite` | 16.7% (2/12) | 2 days ago |
| `TestRegistryWatchSuite` | 25.0% (3/12) | 2 days ago |
| `TestScheduleFunctionalSuite` | 58.3% (7/12) | 1 day ago |
| `TestSizeLimitFunctionalSuite` | 16.7% (2/12) | 1 day ago |
| `TestStandaloneActivityTestSuite` | 41.7% (5/12) | 1 day ago |
| `TestTaskQueueStats_Classic_Suite` | 50.0% (6/12) | 1 day ago |
| `TestTaskQueueStats_Pri_Suite` | 33.3% (4/12) | 1 day ago |
| `TestTaskQueueSuite` | 16.7% (2/12) | 2 days ago |
| `TestUpdateWorkflowSdkSuite` | 25.0% (3/12) | 1 day ago |
| `TestVersionWorkflowSuite` | 8.3% (1/12) | 2 days ago |
| `TestVersioning3FunctionalSuiteV0` | 100.0% (12/12) | 1 day ago |
| `TestVersioning3FunctionalSuiteV2` | 100.0% (12/12) | 1 day ago |
| `TestVersioningFunctionalSuite` | 33.3% (4/12) | 1 day ago |
| `TestWorkerDeploymentSuiteV0` | 75.0% (9/12) | 1 day ago |
| `TestWorkerDeploymentSuiteV2` | 58.3% (7/12) | 2 days ago |
| `TestWorkflowBufferedEventsTestSuite` | 8.3% (1/12) | 3 days ago |
| `TestWorkflowTaskTestSuite` | 50.0% (6/12) | 2 days ago |
| `TestWorkflowTestSuite` | 33.3% (4/12) | 2 days ago |
| `TestWorkflowUpdateSuite` | 16.7% (2/12) | 2 days ago |
</p>
</details>
<details><summary>slack-report.md</summary>
<p>
Flaky Tests Report - Last 3 Days
*CI Success Rate:* 11/12 (91.67%)
*Total Test Runs:* 540855
*Total Failures:* 690
*Failure Rate:* 1.28 per 1000 tests
*CI Breakers:* 0
*Crashes:* 0
*Flaky Tests:* 154
*Timeouts:* 0
*Flaky Tests (top 10):*
• 48.1% (89 failures):
`TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithoutSignal`
• 47.8% (88 failures):
`TestVersioning3FunctionalSuiteV2/TestTransitionDuringTransientTask_WithSignal`
• 47.8% (86 failures):
`TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithSignal`
• 44.2% (76 failures):
`TestVersioning3FunctionalSuiteV0/TestTransitionDuringTransientTask_WithoutSignal`
• 20.0% (3 failures):
`TestMatchingEngine_Classic_Suite/TestMultipleWorkersLesserNumberOfPollersThanTasksNoDBErrors`
• 17.2% (20 failures):
`TestDeploymentVersionSuiteV0/TestStartWorkflowExecution_ReactivateVersionOnPinned`
• 15.0% (17 failures):
`TestDeploymentVersionSuiteV0/TestReactivationSignalCache_Deduplication_StartWorkflow`
• 14.8% (4 failures): `TestNewServer`
• 14.3% (16 failures):
`TestDeploymentVersionSuiteV0/TestUpdateWorkflowExecutionOptions_ReactivateVersionOnPinned`
• 13.5% (15 failures):
`TestDeploymentVersionSuiteV0/TestSignalWithStartWorkflowExecution_ReactivateVersionOnPinned`
</p>
</details>
|
||
|
Stephan Behnke
|
73119cd3ef |
go fix workflow (#9353)
## What changed? 1. Tweak on the loop in Makefile target `fmt-gofix` to retry based on output log. 2. Include `fmt-gofix` in CI. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) |
||
|
Sean Kane
|
2419317efe |
Convert flake report from Python to Go (#9334)
## What changed? * Convert the flake report we use to monitor functional test reliability from invoking the `tringa` cli tool and a Python script to only a Go script that fetches the artifacts, parses the JUnit, and creates the report. * This also fixes an issue where `tringa` only found the last 20 tests, the Go script will iterate through all failures in the last `N` days ## Why? Everything is in a single executable making it easier to test and extend. ## How did you test it? - [X] built - [X] run locally and tested manually - [ ] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Minimal, this is only a tooling change. |
||
|
Stephan Behnke
|
4a2e88a173 |
Parallelize integration tests (#9292)
## What changed? Made integration tests run in parallel. ## Why? Before: ~8min [[run](https://github.com/temporalio/temporal/actions/runs/21930252400/job/63333789136#step:7:1)] 🐢 After: ~3m [[run](https://github.com/temporalio/temporal/actions/runs/22114061618/job/63917852614?pr=9292)] 🐰 ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks They are not known to be flaky; and anecdotally all passed on the first run. |
||
|
Stephan Behnke
|
5b49acfaf9 |
go fix (#9337)
## What changed?
Integrate Go 1.26's new `go fix` into workflow.
NOTE that the changes caused our linter to fire; a [separate
commit](
|
||
|
Roey Berman
|
e0d9f48c84 |
System nexus endpoint (#9002)
## Overview
This feature introduces a special internal Nexus endpoint called
`__temporal_system` that enables adding functionality to workflows
**without requiring new workflow commands and events**. Operations on
this endpoint are routed internally within Temporal's history service
rather than via external HTTP calls.
## Key Components
### 1. System Endpoint Infrastructure
- **Endpoint Name**: `__temporal_system` (constant in
`common/nexus/constants.go:6`)
- **Callback URL**: `temporal://system` for internal routing
- **New History Service RPCs**
(`proto/internal/temporal/server/api/historyservice/v1/service.proto:433-437`):
- `StartNexusOperation` - Starts operations on the system endpoint
- `CancelNexusOperation` - Cancels operations on the system endpoint
### 2. Operation Processor Framework
A new processor pattern (`chasm/nexus_operation_processor.go`) that
allows CHASM libraries to:
- **Validate and transform input**: Processors can validate operation
inputs and set default values
- **Determine routing**: Each processor returns a routing key that
determines which history shard handles the operation
- **Re-serialize input**: Mutated inputs can be re-serialized to persist
default values
**Routing strategies**:
- `NexusOperationRoutingKeyExecution` - Routes to the shard owning a
specific workflow execution
- `NexusOperationRoutingKeyRandom` - Routes to a random shard
### 3. CHASM Library Integration
CHASM libraries can now provide (`chasm/library.go:16-19`):
- **`NexusServices()`**: Regular Nexus service handlers (implement the
actual operation logic)
- **`NexusServiceProcessors()`**: Input processors for validation and
routing
Example from test library (`chasm/lib/tests/nexus_service.go`):
```go
// Service handler - implements the actual operation
TestOperation = nexus.NewSyncOperation("TestOperation",
func(ctx context.Context, input string, options nexus.StartOperationOptions) (string, error) {
return "Hello, " + input, nil
})
// Processor - validates input and determines routing
func (o testOperationProcessor) ProcessInput(ctx chasm.NexusOperationProcessorContext, input string)
(*chasm.NexusOperationProcessorResult, error) {
return &chasm.NexusOperationProcessorResult{
RoutingKey: chasm.NexusOperationRoutingKeyExecution{
NamespaceID: ctx.Namespace.ID().String(),
BusinessID: input, // Route based on input
},
}, nil
}
```
### 4. Execution Flow
When a workflow schedules a Nexus operation on `__temporal_system`
(`components/nexusoperations/executors.go:233-238`):
1. **Input Processing**: The processor validates input and determines
routing
2. **Internal RPC**: Instead of HTTP, calls
`HistoryClient.StartNexusOperation` with the target shard ID
3. **Handler Execution**: The history service invokes the registered
Nexus handler (`service/history/handler.go:2707-2768`)
4. **Result Handling**: Supports both sync (immediate result) and async
(operation token) responses
5. **Workflow Completion**: Results flow back through the same
completion path as external Nexus operations
### 5. Benefits
✅ **No schema changes**: Add functionality without new commands/events
in workflow history
✅ **Consistent API**: Uses existing Nexus operation semantics
(sync/async, callbacks, links)
✅ **Proper routing**: Operations are intelligently routed to the correct
shard
✅ **Input validation**: Type-safe input validation and default value
handling
✅ **Future extensibility**: Foundation for direct client invocation (not
yet implemented)
### 6. Technical Details
- **Error handling** (`components/nexusoperations/executors.go:444`):
Non-retryable service errors are properly handled and fail operations
immediately
- **Metrics**: System operations are tracked separately with
`DestinationTag` set to the endpoint name
- **Link conversion**: Helper functions convert between Nexus SDK links
and protobuf links (`common/nexus/util.go:17-46`)
- **Operation token handling**: Moved link converters to common package
for reuse (`common/nexus/link_converter.go`)
### 7. Current Limitations
- Only accessible from workflows (via `ScheduleNexusOperation` command)
- Direct client invocation not yet implemented
- Headers not supported for system endpoint operations
## Test Coverage
New test (`tests/nexus_workflow_test.go:2763-2843`) demonstrates:
- Scheduling operation on `__temporal_system` endpoint
- Synchronous operation completion
- Result propagation back to workflow
## Architecture
This architecture provides a clean, extensible way to add internal
functionality while maintaining compatibility with Temporal's existing
workflow execution model. The system endpoint acts as a bridge between
workflows and internal CHASM components, enabling:
- **Extensibility**: New operations can be added by implementing CHASM
libraries
- **Type safety**: Input validation happens before operations are routed
- **Scalability**: Intelligent routing ensures operations land on the
correct shard
- **Consistency**: Same execution model as external Nexus operations
|
||
|
Stephan Behnke
|
595713fb01 |
auto-decode for tdbg workflow show (#9294)
## What changed? Adds `--decode` flag to tdbg's `wofkflow show` to decode event history proto payloads to JSON. ## Why? Make on-call investigation easier. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [ ] added new functional test(s) ## Potential risks The decoder ignores errors; so if it fails, it behaves as before. |
||
|
Sean Kane
|
938fc41a8d |
Normalize the test names for reporting (#9293)
## What changed? Remove `(retry 1/2)` from test name for more accurate reporting. ## Why? More accurate reporting. ## How did you test it? - [X] built - [X] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks Reporting only |
||
|
Stephan Behnke
|
914a24a656 |
Move sharding salt to file (#9265)
## What changed? Instead of storing the test sharing salt in a repo var, store it in a file. ## Why? https://github.com/temporalio/temporal/pull/9236 tried to use the repo's variable for storing the test sharding salt; but that is not accessible to PRs from forks. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks PRs could fail due to flaky tests. But then the next run will delete the branch (thereby closing the PR) and open a new one. |
||
|
Sean Kane
|
3a5091770b |
Remove "recent failures" from slack report (#9274)
## What changed? WISOTT ## Why? Cleaner report ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks None, reporting only |