mirror of
https://github.com/temporalio/temporal.git
synced 2026-08-30 18:41:49 -07:00
438847b142
## What changed?
- Adds `phase=error` to the existing `replication_lifecycle` wide event;
no new event type or table.
- Covers state-based replication (`SyncVersionedTransition`,
`VerifyVersionedTransition`, and `SyncWorkflowState`) plus standby
transfer, timer, and outbound queue failures.
- Captures sender, passive execution/apply, verification,
recovery/refetch, namespace refresh, Nack, DLQ, and history-branch
cleanup boundaries.
- Uses one shared error builder with small sender, executable-task, NDC,
and standby-queue adapters.
- Records workflow identity, source task identity, target context,
operation, error, attempt/priority, disposition/recovery, and extensible
diagnostics in `details`.
- Identifies apply provenance as `apply_artifact_source=task_payload` or
`sync_state_refetch`.
- Remains gated by `history.emitReplicationLifecycleEvents` (default
off).
## Why?
Replication failures span the sender, passive executor, recovery loop,
and apply layer. Recording these boundaries in the existing lifecycle
event makes the path of a workflow or replication task directly
traceable without adding another event schema.
## Example traces
The examples below are abridged records captured from the two-cluster
XDC test. Events for the same task correlate on `source_cluster`,
`source_shard`, and `source_task_id`; workflow identity is present on
every record.
A state task that fails on the passive cluster and is written to the
DLQ:
```json
{"phase":"sent","task_type":"sync_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048581,"priority":"High"}
{"phase":"error","task_type":"sync_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048581,"details":{"operation":"passive_task_execution","error":"failed to apply replication task","error_type":"serviceerror.InvalidArgument","apply_artifact_source":"task_payload","attempt":1,"priority":"High","target_cluster":"standby"}}
{"phase":"error","task_type":"sync_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048581,"details":{"operation":"task_execution","error":"failed to apply replication task","terminal":true,"priority":"High","target_cluster":"standby"}}
{"phase":"error","task_type":"sync_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048581,"details":{"operation":"dlq_write","disposition":"dlq","terminal":true,"priority":"High","target_cluster":"standby","target_shard":1}}
```
A verification task that detects missing state, refetches it, and then
verifies successfully:
```json
{"phase":"sent","task_type":"verify_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"priority":"High"}
{"phase":"executing","task_type":"verify_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"attempt":1}
{"phase":"applied","task_type":"verify_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"outcome":"resend_needed"}
{"phase":"error","task_type":"verify_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"details":{"operation":"standby_verification","error":"missing mutable state, resend","error_type":"serviceerror.SyncState","recovery_action":"sync_state","priority":"High","target_cluster":"standby"}}
{"phase":"applied","task_type":"sync_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"outcome":"applied","details":{"apply_artifact_source":"sync_state_refetch"}}
{"phase":"executing","task_type":"verify_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"attempt":1}
{"phase":"applied","task_type":"verify_versioned_transition","workflow_id":"example-workflow","source_cluster":"active","source_shard":1,"source_task_id":1048595,"outcome":"verified"}
```
## How was it tested?
- `go test -tags test_dep ./common/wideevents
./service/history/replication ./service/history/ndc ./service/history
./tests/testcore`
- Changed-lines `golangci-lint`: 0 issues.
- Temporary, uncommitted two-cluster XDC tests forced passive task
failures, DLQ handling, standby verification, and SyncState
refetch/recovery with lifecycle events both enabled and disabled.
- A tiered-processing run confirmed concrete `High` priority on sent and
error records.
## Risks
- Enabling the dynamic config increases event volume; retries and
recovery can produce several error phases for one source task.
- `details.operation`, `details.disposition`, and
`details.recovery_action` distinguish those boundaries.
- Emission is best effort and does not change replication error
propagation, retry, or recovery behavior.
Temporal—durable execution platform
Introduction • Getting Started • Contributing • Temporal Docs • Temporal 101
Introduction
Temporal is a durable execution platform that enables developers to build scalable applications without sacrificing productivity or reliability. The Temporal server executes units of application logic called Workflows in a resilient manner that automatically handles intermittent failures, and retries failed operations.
Temporal is a mature technology that originated as a fork of Uber's Cadence. It is developed by Temporal Technologies, a startup by the creators of Cadence.
Getting Started
Download and Start Temporal Server Locally
Execute the following commands to start a pre-built image along with all the dependencies.
brew install temporal
temporal server start-dev
Refer to Temporal CLI documentation for more installation options.
Run the Samples
Clone or download samples for Go or Java and run them with the local Temporal server. We have a number of HelloWorld type scenarios available, as well as more advanced ones. Note that the sets of samples are currently different between Go and Java.
Use CLI
Use Temporal CLI to interact with the running Temporal server.
temporal operator namespace list
temporal workflow list
Use Temporal Web UI
Try Temporal Web UI by opening http://localhost:8233 for viewing your sample workflows executing on Temporal.
Repository
This repository contains the source code of the Temporal server. To implement Workflows, Activities and Workers, use one of the supported languages.
Contributing
We'd love your help in making Temporal great.
Helpful links to get started:
- work on or propose a new feature
- learn about the Temporal Server architecture
- learn how to build and run the Temporal Server locally
- learn about Temporal Server testing tools and best practices
- join the Temporal community forum and Slack