mirror of
https://github.com/temporalio/temporal.git
synced 2026-08-31 02:51:51 -07:00
## Overview
This feature introduces a special internal Nexus endpoint called
`__temporal_system` that enables adding functionality to workflows
**without requiring new workflow commands and events**. Operations on
this endpoint are routed internally within Temporal's history service
rather than via external HTTP calls.
## Key Components
### 1. System Endpoint Infrastructure
- **Endpoint Name**: `__temporal_system` (constant in
`common/nexus/constants.go:6`)
- **Callback URL**: `temporal://system` for internal routing
- **New History Service RPCs**
(`proto/internal/temporal/server/api/historyservice/v1/service.proto:433-437`):
- `StartNexusOperation` - Starts operations on the system endpoint
- `CancelNexusOperation` - Cancels operations on the system endpoint
### 2. Operation Processor Framework
A new processor pattern (`chasm/nexus_operation_processor.go`) that
allows CHASM libraries to:
- **Validate and transform input**: Processors can validate operation
inputs and set default values
- **Determine routing**: Each processor returns a routing key that
determines which history shard handles the operation
- **Re-serialize input**: Mutated inputs can be re-serialized to persist
default values
**Routing strategies**:
- `NexusOperationRoutingKeyExecution` - Routes to the shard owning a
specific workflow execution
- `NexusOperationRoutingKeyRandom` - Routes to a random shard
### 3. CHASM Library Integration
CHASM libraries can now provide (`chasm/library.go:16-19`):
- **`NexusServices()`**: Regular Nexus service handlers (implement the
actual operation logic)
- **`NexusServiceProcessors()`**: Input processors for validation and
routing
Example from test library (`chasm/lib/tests/nexus_service.go`):
```go
// Service handler - implements the actual operation
TestOperation = nexus.NewSyncOperation("TestOperation",
func(ctx context.Context, input string, options nexus.StartOperationOptions) (string, error) {
return "Hello, " + input, nil
})
// Processor - validates input and determines routing
func (o testOperationProcessor) ProcessInput(ctx chasm.NexusOperationProcessorContext, input string)
(*chasm.NexusOperationProcessorResult, error) {
return &chasm.NexusOperationProcessorResult{
RoutingKey: chasm.NexusOperationRoutingKeyExecution{
NamespaceID: ctx.Namespace.ID().String(),
BusinessID: input, // Route based on input
},
}, nil
}
```
### 4. Execution Flow
When a workflow schedules a Nexus operation on `__temporal_system`
(`components/nexusoperations/executors.go:233-238`):
1. **Input Processing**: The processor validates input and determines
routing
2. **Internal RPC**: Instead of HTTP, calls
`HistoryClient.StartNexusOperation` with the target shard ID
3. **Handler Execution**: The history service invokes the registered
Nexus handler (`service/history/handler.go:2707-2768`)
4. **Result Handling**: Supports both sync (immediate result) and async
(operation token) responses
5. **Workflow Completion**: Results flow back through the same
completion path as external Nexus operations
### 5. Benefits
✅ **No schema changes**: Add functionality without new commands/events
in workflow history
✅ **Consistent API**: Uses existing Nexus operation semantics
(sync/async, callbacks, links)
✅ **Proper routing**: Operations are intelligently routed to the correct
shard
✅ **Input validation**: Type-safe input validation and default value
handling
✅ **Future extensibility**: Foundation for direct client invocation (not
yet implemented)
### 6. Technical Details
- **Error handling** (`components/nexusoperations/executors.go:444`):
Non-retryable service errors are properly handled and fail operations
immediately
- **Metrics**: System operations are tracked separately with
`DestinationTag` set to the endpoint name
- **Link conversion**: Helper functions convert between Nexus SDK links
and protobuf links (`common/nexus/util.go:17-46`)
- **Operation token handling**: Moved link converters to common package
for reuse (`common/nexus/link_converter.go`)
### 7. Current Limitations
- Only accessible from workflows (via `ScheduleNexusOperation` command)
- Direct client invocation not yet implemented
- Headers not supported for system endpoint operations
## Test Coverage
New test (`tests/nexus_workflow_test.go:2763-2843`) demonstrates:
- Scheduling operation on `__temporal_system` endpoint
- Synchronous operation completion
- Result propagation back to workflow
## Architecture
This architecture provides a clean, extensible way to add internal
functionality while maintaining compatibility with Temporal's existing
workflow execution model. The system endpoint acts as a bridge between
workflows and internal CHASM components, enabling:
- **Extensibility**: New operations can be added by implementing CHASM
libraries
- **Type safety**: Input validation happens before operations are routed
- **Scalability**: Intelligent routing ensures operations land on the
correct shard
- **Consistency**: Same execution model as external Nexus operations
233 lines
7.0 KiB
Go
233 lines
7.0 KiB
Go
package chasm
|
|
|
|
import (
|
|
"testing"
|
|
|
|
"github.com/nexus-rpc/sdk-go/nexus"
|
|
"github.com/stretchr/testify/require"
|
|
commonpb "go.temporal.io/api/common/v1"
|
|
persistencespb "go.temporal.io/server/api/persistence/v1"
|
|
"go.temporal.io/server/common/namespace"
|
|
"go.temporal.io/server/common/payloads"
|
|
"go.temporal.io/server/common/primitives/timestamp"
|
|
)
|
|
|
|
type processableInput struct {
|
|
Value int
|
|
}
|
|
|
|
type processableOperation struct {
|
|
}
|
|
|
|
func (o *processableOperation) Name() string {
|
|
return "processable-operation"
|
|
}
|
|
|
|
func (o *processableOperation) ProcessInput(ctx NexusOperationProcessorContext, input *processableInput) (*NexusOperationProcessorResult, error) {
|
|
if input.Value < 0 {
|
|
return nil, nexus.NewHandlerErrorf(nexus.HandlerErrorTypeBadRequest, "value must be non-negative")
|
|
}
|
|
// Mutate to test overwrite behavior.
|
|
input.Value += 1
|
|
return &NexusOperationProcessorResult{
|
|
RoutingKey: NexusOperationRoutingKeyRandom{},
|
|
}, nil
|
|
}
|
|
|
|
func newTestContext() NexusOperationProcessorContext {
|
|
ns := namespace.NewLocalNamespaceForTest(
|
|
&persistencespb.NamespaceInfo{
|
|
Id: "test-namespace-id",
|
|
Name: "test-namespace",
|
|
},
|
|
&persistencespb.NamespaceConfig{
|
|
Retention: timestamp.DurationFromDays(1),
|
|
},
|
|
"active-cluster",
|
|
)
|
|
return NexusOperationProcessorContext{
|
|
Namespace: ns,
|
|
RequestID: "test-request-id",
|
|
}
|
|
}
|
|
|
|
func mustToPayload(t *testing.T, value any) *commonpb.Payload {
|
|
t.Helper()
|
|
ps, err := payloads.Encode(value)
|
|
require.NoError(t, err)
|
|
return ps.Payloads[0]
|
|
}
|
|
|
|
func TestNexusOperationProcessor_ProcessInput(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
processableOp := &processableOperation{}
|
|
|
|
processor := NewRegisterableNexusOperationProcessor(processableOp)
|
|
|
|
tests := []struct {
|
|
name string
|
|
ctx NexusOperationProcessorContext
|
|
input *commonpb.Payload
|
|
checkResult func(*testing.T, *NexusOperationProcessorResult, error)
|
|
}{
|
|
{
|
|
name: "valid input returns routing key, no overwrite",
|
|
ctx: newTestContext(),
|
|
input: mustToPayload(t, processableInput{Value: 23}),
|
|
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
|
|
require.NoError(t, err)
|
|
require.NotNil(t, result)
|
|
require.NotNil(t, result.RoutingKey)
|
|
require.Nil(t, result.ReserializedInputPayload)
|
|
},
|
|
},
|
|
{
|
|
name: "overwrite payload with valid input",
|
|
ctx: func() NexusOperationProcessorContext {
|
|
ctx := newTestContext()
|
|
ctx.ReserializeInputPayload = true
|
|
return ctx
|
|
}(),
|
|
input: mustToPayload(t, processableInput{Value: 23}),
|
|
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
|
|
require.NoError(t, err)
|
|
require.NotNil(t, result)
|
|
require.NotNil(t, result.RoutingKey)
|
|
mutatedInput := processableInput{}
|
|
require.NoError(t, payloads.Decode(&commonpb.Payloads{Payloads: []*commonpb.Payload{result.ReserializedInputPayload}}, &mutatedInput))
|
|
require.Equal(t, 24, mutatedInput.Value)
|
|
},
|
|
},
|
|
{
|
|
name: "invalid input",
|
|
ctx: newTestContext(),
|
|
input: mustToPayload(t, processableInput{Value: -1}),
|
|
checkResult: func(t *testing.T, nopr *NexusOperationProcessorResult, err error) {
|
|
var handlerErr *nexus.HandlerError
|
|
require.ErrorAs(t, err, &handlerErr)
|
|
require.Equal(t, nexus.HandlerErrorTypeBadRequest, handlerErr.Type)
|
|
require.Contains(t, handlerErr.Error(), "value must be non-negative")
|
|
},
|
|
},
|
|
{
|
|
name: "decode error",
|
|
ctx: newTestContext(),
|
|
input: mustToPayload(t, "wrong type"),
|
|
checkResult: func(t *testing.T, nopr *NexusOperationProcessorResult, err error) {
|
|
var handlerErr *nexus.HandlerError
|
|
require.ErrorAs(t, err, &handlerErr)
|
|
require.Equal(t, nexus.HandlerErrorTypeBadRequest, handlerErr.Type)
|
|
require.Contains(t, handlerErr.Error(), "failed to decode input payload")
|
|
},
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
result, err := processor.processInput(tt.ctx, tt.input)
|
|
tt.checkResult(t, result, err)
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestNexusServiceProcessor_ProcessInput(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
processableOp := &processableOperation{}
|
|
|
|
processor := NewNexusServiceProcessor("test-service")
|
|
processor.MustRegisterOperation(processableOp.Name(), NewRegisterableNexusOperationProcessor(processableOp))
|
|
|
|
ctx := newTestContext()
|
|
|
|
tests := []struct {
|
|
name string
|
|
opName string
|
|
input *commonpb.Payload
|
|
checkResult func(*testing.T, *NexusOperationProcessorResult, error)
|
|
}{
|
|
{
|
|
name: "operation not found",
|
|
opName: "nonexistent-operation",
|
|
input: mustToPayload(t, processableInput{Value: 50}),
|
|
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
|
|
var handlerErr *nexus.HandlerError
|
|
require.ErrorAs(t, err, &handlerErr)
|
|
require.Equal(t, nexus.HandlerErrorTypeNotFound, handlerErr.Type)
|
|
require.Contains(t, handlerErr.Error(), `operation "nonexistent-operation" not found`)
|
|
},
|
|
},
|
|
{
|
|
name: "valid input",
|
|
opName: "processable-operation",
|
|
input: mustToPayload(t, processableInput{Value: 50}),
|
|
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
|
|
require.NoError(t, err)
|
|
require.NotNil(t, result)
|
|
require.NotNil(t, result.RoutingKey)
|
|
},
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
result, err := processor.ProcessInput(ctx, tt.opName, tt.input)
|
|
tt.checkResult(t, result, err)
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestNexusEndpointProcessor_ProcessInput(t *testing.T) {
|
|
t.Parallel()
|
|
|
|
processableOp := &processableOperation{}
|
|
|
|
serviceProcessor := NewNexusServiceProcessor("test-service")
|
|
serviceProcessor.MustRegisterOperation(processableOp.Name(), NewRegisterableNexusOperationProcessor(processableOp))
|
|
|
|
processor := NewNexusEndpointProcessor()
|
|
processor.MustRegisterServiceProcessor(serviceProcessor)
|
|
|
|
ctx := newTestContext()
|
|
|
|
tests := []struct {
|
|
name string
|
|
service string
|
|
operation string
|
|
input *commonpb.Payload
|
|
checkResult func(*testing.T, *NexusOperationProcessorResult, error)
|
|
}{
|
|
{
|
|
name: "service not found",
|
|
service: "nonexistent-service",
|
|
operation: "processable-operation",
|
|
input: mustToPayload(t, processableInput{Value: 50}),
|
|
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
|
|
var handlerErr *nexus.HandlerError
|
|
require.ErrorAs(t, err, &handlerErr)
|
|
require.Equal(t, nexus.HandlerErrorTypeNotFound, handlerErr.Type)
|
|
require.Contains(t, handlerErr.Error(), `service "nonexistent-service" not found`)
|
|
},
|
|
},
|
|
{
|
|
name: "valid request",
|
|
service: "test-service",
|
|
operation: "processable-operation",
|
|
input: mustToPayload(t, processableInput{Value: 47}),
|
|
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
|
|
require.NoError(t, err)
|
|
require.NotNil(t, result)
|
|
require.NotNil(t, result.RoutingKey)
|
|
},
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
result, err := processor.ProcessInput(ctx, tt.service, tt.operation, tt.input)
|
|
tt.checkResult(t, result, err)
|
|
})
|
|
}
|
|
}
|