Files
temporal/chasm/nexus_operation_processor_test.go
Roey Berman e0d9f48c84 System nexus endpoint (#9002)
## Overview

This feature introduces a special internal Nexus endpoint called
`__temporal_system` that enables adding functionality to workflows
**without requiring new workflow commands and events**. Operations on
this endpoint are routed internally within Temporal's history service
rather than via external HTTP calls.

## Key Components

### 1. System Endpoint Infrastructure

- **Endpoint Name**: `__temporal_system` (constant in
`common/nexus/constants.go:6`)
- **Callback URL**: `temporal://system` for internal routing
- **New History Service RPCs**
(`proto/internal/temporal/server/api/historyservice/v1/service.proto:433-437`):
  - `StartNexusOperation` - Starts operations on the system endpoint
  - `CancelNexusOperation` - Cancels operations on the system endpoint

### 2. Operation Processor Framework

A new processor pattern (`chasm/nexus_operation_processor.go`) that
allows CHASM libraries to:

- **Validate and transform input**: Processors can validate operation
inputs and set default values
- **Determine routing**: Each processor returns a routing key that
determines which history shard handles the operation
- **Re-serialize input**: Mutated inputs can be re-serialized to persist
default values

**Routing strategies**:
- `NexusOperationRoutingKeyExecution` - Routes to the shard owning a
specific workflow execution
- `NexusOperationRoutingKeyRandom` - Routes to a random shard

### 3. CHASM Library Integration

CHASM libraries can now provide (`chasm/library.go:16-19`):
- **`NexusServices()`**: Regular Nexus service handlers (implement the
actual operation logic)
- **`NexusServiceProcessors()`**: Input processors for validation and
routing

Example from test library (`chasm/lib/tests/nexus_service.go`):

```go
// Service handler - implements the actual operation
TestOperation = nexus.NewSyncOperation("TestOperation",
    func(ctx context.Context, input string, options nexus.StartOperationOptions) (string, error) {
        return "Hello, " + input, nil
    })

// Processor - validates input and determines routing
func (o testOperationProcessor) ProcessInput(ctx chasm.NexusOperationProcessorContext, input string)
    (*chasm.NexusOperationProcessorResult, error) {
    return &chasm.NexusOperationProcessorResult{
        RoutingKey: chasm.NexusOperationRoutingKeyExecution{
            NamespaceID: ctx.Namespace.ID().String(),
            BusinessID:  input, // Route based on input
        },
    }, nil
}
```

### 4. Execution Flow

When a workflow schedules a Nexus operation on `__temporal_system`
(`components/nexusoperations/executors.go:233-238`):

1. **Input Processing**: The processor validates input and determines
routing
2. **Internal RPC**: Instead of HTTP, calls
`HistoryClient.StartNexusOperation` with the target shard ID
3. **Handler Execution**: The history service invokes the registered
Nexus handler (`service/history/handler.go:2707-2768`)
4. **Result Handling**: Supports both sync (immediate result) and async
(operation token) responses
5. **Workflow Completion**: Results flow back through the same
completion path as external Nexus operations

### 5. Benefits

 **No schema changes**: Add functionality without new commands/events
in workflow history
 **Consistent API**: Uses existing Nexus operation semantics
(sync/async, callbacks, links)
 **Proper routing**: Operations are intelligently routed to the correct
shard
 **Input validation**: Type-safe input validation and default value
handling
 **Future extensibility**: Foundation for direct client invocation (not
yet implemented)

### 6. Technical Details

- **Error handling** (`components/nexusoperations/executors.go:444`):
Non-retryable service errors are properly handled and fail operations
immediately
- **Metrics**: System operations are tracked separately with
`DestinationTag` set to the endpoint name
- **Link conversion**: Helper functions convert between Nexus SDK links
and protobuf links (`common/nexus/util.go:17-46`)
- **Operation token handling**: Moved link converters to common package
for reuse (`common/nexus/link_converter.go`)

### 7. Current Limitations

- Only accessible from workflows (via `ScheduleNexusOperation` command)
- Direct client invocation not yet implemented
- Headers not supported for system endpoint operations

## Test Coverage

New test (`tests/nexus_workflow_test.go:2763-2843`) demonstrates:
- Scheduling operation on `__temporal_system` endpoint
- Synchronous operation completion
- Result propagation back to workflow

## Architecture

This architecture provides a clean, extensible way to add internal
functionality while maintaining compatibility with Temporal's existing
workflow execution model. The system endpoint acts as a bridge between
workflows and internal CHASM components, enabling:

- **Extensibility**: New operations can be added by implementing CHASM
libraries
- **Type safety**: Input validation happens before operations are routed
- **Scalability**: Intelligent routing ensures operations land on the
correct shard
- **Consistency**: Same execution model as external Nexus operations
2026-02-16 15:53:51 -08:00

233 lines
7.0 KiB
Go

package chasm
import (
"testing"
"github.com/nexus-rpc/sdk-go/nexus"
"github.com/stretchr/testify/require"
commonpb "go.temporal.io/api/common/v1"
persistencespb "go.temporal.io/server/api/persistence/v1"
"go.temporal.io/server/common/namespace"
"go.temporal.io/server/common/payloads"
"go.temporal.io/server/common/primitives/timestamp"
)
type processableInput struct {
Value int
}
type processableOperation struct {
}
func (o *processableOperation) Name() string {
return "processable-operation"
}
func (o *processableOperation) ProcessInput(ctx NexusOperationProcessorContext, input *processableInput) (*NexusOperationProcessorResult, error) {
if input.Value < 0 {
return nil, nexus.NewHandlerErrorf(nexus.HandlerErrorTypeBadRequest, "value must be non-negative")
}
// Mutate to test overwrite behavior.
input.Value += 1
return &NexusOperationProcessorResult{
RoutingKey: NexusOperationRoutingKeyRandom{},
}, nil
}
func newTestContext() NexusOperationProcessorContext {
ns := namespace.NewLocalNamespaceForTest(
&persistencespb.NamespaceInfo{
Id: "test-namespace-id",
Name: "test-namespace",
},
&persistencespb.NamespaceConfig{
Retention: timestamp.DurationFromDays(1),
},
"active-cluster",
)
return NexusOperationProcessorContext{
Namespace: ns,
RequestID: "test-request-id",
}
}
func mustToPayload(t *testing.T, value any) *commonpb.Payload {
t.Helper()
ps, err := payloads.Encode(value)
require.NoError(t, err)
return ps.Payloads[0]
}
func TestNexusOperationProcessor_ProcessInput(t *testing.T) {
t.Parallel()
processableOp := &processableOperation{}
processor := NewRegisterableNexusOperationProcessor(processableOp)
tests := []struct {
name string
ctx NexusOperationProcessorContext
input *commonpb.Payload
checkResult func(*testing.T, *NexusOperationProcessorResult, error)
}{
{
name: "valid input returns routing key, no overwrite",
ctx: newTestContext(),
input: mustToPayload(t, processableInput{Value: 23}),
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
require.NoError(t, err)
require.NotNil(t, result)
require.NotNil(t, result.RoutingKey)
require.Nil(t, result.ReserializedInputPayload)
},
},
{
name: "overwrite payload with valid input",
ctx: func() NexusOperationProcessorContext {
ctx := newTestContext()
ctx.ReserializeInputPayload = true
return ctx
}(),
input: mustToPayload(t, processableInput{Value: 23}),
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
require.NoError(t, err)
require.NotNil(t, result)
require.NotNil(t, result.RoutingKey)
mutatedInput := processableInput{}
require.NoError(t, payloads.Decode(&commonpb.Payloads{Payloads: []*commonpb.Payload{result.ReserializedInputPayload}}, &mutatedInput))
require.Equal(t, 24, mutatedInput.Value)
},
},
{
name: "invalid input",
ctx: newTestContext(),
input: mustToPayload(t, processableInput{Value: -1}),
checkResult: func(t *testing.T, nopr *NexusOperationProcessorResult, err error) {
var handlerErr *nexus.HandlerError
require.ErrorAs(t, err, &handlerErr)
require.Equal(t, nexus.HandlerErrorTypeBadRequest, handlerErr.Type)
require.Contains(t, handlerErr.Error(), "value must be non-negative")
},
},
{
name: "decode error",
ctx: newTestContext(),
input: mustToPayload(t, "wrong type"),
checkResult: func(t *testing.T, nopr *NexusOperationProcessorResult, err error) {
var handlerErr *nexus.HandlerError
require.ErrorAs(t, err, &handlerErr)
require.Equal(t, nexus.HandlerErrorTypeBadRequest, handlerErr.Type)
require.Contains(t, handlerErr.Error(), "failed to decode input payload")
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := processor.processInput(tt.ctx, tt.input)
tt.checkResult(t, result, err)
})
}
}
func TestNexusServiceProcessor_ProcessInput(t *testing.T) {
t.Parallel()
processableOp := &processableOperation{}
processor := NewNexusServiceProcessor("test-service")
processor.MustRegisterOperation(processableOp.Name(), NewRegisterableNexusOperationProcessor(processableOp))
ctx := newTestContext()
tests := []struct {
name string
opName string
input *commonpb.Payload
checkResult func(*testing.T, *NexusOperationProcessorResult, error)
}{
{
name: "operation not found",
opName: "nonexistent-operation",
input: mustToPayload(t, processableInput{Value: 50}),
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
var handlerErr *nexus.HandlerError
require.ErrorAs(t, err, &handlerErr)
require.Equal(t, nexus.HandlerErrorTypeNotFound, handlerErr.Type)
require.Contains(t, handlerErr.Error(), `operation "nonexistent-operation" not found`)
},
},
{
name: "valid input",
opName: "processable-operation",
input: mustToPayload(t, processableInput{Value: 50}),
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
require.NoError(t, err)
require.NotNil(t, result)
require.NotNil(t, result.RoutingKey)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := processor.ProcessInput(ctx, tt.opName, tt.input)
tt.checkResult(t, result, err)
})
}
}
func TestNexusEndpointProcessor_ProcessInput(t *testing.T) {
t.Parallel()
processableOp := &processableOperation{}
serviceProcessor := NewNexusServiceProcessor("test-service")
serviceProcessor.MustRegisterOperation(processableOp.Name(), NewRegisterableNexusOperationProcessor(processableOp))
processor := NewNexusEndpointProcessor()
processor.MustRegisterServiceProcessor(serviceProcessor)
ctx := newTestContext()
tests := []struct {
name string
service string
operation string
input *commonpb.Payload
checkResult func(*testing.T, *NexusOperationProcessorResult, error)
}{
{
name: "service not found",
service: "nonexistent-service",
operation: "processable-operation",
input: mustToPayload(t, processableInput{Value: 50}),
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
var handlerErr *nexus.HandlerError
require.ErrorAs(t, err, &handlerErr)
require.Equal(t, nexus.HandlerErrorTypeNotFound, handlerErr.Type)
require.Contains(t, handlerErr.Error(), `service "nonexistent-service" not found`)
},
},
{
name: "valid request",
service: "test-service",
operation: "processable-operation",
input: mustToPayload(t, processableInput{Value: 47}),
checkResult: func(t *testing.T, result *NexusOperationProcessorResult, err error) {
require.NoError(t, err)
require.NotNil(t, result)
require.NotNil(t, result.RoutingKey)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := processor.ProcessInput(ctx, tt.service, tt.operation, tt.input)
tt.checkResult(t, result, err)
})
}
}