mirror of
https://github.com/temporalio/temporal.git
synced 2026-08-30 18:41:49 -07:00
37f6e6accb
## What changed?
Pause replication streams when the scheduler is under pressure.
## Why?
This allows us to apply backpressure to replication streams when we
cannot keep up with the load. The schedulers are shared amongst streams
so just keep tracking of tasked tasks is not enough. We use a timer to
notify us if a submit is taking too long rather than recording the time
after the fact so that the backpressure is more reactive.
## How did you test it?
- [ ] built
- [ ] run locally and tested manually
- [ ] covered by existing tests
- [x] added new unit test(s)
- [ ] added new functional test(s)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Backpressure for replication streams**
>
> - Add slow-submission flow control: `stream_receiver` tracks
per-priority slow submission timestamps and
`stream_receiver_flow_controller` pauses when within
`ReplicationReceiverSlowSubmissionWindow` (in addition to outstanding
task count)
> - New dynamic configs:
`history.ReplicationReceiverSubmissionLatencyThreshold`,
`history.ReplicationReceiverSlowSubmissionWindow`,
`history.EnableReplicationReceiverSlowSubmissionFlowControl`, wired
through `configs.Config`
> - Refactor `StreamReceiver`: split scheduler selection into
`getTaskSchedulerPriority`/`getTaskScheduler`, measure `Submit` latency,
and feed `lastSlowSubmission` into flow control signals
> - Tests: expand flow controller tests to cover slow-submission window
logic; adjust stream receiver tests and minor robustness checks
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
9e6ec6ed81. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->