David Porter 871d03858c Make the scheduler idle-close deadline monotonic (#11702)
## What changed?

This adds a lastEventTime tracking value which keeps track of the last
value even if they fall out of the buffer. It's a bugfix for the current
state which can presently cause multiple idle-tasks to get assigned as
the calculation of when to set the idle-task can jump around as actions
are removed from the buffer. By tracking the last seen value, it can be
forced to be a monotonic value.

### LLM summary:

`getLastEventTime` maxed over `Invoker.recentActions()`, which reads
`BufferedStarts` — truncated by `applyCompletedRetention` to the last
`recentActionCount` (10) completed entries, **ordered by `CloseTime`**.
Whenever completion order differs from start order, evicting the start
that held the largest `StartTime` drags the value, and therefore the
idle deadline, backwards.

**Unbounded growth of the mutable state blob and the timer queue.** 

Idle tasks are reaped only via `Validate`, whose predicate
(`idleExpiration.After(scheduledTime)`) drops a task only when the
deadline moved *later*. A flat or regressed deadline invalidates
nothing, so every Generator tick leaves another task behind — one per
action between idle ticks.

### Observed in production

## How did you test it?

- [x] built
- [x] covered by existing tests
- [x] added new unit test(s)

New file `chasm/lib/scheduler/scheduler_idle_deadline_test.go`, 10
tests. Verified they fail without the fix by reverting both production
changes — 9 of 10 fail, reproducing the production numbers:

```
TestGeneratorTask_IdleTask_NotReArmedWhenDeadlineUnchanged
  expected: 1   actual: 21          # 20 no-op ticks -> 21 tasks, one per tick

TestGeneratorTask_IdleTask_DoesNotAccumulateAcrossActionBurst
  idle deadline regressed at action 10:
    2026-08-28 04:43:25.199231 -> 2026-08-28 04:43:25.197731
```

A 1.5ms regression at action 10 — landing exactly where
`recentActionCount = 10` first evicts. The tenth test
(`NotArmedWhileSpecHasWork`) passes before and after by design; it
guards against the skip logic swallowing the non-idle branch.

Coverage: the root cause independent of the idle machinery (recomputed
value regresses on eviction, floored read does not, mark never moves
backwards, nil mark degrades to old behaviour); accumulation (repeated
no-op ticks, and 43 actions completing one at a time through the
retention window); stuck-open safety for the skip guard (deadline
advance re-arms and reaps, pause/unpause re-arms, live spec arms
nothing); `Validate` no longer entering the regressed branch after an
eviction; and a sanity test that the snapshot helper sees armed tasks so
the count assertions can't pass vacuously.

Whole package green, stable under `-count=3 -test.shuffle=on`, `make
lint-code` 0 issues, `go build ./...` and `go test ./chasm/...` clean.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 18:26:30 +00:00
2026-08-26 10:00:40 -07:00
2026-08-26 10:00:40 -07:00
2025-04-07 18:42:37 -07:00

Temporal—durable execution platform

GitHub Release GitHub License Code Coverage Community Go Report Card

Introduction   •   Getting Started   •   Contributing   •   Temporal Docs   •   Temporal 101

Introduction

Temporal is a durable execution platform that enables developers to build scalable applications without sacrificing productivity or reliability. The Temporal server executes units of application logic called Workflows in a resilient manner that automatically handles intermittent failures, and retries failed operations.

Temporal is a mature technology that originated as a fork of Uber's Cadence. It is developed by Temporal Technologies, a startup by the creators of Cadence.

image

Getting Started

Download and Start Temporal Server Locally

Execute the following commands to start a pre-built image along with all the dependencies.

brew install temporal
temporal server start-dev

Refer to Temporal CLI documentation for more installation options.

Run the Samples

Clone or download samples for Go or Java and run them with the local Temporal server. We have a number of HelloWorld type scenarios available, as well as more advanced ones. Note that the sets of samples are currently different between Go and Java.

Use CLI

Use Temporal CLI to interact with the running Temporal server.

temporal operator namespace list
temporal workflow list

Use Temporal Web UI

Try Temporal Web UI by opening http://localhost:8233 for viewing your sample workflows executing on Temporal.

Repository

This repository contains the source code of the Temporal server. To implement Workflows, Activities and Workers, use one of the supported languages.

Contributing

We'd love your help in making Temporal great.

Helpful links to get started:

License

MIT License

Languages
Go 99.5%
Shell 0.2%
PLpgSQL 0.2%
Makefile 0.1%