Queue & supersede policy
The engine executes one run at a time; the supersede policy decides which runs happen when trigger events outpace the executor. This is node-level policy, not engine internals — it lives in peregrin-queue (pure logic, no I/O) and the node daemon drives it around each Runner. The peregrin-node RunSupervisor composes the queue with the engine's ingress channel into the sequential run loop the daemon will own.
supersede: campaign config
supersede:
key: "{repo}:{ref}" # template over the trigger event payload
mode: latest-wins # | fifo
cancel_running: true # cancel the active run vs queue behind it| Field | Type | Default | Meaning |
|---|---|---|---|
key | template | "" | {dotted.path} placeholders resolved against the trigger event's payload; events resolving to the same key are alternative triggers for the same logical run (one branch, one PR, one WAL head). Empty = a constant key, so every event targets the same run. |
mode | latest-wins | fifo | latest-wins | Pending-job dedupe: latest-wins replaces the pending job that shares the key (only the newest trigger for a target runs — the displaced job's queue position is inherited); fifo appends (every trigger runs) up to MAX_PENDING_PER_KEY (32) per key — beyond that the key's oldest pending job is evicted. |
cancel_running | bool | false | true: a new event marks the active run superseded — the queue raises a one-shot cancel signal the caller delivers to the run. false: new events always queue behind the active run. |
Key template grammar
Distinct from ${...} scope resolution — the {path} grammar is a small payload-only interpolation:
{repo},{origin.ref}— dotted-path walk overevent.payload, one key per.segment.- Values stringify like JS:
null→"null", numbers/booleans asString(v), arrays join with,, objects →"[object Object]". - A miss — absent key, non-object step, empty or unclosed placeholder, an interior containing
{— stays verbatim ({missing}in the key is a literal, still deterministic). {{emits a literal{and rescans:{{repo}}→{a/b}.
Queue semantics
Queue::submit(event) resolves the key and returns a Decision (key, disposition, cancel_active):
- Dedupe applies to pending jobs only — under
latest-winsthe new event replaces the same-key pending job in place and the displacedJobcomes back inDisposition::ReplacedPending(job)for observability. The active run is never deduped: same-key events while it runs queue another run (or cancel it, withcancel_running). - Cancel is a one-shot signal: the first
cancel_runningsubmit while a run is active reportscancel_activeand stashes the supersededJob, handed out once bytake_cancel().finish()clears an unread signal — a run that ended is moot to cancel. - Drain is sequential (the gate case):
drain()returnsNonewhile a job is active and otherwise pops the front job and marks it active;finish()releases the slot. Pending jobs iterate in drain order viapending().
Ingress injection channel
External events can't call LocalBus::publish directly — it's &mut and the runner is single-threaded, so a blocking producer (a webhook listener, a WAL poller) would stall the loop (audit I-014). The channel splits that (peregrin-engine, std::sync::mpsc — the engine is synchronous):
peregrin_engine::ingress()returns(Injector, Ingress).Injectoris theSend + Sync, cloneable producer half — any thread or spawned ingress driver pushesEvents without&mutaccess to anything.injectfails withInjectError(carrying the event back) once the run's receiver is dropped.Ingressis the runner's receiver —Runner::with_ingressattaches it and the run loop drains it intopendingonce per iteration.
Ordering guarantee: injected events serialize after every event already pending at the drain — including the triggers of the step currently executing — and before anything nodes emit afterward. A workflow.cancel pushed through the injector is honored at the next iteration boundary, exactly like a node-emitted one (it is consumed by the loop, not published — the log shows workflow.cancelled).
Two edge rulings:
- Events pushed before
run()starts are delivered — the runner drains on the first iteration too (RunSupervisoruses this to deliver the trigger event itself). - Events arriving while the last step executes still land: the loop drains before its exit check, so only an event arriving after the final iteration can be dropped — a quiescent run exits; the channel does not keep it alive.
RunSupervisor composition
peregrin_node::supervisor::RunSupervisor is the composable wrapper the daemon drives: submit(event) → SubmitOutcome { decision, start, cancelled_active }; finish() → Option<PendingRun>.
startisSome(PendingRun)only when no run is active — it carries a fresh(injector, ingress)pair for its run with the trigger event already queued on it; handingresstoRunner::with_ingress, cloneinjectorfor external producers.- Under
cancel_running, a superseding submit pushesEvent::cancel("superseded by a newer trigger for key '<key>'")into the active run's own ingress channel — the cancellation surface the single-threaded runner actually honors. - The runner itself stays
!Send(Rc internals) — runs execute on the loop thread; the channel is the entire cross-thread input surface.
Superseded-run observability
When latest-wins displaces a pending job — or a fifo-bounded key evicts its oldest pending entry — the displaced trigger does not vanish silently. RunSupervisor keeps the displaced Job per supersede key and injects a marker event into the next replacement run for that key, alongside its trigger:
- Kind
push.superseded, source__runner__, payload carrying the displaced trigger'srepo/sha(where the event payload has them), thekey,superseded: true, and asuperseded_byobject naming the replacement event's kind. - Multiple displacements for one key accumulate in order and all land on the same replacement run; other keys never see them.
- Because the marker is an ordinary injected event, it is logged in the run's event log and persisted to
--historyJSONL like any other event — the displaced SHA is auditable after the fact.
No GitHub commit status is posted automatically: status posting is a campaign-level concern (peregrin-sink-github-status needs campaign node execution context, which the queue layer doesn't have). A campaign that wants a "superseded" status on the displaced SHA adds a github-status node matching the push.superseded event type.
Durable queue (--queue-store)
Without a store the queue is memory-only — a restart loses every pending trigger. campaign run --queue-store <path> (supervisor mode only) opens a sqlite QueueStore (peregrin-queue-sqlite) and makes the queue durable:
- What persists — pending submissions grouped by supersede key (per-key writes, so peers never clobber each other) plus the in-flight marker. Written through on every submit/drain/finish.
- What replays — on boot,
RunSupervisor::with_storeloads the snapshot and runs it throughQueue::restorebefore the loop accepts new triggers: pending jobs restore in drain order and the interrupted run's marker re-queues at the front (at-least-once — it gets another shot). Underlatest-winsa newer same-key pending entry still supersedes it, exactly as live. - Re-validation — persisted rows aren't a trust boundary: each entry's stored key must match
policy.key_of(event)(a changedsupersede:block or corrupt row drops it, logged), and restored entries go through the same dedupe/bound path as live submits, so a restored queue can't hold a state submits couldn't produce. - Failure policy — an unreadable store fails the boot (queued triggers would be silently lost otherwise); per-entry corruption is skipped with a warning; write failures mid-run are logged and degrade durability rather than killing runs.
The store file is ordinary sqlite — queue_pending(key, seq, job) rows plus a singleton queue_running row; safe to inspect with sqlite3.
Under peregrin campaign serve --queue-store-dir <dir> each hosted campaign gets its own store at <dir>/<campaign>.db — same schema, same semantics, independent namespaces. Likewise --history-dir namespaces run history as <dir>/<campaign>.jsonl. See Serve a fleet of campaigns.