System 1, by contract
Kahneman's fast, automatic, effortless mode is the latency contract: single-pass answers on a sub-50 ms path, no chain-of-thought in the loop. The whirlwind is the product metaphor.
$ saci --status
S.A.C.I.
A decision layer that turns a program's state into routed, calibrated choices — and says "I don't know" when it doesn't. in training
the name
Tying knots
The generator operates on nodes and edges — states to typed options. Resolving a tangled decision is literally untying knots in a graph, which is what the engine does.
Adaptive redirection
The Saci misdirects the obvious route. Same idea here: alternative paths and dynamic routing that a rigid rules engine would never see — including abstaining when confidence is low.
An independent build
State plus typed questions plus calibrated answers is a general idea with several implementations, including commercial ones. This one is built and measured here, on its own terms.
architecture
State, routing and execution are three different jobs, and keeping them in three different places is what makes the system auditable. The model never owns a consequence.
- Program state
- Holds what is known, what is stale, and what the current step needs to resolve — owned by the application, never by the model.
- S.A.C.I.
- Given that state and a set of typed questions, returns typed answers with probabilities and a confidence band.
- Application code
- Owns the consequences. It reads the band, not the raw answer, and decides to proceed, escalate or ask.
roadmap
Absorbed from the open implementations of the same pattern, each pre-registered with a
status. Nothing moves to adopted without a measurement.
A1 in progress Calibration-shaped reward
Reinforcement stage whose reward is outcome minus the probability assigned to the chosen option. Published evidence from an independent implementation: final ECE 0.021 with a 6-point accuracy gain, where the same loop with a plain outcome reward ends at ECE 0.216 — the calibration term is what preserves calibration. A lightweight rebalanced retrain is running now.
A2 planned Learned abstention slot
An explicit "insufficient evidence" candidate in the answer space, so abstention is learned rather than bolted on post-hoc.
A3 planned Parallel multi-question broadcast
Answer every question for a state in one pass instead of one call per question — independently measured at 5.6–7.0× latency reduction.
A4 planned Small distill, decision-only export
A compact student model with only the decision head exported — enough for routing tasks that never need generation.
A5 planned Synthetic template expansion
Generated slot templates to widen domain coverage, and the prerequisite for running domain-specialised benchmarks.
A7 adopted NLI framing of candidate labels
Rephrasing candidates as statements rather than bare labels. Measured +22 points on the intent question and +22 on the public benchmark's easy split at zero cost elsewhere — with the caveat that framing everything breaks the routing question, so the word-only variant is the one in use.
next consumers
Every place the harness currently guesses with a heuristic is a candidate for a typed question with a calibrated answer.
clarify gateshadowinjection screenservingcache admitcandidatemodel routingservingbudget tiercandidatecompaction triggershadowdelegation hintcandidateredundancy probecandidate
open items
Training is still running. These are the slots that fill in next — listed rather than quietly omitted, because a research page that only shows finished work is a brochure.
- Active-learning queue: 953 uncertain samples mined from shadow logs, waiting on human annotation.
- Judge agreement: 0.198 today against an 0.85 bar — blocked on annotating the queue above.
- Feature-model promotion: waiting on the second half of the pre-registered A/B, which needs real traffic.
- Injection head v4: golden F1 0.911 beats v3 at 0.891, held back until the functional suite passes.
- Embedding variant: standalone retrieval loses to the general-purpose baseline; fusion is the next design to measure, held until the evaluation queued ahead of it reports.
- Contextual-bandit routing: the router now logs the probability of every route it takes, with exploration held at zero until the source of some unpaired router decisions is traced; the offline replay gate is pre-registered, not yet run.
- Reinforcement stage (calibration-shaped reward) — running; result pending.
- Public benchmark on the domain-specialised task set — blocked on the synthetic-template path.