Commit
document budget cap, gavel, spine, peer prediction, provenance, re-run
commit
ee8b561
2 changed files with +18 and −5
Jump to a changed file
- README.md +15 −5
- docs/README.md +3 −0
modified README.md +15 −5
| @@ -32,6 +32,12 @@Everything a reviewer needs to _trust_ the answer is on screen: the critique mat | ||
| 32 | 32 | | **Dissent report** | The chairman explicitly records unresolved disagreements and which model held which position. | |
| 33 | 33 | | **Self-preference mitigation** | Answers are anonymized to the chairman too; a warning badge flags when the chairman shares a provider family with a council member. | |
| 34 | 34 | | **Cost transparency** | Live cost meter during the debate + a per-model breakdown chart after, with per-user monthly spend. | |
| 35 | +| **Budget cap** | An optional per-debate spend ceiling. Crossing it skips the remaining rounds and goes straight to synthesis - a cheaper answer instead of a runaway bill. | | |
| 36 | +| **The gavel** | Conclude a running debate gracefully: it finishes the phase in flight, records the partial round, and synthesizes what exists - unlike cancel, which aborts and wastes the spend. | | |
| 37 | +| **Pressure response ("spine")** | Per model, per round, cross the critique it received with whether it revised: _revised_, _defended_, _caved_ (changed a well-scored answer), _stonewalled_ (kept a poorly-scored one). Surfaces sycophancy and stubbornness with zero extra LLM calls. | | |
| 38 | +| **Peer prediction** | Each reviewer also predicts the council's average score for an answer. Comparing prediction to reality splits dissent into informed contrarians (low score, accurate read) and miscalibrated reviewers (low score, wrong read), with a per-reviewer "council read" calibration column. | | |
| 39 | +| **Claim provenance** | An advisory post-synthesis audit traces every substantive claim in the final answer back to the council's answers and flags unsourced "chairman additions" - synthesis hallucination detection - run on the cheap convergence model. | | |
| 40 | +| **Re-run & key recovery** | Re-run any past debate (`/debate?from=<id>` prefills its question and council). A start that fails for a missing key drops you into the key dialog with the composed debate intact, not a dead end. | | |
| 35 | 41 | | **BYOK, two modes** | Bring your own OpenRouter key - saved (AES-256-GCM encrypted at rest) or session-only (encrypted HttpOnly cookie). No server-paid key. | |
| 36 | 42 | | **Portfolio demo mode** | Public, no-login page that replays real recorded debates through the full UI with simulated streaming. | |
| 37 | 43 | | **Export & share** | Download a Markdown deliberation report, or mint an unlisted share link. | |
| @@ -106,7 +112,7 @@flowchart TB | ||
| 106 | 112 | API -- "replay snapshot" --> DB |
| 107 | 113 | ``` |
| 108 | 114 | |
| 109 | -A single typed **`DebateEvent`** discriminated union (`stage_started`, `token_delta`, `answer_completed`, `critique_completed`, `convergence_result`, `model_failed`, `synthesis_completed`, `debate_completed`, ...) is the one source of truth: the SSE endpoint serializes it, the persistence layer reacts to it, and the client reducer folds it into the `DebateView` that every component renders - live streaming and recorded replay share the exact same components. | |
| 115 | +A single typed **`DebateEvent`** discriminated union (`stage_started`, `token_delta`, `answer_completed`, `critique_completed`, `convergence_result`, `model_failed`, `synthesis_completed`, `provenance_completed`, `budget_reached`, `gavel_struck`, `debate_completed`, ...) is the one source of truth: the SSE endpoint serializes it, the persistence layer reacts to it, and the client reducer folds it into the `DebateView` that every component renders - live streaming and recorded replay share the exact same components. | |
| 110 | 116 | |
| 111 | 117 | ### The debate state machine |
| 112 | 118 | |
| @@ -119,7 +125,10 @@stateDiagram-v2 | ||
| 119 | 125 | Revision --> Convergence: revise or defend (JSON + changelog) |
| 120 | 126 | Convergence --> Critique: score < threshold & rounds left |
| 121 | 127 | Convergence --> Synthesis: score ≥ threshold OR max rounds |
| 122 | - Synthesis --> [*]: final answer + dissent report | |
| 128 | + Critique --> Synthesis: gavel struck (partial round recorded) | |
| 129 | + Convergence --> Synthesis: budget cap reached OR gavel struck | |
| 130 | + Synthesis --> Provenance: trace claims (advisory) | |
| 131 | + Provenance --> [*]: final answer + dissent + claim check | |
| 123 | 132 | ``` |
| 124 | 133 | |
| 125 | 134 | ### Project layout |
| @@ -132,7 +141,8 @@src/ | ||
| 132 | 141 | │ ├── anonymize.ts # seeded PRNG + Fisher-Yates shuffle |
| 133 | 142 | │ ├── json-repair.ts # defensive parse + Zod validate |
| 134 | 143 | │ ├── convergence.ts # pure early-stop decision |
| 135 | -│ ├── scoring.ts # critique score matrix | |
| 144 | +│ ├── scoring.ts # critique score matrix + peer-prediction analytics | |
| 145 | +│ ├── spine.ts # per-model pressure response (revised/defended/caved/stonewalled) | |
| 136 | 146 | │ ├── llm-client.ts # the LlmClient interface the engine depends on |
| 137 | 147 | │ ├── mock-client.ts # deterministic offline model |
| 138 | 148 | │ └── *.test.ts # Vitest unit tests |
| @@ -163,7 +173,7 @@pnpm lint # ESLint | ||
| 163 | 173 | pnpm build # production build |
| 164 | 174 | ``` |
| 165 | 175 | |
| 166 | -The orchestrator suite drives full debates through the mock client and asserts on the emitted event stream: happy path, early convergence, max-rounds, model drop-out (and the ≥2-healthy floor), JSON-repair recovery, chairman provider-conflict detection, and cancellation. Around 100 unit tests cover the engine plus the lib layer (crypto, rate limiting, cost estimation, the event reducer, Markdown export, and the demo fixtures). CI also runs a schema + seed smoke test against a real Postgres. | |
| 176 | +The orchestrator suite drives full debates through the mock client and asserts on the emitted event stream: happy path, early convergence, max-rounds, model drop-out (and the ≥2-healthy floor), JSON-repair recovery, chairman provider-conflict detection, budget-cap and gavel early-exits, the advisory provenance audit, and cancellation. Around 130 unit tests cover the engine plus the lib layer (crypto, rate limiting, cost estimation, the event reducer, Markdown export, spine and peer-prediction analytics, and the demo fixtures). CI also runs a schema + seed smoke test against a real Postgres. | |
| 167 | 177 | |
| 168 | 178 | ## ⌨️ Run the engine from the terminal |
| 169 | 179 | |
| @@ -173,7 +183,7 @@Because `src/core` is framework-agnostic, the same orchestrator the web app driv | ||
| 173 | 183 | pnpm debate "Is a modular monolith better than microservices for a small team?" |
| 174 | 184 | ``` |
| 175 | 185 | |
| 176 | -It streams each stage to stdout and prints the chairman's final answer, dissent, and cost, all on the deterministic mock client. | |
| 186 | +It streams each stage to stdout and prints the chairman's final answer, dissent, any unsourced "chairman additions" from the provenance audit, and cost, all on the deterministic mock client. | |
| 177 | 187 | |
| 178 | 188 | ## 🚢 Deployment (Linux VPS + Caddy) |
| 179 | 189 |
modified docs/README.md +3 −0
| @@ -9,3 +9,6 @@Suggested captures (run `pnpm dev` with `MOCK_LLM=1`, open `/demo`): | ||
| 9 | 9 | - `revision-diff.png` — an inline revision diff with the changelog. |
| 10 | 10 | - `demo-playback.gif` — the demo playing back with streaming + controls. |
| 11 | 11 | - `cost-breakdown.png` — the per-model cost chart. |
| 12 | +- `claim-check.png` - the final answer with claims traced back to the council and unsourced chairman additions flagged. | |
| 13 | +- `under-pressure.png` - the pressure-response panel with each model's revised/defended/caved/stonewalled verdicts. | |
| 14 | +- `critique-prediction.png` - the critique matrix with informed-contrarian (violet) and miscalibrated (amber) outlines. |