Side A vendors an entire generic 'durable' RocksDB collections crate (with unused Entry/nested-collection API, README, docs, LICENSE, benchmarks, examples) just to back a simple string->string JSON store, adding large dependency/boilerplate weight disproportionate to the actual server-side change (entity_store.rs + streaming replay), much of which reads as generated filler. Side B's patch is tightly scoped to real, used functionality: CI/CD deploy pipeline, Dockerfile/fly.toml for production, a working live SSE audit dashboard, and concrete correctness fixes (require OPENROUTER_API_KEY for contested rankings, dedupe redundant git-object hashing, GITHUB_TOKEN auth), all backed by updated tests exercising the new behavior.
constitution · epochs · watch · epoch 3
comparison
c_2dc96aace098 (tommy-mor) vs c_7a129e904906 (tommy-mor)
download prompt · raw event · cmp_2d4005ebfe0c57
council reasoning
A’s lasting value is architectural: entity JSON moves out of in-memory GlobalTree into a RocksDB-backed EntityStore, and event_log gains true line-at-a-time replay so startup no longer materializes the full log/payloads in RAM—concrete reducer/state/reddit call-site changes that change scalability. B is real product/ops work (Fly/Dockerfile/CI deploy, /watch audit SSE, production repo/contributor roots, emission retry), but it is mostly deployment, UI, and configuration on an existing process rather than a deeper data-plane redesign.
Side A makes substantive architectural changes: it introduces a RocksDB-backed durable storage crate, moves large Reddit JSON payloads out of the in-memory tree into a persistent entity store, and replaces whole-log loading with streaming event replay to reduce startup memory, updating replay and integration tests accordingly. Side B primarily adds deployment infrastructure, a live dashboard, SSE status/audit UI, and CI/CD configuration, which improves operations and visibility but has much less impact on the project's core functionality and architecture.
sides
A — c_2dc96aace098 (tommy-mor)
message
[285b64d4] Offload Reddit payloads to RocksDB and stream event log replay. Vendor durable as a workspace crate, store entity JSON in entity_db instead of GlobalTree, and replay events.jsonl one line at a time to cut startup RAM. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/Cargo.lock b/Cargo.lock index e55d87f32ab32064686431c7082ef8c9ca872d63..8fc09f9ac978bd7ccf57f177989057b606677db8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -11,6 +11,18 @@ dependencies = [ "memchr", ] +[[package]] +name = "anes" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b46cbb362ab8752921c97e041f5e366ee6297bd428a31275b9fcf1e380f7299" + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + [[package]] name = "anyhow" version = "1.0.102" @@ -56,6 +68,12 @@ version = "1.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + [[package]] name = "axum" version = "0.7.9" @@ -153,6 +171,75 @@ version = "0.22.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" +[[package]] +name = "bincode" +version = "1.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad" +dependencies = [ + "serde", +] + +[[package]] +name = "bindgen" +version = "0.65.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfdf7b466f9a4903edc73f95d6d2bcd5baf8ae620638762244d3f60143643cc5" +dependencies = [ + "bitflags 1.3.2", + "cexpr", + "clang-sys", + "lazy_static", + "lazycell", + "peeking_take_while", + "prettyplease", + "proc-macro2", + "quote", + "regex", + "rustc-hash 1.1.0", + "shlex", + "syn", +] + +[[package]] +name = "bindgen" +version = "0.72.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "993776b509cfb49c750f11b8f07a46fa23e0a1386ffc01fb1e7d343efc387895" +dependencies = [ + "bitflags 2.11.1", + "cexpr", + "clang-sys", + "itertools 0.13.0", + "proc-macro2", + "quote", + "regex", + "rustc-hash 2.1.2", + "shlex", + "syn", +] + +[[package]] +name = "bit-set" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3" +dependencies = [ + "bit-vec", +] + +[[package]] +name = "bit-vec" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + [[package]] name = "bitflags" version = "2.11.1" @@ -171,6 +258,22 @@ version = "1.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33" +[[package]] +name = "bzip2-sys" +version = "0.1.13+1.0.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "225bff33b2141874fe80d71e07d6eec4f85c5c216453dd96388240f96e1acc14" +dependencies = [ + "cc", + "pkg-config", +] + +[[package]] +name = "cast" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5" + [[package]] name = "cc" version = "1.2.62" @@ -178,15 +281,89 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a1dce859f0832a7d088c4f1119888ab94ef4b5d6795d1ce05afb7fe159d79f98" dependencies = [ "find-msvc-tools", + "jobserver", + "libc", "shlex", ] +[[package]] +name = "cexpr" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6fac387a98bb7c37292057cffc56d62ecb629900026402633ae9160df93a8766" +dependencies = [ + "nom", +] + [[package]] name = "cfg-if" version = "1.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" +[[package]] +name = "ciborium" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "42e69ffd6f0917f5c029256a24d0161db17cea3997d185db0d35926308770f0e" +dependencies = [ + "ciborium-io", + "ciborium-ll", + "serde", +] + +[[package]] +name = "ciborium-io" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05afea1e0a06c9be33d539b876f1ce3692f4afea2cb41f740e7743225ed1c757" + +[[package]] +name = "ciborium-ll" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57663b653d948a338bfb3eeba9bb2fd5fcfaecb9e199e87e1eda4d9e8b240fd9" +dependencies = [ + "ciborium-io", + "half", +] + +[[package]] +name = "clang-sys" +version = "1.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b023947811758c97c59bf9d1c188fd619ad4718dcaa767947df1cadb14f39f4" +dependencies = [ + "glob", + "libc", + "libloading", +] + +[[package]] +name = "clap" +version = "4.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51" +dependencies = [ + "clap_builder", +] + +[[package]] +name = "clap_builder" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f" +dependencies = [ + "anstyle", + "clap_lex", +] + +[[package]] +name = "clap_lex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" + [[package]] name = "cookie" version = "0.18.1" @@ -224,6 +401,73 @@ version = "0.8.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" +[[package]] +name = "criterion" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2b12d017a929603d80db1831cd3a24082f8137ce19c69e6447f54f5fc8d692f" +dependencies = [ + "anes", + "cast", + "ciborium", + "clap", + "criterion-plot", + "is-terminal", + "itertools 0.10.5", + "num-traits", + "once_cell", + "oorandom", + "plotters", + "rayon", + "regex", + "serde", + "serde_derive", + "serde_json", + "tinytemplate", + "walkdir", +] + +[[package]] +name = "criterion-plot" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6b50826342786a51a89e2da3a28f1c32b06e387201bc2d19791f622c673706b1" +dependencies = [ + "cast", + "itertools 0.10.5", +] + +[[package]] +name = "crossbeam-deque" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9dd111b7b7f7d55b72c0a6ae361660ee5853c9af73f70c3c2ef6858b950e2e51" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28" + +[[package]] +name = "crunchy" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5" + [[package]] name = "deranged" version = "0.5.8" @@ -250,6 +494,25 @@ version = "0.15.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b" +[[package]] +name = "durable" +version = "0.1.0" +dependencies = [ + "bincode", + "criterion", + "proptest", + "rocksdb", + "serde", + "tempfile", + "thiserror", +] + +[[package]] +name = "either" +version = "1.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91622ff5e7162018101f2fea40d6ebf4a78bbe5a49736a2020649edf9693679e" + [[package]] name = "encoding_rs" version = "0.8.35" @@ -373,6 +636,18 @@ dependencies = [ "wasi", ] +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "libc", + "r-efi 5.3.0", + "wasip2", +] + [[package]] name = "getrandom" version = "0.4.2" @@ -381,11 +656,17 @@ checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555" dependencies = [ "cfg-if", "libc", - "r-efi", + "r-efi 6.0.0", "wasip2", "wasip3", ] +[[package]] +name = "glob" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280" + [[package]] name = "h2" version = "0.4.14" @@ -405,6 +686,17 @@ dependencies = [ "tracing", ] +[[package]] +name = "half" +version = "2.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ea2d84b969582b4b1864a92dc5d27cd2b77b622a8d79306834f1be5ba20d84b" +dependencies = [ + "cfg-if", + "crunchy", + "zerocopy", +] + [[package]] name = "hashbrown" version = "0.15.5" @@ -426,6 +718,12 @@ version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" +[[package]] +name = "hermit-abi" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc0fef456e4baa96da950455cd02c081ca953b141298e41db3fc7e36b1da849c" + [[package]] name = "http" version = "1.4.1" @@ -676,12 +974,51 @@ version = "2.12.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2" +[[package]] +name = "is-terminal" +version = "0.4.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3640c1c38b8e4e43584d8df18be5fc6b0aa314ce6ebf51b53313d4306cca8e46" +dependencies = [ + "hermit-abi", + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "itertools" +version = "0.10.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b0fd2260e829bddf4cb6ea802289de2f86d6a7a690192fbe91b3f46e0f2c8473" +dependencies = [ + "either", +] + +[[package]] +name = "itertools" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186" +dependencies = [ + "either", +] + [[package]] name = "itoa" version = "1.0.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" +[[package]] +name = "jobserver" +version = "0.1.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33" +dependencies = [ + "getrandom 0.3.4", + "libc", +] + [[package]] name = "js-sys" version = "0.3.99" @@ -700,6 +1037,12 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" +[[package]] +name = "lazycell" +version = "1.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "830d08ce1d1d941e6b30645f1a0eb5643013d835ce3779a5fc208261dbe10f55" + [[package]] name = "leb128fmt" version = "0.1.0" @@ -712,6 + … preview truncated; 197,635 characters omitted
B — c_7a129e904906 (tommy-mor)
message
[4e327784] deploy live constitution dashboard Expose auditable progress and event streaming, configure the production roots and runtime, and make tested main-branch commits the deployment authority. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/.dockerignore b/.dockerignore
new file mode 100644
index 0000000000000000000000000000000000000000..c9d63a722beba0a0297fc853089332c460ab78dd
--- /dev/null
+++ b/.dockerignore
@@ -0,0 +1,7 @@
+.git
+.venv
+.hypothesis
+__pycache__
+tests
+*.json
+*.bsp
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
new file mode 100644
index 0000000000000000000000000000000000000000..76dcdf82d53177c1e47d86b23a54523239d232a6
--- /dev/null
+++ b/.github/workflows/deploy.yml
@@ -0,0 +1,49 @@
+name: Test and deploy
+
+on:
+ push:
+ branches: [main]
+
+concurrency:
+ group: production
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ test:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: astral-sh/setup-uv@v6
+ with:
+ enable-cache: true
+
+ - name: Run Python tests
+ run: uv run pytest -q
+
+ - name: Install Babashka
+ run: |
+ curl -fsSL https://raw.githubusercontent.com/babashka/babashka/master/install \
+ | sudo bash -s -- --dir /usr/local/bin
+
+ - name: Run process integration tests
+ run: bb TEST.sh
+
+ deploy:
+ needs: test
+ runs-on: ubuntu-latest
+ environment:
+ name: production
+ url: https://token.slug.social
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: superfly/flyctl-actions/setup-flyctl@master
+
+ - name: Deploy to Fly
+ run: flyctl deploy --remote-only
+ env:
+ FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000000000000000000000000000000000000..c9a5c00782371c19ad5ab5c58cf6f5a8ffec0141
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,17 @@
+FROM ghcr.io/astral-sh/uv:python3.11-bookworm-slim
+
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends git ca-certificates \
+ && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+COPY pyproject.toml uv.lock ./
+RUN uv sync --frozen --no-install-project
+
+COPY constitution.py ./
+
+ENV PATH="/app/.venv/bin:${PATH}" \
+ PYTHONUNBUFFERED="1"
+
+EXPOSE 8080
+CMD ["python", "constitution.py"]
diff --git a/constitution.py b/constitution.py
index 4bee9f83663ab7fb36db95129b92b64b4ef57258..a58257e1881b21d1d6fa8e68a3faa222e4f661ef 100644
--- a/constitution.py
+++ b/constitution.py
@@ -24,12 +24,12 @@ A daily GitHub Action backs up the JSONL ledger to the same repo.
Run: uv run constitution.py
"""
-from decimal import Decimal, getcontext
+from decimal import Decimal, getcontext, DefaultContext
from datetime import datetime, timezone
from fastapi import FastAPI, Request, Response
from fastapi.responses import PlainTextResponse, HTMLResponse
from starlette.middleware.sessions import SessionMiddleware
-import json, time, os, asyncio, httpx, pathlib, subprocess, hashlib, re, fcntl
+import json, time, os, asyncio, httpx, pathlib, subprocess, hashlib, re, fcntl, base64
import sympy as sp # type: ignore[reportMissingImports]
from tenacity import retry, retry_if_exception, stop_after_attempt, wait_exponential
from evaleval import (
@@ -37,6 +37,7 @@ from evaleval import (
exec_event, One, Two, Three, Selector, MORPH, PREPEND,
)
+DefaultContext.prec = 50
getcontext().prec = 50
app = FastAPI()
@@ -129,14 +130,47 @@ OPENROUTER_BASE_URL = os.environ.get("OPENROUTER_BASE_URL", "https://openrouter.
# using the exact same source; their normalized values are committed to every
# discovery event.
DEFAULT_REPOSITORIES = [
+ {
+ "id": "constitution",
+ "url": "https://github.com/sortersocial/constitution.git",
+ "refs": ["refs/heads/**"],
+ },
{
"id": "slug",
- "url": "https://github.com/tommy-mor/slug.git",
+ "url": "https://github.com/sortersocial/slug.git",
+ "refs": ["refs/heads/**"],
+ },
+ {
+ "id": "sorter",
+ "url": "https://github.com/sorterisntonline/sorter.git",
+ "refs": ["refs/heads/**"],
+ },
+ {
+ "id": "sorter2",
+ "url": "https://github.com/sortersocial/sorter2.git",
+ "refs": ["refs/heads/**"],
+ },
+ {
+ "id": "sorter-oldest",
+ "url": "https://github.com/tommy-mor/sorter.git",
"refs": ["refs/heads/**"],
},
]
DEFAULT_CONTRIBUTORS = {
"tommy-mor": ["thmorriss@gmail.com"],
+ "christopher-whitman": [
+ "chris@cwwhitman.com",
+ "7566903+cwwhitman@users.noreply.github.com",
+ ],
+ "jake-chvatal": [
+ "jake+github@uln.industries",
+ "jakechvatal@gmail.com",
+ "jake@isnt.online",
+ ],
+ "lara": ["me@lara.lv"],
+ "nat-reid": ["nathanielreid@gmail.com"],
+ "zod": ["jason.p.mcel@gmail.com", "me@zod.tf"],
+ "jovan": ["jovan@slug.social", "jovan@getcivicai.com"],
}
REPOSITORIES = json.loads(
@@ -147,6 +181,7 @@ CONTRIBUTORS = json.loads(
)
GIT_MIRROR_DIR = pathlib.Path(os.environ.get("GIT_MIRROR_DIR", "/data/git"))
GIT_TIMEOUT_SECONDS = int(os.environ.get("GIT_TIMEOUT_SECONDS", "120"))
+GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")
# Council model IDs: slug.social garden rank under this parent (bodies = OpenRouter URLs), then top-up from OpenRouter list.
SLUG_SOCIAL_BASE_URL = os.environ.get("SLUG_SOCIAL_BASE_URL", "https://slug.social").rstrip("/")
@@ -661,20 +696,30 @@ def _git(repo: pathlib.Path | None, *args: str, input_bytes: bytes | None = None
if repo is not None:
command += ["-C", str(repo)]
command += list(args)
+ git_env = {
+ **os.environ,
+ "GIT_CONFIG_NOSYSTEM": "1",
+ "GIT_CONFIG_GLOBAL": os.devnull,
+ "GIT_NO_REPLACE_OBJECTS": "1",
+ "LC_ALL": "C",
+ "TZ": "UTC",
+ }
+ if GITHUB_TOKEN:
+ credential = base64.b64encode(
+ f"x-access-token:{GITHUB_TOKEN}".encode()
+ ).decode()
+ git_env.update({
+ "GIT_CONFIG_COUNT": "1",
+ "GIT_CONFIG_KEY_0": "http.https://github.com/.extraHeader",
+ "GIT_CONFIG_VALUE_0": f"Authorization: Basic {credential}",
+ })
try:
result = subprocess.run(
command,
input=input_bytes,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
- env={
- **os.environ,
- "GIT_CONFIG_NOSYSTEM": "1",
- "GIT_CONFIG_GLOBAL": os.devnull,
- "GIT_NO_REPLACE_OBJECTS": "1",
- "LC_ALL": "C",
- "TZ": "UTC",
- },
+ env=git_env,
timeout=GIT_TIMEOUT_SECONDS,
check=False,
)
@@ -844,9 +889,15 @@ def _build_discovery(epoch_n: int, boundary_ms: int, events: list) -> GitDiscove
canonical_location = min(
locations[qualified_oid], key=lambda x: (x[0], x[1])
)
+ # One commit may be reachable from dozens of refs in the same mirror.
+ # Verify its object once per repository, not once per source ref.
+ object_locations = {
+ (str(m), raw_oid): (m, raw_oid)
+ for _, _, m, raw_oid in locations[qualified_oid]
+ }
object_hashes = {
hashlib.sha256(_git(m, "cat-file", "commit", raw_oid)).hexdigest()
- for _, _, m, raw_oid in locations[qualified_oid]
+ for m, raw_oid in object_locations.values()
}
if len(object_hashes) != 1:
raise RuntimeError(f"conflicting Git objects share OID {qualified_oid}")
@@ -994,11 +1045,55 @@ async def discover_repositories(epoch_n: int, boundary_ms: int) -> GitDiscovery:
SSE_CLIENTS = []
+AUDIT_HISTORY = []
+AUDIT_SEQUENCE = 0
+PROCESS_STATE = {
+ "running": False,
+ "phase": "idle",
+ "progress": 100,
+ "message": "Waiting for the next epoch",
+}
+
+
+def _sse_event(event_name: str, payload: dict) -> str:
+ return (
+ f"event: {event_name}\n"
+ f"data: {json.dumps(payload, separators=(',', ':'))}\n\n"
+ )
+
+
+async def broadcast_audit(
+ kind: str,
+ message: str,
+ *,
+ progress: int | None = None,
+ phase: str | None = None,
+) -> dict:
+ global AUDIT_SEQUENCE
+ AUDIT_SEQUENCE += 1
+ if progress is not None:
+ PROCESS_STATE["progress"] = max(0, min(100, int(progress)))
+ if phase is not None:
+ PROCESS_STATE["phase"] = phase
+ PROCESS_STATE["message"] = message
+ payload = {
+ "id": AUDIT_SEQUENCE,
+ "timestamp_ms": int(time.time() * 1000),
+ "kind": kind,
+ "message": message,
+ **PROCESS_STATE,
+ }
+ AUDIT_HISTORY.append(payload)
+ del AUDIT_HISTORY[:-200]
+ wire = _sse_event("audit", payload)
+ for queue in list(SSE_CLIENTS):
+ await queue.put(wire)
+ return payload
async def broadcast_js(js: str):
"""Send a JS snippet to all connected SSE clients."""
- for queue in SSE_CLIENTS:
+ for queue in list(SSE_CLIENTS):
await queue.put(js)
@@ -1006,10 +1101,29 @@ async def rank_commits(commits: list[dict]):
if not commits:
return {}, []
- models = await fetch_top_models(n=3)
contributors = sorted(set(c["contributor"] for c in commits))
- if len(contributors) > 1 and not models:
+ if len(contributors) == 1:
+ await broadcast_audit(
+ "ranking",
+ f"Only {contributors[0]} is eligible; rank is 1.0",
+ progress=90,
+ phase="finalizing",
+ )
+ return {contributors[0]: Decimal("1")}, []
+ if not (OPENROUTER_API_KEY or "").strip():
+ raise RuntimeError(
+ "OPENROUTER_API_KEY is required when multiple contributors need ranking"
+ )
+
+ models = await fetch_top_models(n=3)
+ if not models:
raise RuntimeError("no council models available for contributor ranking")
+ await broadcast_audit(
+ "council",
+ f"Council selected: {', '.join(models)}",
+ progress=35,
+ phase="ranking",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-log")][PREPEND][
["div.log-council", f"Council: {', '.join(models)} — {len(commits)} commits"]
]))
@@ -1035,6 +1149,11 @@ async def rank_commits(commits: list[dict]):
async def compare_fn(i, j):
a1, a2 = authors[i], authors[j]
+ await broadcast_audit(
+ "comparison",
+ f"Comparing {a1} with {a2}",
+ phase="ranking",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-status")][MORPH][
["div#emission-status", f"Comparing {a1} vs {a2}…"]
]))
@@ -1050,6 +1169,11 @@ async def rank_commits(commits: list[dict]):
if winner_weight <= 0 or loser_weight <= 0:
raise ValueError("ratio weights must be positive")
results.append((w, l, winner_weight, loser_weight))
+ await broadcast_audit(
+ "vote",
+ f"{model}: {authors[w]} over {authors[l]} ({result['ratio']})",
+ phase="ranking",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-log")][PREPEND][
["div.log-vote",
["span.model", model], " — ",
@@ -1059,6 +1183,11 @@ async def rank_commits(commits: list[dict]):
]
]))
except Exception as e:
+ await broadcast_audit(
+ "error",
+ f"{model} failed: {e}",
+ phase="error",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-log")][PREPEND][
["div.log-error", f"⚠ {model}: {e}"]
]))
@@ -1068,8 +1197,13 @@ async def rank_commits(commits: list[dict]):
async def progress_fn(ev):
if ev["phase"] == "spanning_tree":
label = f"Spanning tree: {ev['step']}/{ev['total']}"
+
… preview truncated; 28,089 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.