Side B fixes a genuine correctness bug (feed catch-up relying on timestamps that can collide or move backwards) with a principled ingest-order anchor, deduplicates duplicated code paths into a shared rpc_feed helper, and adds thorough targeted integration tests for multi-user/private-room visibility. Side A is largely infrastructure/deploy plumbing (Dockerfile, fly.toml, CI) plus a UI/dashboard feature with generated CSS/JS, which is useful operationally but lower in durable design value and correctness impact than B's bugfix.
constitution · epochs · watch · epoch 3
comparison
c_7a129e904906 (tommy-mor) vs c_0a9a8eab32ba (tommy-mor)
download prompt · raw event · cmp_fe7a38359237d3
council reasoning
A lands durable production infrastructure (Dockerfile, fly.toml, gated deploy workflow), epoch retry/error handling, GitHub-authenticated discovery, and a tested live audit SSE/watch surface that makes the constitution process operable. B is a precise, well-tested feed fix (ingest-index anchors, private-room visibility, room on FeedPost) but is a narrower correctness patch than A’s end-to-end production and observability delivery.
Side B fixes a core correctness issue in feed retrieval by anchoring implicit catch-up to durable ingest order instead of timestamps, preserving correct behavior across same-millisecond posts, clock rollback, and permission changes. It also refactors the feed logic into a shared helper, includes room metadata, and adds thorough integration tests covering multi-user visibility and revoked access, whereas Side A is largely deployment, monitoring, and operational infrastructure with only a few smaller correctness improvements (such as deduplicating Git object verification and authenticated Git fetches).
sides
A — c_7a129e904906 (tommy-mor)
message
[4e327784] deploy live constitution dashboard Expose auditable progress and event streaming, configure the production roots and runtime, and make tested main-branch commits the deployment authority. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/.dockerignore b/.dockerignore
new file mode 100644
index 0000000000000000000000000000000000000000..c9d63a722beba0a0297fc853089332c460ab78dd
--- /dev/null
+++ b/.dockerignore
@@ -0,0 +1,7 @@
+.git
+.venv
+.hypothesis
+__pycache__
+tests
+*.json
+*.bsp
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
new file mode 100644
index 0000000000000000000000000000000000000000..76dcdf82d53177c1e47d86b23a54523239d232a6
--- /dev/null
+++ b/.github/workflows/deploy.yml
@@ -0,0 +1,49 @@
+name: Test and deploy
+
+on:
+ push:
+ branches: [main]
+
+concurrency:
+ group: production
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ test:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: astral-sh/setup-uv@v6
+ with:
+ enable-cache: true
+
+ - name: Run Python tests
+ run: uv run pytest -q
+
+ - name: Install Babashka
+ run: |
+ curl -fsSL https://raw.githubusercontent.com/babashka/babashka/master/install \
+ | sudo bash -s -- --dir /usr/local/bin
+
+ - name: Run process integration tests
+ run: bb TEST.sh
+
+ deploy:
+ needs: test
+ runs-on: ubuntu-latest
+ environment:
+ name: production
+ url: https://token.slug.social
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: superfly/flyctl-actions/setup-flyctl@master
+
+ - name: Deploy to Fly
+ run: flyctl deploy --remote-only
+ env:
+ FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000000000000000000000000000000000000..c9a5c00782371c19ad5ab5c58cf6f5a8ffec0141
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,17 @@
+FROM ghcr.io/astral-sh/uv:python3.11-bookworm-slim
+
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends git ca-certificates \
+ && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+COPY pyproject.toml uv.lock ./
+RUN uv sync --frozen --no-install-project
+
+COPY constitution.py ./
+
+ENV PATH="/app/.venv/bin:${PATH}" \
+ PYTHONUNBUFFERED="1"
+
+EXPOSE 8080
+CMD ["python", "constitution.py"]
diff --git a/constitution.py b/constitution.py
index 4bee9f83663ab7fb36db95129b92b64b4ef57258..a58257e1881b21d1d6fa8e68a3faa222e4f661ef 100644
--- a/constitution.py
+++ b/constitution.py
@@ -24,12 +24,12 @@ A daily GitHub Action backs up the JSONL ledger to the same repo.
Run: uv run constitution.py
"""
-from decimal import Decimal, getcontext
+from decimal import Decimal, getcontext, DefaultContext
from datetime import datetime, timezone
from fastapi import FastAPI, Request, Response
from fastapi.responses import PlainTextResponse, HTMLResponse
from starlette.middleware.sessions import SessionMiddleware
-import json, time, os, asyncio, httpx, pathlib, subprocess, hashlib, re, fcntl
+import json, time, os, asyncio, httpx, pathlib, subprocess, hashlib, re, fcntl, base64
import sympy as sp # type: ignore[reportMissingImports]
from tenacity import retry, retry_if_exception, stop_after_attempt, wait_exponential
from evaleval import (
@@ -37,6 +37,7 @@ from evaleval import (
exec_event, One, Two, Three, Selector, MORPH, PREPEND,
)
+DefaultContext.prec = 50
getcontext().prec = 50
app = FastAPI()
@@ -129,14 +130,47 @@ OPENROUTER_BASE_URL = os.environ.get("OPENROUTER_BASE_URL", "https://openrouter.
# using the exact same source; their normalized values are committed to every
# discovery event.
DEFAULT_REPOSITORIES = [
+ {
+ "id": "constitution",
+ "url": "https://github.com/sortersocial/constitution.git",
+ "refs": ["refs/heads/**"],
+ },
{
"id": "slug",
- "url": "https://github.com/tommy-mor/slug.git",
+ "url": "https://github.com/sortersocial/slug.git",
+ "refs": ["refs/heads/**"],
+ },
+ {
+ "id": "sorter",
+ "url": "https://github.com/sorterisntonline/sorter.git",
+ "refs": ["refs/heads/**"],
+ },
+ {
+ "id": "sorter2",
+ "url": "https://github.com/sortersocial/sorter2.git",
+ "refs": ["refs/heads/**"],
+ },
+ {
+ "id": "sorter-oldest",
+ "url": "https://github.com/tommy-mor/sorter.git",
"refs": ["refs/heads/**"],
},
]
DEFAULT_CONTRIBUTORS = {
"tommy-mor": ["thmorriss@gmail.com"],
+ "christopher-whitman": [
+ "chris@cwwhitman.com",
+ "7566903+cwwhitman@users.noreply.github.com",
+ ],
+ "jake-chvatal": [
+ "jake+github@uln.industries",
+ "jakechvatal@gmail.com",
+ "jake@isnt.online",
+ ],
+ "lara": ["me@lara.lv"],
+ "nat-reid": ["nathanielreid@gmail.com"],
+ "zod": ["jason.p.mcel@gmail.com", "me@zod.tf"],
+ "jovan": ["jovan@slug.social", "jovan@getcivicai.com"],
}
REPOSITORIES = json.loads(
@@ -147,6 +181,7 @@ CONTRIBUTORS = json.loads(
)
GIT_MIRROR_DIR = pathlib.Path(os.environ.get("GIT_MIRROR_DIR", "/data/git"))
GIT_TIMEOUT_SECONDS = int(os.environ.get("GIT_TIMEOUT_SECONDS", "120"))
+GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")
# Council model IDs: slug.social garden rank under this parent (bodies = OpenRouter URLs), then top-up from OpenRouter list.
SLUG_SOCIAL_BASE_URL = os.environ.get("SLUG_SOCIAL_BASE_URL", "https://slug.social").rstrip("/")
@@ -661,20 +696,30 @@ def _git(repo: pathlib.Path | None, *args: str, input_bytes: bytes | None = None
if repo is not None:
command += ["-C", str(repo)]
command += list(args)
+ git_env = {
+ **os.environ,
+ "GIT_CONFIG_NOSYSTEM": "1",
+ "GIT_CONFIG_GLOBAL": os.devnull,
+ "GIT_NO_REPLACE_OBJECTS": "1",
+ "LC_ALL": "C",
+ "TZ": "UTC",
+ }
+ if GITHUB_TOKEN:
+ credential = base64.b64encode(
+ f"x-access-token:{GITHUB_TOKEN}".encode()
+ ).decode()
+ git_env.update({
+ "GIT_CONFIG_COUNT": "1",
+ "GIT_CONFIG_KEY_0": "http.https://github.com/.extraHeader",
+ "GIT_CONFIG_VALUE_0": f"Authorization: Basic {credential}",
+ })
try:
result = subprocess.run(
command,
input=input_bytes,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
- env={
- **os.environ,
- "GIT_CONFIG_NOSYSTEM": "1",
- "GIT_CONFIG_GLOBAL": os.devnull,
- "GIT_NO_REPLACE_OBJECTS": "1",
- "LC_ALL": "C",
- "TZ": "UTC",
- },
+ env=git_env,
timeout=GIT_TIMEOUT_SECONDS,
check=False,
)
@@ -844,9 +889,15 @@ def _build_discovery(epoch_n: int, boundary_ms: int, events: list) -> GitDiscove
canonical_location = min(
locations[qualified_oid], key=lambda x: (x[0], x[1])
)
+ # One commit may be reachable from dozens of refs in the same mirror.
+ # Verify its object once per repository, not once per source ref.
+ object_locations = {
+ (str(m), raw_oid): (m, raw_oid)
+ for _, _, m, raw_oid in locations[qualified_oid]
+ }
object_hashes = {
hashlib.sha256(_git(m, "cat-file", "commit", raw_oid)).hexdigest()
- for _, _, m, raw_oid in locations[qualified_oid]
+ for m, raw_oid in object_locations.values()
}
if len(object_hashes) != 1:
raise RuntimeError(f"conflicting Git objects share OID {qualified_oid}")
@@ -994,11 +1045,55 @@ async def discover_repositories(epoch_n: int, boundary_ms: int) -> GitDiscovery:
SSE_CLIENTS = []
+AUDIT_HISTORY = []
+AUDIT_SEQUENCE = 0
+PROCESS_STATE = {
+ "running": False,
+ "phase": "idle",
+ "progress": 100,
+ "message": "Waiting for the next epoch",
+}
+
+
+def _sse_event(event_name: str, payload: dict) -> str:
+ return (
+ f"event: {event_name}\n"
+ f"data: {json.dumps(payload, separators=(',', ':'))}\n\n"
+ )
+
+
+async def broadcast_audit(
+ kind: str,
+ message: str,
+ *,
+ progress: int | None = None,
+ phase: str | None = None,
+) -> dict:
+ global AUDIT_SEQUENCE
+ AUDIT_SEQUENCE += 1
+ if progress is not None:
+ PROCESS_STATE["progress"] = max(0, min(100, int(progress)))
+ if phase is not None:
+ PROCESS_STATE["phase"] = phase
+ PROCESS_STATE["message"] = message
+ payload = {
+ "id": AUDIT_SEQUENCE,
+ "timestamp_ms": int(time.time() * 1000),
+ "kind": kind,
+ "message": message,
+ **PROCESS_STATE,
+ }
+ AUDIT_HISTORY.append(payload)
+ del AUDIT_HISTORY[:-200]
+ wire = _sse_event("audit", payload)
+ for queue in list(SSE_CLIENTS):
+ await queue.put(wire)
+ return payload
async def broadcast_js(js: str):
"""Send a JS snippet to all connected SSE clients."""
- for queue in SSE_CLIENTS:
+ for queue in list(SSE_CLIENTS):
await queue.put(js)
@@ -1006,10 +1101,29 @@ async def rank_commits(commits: list[dict]):
if not commits:
return {}, []
- models = await fetch_top_models(n=3)
contributors = sorted(set(c["contributor"] for c in commits))
- if len(contributors) > 1 and not models:
+ if len(contributors) == 1:
+ await broadcast_audit(
+ "ranking",
+ f"Only {contributors[0]} is eligible; rank is 1.0",
+ progress=90,
+ phase="finalizing",
+ )
+ return {contributors[0]: Decimal("1")}, []
+ if not (OPENROUTER_API_KEY or "").strip():
+ raise RuntimeError(
+ "OPENROUTER_API_KEY is required when multiple contributors need ranking"
+ )
+
+ models = await fetch_top_models(n=3)
+ if not models:
raise RuntimeError("no council models available for contributor ranking")
+ await broadcast_audit(
+ "council",
+ f"Council selected: {', '.join(models)}",
+ progress=35,
+ phase="ranking",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-log")][PREPEND][
["div.log-council", f"Council: {', '.join(models)} — {len(commits)} commits"]
]))
@@ -1035,6 +1149,11 @@ async def rank_commits(commits: list[dict]):
async def compare_fn(i, j):
a1, a2 = authors[i], authors[j]
+ await broadcast_audit(
+ "comparison",
+ f"Comparing {a1} with {a2}",
+ phase="ranking",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-status")][MORPH][
["div#emission-status", f"Comparing {a1} vs {a2}…"]
]))
@@ -1050,6 +1169,11 @@ async def rank_commits(commits: list[dict]):
if winner_weight <= 0 or loser_weight <= 0:
raise ValueError("ratio weights must be positive")
results.append((w, l, winner_weight, loser_weight))
+ await broadcast_audit(
+ "vote",
+ f"{model}: {authors[w]} over {authors[l]} ({result['ratio']})",
+ phase="ranking",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-log")][PREPEND][
["div.log-vote",
["span.model", model], " — ",
@@ -1059,6 +1183,11 @@ async def rank_commits(commits: list[dict]):
]
]))
except Exception as e:
+ await broadcast_audit(
+ "error",
+ f"{model} failed: {e}",
+ phase="error",
+ )
await broadcast_js(exec_event(Three[Selector("#emission-log")][PREPEND][
["div.log-error", f"⚠ {model}: {e}"]
]))
@@ -1068,8 +1197,13 @@ async def rank_commits(commits: list[dict]):
async def progress_fn(ev):
if ev["phase"] == "spanning_tree":
label = f"Spanning tree: {ev['step']}/{ev['total']}"
+
… preview truncated; 28,089 characters omittedB — c_0a9a8eab32ba (tommy-mor)
message
[c94456ff] Make feed catch-up stable and permission-aware Anchor implicit feeds to durable ingest order and cover multi-user private-room visibility so concurrent posts are not missed or leaked. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/cli/src/main.rs b/cli/src/main.rs
index abb5a55b49f60fe28fbfd4ec02715cb94ea0b4ec..c4f1494df3aedbd8b883aea6249579aa8336abfe 100644
--- a/cli/src/main.rs
+++ b/cli/src/main.rs
@@ -878,6 +878,30 @@ mod tests {
"graph: 4 items, 3/6 pairs (50.0% density), 1 component, connected"
);
}
+
+ #[test]
+ fn feed_without_since_uses_logged_in_delegate_from_env() {
+ let key = "SLUG_DELEGATE";
+ let previous = std::env::var_os(key);
+ let expected = "00000000-0000-0000-0000-0000000000ee:test:local/model";
+ std::env::set_var(key, expected);
+
+ let cli = Cli::try_parse_from(["slugsocial", "feed"]).expect("parse feed");
+
+ match previous {
+ Some(value) => std::env::set_var(key, value),
+ None => std::env::remove_var(key),
+ }
+ match cli.cmd {
+ Some(Command::Feed {
+ delegate, since, ..
+ }) => {
+ assert_eq!(delegate.as_deref(), Some(expected));
+ assert!(since.is_none());
+ }
+ _ => panic!("expected feed command"),
+ }
+ }
}
async fn run_scoped(base: &str, room: &str, sub: ScopedCmd) -> Result<()> {
@@ -1626,7 +1650,15 @@ async fn run() -> Result<()> {
} else {
for p in &resp.posts {
let ago = slug_types::timeago::timeago(now_ms, p.ts);
- println!("<post id=\"{}\" ts=\"{}\">", p.id, ago);
+ let thread_attr = p
+ .thread
+ .as_deref()
+ .map(|thread| format!(" thread=\"{thread}\""))
+ .unwrap_or_default();
+ println!(
+ "<post id=\"{}\" ts=\"{}\" room=\"{}\"{}>",
+ p.id, ago, p.room, thread_attr
+ );
print!("{}", p.body);
if !p.body.ends_with('\n') { println!(); }
println!("</post>");
diff --git a/server/src/api/rpc.rs b/server/src/api/rpc.rs
index afc4f95bef160c1e38ecff2c096d6440cd94e2b3..46d748f918d9b225acc4ecedfe5a1793089407b5 100644
--- a/server/src/api/rpc.rs
+++ b/server/src/api/rpc.rs
@@ -72,6 +72,80 @@ fn can_view_scope(reduced: &ReducerState, scope: &ScopeId, principal: Option<&st
}
}
+/// Build a feed in durable ingest order.
+///
+/// An implicit feed boundary is an ingest position, not only its millisecond timestamp. Two users
+/// can post in the same millisecond, and wall-clock timestamps can move backwards during replay.
+/// Explicit `since` remains a timestamp query for API compatibility, but scans the whole ordered
+/// ledger rather than assuming timestamps are monotonic.
+fn rpc_feed(
+ reduced: &ReducerState,
+ viewer: &str,
+ delegate: Option<String>,
+ requested_since: Option<i64>,
+ implicit_anchor: Option<(usize, i64)>,
+ limit: usize,
+) -> FeedResponse {
+ let since = requested_since.or_else(|| implicit_anchor.map(|(_, ts)| ts));
+ let implicit_anchor_index = requested_since
+ .is_none()
+ .then(|| implicit_anchor.map(|(index, _)| index))
+ .flatten();
+
+ let matching: Vec<&str> = reduced
+ .ingests_ordered
+ .iter()
+ .enumerate()
+ .rev()
+ .filter(|(index, id)| {
+ reduced.ingests_by_id.get(id.as_str()).is_some_and(|ing| {
+ match requested_since {
+ Some(cutoff) => ing.ts > cutoff,
+ None => implicit_anchor_index.is_none_or(|anchor| *index > anchor),
+ }
+ })
+ })
+ .map(|(_, id)| id.as_str())
+ .filter(|id| {
+ reduced.ingests_by_id.get(*id).is_some_and(|ing| {
+ let scope = scope_from_room_wire(&ing.room_id);
+ can_view_scope(reduced, &scope, Some(viewer))
+ })
+ })
+ .filter(|id| !reduced.redacted_posts.contains(*id))
+ .collect();
+
+ let total = matching.len();
+ let posts = matching
+ .into_iter()
+ .take(limit)
+ .filter_map(|id| reduced.ingests_by_id.get(id))
+ .map(|ing| {
+ let scope = scope_from_room_wire(&ing.room_id);
+ let thread_post_index = reduced.try_thread_post_index_chronological(
+ &scope,
+ &ing.thread_tag,
+ &ing.id,
+ );
+ FeedPost {
+ ts: ing.ts,
+ id: ing.id.clone(),
+ room: ing.room_id.clone(),
+ thread: Some(ing.thread_tag.clone()),
+ thread_post_index,
+ body: ing.raw.clone(),
+ }
+ })
+ .collect();
+
+ FeedResponse {
+ delegate,
+ since,
+ posts,
+ total,
+ }
+}
+
fn principal_from_optional_bearer(headers: &HeaderMap, reduced: &ReducerState) -> Result<Option<String>, RpcErr> {
if headers.contains_key(axum::http::header::AUTHORIZATION) {
verify_bearer_principal(headers, reduced)
@@ -1506,60 +1580,27 @@ pub async fn handle_rpc_batch(
Some("this delegate is not bound to your signed-in account".into()),
)
} else {
- let since_default = reduced
+ let implicit_anchor = reduced
.ingests_ordered
.iter()
+ .enumerate()
.rev()
- .filter_map(|id| reduced.ingests_by_id.get(id))
- .find(|ing| {
- if ing.delegate.as_deref() != Some(delegate_stored.as_str()) {
- return false;
- }
- let scope = scope_from_room_wire(&ing.room_id);
- can_view_scope(&reduced, &scope, Some(viewer.as_str()))
- })
- .map(|ing| ing.ts);
- let since = since.or(since_default);
- let cutoff = since.unwrap_or(0);
- let limit = limit.unwrap_or(DEFAULT_LIMIT).min(MAX_LIMIT);
- let matching: Vec<&str> = reduced.ingests_ordered.iter().rev()
- .map(|id| id.as_str())
- .take_while(|id| reduced.ingests_by_id.get(*id).is_some_and(|ing| ing.ts > cutoff))
- .filter(|id| {
- reduced.ingests_by_id.get(*id).is_some_and(|ing| {
- let scope = scope_from_room_wire(&ing.room_id);
- can_view_scope(&reduced, &scope, Some(viewer.as_str()))
+ .find_map(|(index, id)| {
+ reduced.ingests_by_id.get(id).and_then(|ing| {
+ (ing.delegate.as_deref()
+ == Some(delegate_stored.as_str()))
+ .then_some((index, ing.ts))
})
- })
- .filter(|id| !reduced.redacted_posts.contains(*id))
- .collect();
- let total = matching.len();
- let posts: Vec<FeedPost> = matching.into_iter()
- .take(limit)
- .filter_map(|id| reduced.ingests_by_id.get(id))
- .map(|ing| {
- let scope = scope_from_room_wire(&ing.room_id);
- let thread_post_index = reduced
- .try_thread_post_index_chronological(
- &scope,
- &ing.thread_tag,
- &ing.id,
- );
- FeedPost {
- ts: ing.ts,
- id: ing.id.clone(),
- thread: Some(ing.thread_tag.clone()),
- thread_post_index,
- body: ing.raw.clone(),
- }
- })
- .collect();
- line_ok(RpcResult::Feed(FeedResponse {
- delegate: Some(delegate_stored),
+ });
+ let limit = limit.unwrap_or(DEFAULT_LIMIT).min(MAX_LIMIT);
+ line_ok(RpcResult::Feed(rpc_feed(
+ &reduced,
+ &viewer,
+ Some(delegate_stored),
since,
- posts,
- total,
- }))
+ implicit_anchor,
+ limit,
+ )))
};
drop(reduced);
line
@@ -1567,60 +1608,25 @@ pub async fn handle_rpc_batch(
None => {
// Session catch-up: last time *you* posted anything (delegate or not), so revisiting
// an old chat with only a token still gets a sane cutoff.
- let since_default = reduced
+ let implicit_anchor = reduced
.ingests_ordered
.iter()
+ .enumerate()
.rev()
- .filter_map(|id| reduced.ingests_by_id.get(id))
- .find(|ing| {
- if ing.principal != viewer {
- return false;
- }
- let scope = scope_from_room_wire(&ing.room_id);
- can_view_scope(&reduced, &scope, Some(viewer.as_str()))
- })
- .map(|ing| ing.ts);
- let since = since.or(since_default);
- let cutoff = since.unwrap_or(0);
- let limit = limit.unwrap_or(DEFAULT_LIMIT).min(MAX_LIMIT);
- let matching: Vec<&str> = reduced.ingests_ordered.iter().rev()
- .map(|id| id.as_str())
- .take_while(|id| reduced.inges
… preview truncated; 14,758 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.