{"messages":[{"content":"You are a constitutional council ranking individual git commits for ownership allocation.\n\nCompare these two commits. Decide which contributed more lasting value to the project.\n\nJudge substance, not spectacle:\n- Prefer correct, lasting design and real bugfixes over churn, formatting, renames, or generated noise.\n- Prefer clarity and necessity over sheer line count. A small precise change can beat a large diffuse one.\n- Do not favor a side merely because its patch is longer or noisier.\n- Weight what the change does for the project, not the contributor's name.\n\nReturn ONLY a JSON object: {\"winner\": \"A\" or \"B\", \"ratio\": \"N:M\", \"explanation\": \"...\"}\nThe explanation must cite concrete differences in the patches (1-3 sentences).\n\nSide A — contributor: tommy-mor\nSide A — commit message:\n[9e20d06c] Add sorterc dev tool for offline DSL compile and JSONL lint.\n\nIntroduce a workspace-only binary that validates .sorter files into ranking JSON and scans events.jsonl for corrupt or unreplayable ingests.\n\nCo-authored-by: Cursor <cursoragent@cursor.com>\n\nSide A — unified diff (full patch):\ndiff --git a/Cargo.lock b/Cargo.lock\nindex bf8153d9c723af97122c9ffdd4a7cfe82e853bb6..a07734f089b466440c3ae6fc1087ce85fc24ce62 100644\n--- a/Cargo.lock\n+++ b/Cargo.lock\n@@ -1826,6 +1826,17 @@ dependencies = [\n  \"windows-sys 0.60.2\",\n ]\n \n+[[package]]\n+name = \"sorterc\"\n+version = \"0.0.1\"\n+dependencies = [\n+ \"anyhow\",\n+ \"clap\",\n+ \"serde\",\n+ \"serde_json\",\n+ \"slugsocial-server\",\n+]\n+\n [[package]]\n name = \"spin\"\n version = \"0.9.8\"\ndiff --git a/Cargo.toml b/Cargo.toml\nindex 149cbf07901eab57c593184ff8719a75d530f1da..25337acdd61e44b20f354c78fed4a88caf896280 100644\n--- a/Cargo.toml\n+++ b/Cargo.toml\n@@ -1,5 +1,5 @@\n [workspace]\n-members = [\"server\", \"cli\"]\n+members = [\"server\", \"cli\", \"sorterc\"]\n resolver = \"2\"\n \n \ndiff --git a/agents.md b/agents.md\nindex d8b801e454fdf37e7ac6038b91a69f83b0746d59..ce646ed3cd7123be4732dccec4a6800467e651e7 100644\n--- a/agents.md\n+++ b/agents.md\n@@ -93,6 +93,17 @@ SLUG_GOOGLE_CLIENT_SECRET=mock\n \n After OAuth completes, the pending-session poll returns a `slug_…` bearer token for API calls.\n \n+### Dev-only offline tooling\n+\n+**`sorterc`** — workspace binary, not published via npm. Compiles `.sorter` files and lints `events.jsonl` without a server:\n+\n+```\n+cargo run -p sorterc -- compile path/to/doc.sorter [--base events.jsonl] [--room public] [--pretty]\n+cargo run -p sorterc -- scan path/to/events.jsonl [--pretty]\n+```\n+\n+`compile` validates DSL, simulates ingest against empty (or `--base`) reducer state, and prints JSON rankings. `scan` reports corrupt JSONL lines and ingests that fail DSL replay.\n+\n ### Testing\n \n - **Rust tests:** `cargo nextest run --workspace` (163 tests; requires `cargo-nextest`)\ndiff --git a/server/src/lib.rs b/server/src/lib.rs\nindex c1d477d21aea03aff00e6f0689b0b4379d0d68d2..ad8e31099c807fb5844acb16cd5086a2f19327a7 100644\n--- a/server/src/lib.rs\n+++ b/server/src/lib.rs\n@@ -10,6 +10,7 @@ pub mod form_template;\n pub mod html;\n pub mod identity;\n pub mod middleware;\n+pub mod offline;\n pub mod path_types;\n pub mod ranking;\n pub mod reducer;\ndiff --git a/server/src/offline.rs b/server/src/offline.rs\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..54ad0ded096a305ef8454ab2cdd1c3af71b14f5d\n--- /dev/null\n+++ b/server/src/offline.rs\n@@ -0,0 +1,333 @@\n+//! Offline `.sorter` compilation and JSONL diagnostics (no network, no auth).\n+\n+use std::collections::HashSet;\n+use std::path::Path;\n+\n+use serde::Serialize;\n+use slug_types::{CheckScopeRanking, RankComponent, RankRow, paths::GardenItemUrl};\n+\n+use crate::{\n+    api::{resolve_item, validate_ingest_document},\n+    dsl,\n+    events::{Event, Ingest},\n+    path_types::ItemId,\n+    reducer::{ReducerState, ScopeId, scope_from_room_wire},\n+    scope_rank::build_children_rankings,\n+};\n+\n+#[derive(Debug, Clone, Serialize)]\n+pub struct CompileStats {\n+    pub items: usize,\n+    pub votes: usize,\n+    pub prose_blocks: usize,\n+}\n+\n+#[derive(Debug, Serialize)]\n+pub struct CompileResult {\n+    pub ok: bool,\n+    pub threads: Vec<String>,\n+    pub rankings: Vec<CheckScopeRanking>,\n+    pub stats: CompileStats,\n+}\n+\n+#[derive(Debug, Clone, Serialize)]\n+pub struct CompileError {\n+    pub ok: bool,\n+    pub error: String,\n+    #[serde(skip_serializing_if = \"Option::is_none\")]\n+    pub hint: Option<String>,\n+}\n+\n+#[derive(Debug, Clone, Serialize)]\n+pub struct BadJsonLine {\n+    pub line: usize,\n+    pub message: String,\n+}\n+\n+#[derive(Debug, Clone, Serialize)]\n+pub struct MalformedIngest {\n+    pub line: usize,\n+    pub id: String,\n+    pub room_id: String,\n+    pub thread_tag: String,\n+    pub reason: String,\n+}\n+\n+#[derive(Debug, Clone, Serialize)]\n+pub struct ScanResult {\n+    pub ok: bool,\n+    pub path: String,\n+    pub total_lines: usize,\n+    pub parsed_events: usize,\n+    pub bad_json_lines: Vec<BadJsonLine>,\n+    pub malformed_ingests: Vec<MalformedIngest>,\n+    pub skipped_ingests: usize,\n+}\n+\n+fn document_stats(doc: &dsl::Document) -> CompileStats {\n+    let mut items = 0usize;\n+    let mut votes = 0usize;\n+    let mut prose_blocks = 0usize;\n+    for stmt in &doc.statements {\n+        match stmt {\n+            dsl::Stmt::Item { .. } => items += 1,\n+            dsl::Stmt::Vote { .. } => votes += 1,\n+            dsl::Stmt::Prose { .. } => prose_blocks += 1,\n+        }\n+    }\n+    CompileStats {\n+        items,\n+        votes,\n+        prose_blocks,\n+    }\n+}\n+\n+fn threads_in_document(text: &str) -> Vec<String> {\n+    let mut out = HashSet::new();\n+    for line in text.lines() {\n+        let trimmed = line.trim();\n+        if !trimmed.starts_with('#') {\n+            continue;\n+        }\n+        let rest = trimmed.trim_start_matches('#').trim();\n+        if rest.is_empty() {\n+            continue;\n+        }\n+        let tag = rest.split_whitespace().next().unwrap_or(rest);\n+        let tag = tag.split(':').next().unwrap_or(tag).trim();\n+        if tag.is_empty() {\n+            continue;\n+        }\n+        out.insert(format!(\"#{}\", crate::canonical_path::canonicalize_tag(tag)));\n+    }\n+    let mut tags: Vec<String> = out.into_iter().collect();\n+    tags.sort();\n+    tags\n+}\n+\n+fn voted_parent_scopes(doc: &dsl::Document) -> Vec<ItemId> {\n+    let mut parents = HashSet::new();\n+    for stmt in &doc.statements {\n+        if let dsl::Stmt::Vote { item1, item2, .. } = stmt {\n+            if let (Ok(a), Ok(b)) = (resolve_item(item1), resolve_item(item2)) {\n+                if let Some(p) = a.parent() {\n+                    parents.insert(p);\n+                }\n+                if let Some(p) = b.parent() {\n+                    parents.insert(p);\n+                }\n+            }\n+        }\n+    }\n+    let mut out: Vec<ItemId> = parents.into_iter().collect();\n+    out.sort();\n+    out\n+}\n+\n+fn rankings_for_simulated(\n+    simulated: &ReducerState,\n+    scope: &ScopeId,\n+    room_wire: &str,\n+    doc: &dsl::Document,\n+) -> Vec<CheckScopeRanking> {\n+    voted_parent_scopes(doc)\n+        .iter()\n+        .map(|parent| {\n+            let scoped_content = simulated\n+                .content_for_scope(&scope)\n+                .unwrap_or_else(|| simulated.public());\n+            let scoped = build_children_rankings(scoped_content, parent);\n+            let components: Vec<RankComponent> = scoped\n+                .component_rankings\n+                .into_iter()\n+                .map(|comp| RankComponent {\n+                    pairs: comp.pairs,\n+                    ranking: comp\n+                        .ranked\n+                        .into_iter()\n+                        .map(|r| RankRow {\n+                            item: GardenItemUrl::from_stored(&r.item, room_wire),\n+                            score: r.score,\n+                            percent: None,\n+                        })\n+                        .collect(),\n+                })\n+                .collect();\n+            CheckScopeRanking {\n+                parent: GardenItemUrl::from_stored(parent, room_wire).into_inner(),\n+                components,\n+                unranked_items: scoped\n+                    .unranked_items\n+                    .into_iter()\n+                    .map(|it| GardenItemUrl::from_stored(&it, room_wire))\n+                    .collect(),\n+            }\n+        })\n+        .collect()\n+}\n+\n+/// Validate and simulate one `.sorter` document against optional base reducer state.\n+pub fn compile_document(\n+    base: &ReducerState,\n+    room: &str,\n+    text: &str,\n+) -> Result<CompileResult, CompileError> {\n+    let room_key = room.trim();\n+    let scope = scope_from_room_wire(room_key);\n+    let validated = validate_ingest_document(base, text, &scope).map_err(|(_, message, hint)| {\n+        CompileError {\n+            ok: false,\n+            error: message,\n+            hint,\n+        }\n+    })?;\n+\n+    let event = Event::Ingest(Ingest {\n+        ts: validated.ts,\n+        id: uuid::Uuid::new_v4().to_string(),\n+        raw: validated.raw_text.clone(),\n+        principal: \"offline\".to_string(),\n+        delegate: None,\n+        room_id: room_key.to_string(),\n+        thread_tag: \"offline\".to_string(),\n+    });\n+\n+    let mut simulated = base.clone();\n+    simulated.apply_event(event);\n+\n+    Ok(CompileResult {\n+        ok: true,\n+        threads: threads_in_document(text),\n+        rankings: rankings_for_simulated(&simulated, &scope, room_key, &validated.doc),\n+        stats: document_stats(&validated.doc),\n+    })\n+}\n+\n+fn ingest_parse_error(raw: &str) -> Option<String> {\n+    dsl::parse_full(raw).err().map(|e| e.to_string())\n+}\n+\n+fn load_events_from_jsonl(path: &Path) -> Result<(Vec<(usize, Event)>, Vec<BadJsonLine>), std::io::Error> {\n+    let text = std::fs::read_to_string(path)?;\n+    let mut events = Vec::new();\n+    let mut bad_json_lines = Vec::new();\n+    for (idx, line) in text.lines().enumerate() {\n+        let line_no = idx + 1;\n+        let trimmed = line.trim();\n+        if trimmed.is_empty() {\n+            continue;\n+        }\n+        match serde_json::from_str::<Event>(trimmed) {\n+            Ok(ev) => events.push((line_no, ev)),\n+            Err(e) => bad_json_lines.push(BadJsonLine {\n+                line: line_no,\n+                message: e.to_string(),\n+            }),\n+        }\n+    }\n+    Ok((events, bad_json_lines))\n+}\n+\n+/// Replay a JSONL event log into reducer state (same rules as server boot).\n+pub fn load_reducer_from_jsonl(path: &Path) -> Result<(ReducerState, Vec<BadJsonLine>), std::io::Error> {\n+    let (events, bad_json_lines) = load_events_from_jsonl(path)?;\n+    let mut state = ReducerState::default();\n+    for (_line_no, ev) in events {\n+        state.apply_event(ev);\n+    }\n+    Ok((state, bad_json_lines))\n+}\n+\n+/// Scan an events.jsonl for corrupt JSON lines and ingests that fail DSL replay.\n+pub fn scan_jsonl(path: &Path) -> Result<ScanResult, std::io::Error> {\n+    let text = std::fs::read_to_string(path)?;\n+    let total_lines = text.lines().count();\n+    let (events, bad_json_lines) = load_events_from_jsonl(path)?;\n+\n+    let mut malformed_ingests = Vec::new();\n+    let mut skipped_ingests = 0usize;\n+    let mut state = ReducerState::default();\n+    let parsed_events = events.len();\n+\n+    for (line_no, ev) in events {\n+        if let Event::Ingest(ref ing) = ev {\n+            if let Some(reason) = ingest_parse_error(&ing.raw) {\n+                malformed_ingests.push(MalformedIngest {\n+                    line: line_no,\n+                    id: ing.id.clone(),\n+                    room_id: ing.room_id.clone(),\n+                    thread_tag: ing.thread_tag.clone(),\n+                    reason,\n+                });\n+            }\n+            let before = state.ingests_by_id.len();\n+            state.apply_event(ev);\n+            if state.ingests_by_id.len() == before {\n+                skipped_ingests += 1;\n+            }\n+        } else {\n+            state.apply_event(ev);\n+        }\n+    }\n+\n+    let ok = bad_json_lines.is_empty() && malformed_ingests.is_empty() && skipped_ingests == 0;\n+\n+    Ok(ScanResult {\n+        ok,\n+        path: path.display().to_string(),\n+        total_lines,\n+        parsed_events,\n+        bad_json_lines,\n+        malformed_ingests,\n+        skipped_ingests,\n+    })\n+}\n+\n+#[cfg(test)]\n+mod tests {\n+    use super::*;\n+\n+    const TUTORIAL: &str = include_str!(\"../tests/fixtures/tutorial.sorter\");\n+\n+    #[test]\n+    fn compile_tutorial_fixture_emits_rankings() {\n+        let result = compile_document(&ReducerState::default(), \"public\", TUTORIAL).unwrap();\n+        assert!(result.ok);\n+        assert!(!result.threads.is_empty());\n+        assert!(result.stats.items >= 6);\n+        assert!(result.stats.votes >= 6);\n+        assert!(!result.rankings.is_empty());\n+    }\n+\n+    #[test]\n+    fn compile_rejects_vote_on_missing_item() {\n+        let err = compile_document(\n+            &ReducerState::default(),\n+            \"public\",\n+            \"{ reason }\\n~/missing/a 2:1 ~/missing/b\",\n+        )\n+        .unwrap_err();\n+        assert!(!err.ok);\n+        assert!(err.error.contains(\"undefined\"));\n+    }\n+\n+    #[test]\n+    fn scan_empty_jsonl_is_ok() {\n+        let dir = tempfile::tempdir().unwrap();\n+        let path = dir.path().join(\"events.jsonl\");\n+        std::fs::write(&path, \"\").unwrap();\n+        let report = scan_jsonl(&path).unwrap();\n+        assert!(report.ok);\n+        assert!(report.bad_json_lines.is_empty());\n+    }\n+\n+    #[test]\n+    fn scan_reports_bad_json_line() {\n+        let dir = tempfile::tempdir().unwrap();\n+        let path = dir.path().join(\"events.jsonl\");\n+        std::fs::write(&path, \"{not json}\\n\").unwrap();\n+        let report = scan_jsonl(&path).unwrap();\n+        assert!(!report.ok);\n+        assert_eq!(report.bad_json_lines.len(), 1);\n+    }\n+}\ndiff --git a/sorterc/Cargo.toml b/sorterc/Cargo.toml\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..92d477aff9b53291fb1a266db83065c1c800791d\n--- /dev/null\n+++ b/sorterc/Cargo.toml\n@@ -0,0 +1,18 @@\n+[package]\n+name = \"sorterc\"\n+version = \"0.0.1\"\n+edition = \"2021\"\n+license = \"MIT\"\n+publish = false\n+description = \"Offline .sorter compiler and events.jsonl linter (dev only)\"\n+\n+[[bin]]\n+name = \"sorterc\"\n+path = \"src/main.rs\"\n+\n+[dependencies]\n+anyhow = \"1\"\n+clap = { version = \"4\", features = [\"derive\"] }\n+serde = { version = \"1\", features = [\"derive\"] }\n+serde_json = \"1\"\n+slugsocial-server = { path = \"../server\" }\ndiff --git a/sorterc/readme.md b/sorterc/readme.md\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..1ebcc3fc935541ea9e47e0458ec67a750fe19fff\n--- /dev/null\n+++ b/sorterc/readme.md\n@@ -0,0 +1,92 @@\n+# sorterc\n+\n+Dev-only offline tooling for the slug `.sorter` DSL and `events.jsonl` event log.\n+\n+`sorterc` is **not** published via npm and does not talk to slug.social. It reuses the same parser, validator, and ranking code as the server, but runs entirely on local files.\n+\n+## Build\n+\n+From the repo root:\n+\n+```bash\n+cargo build -p sorterc\n+cargo run -p sorterc -- --help\n+```\n+\n+## Commands\n+\n+### `compile` — evaluate a `.sorter` document\n+\n+Reads a `.sorter` file (or `-` for stdin), validates the DSL, simulates one ingest against reducer state, and prints JSON rankings to stdout.\n+\n+```bash\n+cargo run -p sorterc -- compile path/to/doc.sorter\n+cargo run -p sorterc -- compile path/to/doc.sorter --pretty\n+cargo run -p sorterc -- compile - --pretty                    # stdin\n+cargo run -p sorterc -- compile doc.sorter --base events.jsonl  # seed garden from log\n+cargo run -p sorterc -- compile doc.sorter --room public         # default room\n+```\n+\n+**Flags**\n+\n+| Flag | Description |\n+|------|-------------|\n+| `--base PATH` | Replay an `events.jsonl` first, then compile against that garden state |\n+| `--room ID` | Room wire id (`public` or private room id). Default: `public` |\n+| `--pretty` | Pretty-print JSON |\n+\n+**Success output** (shape):\n+\n+```json\n+{\n+  \"ok\": true,\n+  \"threads\": [\"#my-thread\"],\n+  \"rankings\": [ … ],\n+  \"stats\": { \"items\": 3, \"votes\": 2, \"prose_blocks\": 5 }\n+}\n+```\n+\n+Rankings use the same structure as the server's dry-run check: parent scope, connected components, scores, unranked items.\n+\n+**Error output** exits with code 1:\n+\n+```json\n+{\n+  \"ok\": false,\n+  \"error\": \"parse error\",\n+  \"hint\": \"…\"\n+}\n+```\n+\n+### `scan` — lint an `events.jsonl`\n+\n+Reads a JSONL event log and reports problems without starting a server.\n+\n+```bash\n+cargo run -p sorterc -- scan events.jsonl\n+cargo run -p sorterc -- scan events.jsonl --pretty\n+```\n+\n+Reports:\n+\n+- **bad JSON lines** — lines that are not valid JSON\n+- **malformed ingests** — ingest events whose `raw` DSL fails to parse\n+- **skipped ingests** — ingests dropped during replay (same behavior as server boot)\n+\n+Exits 0 when clean, 1 when any issue is found.\n+\n+## Typical uses\n+\n+- Iterate on `.sorter` files in an editor and pipe through `compile` to see rankings instantly\n+- Verify a downloaded or edited `events.jsonl` before uploading to Fly\n+- Debug \"malformed ingest\" warnings from production boot logs\n+- CI or pre-commit checks on fixture docs (no OAuth, no network)\n+\n+## What it does not do\n+\n+- Post to slug.social or append to a live log\n+- Authenticate users or bind agents\n+- Run browser/UI tests\n+- Replace `slugsocial public check` for operators who want the full RPC path against a running server\n+\n+For live server dry-run against current garden state, use `npx slugsocial public check` or `POST /try/check` in the browser.\ndiff --git a/sorterc/src/main.rs b/sorterc/src/main.rs\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..71382c180085cb0ad71043c852f8db5d3a48a284\n--- /dev/null\n+++ b/sorterc/src/main.rs\n@@ -0,0 +1,117 @@\n+use std::path::{Path, PathBuf};\n+\n+use anyhow::{bail, Context, Result};\n+use clap::{Parser, Subcommand};\n+use slugsocial_server::{\n+    offline::{self, CompileError, CompileResult, ScanResult},\n+    reducer::ReducerState,\n+};\n+\n+#[derive(Parser)]\n+#[command(\n+    name = \"sorterc\",\n+    about = \"Offline .sorter compiler and events.jsonl linter (dev only)\",\n+    version\n+)]\n+struct Cli {\n+    #[command(subcommand)]\n+    cmd: Command,\n+}\n+\n+#[derive(Subcommand)]\n+enum Command {\n+    /// Parse and simulate a .sorter document; emit ranking JSON to stdout.\n+    Compile {\n+        /// `.sorter` file, or `-` for stdin.\n+        file: PathBuf,\n+        /// Room wire id (`public` or private room id).\n+        #[arg(long, default_value = \"public\")]\n+        room: String,\n+        /// Optional events.jsonl to replay before compiling (seed garden state).\n+        #[arg(long)]\n+        base: Option<PathBuf>,\n+        /// Pretty-print JSON.\n+        #[arg(long)]\n+        pretty: bool,\n+    },\n+    /// Scan an events.jsonl for corrupt JSON lines and malformed ingests.\n+    Scan {\n+        file: PathBuf,\n+        #[arg(long)]\n+        pretty: bool,\n+    },\n+}\n+\n+fn read_input(path: &Path) -> Result<String> {\n+    if path.as_os_str() == \"-\" {\n+        use std::io::Read;\n+        let mut buf = String::new();\n+        std::io::stdin().read_to_string(&mut buf)?;\n+        Ok(buf)\n+    } else {\n+        std::fs::read_to_string(path)\n+            .with_context(|| format!(\"read {}\", path.display()))\n+    }\n+}\n+\n+fn load_base_state(base: Option<&Path>) -> Result<ReducerState> {\n+    let Some(path) = base else {\n+        return Ok(ReducerState::default());\n+    };\n+    let (state, bad_lines) = offline::load_reducer_from_jsonl(path)\n+        .with_context(|| format!(\"load base jsonl {}\", path.display()))?;\n+    if !bad_lines.is_empty() {\n+        bail!(\n+            \"base jsonl has {} corrupt line(s); fix or omit --base\",\n+            bad_lines.len()\n+        );\n+    }\n+    Ok(state)\n+}\n+\n+fn print_json<T: serde::Serialize>(value: &T, pretty: bool) -> Result<()> {\n+    if pretty {\n+        println!(\"{}\", serde_json::to_string_pretty(value)?);\n+    } else {\n+        println!(\"{}\", serde_json::to_string(value)?);\n+    }\n+    Ok(())\n+}\n+\n+fn run_compile(file: PathBuf, room: String, base: Option<PathBuf>, pretty: bool) -> Result<()> {\n+    let text = read_input(&file)?;\n+    let base_state = load_base_state(base.as_deref())?;\n+    match offline::compile_document(&base_state, &room, &text) {\n+        Ok(result) => {\n+            print_json::<CompileResult>(&result, pretty)?;\n+            Ok(())\n+        }\n+        Err(err) => {\n+            print_json::<CompileError>(&err, pretty)?;\n+            std::process::exit(1);\n+        }\n+    }\n+}\n+\n+fn run_scan(file: PathBuf, pretty: bool) -> Result<()> {\n+    let report = offline::scan_jsonl(&file)\n+        .with_context(|| format!(\"scan {}\", file.display()))?;\n+    print_json::<ScanResult>(&report, pretty)?;\n+    if !report.ok {\n+        std::process::exit(1);\n+    }\n+    Ok(())\n+}\n+\n+fn main() -> Result<()> {\n+    let cli = Cli::parse();\n+    match cli.cmd {\n+        Command::Compile {\n+            file,\n+            room,\n+            base,\n+            pretty,\n+        } => run_compile(file, room, base, pretty),\n+        Command::Scan { file, pretty } => run_scan(file, pretty),\n+    }\n+}\n\n\nSide B — contributor: tommy-mor\nSide B — commit message:\n[8d8230d1] reddit\n\nSide B — unified diff (full patch):\ndiff --git a/.gitignore b/.gitignore\nindex 4c7073f9fac0c30fd2050d79a60ef447af58ebeb..ada462e900d24a3a6d08165d158c80f79c35a5a5 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -7,3 +7,4 @@\n data/\n repomix-output.xml\n dev-data/\n+.env\ndiff --git a/server/Cargo.toml b/server/Cargo.toml\nindex 7906a8547d56b8e6a48ef59c37aa82a8510fdee9..4677fedcb45292eebebe7e9cf6ce2f5738f18ddf 100644\n--- a/server/Cargo.toml\n+++ b/server/Cargo.toml\n@@ -16,6 +16,7 @@ tower = \"0.5\"\n tower-http = { version = \"0.5\", features = [\"trace\"] }\n tracing = \"0.1\"\n tracing-subscriber = { version = \"0.3\", features = [\"env-filter\"] }\n+reqwest = { version = \"0.12\", features = [\"json\"] }\n \n [dev-dependencies]\n reqwest = { version = \"0.12\", features = [\"json\"] }\ndiff --git a/server/src/html/mod.rs b/server/src/html/mod.rs\nindex 2864407ed6e8ec284a1dc663acf1805534566b62..df6505021d9f446c2b453e20e3eb3cf696a111f9 100644\n--- a/server/src/html/mod.rs\n+++ b/server/src/html/mod.rs\n@@ -272,5 +272,16 @@ pub async fn home(State(state): State<AppState>, uri: Uri) -> impl IntoResponse\n \n pub async fn browse(State(state): State<AppState>, uri: Uri) -> impl IntoResponse {\n     let item = ItemId::from_browse_uri(uri.path()).unwrap_or(ItemId::root());\n+    if item.as_str().starts_with(\"reddit.com\") {\n+        let needs_fetch = {\n+            let tree = state.tree.read().await;\n+            tree.get(&item)\n+                .map(|n| n.data.is_none())\n+                .unwrap_or(true)\n+        };\n+        if needs_fetch {\n+            state.reddit.request_fetch(item.clone());\n+        }\n+    }\n     item_page(state, uri, item).await\n }\ndiff --git a/server/src/reddit.rs b/server/src/reddit.rs\nindex d203dca09245daf869b3aa942898447700ae69fb..90053ad03b1d7c8e94f325dd4ee64c2b4f7da900 100644\n--- a/server/src/reddit.rs\n+++ b/server/src/reddit.rs\n@@ -1,4 +1,12 @@\n-//! Reddit API import (async, decoupled from UI request path).\n+//! Reddit API import via a single background worker (rate limits, dedup, backoff).\n+\n+use std::collections::{HashMap, HashSet};\n+use std::sync::Arc;\n+use std::time::{Duration, Instant};\n+\n+use reqwest::{header, Client, StatusCode};\n+use serde::Deserialize;\n+use tokio::sync::{mpsc, RwLock};\n \n use crate::{\n     path_types::ItemId,\n@@ -10,12 +18,401 @@ pub fn ensure_partial_tree(tree: &mut GlobalTree, id: &ItemId) {\n     tree.ensure_path(id);\n }\n \n-/// Placeholder for Reddit JSON import. Returns entity data when implemented.\n-pub async fn fetch_reddit_entity(_id: &ItemId) -> Option<EntityData> {\n-    None\n+pub struct RedditCommand {\n+    pub id: ItemId,\n+}\n+\n+#[derive(Clone)]\n+pub struct RedditBroker {\n+    tx: mpsc::Sender<RedditCommand>,\n+}\n+\n+#[derive(Clone)]\n+struct RedditCredentials {\n+    client_id: String,\n+    client_secret: String,\n+}\n+\n+struct OAuthToken {\n+    access_token: String,\n+    expires_at: Instant,\n+}\n+\n+impl RedditBroker {\n+    pub fn spawn(tree: Arc<RwLock<GlobalTree>>, user_agent: &str) -> Self {\n+        let (tx, rx) = mpsc::channel(100);\n+\n+        let mut headers = header::HeaderMap::new();\n+        headers.insert(\n+            header::USER_AGENT,\n+            header::HeaderValue::from_str(user_agent).expect(\"valid user agent\"),\n+        );\n+\n+        let client = Client::builder()\n+            .default_headers(headers)\n+            .timeout(Duration::from_secs(15))\n+            .build()\n+            .expect(\"reqwest client\");\n+\n+        let creds = RedditCredentials::from_env();\n+        tokio::spawn(reddit_worker(rx, tree, client, creds));\n+\n+        Self { tx }\n+    }\n+\n+    /// Fire-and-forget: queue a fetch; worker updates the tree when done.\n+    pub fn request_fetch(&self, id: ItemId) {\n+        let _ = self.tx.try_send(RedditCommand { id });\n+    }\n+}\n+\n+impl RedditCredentials {\n+    fn from_env() -> Option<Self> {\n+        let client_id = std::env::var(\"REDDIT_CLIENT_ID\").ok()?;\n+        let client_secret = std::env::var(\"REDDIT_CLIENT_SECRET\").ok()?;\n+        if client_id.is_empty() || client_secret.is_empty() {\n+            return None;\n+        }\n+        Some(Self {\n+            client_id,\n+            client_secret,\n+        })\n+    }\n+}\n+\n+pub fn default_user_agent() -> String {\n+    std::env::var(\"REDDIT_USER_AGENT\").unwrap_or_else(|_| {\n+        \"web:sorter2.social:v0.0.1 (by /u/sorter2)\".to_string()\n+    })\n }\n \n-/// Apply fetched entity data to a node (called from async worker).\n-pub fn apply_entity(tree: &mut GlobalTree, id: &ItemId, data: EntityData) {\n-    tree.set_entity_data(id, data);\n+async fn reddit_worker(\n+    mut rx: mpsc::Receiver<RedditCommand>,\n+    tree: Arc<RwLock<GlobalTree>>,\n+    client: Client,\n+    creds: Option<RedditCredentials>,\n+) {\n+    let mut in_flight = HashSet::new();\n+    let mut recently_fetched: HashMap<ItemId, Instant> = HashMap::new();\n+    let mut current_delay = Duration::from_secs(1);\n+    let mut oauth: Option<OAuthToken> = None;\n+    let cache_ttl = Duration::from_secs(300);\n+\n+    while let Some(cmd) = rx.recv().await {\n+        let now = Instant::now();\n+        recently_fetched.retain(|_, t| now.duration_since(*t) < cache_ttl);\n+\n+        if in_flight.contains(&cmd.id) || recently_fetched.contains_key(&cmd.id) {\n+            continue;\n+        }\n+\n+        in_flight.insert(cmd.id.clone());\n+        let fetch_id = cmd.id.clone();\n+\n+        tokio::time::sleep(current_delay).await;\n+\n+        if let Some(c) = &creds {\n+            oauth = ensure_oauth_token(&client, c, oauth.take()).await;\n+        }\n+\n+        let token = oauth.as_ref().map(|t| t.access_token.as_str());\n+        let use_oauth = token.is_some();\n+\n+        match do_fetch(&client, &fetch_id, use_oauth, token).await {\n+            Ok(FetchOutcome::Entity(data)) => {\n+                let mut w = tree.write().await;\n+                w.set_entity_data(&fetch_id, data);\n+                recently_fetched.insert(fetch_id.clone(), Instant::now());\n+                current_delay = Duration::from_millis(600);\n+            }\n+            Ok(FetchOutcome::NotFound) => {\n+                recently_fetched.insert(fetch_id.clone(), Instant::now());\n+            }\n+            Ok(FetchOutcome::RateLimited { reset_secs }) => {\n+                let wait = Duration::from_secs(reset_secs.max(1));\n+                tracing::warn!(\n+                    \"Reddit rate limit for {}; sleeping {}s\",\n+                    fetch_id,\n+                    wait.as_secs()\n+                );\n+                tokio::time::sleep(wait).await;\n+                current_delay = (current_delay * 2).min(Duration::from_secs(60));\n+            }\n+            Err(e) => {\n+                tracing::warn!(\"Reddit fetch failed for {}: {}\", fetch_id, e);\n+                current_delay = (current_delay * 2).min(Duration::from_secs(60));\n+            }\n+        }\n+\n+        in_flight.remove(&fetch_id);\n+    }\n+}\n+\n+enum FetchOutcome {\n+    Entity(EntityData),\n+    NotFound,\n+    RateLimited { reset_secs: u64 },\n+}\n+\n+async fn ensure_oauth_token(\n+    client: &Client,\n+    creds: &RedditCredentials,\n+    existing: Option<OAuthToken>,\n+) -> Option<OAuthToken> {\n+    if let Some(t) = existing {\n+        if Instant::now() < t.expires_at - Duration::from_secs(60) {\n+            return Some(t);\n+        }\n+    }\n+\n+    let resp = client\n+        .post(\"https://www.reddit.com/api/v1/access_token\")\n+        .basic_auth(&creds.client_id, Some(&creds.client_secret))\n+        .form(&[(\"grant_type\", \"client_credentials\")])\n+        .send()\n+        .await;\n+\n+    let resp = match resp {\n+        Ok(r) => r,\n+        Err(e) => {\n+            tracing::warn!(\"Reddit OAuth token request failed: {e}\");\n+            return None;\n+        }\n+    };\n+\n+    if !resp.status().is_success() {\n+        tracing::warn!(\"Reddit OAuth token HTTP {}\", resp.status());\n+        return None;\n+    }\n+\n+    #[derive(Deserialize)]\n+    struct TokenResponse {\n+        access_token: String,\n+        expires_in: u64,\n+    }\n+\n+    let body: TokenResponse = match resp.json().await {\n+        Ok(b) => b,\n+        Err(e) => {\n+            tracing::warn!(\"Reddit OAuth token parse failed: {e}\");\n+            return None;\n+        }\n+    };\n+\n+    Some(OAuthToken {\n+        access_token: body.access_token,\n+        expires_at: Instant::now() + Duration::from_secs(body.expires_in),\n+    })\n+}\n+\n+async fn do_fetch(\n+    client: &Client,\n+    id: &ItemId,\n+    use_oauth: bool,\n+    bearer: Option<&str>,\n+) -> Result<FetchOutcome, String> {\n+    let url = map_item_to_reddit_api(id, use_oauth);\n+    if url.is_empty() {\n+        return Ok(FetchOutcome::NotFound);\n+    }\n+\n+    let mut req = client.get(&url);\n+    if let Some(token) = bearer {\n+        req = req.bearer_auth(token);\n+    }\n+\n+    let resp = req.send().await.map_err(|e| e.to_string())?;\n+\n+    if resp.status() == StatusCode::TOO_MANY_REQUESTS {\n+        let reset = rate_limit_reset_secs(&resp);\n+        return Ok(FetchOutcome::RateLimited { reset_secs: reset });\n+    }\n+\n+    if resp.status() == StatusCode::SERVICE_UNAVAILABLE {\n+        return Err(\"Reddit unavailable (503)\".to_string());\n+    }\n+\n+    if !resp.status().is_success() {\n+        return Ok(FetchOutcome::NotFound);\n+    }\n+\n+    if rate_limit_remaining(&resp) == Some(0) {\n+        let reset = rate_limit_reset_secs(&resp);\n+        return Ok(FetchOutcome::RateLimited { reset_secs: reset });\n+    }\n+\n+    let bytes = resp.bytes().await.map_err(|e| e.to_string())?;\n+    Ok(parse_reddit_json(id, &bytes)\n+        .map(FetchOutcome::Entity)\n+        .unwrap_or(FetchOutcome::NotFound))\n+}\n+\n+fn rate_limit_remaining(resp: &reqwest::Response) -> Option<u64> {\n+    resp.headers()\n+        .get(\"x-ratelimit-remaining\")\n+        .and_then(|v| v.to_str().ok())\n+        .and_then(|s| s.parse::<f64>().ok())\n+        .map(|f| f.floor() as u64)\n+}\n+\n+fn rate_limit_reset_secs(resp: &reqwest::Response) -> u64 {\n+    resp.headers()\n+        .get(\"x-ratelimit-reset\")\n+        .and_then(|v| v.to_str().ok())\n+        .and_then(|s| s.parse::<f64>().ok())\n+        .map(|f| f.ceil() as u64)\n+        .unwrap_or(5)\n+}\n+\n+/// Map canonical item id to Reddit JSON API URL.\n+pub fn map_item_to_reddit_api(id: &ItemId, oauth: bool) -> String {\n+    let path = id.as_str();\n+    if !path.starts_with(\"reddit.com/\") && path != \"reddit.com\" {\n+        return String::new();\n+    }\n+\n+    let base = if oauth {\n+        \"https://oauth.reddit.com\"\n+    } else {\n+        \"https://www.reddit.com\"\n+    };\n+\n+    let segments: Vec<&str> = path.split('/').collect();\n+\n+    if let Some(i) = segments.iter().position(|&p| p == \"comments\") {\n+        if segments.len() > i + 1 {\n+            let api_path = segments[1..=i + 1].join(\"/\");\n+            return format!(\"{base}/{api_path}.json?raw_json=1\");\n+        }\n+    }\n+\n+    if segments.len() == 3 && segments[1] == \"r\" {\n+        return format!(\"{base}/r/{}/about.json?raw_json=1\", segments[2]);\n+    }\n+\n+    String::new()\n+}\n+\n+fn parse_reddit_json(id: &ItemId, bytes: &[u8]) -> Option<EntityData> {\n+    let v: serde_json::Value = serde_json::from_slice(bytes).ok()?;\n+    let segments: Vec<&str> = id.as_str().split('/').collect();\n+\n+    if segments.iter().any(|&p| p == \"comments\") {\n+        parse_post_listing(&v)\n+    } else {\n+        parse_subreddit_about(&v)\n+    }\n+}\n+\n+fn parse_subreddit_about(v: &serde_json::Value) -> Option<EntityData> {\n+    let data = v.get(\"data\")?;\n+    let title = data\n+        .get(\"title\")\n+        .or_else(|| data.get(\"display_name\"))\n+        .and_then(|t| t.as_str())?\n+        .to_string();\n+    let body_html = data\n+        .get(\"public_description_html\")\n+        .or_else(|| data.get(\"public_description\"))\n+        .and_then(|t| t.as_str())\n+        .map(|s| s.to_string());\n+    let thumb_url = data\n+        .get(\"icon_img\")\n+        .or_else(|| data.get(\"community_icon\"))\n+        .and_then(|t| t.as_str())\n+        .filter(|s| !s.is_empty())\n+        .map(|s| s.to_string());\n+\n+    Some(EntityData {\n+        title,\n+        author: None,\n+        body_html,\n+        thumb_url,\n+    })\n+}\n+\n+fn parse_post_listing(v: &serde_json::Value) -> Option<EntityData> {\n+    let listing = v.as_array()?.first()?;\n+    let child = listing\n+        .pointer(\"/data/children/0/data\")?;\n+    let title = child.get(\"title\")?.as_str()?.to_string();\n+    let author = child\n+        .get(\"author\")\n+        .and_then(|a| a.as_str())\n+        .filter(|a| *a != \"[deleted]\")\n+        .map(|s| s.to_string());\n+    let body_html = child\n+        .get(\"selftext_html\")\n+        .and_then(|t| t.as_str())\n+        .filter(|s| !s.is_empty())\n+        .map(|s| s.to_string());\n+    let thumb_url = child\n+        .get(\"thumbnail\")\n+        .and_then(|t| t.as_str())\n+        .filter(|s| s.starts_with(\"http\"))\n+        .map(|s| s.to_string());\n+\n+    Some(EntityData {\n+        title,\n+        author,\n+        body_html,\n+        thumb_url,\n+    })\n+}\n+\n+#[cfg(test)]\n+mod tests {\n+    use super::*;\n+\n+    #[test]\n+    fn map_subreddit_about_url() {\n+        let id = ItemId::parse(\"reddit.com/r/rust\").unwrap();\n+        assert_eq!(\n+            map_item_to_reddit_api(&id, false),\n+            \"https://www.reddit.com/r/rust/about.json?raw_json=1\"\n+        );\n+        assert_eq!(\n+            map_item_to_reddit_api(&id, true),\n+            \"https://oauth.reddit.com/r/rust/about.json?raw_json=1\"\n+        );\n+    }\n+\n+    #[test]\n+    fn map_post_url() {\n+        let id =\n+            ItemId::parse(\"reddit.com/r/amitheasshole/comments/1trnvdl\").unwrap();\n+        assert_eq!(\n+            map_item_to_reddit_api(&id, false),\n+            \"https://www.reddit.com/r/amitheasshole/comments/1trnvdl.json?raw_json=1\"\n+        );\n+    }\n+\n+    #[test]\n+    fn map_non_reddit_empty() {\n+        let id = ItemId::opaque(\"example.com/foo\");\n+        assert!(map_item_to_reddit_api(&id, false).is_empty());\n+    }\n+\n+    #[test]\n+    fn parse_subreddit_fixture() {\n+        let json = r#\"{\"kind\":\"t5\",\"data\":{\"title\":\"Rust\",\"display_name\":\"rust\",\"public_description\":\"systems\"}}\"#;\n+        let entity = parse_reddit_json(\n+            &ItemId::parse(\"reddit.com/r/rust\").unwrap(),\n+            json.as_bytes(),\n+        )\n+        .unwrap();\n+        assert_eq!(entity.title, \"Rust\");\n+    }\n+\n+    #[test]\n+    fn parse_post_fixture() {\n+        let json = r#\"[{\"kind\":\"Listing\",\"data\":{\"children\":[{\"kind\":\"t3\",\"data\":{\"title\":\"AITA\",\"author\":\"op\",\"selftext_html\":\"&lt;p&gt;hi&lt;/p&gt;\",\"thumbnail\":\"https://b.thumbs.redditmedia.com/x.jpg\"}}]}}]\"#;\n+        let entity = parse_reddit_json(\n+            &ItemId::parse(\"reddit.com/r/x/comments/abc\").unwrap(),\n+            json.as_bytes(),\n+        )\n+        .unwrap();\n+        assert_eq!(entity.title, \"AITA\");\n+        assert_eq!(entity.author.as_deref(), Some(\"op\"));\n+    }\n }\ndiff --git a/server/src/state.rs b/server/src/state.rs\nindex cc1722f5a5bf4d415f2327ea585c488a15a75592..8c03aa60c15aee803a534439400b69935b1a3d84 100644\n--- a/server/src/state.rs\n+++ b/server/src/state.rs\n@@ -5,9 +5,10 @@ use tokio::sync::RwLock;\n use crate::{\n     event_log::EventLog,\n     events::Event,\n+    journal::JournalClient,\n     path_types::ItemId,\n+    reddit::{default_user_agent, RedditBroker},\n     reducer::{GlobalTree, VoteData},\n-    journal::JournalClient,\n     views::ViewStore,\n };\n \n@@ -73,6 +74,7 @@ pub struct AppState {\n     pub views: ViewStore,\n     pub tree: Arc<RwLock<GlobalTree>>,\n     journal: JournalClient,\n+    pub reddit: RedditBroker,\n }\n \n impl AppState {\n@@ -112,6 +114,7 @@ impl AppState {\n \n         let tree = Arc::new(RwLock::new(tree));\n         let journal = JournalClient::spawn(tree.clone(), event_log.clone());\n+        let reddit = RedditBroker::spawn(tree.clone(), &default_user_agent());\n \n         Self {\n             cfg: Arc::new(cfg),\n@@ -119,6 +122,7 @@ impl AppState {\n             views,\n             tree,\n             journal,\n+            reddit,\n         }\n     }\n \n@@ -127,8 +131,11 @@ impl AppState {\n             id: id.as_str().to_string(),\n         };\n         self.event_log.append(&event).await.map_err(|e| e.to_string())?;\n-        let mut w = self.tree.write().await;\n-        w.ensure_path(id);\n+        {\n+            let mut w = self.tree.write().await;\n+            w.ensure_path(id);\n+        }\n+        self.reddit.request_fetch(id.clone());\n         Ok(())\n     }\n \ndiff --git a/todo b/todo\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..d196e8cb4cc80ccb95eeff01c73607d520e13212\n--- /dev/null\n+++ b/todo\n@@ -0,0 +1,7 @@\n+reddit import (only on explicit request)\n+reddit rendering\n+vote redering\n+pair chosing\n+nsfw gate\n+\n+logins (uuid user, two sides, oauths, and pseudonyms)\n","role":"user"}],"model":"~anthropic/claude-sonnet-latest"}