constitution · epochs · watch · epoch 3

comparison

c_8dc1a8119370 (tommy-mor) vs c_0c466eea1ed4 (tommy-mor)

download prompt · raw event · cmp_32962c2af1e113

council reasoning

~anthropic/claude-sonnet-latest · winner A · 6:4 · permalink

Side A rips out a genuinely broken, overengineered keystroke-transition parser (1800+ lines of fragile graph/backtracking logic plus a Playwright race test) and replaces it with a small, correct, testable subreddit-URL parser and simple paste-and-go UI, net-simplifying the codebase while fixing real reliability issues. Side B adds a reasonable but purely auxiliary dev CLI (sorterc) that duplicates server logic for offline linting/compiling—useful tooling but lower-stakes and more speculative than fixing a broken core UX flow shipped to users.

~x-ai/grok-latest · winner A · 3:1 · permalink

A replaces an unreliable ~1800-line keystroke graph parser plus race-prone autocomplete UI/tests with a small correct paste-and-go URL extractor and redirect, deleting whole modules (parser_action, parser_race) and lasting tech debt. B adds useful but peripheral offline dev tooling (sorterc + offline.rs compile/scan) that does not change core product behavior.

openai/gpt-chat-latest · winner B · 4:1 · permalink

Side B adds a substantial new offline capability: a reusable `offline` module plus a new `sorterc` workspace binary that compiles `.sorter` documents, replays JSONL event logs, reports malformed ingests, and includes tests and CLI integration. Side A mainly replaces a complex autocomplete/parser UI with a simpler paste-and-go flow by deleting the transition graph and related UI logic, which simplifies maintenance but also removes interactive functionality rather than adding broadly reusable project infrastructure.

sides

A — c_8dc1a8119370 (tommy-mor)

message

[529cc941] Replace autocomplete parser with paste-and-go navigate.

The keystroke transition graph was unreliable; a textarea plus Go button now parses pasted Reddit URLs and redirects to the subreddit ranking scope.

Co-authored-by: Cursor <cursoragent@cursor.com>

diff preview

diff --git a/AGENTS.md b/AGENTS.md
index 77f2e31d4e860a92a77255ca5106c8b6c4510ee7..36ee4c0ec700bffbe226ee775ba9cf59cf35c770 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -11,7 +11,6 @@ Single Rust web app **`sorter2-server`**: pairwise voting, rank-centrality ranki
 - **Rust 1.88+** is required (some transitive crates need a recent Cargo). The image may ship older `/usr/local/cargo` (1.83); use **rustup** and `rustup default 1.88.0` before building.
 - **System packages** for builds: `pkg-config`, `libssl-dev` (for `reqwest` / OpenSSL in integration tests and release builds).
 - **Clojure CLI 1.12.0.1530** (optional but used in CI): install from https://clojure.org/guides/install_clojure — needed for `./scripts/clj-test.sh` / Kaocha tests.
-- **Playwright browser** for the spel browser test (`test/parser_race.clj`): install once with `clojure -M -e "(com.microsoft.playwright.CLI/main (into-array String [\"install\" \"chromium\" \"--with-deps\"]))"`. The browser binary is cached under `~/.cache/ms-playwright`.
 
 ### Commands (see also `TEST.sh`)
 
diff --git a/server/src/api/ui_html.rs b/server/src/api/ui_html.rs
index 1339048c955c0a3eb5120381aaf031424cbed540..c4ab9d65c7b3cd42a5b4d093ba429993c101e9a8 100644
--- a/server/src/api/ui_html.rs
+++ b/server/src/api/ui_html.rs
@@ -8,7 +8,7 @@ use std::collections::HashMap;
 use crate::{
     html::{js_string_literal, ranking_panel, JsBuilder},
     parser::parse_reddit_url,
-    parser_render::parser_panel_morph,
+    parser_render::navigate_panel,
     state::AppState,
     ui_action::{parse_html_ui_from_form, HtmlUiAction},
 };
@@ -57,18 +57,23 @@ pub async fn post_ui_html(
                 .morph_selector("#ranking-panel", panel)
                 .into_response()
         }
-        HtmlUiAction::ParseQuery { query } => {
-            let action = parse_reddit_url(&query);
-            let panel = parser_panel_morph(&query, &action);
-            let mut js = JsBuilder::new().morph_selector("#parser-panel", panel);
-            if let Some(comp) = action.primary_completion() {
-                js = js.raw(&format!(
-                    "var __pi=document.getElementById('parser-input'); if(__pi){{__pi.dataset.completion={};}}",
-                    js_string_literal(comp)
-                ));
+        HtmlUiAction::ParseQuery { query } => match parse_reddit_url(&query) {
+            Ok(subreddit) => {
+                let dest = format!("/?sub={subreddit}");
+                JsBuilder::new()
+                    .raw(&format!(
+                        "window.location.href={};",
+                        js_string_literal(&dest)
+                    ))
+                    .into_response()
             }
-            js.into_response()
-        }
+            Err(message) => {
+                let panel = navigate_panel(&query, Some(&message));
+                JsBuilder::new()
+                    .morph_selector("#parser-panel", panel)
+                    .into_response()
+            }
+        },
     }
 }
 
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index 94a5cfb561d1c8464bd2782f7ffd4e0965ccf95d..9650d333d29c4ac94ceb407aee3ee00399c7f40b 100644
--- a/server/src/html/mod.rs
+++ b/server/src/html/mod.rs
@@ -11,8 +11,7 @@ use serde::Deserialize;
 
 use crate::{
     form_template::template_json_compact,
-    parser_action::ParserAction,
-    parser_render::parser_panel,
+    parser_render::navigate_panel,
     ranking::{top_bottom, RankedItem},
     reducer::GroupState,
     state::{normalize_scope, AppState},
@@ -336,10 +335,9 @@ pub async fn home(
     let empty = GroupState::new();
     let group = groups.get(&scope).unwrap_or(&empty);
 
-    let empty_action = ParserAction::suggest(String::new(), None);
     let body = html! {
         h1 { "sorter2" }
-        (parser_panel("", &empty_action))
+        (navigate_panel("", None))
         (vote_panel(&scope))
         (ranking_panel(&scope, group))
     };
diff --git a/server/src/lib.rs b/server/src/lib.rs
index fa423640d598f4ba97a5885d228e78d7b97f7a22..de8bca48cbf689cad22337883e7791966e7c4919 100644
--- a/server/src/lib.rs
+++ b/server/src/lib.rs
@@ -4,7 +4,6 @@ pub mod events;
 pub mod form_template;
 pub mod html;
 pub mod parser;
-pub mod parser_action;
 pub mod parser_render;
 pub mod path_types;
 pub mod ranking;
diff --git a/server/src/parser.rs b/server/src/parser.rs
index ea437c4434045b44cba04a2b3416df850db518e7..50571a59f0d3ece1e2538f88e00ef46ec40ea545 100644
--- a/server/src/parser.rs
+++ b/server/src/parser.rs
@@ -1,1811 +1,87 @@
-use std::collections::HashMap;
-use std::sync::OnceLock;
+//! Extract a subreddit name from a pasted Reddit URL or path.
 
-use crate::parser_action::{GuideOption, ParserAction, ScrollingSuggestion, Suggestion};
-
-// --- Core Abstractions ---
-
-/// Unique identifier for nodes in the graph
-type NodeId = &'static str;
-
-/// Pattern matching for edges
-#[derive(Debug, Clone)]
-pub enum EdgePattern {
-    /// Matches exact literal string
-    Literal(&'static str),
-    
-    /// Matches any prefix of a string and suggests the full string
-    /// e.g., PrefixOf("reddit.com") matches "r", "re", "red", "reddit", "reddit.com"
-    PrefixOf(&'static str),
-    
-    /// Captures a variable segment (e.g., subreddit name, username)
-    Variable(&'static str),
-    
-    /// Matches any string (wildcard)
-    Any,
-}
-
-impl EdgePattern {
-    /// Try to match this pattern against input, return (consumed_chars, captured_value)
-    fn matches(&self, input: &str) -> Option<(usize, Option<String>)> {
-        match self {
-            EdgePattern::Literal(lit) => {
-                if input.starts_with(lit) {
-                    Some((lit.len(), None))
-                } else {
-                    None
-                }
-            }
-            EdgePattern::PrefixOf(target) => {
-                // Check if input is a prefix of target
-                if target.starts_with(input) && !input.is_empty() {
-                    // It's a valid prefix
-                    Some((input.len(), None))
-                } else if input.starts_with(target) {
-                    // Full match
-                    Some((target.len(), None))
-                } else {
-                    None
-                }
-            }
-            EdgePattern::Variable(var_name) => {
-                // Consume until next '/' or end of string
-                let end = input.find('/').unwrap_or(input.len());
-                if end > 0 {
-                    let captured = input[..end].to_string();
-                    // Validate based on variable type
-                    if is_valid_variable(var_name, &captured) {
-                        Some((end, Some(captured)))
-                    } else {
-                        None
-                    }
-                } else {
-                    None
-                }
-            }
-            EdgePattern::Any => {
-                // Match everything until next '/' or end
-                let end = input.find('/').unwrap_or(input.len());
-                if end > 0 {
-                    Some((end, Some(input[..end].to_string())))
-                } else {
-                    None
-                }
-            }
-        }
-    }
-    
-    /// Get the completion suggestion for this pattern
-    fn completion(&self, partial: &str) -> Option<String> {
-        match self {
-            EdgePattern::PrefixOf(target) => {
-                if target.starts_with(partial) && partial != *target {
-                    Some(target.to_string())
-                } else {
-                    None
-                }
-            }
-            _ => None,
-        }
+pub fn parse_reddit_url(query: &str) -> Result<String, String> {
+    let q = query.trim();
+    if q.is_empty() {
+        return Err("Paste a Reddit URL or r/subreddit path".into());
     }
-}
 
-/// Edge in the graph
-pub struct Edge {
-    pattern: EdgePattern,
-    target: NodeId,
-    /// Optional description for autocomplete
-    description: Option<&'static str>,
-}
-
-/// Handler function for generating UI actions (Send + Sync so the graph can live in `OnceLock`).
-type Handler = Box<dyn Fn(&str, &str, &HashMap<String, String>) -> ParserAction + Send + Sync>;
-
-/// Node in the graph
-pub struct Node {
-    #[allow(dead_code)]
-    id: NodeId,
-    edges: Vec<Edge>,
-    handler: Option<Handler>,
-}
-
-/// The composable parser graph (immutable after `build`).
-pub struct Graph {
-    nodes: HashMap<NodeId, Node>,
-    root: NodeId,
-}
-
-// --- Graph Builder (Fluent API) ---
-
-pub struct GraphBuilder {
-    nodes: HashMap<NodeId, Node>,
-    current_node: Option<NodeId>,
-    root: NodeId,
-}
-
-impl GraphBuilder {
-    pub fn new() -> Self {
-        let mut nodes = HashMap::new();
-        nodes.insert(
-            "root",
-            Node {
-                id: "root",
-                edges: Vec::new(),
-                handler: None,
-            },
-        );
-
-        GraphBuilder {
-            nodes,
-            current_node: Some("root"),
-            root: "root",
-        }
+    if let Some(sub) = subreddit_after_prefix(q, "r/") {
+        return Ok(sub);
     }
 
-    /// Select a node to add edges to
-    pub fn at(mut self, node_id: NodeId) -> Self {
-        self.nodes.entry(node_id).or_insert_with(|| Node {
-            id: node_id,
-            edges: Vec::new(),
-            handler: None,
-        });
-        self.current_node = Some(node_id);
-        self
+    if let Some(sub) = subreddit_from_path_segment(q, "/r/") {
+        return Ok(sub);
     }
-    
-    /// Add an edge from the current node
-    pub fn edge(self, pattern: EdgePattern, target: NodeId) -> Self {
-        self.edge_with_desc(pattern, target, None)
-    }
-    
-    /// Add an edge with description
-    pub fn edge_with_desc(
-        mut self,
-        pattern: EdgePattern,
-        target: NodeId,
-        desc: Option<&'static str>,
-    ) -> Self {
-        let current = self.current_node.expect("No current node selected");
-
-        self.nodes.entry(target).or_insert_with(|| Node {
-            id: target,
-            edges: Vec::new(),
-            handler: None,
-        });
-
-        if let Some(node) = self.nodes.get_mut(current) {
-            node.edges.push(Edge {
-                pattern,
-                target,
-                description: desc,
-            });
-        }
 
-        self
-    }
-
-    /// Set handler for current node
-    pub fn handler<F>(mut self, handler: F) -> Self
-    where
-        F: Fn(&str, &str, &HashMap<String, String>) -> ParserAction + Send + Sync + 'static,
-    {
-        let current = self.current_node.expect("No current node selected");
-        if let Some(node) = self.nodes.get_mut(current) {
-            node.handler = Some(Box::new(handler));
-        }
-        self
-    }
-    
-    /// Build the final graph
-    pub fn build(self) -> Graph {
-        Graph {
-            nodes: self.nodes,
-            root: self.root,
-        }
-    }
+    Err("Could not find a subreddit in that URL".into())
 }
 
-// --- Parser Implementation ---
-
-impl Graph {
-    pub fn parse(&self, input: &str) -> ParserAction {
-        let normalized = input.trim().to_lowercase();
-        let mut state = ParserState {
-            input: &normalized,
-            cursor: 0,
-            current_node_id: self.root,
-            context: HashMap::new(),
-            original_query: input.to_string(),
-            current_prefix: String::new(),
-        };
-        
-        self.parse_recursive(&mut state)
-    }
-    
-    fn parse_recursive(&self, state: &mut ParserState) -> ParserAction {
-        let node = self
-            .nodes
-            .get(state.current_node_id)
-            .expect("Node not found in graph");
-
-        // If we've consumed all input, check for handler or suggestions
-        if state.cursor >= state.input.len() {
-            if let Some(handler) = &node.han

… preview truncated; 92,533 characters omitted

download full diff A

B — c_0c466eea1ed4 (tommy-mor)

message

[9e20d06c] Add sorterc dev tool for offline DSL compile and JSONL lint.

Introduce a workspace-only binary that validates .sorter files into ranking JSON and scans events.jsonl for corrupt or unreplayable ingests.

Co-authored-by: Cursor <cursoragent@cursor.com>

diff preview

diff --git a/Cargo.lock b/Cargo.lock
index bf8153d9c723af97122c9ffdd4a7cfe82e853bb6..a07734f089b466440c3ae6fc1087ce85fc24ce62 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -1826,6 +1826,17 @@ dependencies = [
  "windows-sys 0.60.2",
 ]
 
+[[package]]
+name = "sorterc"
+version = "0.0.1"
+dependencies = [
+ "anyhow",
+ "clap",
+ "serde",
+ "serde_json",
+ "slugsocial-server",
+]
+
 [[package]]
 name = "spin"
 version = "0.9.8"
diff --git a/Cargo.toml b/Cargo.toml
index 149cbf07901eab57c593184ff8719a75d530f1da..25337acdd61e44b20f354c78fed4a88caf896280 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,5 +1,5 @@
 [workspace]
-members = ["server", "cli"]
+members = ["server", "cli", "sorterc"]
 resolver = "2"
 
 
diff --git a/agents.md b/agents.md
index d8b801e454fdf37e7ac6038b91a69f83b0746d59..ce646ed3cd7123be4732dccec4a6800467e651e7 100644
--- a/agents.md
+++ b/agents.md
@@ -93,6 +93,17 @@ SLUG_GOOGLE_CLIENT_SECRET=mock
 
 After OAuth completes, the pending-session poll returns a `slug_…` bearer token for API calls.
 
+### Dev-only offline tooling
+
+**`sorterc`** — workspace binary, not published via npm. Compiles `.sorter` files and lints `events.jsonl` without a server:
+
+```
+cargo run -p sorterc -- compile path/to/doc.sorter [--base events.jsonl] [--room public] [--pretty]
+cargo run -p sorterc -- scan path/to/events.jsonl [--pretty]
+```
+
+`compile` validates DSL, simulates ingest against empty (or `--base`) reducer state, and prints JSON rankings. `scan` reports corrupt JSONL lines and ingests that fail DSL replay.
+
 ### Testing
 
 - **Rust tests:** `cargo nextest run --workspace` (163 tests; requires `cargo-nextest`)
diff --git a/server/src/lib.rs b/server/src/lib.rs
index c1d477d21aea03aff00e6f0689b0b4379d0d68d2..ad8e31099c807fb5844acb16cd5086a2f19327a7 100644
--- a/server/src/lib.rs
+++ b/server/src/lib.rs
@@ -10,6 +10,7 @@ pub mod form_template;
 pub mod html;
 pub mod identity;
 pub mod middleware;
+pub mod offline;
 pub mod path_types;
 pub mod ranking;
 pub mod reducer;
diff --git a/server/src/offline.rs b/server/src/offline.rs
new file mode 100644
index 0000000000000000000000000000000000000000..54ad0ded096a305ef8454ab2cdd1c3af71b14f5d
--- /dev/null
+++ b/server/src/offline.rs
@@ -0,0 +1,333 @@
+//! Offline `.sorter` compilation and JSONL diagnostics (no network, no auth).
+
+use std::collections::HashSet;
+use std::path::Path;
+
+use serde::Serialize;
+use slug_types::{CheckScopeRanking, RankComponent, RankRow, paths::GardenItemUrl};
+
+use crate::{
+    api::{resolve_item, validate_ingest_document},
+    dsl,
+    events::{Event, Ingest},
+    path_types::ItemId,
+    reducer::{ReducerState, ScopeId, scope_from_room_wire},
+    scope_rank::build_children_rankings,
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CompileStats {
+    pub items: usize,
+    pub votes: usize,
+    pub prose_blocks: usize,
+}
+
+#[derive(Debug, Serialize)]
+pub struct CompileResult {
+    pub ok: bool,
+    pub threads: Vec<String>,
+    pub rankings: Vec<CheckScopeRanking>,
+    pub stats: CompileStats,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CompileError {
+    pub ok: bool,
+    pub error: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub hint: Option<String>,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct BadJsonLine {
+    pub line: usize,
+    pub message: String,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct MalformedIngest {
+    pub line: usize,
+    pub id: String,
+    pub room_id: String,
+    pub thread_tag: String,
+    pub reason: String,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct ScanResult {
+    pub ok: bool,
+    pub path: String,
+    pub total_lines: usize,
+    pub parsed_events: usize,
+    pub bad_json_lines: Vec<BadJsonLine>,
+    pub malformed_ingests: Vec<MalformedIngest>,
+    pub skipped_ingests: usize,
+}
+
+fn document_stats(doc: &dsl::Document) -> CompileStats {
+    let mut items = 0usize;
+    let mut votes = 0usize;
+    let mut prose_blocks = 0usize;
+    for stmt in &doc.statements {
+        match stmt {
+            dsl::Stmt::Item { .. } => items += 1,
+            dsl::Stmt::Vote { .. } => votes += 1,
+            dsl::Stmt::Prose { .. } => prose_blocks += 1,
+        }
+    }
+    CompileStats {
+        items,
+        votes,
+        prose_blocks,
+    }
+}
+
+fn threads_in_document(text: &str) -> Vec<String> {
+    let mut out = HashSet::new();
+    for line in text.lines() {
+        let trimmed = line.trim();
+        if !trimmed.starts_with('#') {
+            continue;
+        }
+        let rest = trimmed.trim_start_matches('#').trim();
+        if rest.is_empty() {
+            continue;
+        }
+        let tag = rest.split_whitespace().next().unwrap_or(rest);
+        let tag = tag.split(':').next().unwrap_or(tag).trim();
+        if tag.is_empty() {
+            continue;
+        }
+        out.insert(format!("#{}", crate::canonical_path::canonicalize_tag(tag)));
+    }
+    let mut tags: Vec<String> = out.into_iter().collect();
+    tags.sort();
+    tags
+}
+
+fn voted_parent_scopes(doc: &dsl::Document) -> Vec<ItemId> {
+    let mut parents = HashSet::new();
+    for stmt in &doc.statements {
+        if let dsl::Stmt::Vote { item1, item2, .. } = stmt {
+            if let (Ok(a), Ok(b)) = (resolve_item(item1), resolve_item(item2)) {
+                if let Some(p) = a.parent() {
+                    parents.insert(p);
+                }
+                if let Some(p) = b.parent() {
+                    parents.insert(p);
+                }
+            }
+        }
+    }
+    let mut out: Vec<ItemId> = parents.into_iter().collect();
+    out.sort();
+    out
+}
+
+fn rankings_for_simulated(
+    simulated: &ReducerState,
+    scope: &ScopeId,
+    room_wire: &str,
+    doc: &dsl::Document,
+) -> Vec<CheckScopeRanking> {
+    voted_parent_scopes(doc)
+        .iter()
+        .map(|parent| {
+            let scoped_content = simulated
+                .content_for_scope(&scope)
+                .unwrap_or_else(|| simulated.public());
+            let scoped = build_children_rankings(scoped_content, parent);
+            let components: Vec<RankComponent> = scoped
+                .component_rankings
+                .into_iter()
+                .map(|comp| RankComponent {
+                    pairs: comp.pairs,
+                    ranking: comp
+                        .ranked
+                        .into_iter()
+                        .map(|r| RankRow {
+                            item: GardenItemUrl::from_stored(&r.item, room_wire),
+                            score: r.score,
+                            percent: None,
+                        })
+                        .collect(),
+                })
+                .collect();
+            CheckScopeRanking {
+                parent: GardenItemUrl::from_stored(parent, room_wire).into_inner(),
+                components,
+                unranked_items: scoped
+                    .unranked_items
+                    .into_iter()
+                    .map(|it| GardenItemUrl::from_stored(&it, room_wire))
+                    .collect(),
+            }
+        })
+        .collect()
+}
+
+/// Validate and simulate one `.sorter` document against optional base reducer state.
+pub fn compile_document(
+    base: &ReducerState,
+    room: &str,
+    text: &str,
+) -> Result<CompileResult, CompileError> {
+    let room_key = room.trim();
+    let scope = scope_from_room_wire(room_key);
+    let validated = validate_ingest_document(base, text, &scope).map_err(|(_, message, hint)| {
+        CompileError {
+            ok: false,
+            error: message,
+            hint,
+        }
+    })?;
+
+    let event = Event::Ingest(Ingest {
+        ts: validated.ts,
+        id: uuid::Uuid::new_v4().to_string(),
+        raw: validated.raw_text.clone(),
+        principal: "offline".to_string(),
+        delegate: None,
+        room_id: room_key.to_string(),
+        thread_tag: "offline".to_string(),
+    });
+
+    let mut simulated = base.clone();
+    simulated.apply_event(event);
+
+    Ok(CompileResult {
+        ok: true,
+        threads: threads_in_document(text),
+        rankings: rankings_for_simulated(&simulated, &scope, room_key, &validated.doc),
+        stats: document_stats(&validated.doc),
+    })
+}
+
+fn ingest_parse_error(raw: &str) -> Option<String> {
+    dsl::parse_full(raw).err().map(|e| e.to_string())
+}
+
+fn load_events_from_jsonl(path: &Path) -> Result<(Vec<(usize, Event)>, Vec<BadJsonLine>), std::io::Error> {
+    let text = std::fs::read_to_string(path)?;
+    let mut events = Vec::new();
+    let mut bad_json_lines = Vec::new();
+    for (idx, line) in text.lines().enumerate() {
+        let line_no = idx + 1;
+        let trimmed = line.trim();
+        if trimmed.is_empty() {
+            continue;
+        }
+        match serde_json::from_str::<Event>(trimmed) {
+            Ok(ev) => events.push((line_no, ev)),
+            Err(e) => bad_json_lines.push(BadJsonLine {
+                line: line_no,
+                message: e.to_string(),
+            }),
+        }
+    }
+    Ok((events, bad_json_lines))
+}
+
+/// Replay a JSONL event log into reducer state (same rules as server boot).
+pub fn load_reducer_from_jsonl(path: &Path) -> Result<(ReducerState, Vec<BadJsonLine>), std::io::Error> {
+    let (events, bad_json_lines) = load_events_from_jsonl(path)?;
+    let mut state = ReducerState::default();
+    for (_line_no, ev) in events {
+        state.apply_event(ev);
+    }
+    Ok((state, bad_json_lines))
+}
+
+/// Scan an events.jsonl for corrupt JSON lines and ingests that fail DSL replay.
+pub fn scan_jsonl(path: &Path) -> Result<ScanResult, std::io::Error> {
+    let text = std::fs::read_to_string(path)?;
+    let total_lines = text.lines().count();
+    let (events, bad_json_lines) = load_events_from_jsonl(path)?;
+
+    let mut malformed_ingests = Vec::new();
+    let mut skipped_ingests = 0usize;
+    let mut state = ReducerState::default();
+    let parsed_events = events.len();
+
+    for (line_no, ev) in events {
+        if let Event::Ingest(ref ing) = ev {
+            if let Some(reason) = ingest_parse_error(&ing.raw) {
+                malformed_ingests.push(MalformedIngest {
+                    line: line_no,
+                    id: ing.id.clone(),
+                    room_id: ing.room_id.clone(),
+                    thread_tag: ing.thread_tag.clone(),
+                    reason,
+                });
+            }
+            let before = state.ingests_by_id.len();
+            state.apply_event(ev);
+            if state.ingests_by_id.len() == before {
+                skipped_ingests += 1;
+            }
+        } else {
+            state.apply_event(ev);
+        }
+    }
+
+    let ok = bad_json_lines.is_empty() && malformed_ingests.is_empty() && skipped_ingests == 0;
+
+    Ok(ScanResult {
+        ok,
+        path: path.display().to_string(),
+        total_lines,
+        parsed_events,
+        bad_json_lines,
+        malformed_ingests,
+        skipped_ingests,
+    })
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    const TUTORIAL: &str = include_str!("../tests/fixtures/tutorial.sorter");
+
+    #[test]
+    fn compile_tutorial_fixture_emits_rankings() {
+        let result = compile_document(&ReducerState::default(), "public", TUTORIAL).unwrap();
+        assert!(result.ok);
+        assert!(!result.threads.is_empty());
+        assert!(result.stats.items >= 6);
+        assert!(result.stats.votes >= 6);
+        assert!(!result.rankings.is_empty());
+    }
+
+    #[test]
+    fn compile_rejects_vote_on_missing_item() {
+        let err = compile_document(
+            &ReducerState::default(),
+            "public",
+            "{ reason }\n~/missing/a 2:1 ~/missing/b",
+        )
+        .unwrap_err();
+        assert!(!err.ok);
+        assert!(err.error.contains("undefined"));
+    }
+
+    #[test]
+    fn scan

… preview truncated; 7,950 characters omitted

download full diff B

Hardlinks — judgments / attempts / prompt

prompt download

judgments

attempts

Prompt text is loaded only by the download route.