Side A performs a genuine architectural consolidation—moving path canonicalization/URL construction into a shared slug-types crate with correct-by-construction newtypes (GardenItemUrl, ForumThreadUrl, TildeOntologyPath), replacing scattered string-formatting helpers with typed APIs across server and RPC code, which reduces duplication and future URL-shape bugs. Side B adds a useful but auxiliary dev-only CLI tool (sorterc) for offline DSL compiling/linting, which is valuable tooling but doesn't touch core correctness or design of the production system. Both are solid, but A's refactor has broader lasting impact on the codebase's core type safety and maintainability.
constitution · epochs · watch · epoch 3
comparison
c_5e9a63e9d276 (tommy-mor) vs c_0c466eea1ed4 (tommy-mor)
download prompt · raw event · cmp_636ce4b32c52a5
council reasoning
A centralizes path canonicalization and room-aware href construction into shared slug-types newtypes (GardenItemUrl, ForumThreadUrl, TildeOntologyPath) and threads them through API/RPC response types, replacing ad-hoc string helpers with lasting domain structure. B adds a useful but peripheral workspace-only sorterc binary for offline DSL compile/JSONL lint that mostly wraps existing server logic. Core path-identity typing in the shared schema outweighs a dev-only tooling surface.
Side A consolidates canonicalization, path normalization, and room-aware URL generation into a shared `types::paths` module, replacing scattered string helpers with strongly typed `GardenItemUrl`, `ForumThreadUrl`, and `TildeOntologyPath` throughout RPCs and API types. This removes duplicated logic, makes wire formats correct-by-construction across many interfaces, and centralizes core path semantics, whereas Side B adds a useful developer-only offline compiler/linter (`sorterc`) without changing the project's core runtime design.
sides
A — c_5e9a63e9d276 (tommy-mor)
message
[a888d56c] refactor: centralize path identity in slug-types Move canonicalization and CanonicalItemUrl into types::paths with GardenItemUrl, ForumThreadUrl, and TildeOntologyPath for JSON hrefs. Server canonical_path and path_types re-export slug-types; RPC and validation build hrefs via those types instead of string helpers. Made-with: Cursor
diff preview
diff --git a/server/src/api/helpers.rs b/server/src/api/helpers.rs
index 9b71491e9f9efc44a2a4beba09be8f64bd2ff2ee..03b3e77911ccd662bec8635345dafe2593cf242e 100644
--- a/server/src/api/helpers.rs
+++ b/server/src/api/helpers.rs
@@ -4,12 +4,12 @@ use axum::{
Json,
};
use sha2::{Digest, Sha256};
+use slug_types::paths::{CanonicalItemUrl, GardenItemUrl};
use slug_types::*;
use std::collections::HashMap;
use crate::{
canonical_path::canonicalize_item,
- path_types::CanonicalItemUrl,
ranking::connected_components_from_voted_pairs,
};
@@ -30,64 +30,6 @@ pub fn now_ms() -> i64 {
t.as_millis() as i64
}
-/// Serialize a canonical item for JSON: absolute URLs stay as-is; bare paths get a `/` prefix.
-pub fn item_path_for_api(item: &str) -> String {
- if item.starts_with("http://") || item.starts_with("https://") {
- item.to_string()
- } else {
- format!("/{}", item)
- }
-}
-
-/// Same as [`item_path_for_api`], but for private rooms ontology items are prefixed with
-/// `/r/{short}/{slug}` so the URL matches the web app (`/r/…/~/…` routes).
-pub fn item_path_for_api_in_room(item: &str, room_wire: &str) -> String {
- let room = room_wire.trim();
- if room.is_empty() || room == "public" {
- return item_path_for_api(item);
- }
- let Some((short, slug)) = room.split_once('/') else {
- return item_path_for_api(item);
- };
- if short.is_empty() || slug.is_empty() {
- return item_path_for_api(item);
- }
- let Some(c) = CanonicalItemUrl::parse(item) else {
- return item_path_for_api(item);
- };
- let root = CanonicalItemUrl::ontology_root();
- let item_norm = c.as_str().trim_end_matches('/');
- let root_norm = root.as_str().trim_end_matches('/');
- if let Some(tail) = c.tilde_tail() {
- return if tail.is_empty() {
- format!("https://slug.social/r/{short}/{slug}/~")
- } else {
- format!("https://slug.social/r/{short}/{slug}/~/{}", tail)
- };
- }
- if item_norm == root_norm {
- return format!("https://slug.social/r/{short}/{slug}/~");
- }
- item_path_for_api(item)
-}
-
-/// Absolute thread URL for forum JSON (`/t/…` vs `/r/…/t/…`).
-pub fn forum_thread_web_url(room_wire: &str, thread_tag: &str) -> String {
- let room = room_wire.trim();
- let tag = thread_tag.trim().trim_start_matches('#');
- if room.is_empty() || room == "public" {
- format!("https://slug.social/t/{tag}")
- } else if let Some((short, slug)) = room.split_once('/') {
- if short.is_empty() || slug.is_empty() {
- format!("https://slug.social/t/{tag}")
- } else {
- format!("https://slug.social/r/{short}/{slug}/t/{tag}")
- }
- } else {
- format!("https://slug.social/t/{tag}")
- }
-}
-
/// Resolve an item path as a first-class canonical path.
pub fn resolve_item(item: &str) -> Result<String, String> {
let canonical = canonicalize_item(item);
@@ -109,14 +51,12 @@ pub fn parse_parent_specs(parent: Option<&String>) -> Vec<String> {
}
/// Apply offset+limit pagination to the flattened component rankings.
-/// Items are flattened in component order (largest component first), then unranked last.
-/// Returns (components, unranked_items) after the window.
pub fn paginate_rankings(
components: Vec<RankComponent>,
- unranked_items: Vec<String>,
+ unranked_items: Vec<GardenItemUrl>,
offset: usize,
limit: Option<usize>,
-) -> (Vec<RankComponent>, Vec<String>) {
+) -> (Vec<RankComponent>, Vec<GardenItemUrl>) {
let mut remaining_skip = offset;
let mut remaining_take = limit.unwrap_or(usize::MAX);
let mut out_components: Vec<RankComponent> = Vec::new();
@@ -141,7 +81,7 @@ pub fn paginate_rankings(
});
}
- let out_unranked: Vec<String> = if remaining_take > 0 {
+ let out_unranked: Vec<GardenItemUrl> = if remaining_take > 0 {
unranked_items
.into_iter()
.skip(remaining_skip)
@@ -183,11 +123,9 @@ pub fn is_pair_voted(group: &crate::reducer::GroupState, a: &str, b: &str) -> bo
group.voted_pairs.contains(&(i, j))
}
-/// Compute graph connectivity stats for a set of items within the ranking group.
pub fn compute_connectivity_stats(group: &crate::reducer::GroupState, pool: &[String]) -> ConnectivityStats {
let n = pool.len();
- // Map pool items to global indices (items not yet in the group get no index)
let global_idxs: Vec<Option<usize>> = pool
.iter()
.map(|it| {
@@ -197,7 +135,6 @@ pub fn compute_connectivity_stats(group: &crate::reducer::GroupState, pool: &[St
.collect();
let present: Vec<usize> = global_idxs.iter().filter_map(|x| *x).collect();
- // Build local index mapping for items that exist in the ranking group
let global_to_local: HashMap<usize, usize> = present
.iter()
.enumerate()
@@ -213,7 +150,6 @@ pub fn compute_connectivity_stats(group: &crate::reducer::GroupState, pool: &[St
}),
);
- // Items not in the ranking group at all are also isolates
let items_not_in_group = global_idxs.iter().filter(|x| x.is_none()).count();
let num_components = comps.len() + isolates.len() + items_not_in_group;
@@ -237,52 +173,3 @@ pub fn vote_touches_path(a: &str, b: &str, parent_canon: &str) -> bool {
let under = |item: &str| item == parent_canon || item.starts_with(&format!("{}/", parent_canon));
under(a) || under(b)
}
-
-#[cfg(test)]
-mod wire_url_tests {
- use super::{forum_thread_web_url, item_path_for_api_in_room};
-
- #[test]
- fn public_room_unchanged() {
- let u = "https://slug.social/~/a/b";
- assert_eq!(item_path_for_api_in_room(u, "public"), u);
- }
-
- #[test]
- fn private_room_prefixes_ontology() {
- assert_eq!(
- item_path_for_api_in_room("https://slug.social/~/topic/x", "9ab12cd/my-room"),
- "https://slug.social/r/9ab12cd/my-room/~/topic/x"
- );
- }
-
- #[test]
- fn private_room_ontology_root() {
- assert_eq!(
- item_path_for_api_in_room("https://slug.social/~", "9ab12cd/my-room"),
- "https://slug.social/r/9ab12cd/my-room/~"
- );
- assert_eq!(
- item_path_for_api_in_room("https://slug.social/~/", "9ab12cd/my-room"),
- "https://slug.social/r/9ab12cd/my-room/~"
- );
- }
-
- #[test]
- fn external_url_untouched_in_private_room() {
- let u = "https://example.com/z";
- assert_eq!(item_path_for_api_in_room(u, "9ab12cd/my-room"), u);
- }
-
- #[test]
- fn forum_web_public_vs_room() {
- assert_eq!(
- forum_thread_web_url("public", "debate"),
- "https://slug.social/t/debate"
- );
- assert_eq!(
- forum_thread_web_url("9ab12cd/my-room", "#debate"),
- "https://slug.social/r/9ab12cd/my-room/t/debate"
- );
- }
-}
diff --git a/server/src/api/mod.rs b/server/src/api/mod.rs
index 042aa248305f9362a3be78f9eea2a5abf6ba707a..cf22cb0129366c3aed031bc86f3197a4321cb806 100644
--- a/server/src/api/mod.rs
+++ b/server/src/api/mod.rs
@@ -24,8 +24,7 @@ pub use auth::{
pub use helpers::{
api_error, compute_connectivity_stats, is_pair_voted, now_ms, paginate_rankings,
- parse_parent_specs, pick_random_distinct, sha256_hex, resolve_item, vote_touches_path,
- item_path_for_api,
+ parse_parent_specs, pick_random_distinct, resolve_item, sha256_hex, vote_touches_path,
};
pub use rpc::handle_rpc_batch;
diff --git a/server/src/api/rpc.rs b/server/src/api/rpc.rs
index 5b91f5836625eedbb1cd9423168046e3fb576c17..5f7d50188f1381267402f2e57e671234ef5db2fd 100644
--- a/server/src/api/rpc.rs
+++ b/server/src/api/rpc.rs
@@ -8,6 +8,7 @@ use axum::{
Json,
};
use rand::seq::SliceRandom;
+use slug_types::paths::{ForumThreadUrl, GardenItemUrl, TildeOntologyPath};
use slug_types::*;
use crate::{
@@ -27,9 +28,8 @@ use crate::{
use super::auth::verify_bearer_principal;
use super::helpers::{
- compute_connectivity_stats, forum_thread_web_url, is_pair_voted, item_path_for_api,
- item_path_for_api_in_room, now_ms, paginate_rankings, parse_parent_specs, pick_random_distinct,
- resolve_item, vote_touches_path,
+ compute_connectivity_stats, is_pair_voted, now_ms, paginate_rankings, parse_parent_specs,
+ pick_random_distinct, resolve_item, vote_touches_path,
};
use super::validate::{normalize_room_and_thread, validate_ingest_document};
@@ -184,7 +184,7 @@ fn compute_scope_rank_changes(
};
if changed {
changes.push(RankChange {
- item: item_path_for_api_in_room(&item, room_wire),
+ item: GardenItemUrl::from_storage_str(&item, room_wire),
before: b,
after: a,
});
@@ -206,7 +206,7 @@ fn compute_scope_rank_changes(
parent: if parent.is_empty() {
"/".to_string()
} else {
- item_path_for_api_in_room(parent, room_wire)
+ GardenItemUrl::from_storage_str(parent, room_wire).into_inner()
},
changes,
})
@@ -302,7 +302,7 @@ fn build_rank_response_for_content(
.ranked
.into_iter()
.map(|r| RankRow {
- item: item_path_for_api_in_room(r.item.as_str(), room_wire),
+ item: GardenItemUrl::from_stored(&r.item, room_wire),
percent: if want_percent {
Some((r.score / max_score) * 100.0)
} else {
@@ -315,10 +315,10 @@ fn build_rank_response_for_content(
})
.collect();
- let prefixed_unranked: Vec<String> = rankings
+ let prefixed_unranked: Vec<GardenItemUrl> = rankings
.unranked_items
.into_iter()
- .map(|s| item_path_for_api_in_room(s.as_str(), room_wire))
+ .map(|s| GardenItemUrl::from_stored(&s, room_wire))
.collect();
let (components, unranked_items) = if offset > 0 || limit.is_some() {
@@ -537,13 +537,13 @@ async fn rpc_post(
(
"npx slugsocial public garden pair".to_string(),
"npx slugsocial public garden rank".to_string(),
- forum_thread_web_url("public", &thread_id),
+ ForumThreadUrl::from_room_tag("public", &thread_id),
)
} else {
(
format!("npx slugsocial private {room_key} garden pair"),
format!("npx slugsocial private {room_key} garden rank"),
- forum_thread_web_url(&room_key, &thread_id),
+ ForumThreadUrl::from_room_tag(&room_key, &thread_id),
)
};
@@ -664,7 +664,7 @@ async fn rpc_check(
.ranked
.into_iter()
.map(|r| RankRow {
- item: item_path_for_api_in_room(r.item.as_str(), &room_key),
+ item: GardenItemUrl::from_stored(&r.item, &room_key),
score: r.score,
percent: None,
})
@@ -672,12 +672,12 @@ async fn rpc_check(
})
.collect();
CheckScopeRanking {
- parent: item_path_for_api_in_room(parent.as_str(), &room_key),
+ parent: GardenItemUrl::from_stored(parent, &room_key).into_inner(),
components,
unranked_items: scoped
.unranked_items
.into_iter()
- .map(|it| item_path_for_api_in_room(it.as_str(), &room_key))
+ .map(|it| GardenItemUrl::from_stored(&it, &room_key))
.collect(),
}
})
@@ -687,13 +687,13 @@ async fn rpc_check(
vec![
"npx slugsocial public forum post <TAG> --delegate <uuid:rig:
… preview truncated; 46,249 characters omittedB — c_0c466eea1ed4 (tommy-mor)
message
[9e20d06c] Add sorterc dev tool for offline DSL compile and JSONL lint. Introduce a workspace-only binary that validates .sorter files into ranking JSON and scans events.jsonl for corrupt or unreplayable ingests. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/Cargo.lock b/Cargo.lock
index bf8153d9c723af97122c9ffdd4a7cfe82e853bb6..a07734f089b466440c3ae6fc1087ce85fc24ce62 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -1826,6 +1826,17 @@ dependencies = [
"windows-sys 0.60.2",
]
+[[package]]
+name = "sorterc"
+version = "0.0.1"
+dependencies = [
+ "anyhow",
+ "clap",
+ "serde",
+ "serde_json",
+ "slugsocial-server",
+]
+
[[package]]
name = "spin"
version = "0.9.8"
diff --git a/Cargo.toml b/Cargo.toml
index 149cbf07901eab57c593184ff8719a75d530f1da..25337acdd61e44b20f354c78fed4a88caf896280 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,5 +1,5 @@
[workspace]
-members = ["server", "cli"]
+members = ["server", "cli", "sorterc"]
resolver = "2"
diff --git a/agents.md b/agents.md
index d8b801e454fdf37e7ac6038b91a69f83b0746d59..ce646ed3cd7123be4732dccec4a6800467e651e7 100644
--- a/agents.md
+++ b/agents.md
@@ -93,6 +93,17 @@ SLUG_GOOGLE_CLIENT_SECRET=mock
After OAuth completes, the pending-session poll returns a `slug_…` bearer token for API calls.
+### Dev-only offline tooling
+
+**`sorterc`** — workspace binary, not published via npm. Compiles `.sorter` files and lints `events.jsonl` without a server:
+
+```
+cargo run -p sorterc -- compile path/to/doc.sorter [--base events.jsonl] [--room public] [--pretty]
+cargo run -p sorterc -- scan path/to/events.jsonl [--pretty]
+```
+
+`compile` validates DSL, simulates ingest against empty (or `--base`) reducer state, and prints JSON rankings. `scan` reports corrupt JSONL lines and ingests that fail DSL replay.
+
### Testing
- **Rust tests:** `cargo nextest run --workspace` (163 tests; requires `cargo-nextest`)
diff --git a/server/src/lib.rs b/server/src/lib.rs
index c1d477d21aea03aff00e6f0689b0b4379d0d68d2..ad8e31099c807fb5844acb16cd5086a2f19327a7 100644
--- a/server/src/lib.rs
+++ b/server/src/lib.rs
@@ -10,6 +10,7 @@ pub mod form_template;
pub mod html;
pub mod identity;
pub mod middleware;
+pub mod offline;
pub mod path_types;
pub mod ranking;
pub mod reducer;
diff --git a/server/src/offline.rs b/server/src/offline.rs
new file mode 100644
index 0000000000000000000000000000000000000000..54ad0ded096a305ef8454ab2cdd1c3af71b14f5d
--- /dev/null
+++ b/server/src/offline.rs
@@ -0,0 +1,333 @@
+//! Offline `.sorter` compilation and JSONL diagnostics (no network, no auth).
+
+use std::collections::HashSet;
+use std::path::Path;
+
+use serde::Serialize;
+use slug_types::{CheckScopeRanking, RankComponent, RankRow, paths::GardenItemUrl};
+
+use crate::{
+ api::{resolve_item, validate_ingest_document},
+ dsl,
+ events::{Event, Ingest},
+ path_types::ItemId,
+ reducer::{ReducerState, ScopeId, scope_from_room_wire},
+ scope_rank::build_children_rankings,
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CompileStats {
+ pub items: usize,
+ pub votes: usize,
+ pub prose_blocks: usize,
+}
+
+#[derive(Debug, Serialize)]
+pub struct CompileResult {
+ pub ok: bool,
+ pub threads: Vec<String>,
+ pub rankings: Vec<CheckScopeRanking>,
+ pub stats: CompileStats,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CompileError {
+ pub ok: bool,
+ pub error: String,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub hint: Option<String>,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct BadJsonLine {
+ pub line: usize,
+ pub message: String,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct MalformedIngest {
+ pub line: usize,
+ pub id: String,
+ pub room_id: String,
+ pub thread_tag: String,
+ pub reason: String,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct ScanResult {
+ pub ok: bool,
+ pub path: String,
+ pub total_lines: usize,
+ pub parsed_events: usize,
+ pub bad_json_lines: Vec<BadJsonLine>,
+ pub malformed_ingests: Vec<MalformedIngest>,
+ pub skipped_ingests: usize,
+}
+
+fn document_stats(doc: &dsl::Document) -> CompileStats {
+ let mut items = 0usize;
+ let mut votes = 0usize;
+ let mut prose_blocks = 0usize;
+ for stmt in &doc.statements {
+ match stmt {
+ dsl::Stmt::Item { .. } => items += 1,
+ dsl::Stmt::Vote { .. } => votes += 1,
+ dsl::Stmt::Prose { .. } => prose_blocks += 1,
+ }
+ }
+ CompileStats {
+ items,
+ votes,
+ prose_blocks,
+ }
+}
+
+fn threads_in_document(text: &str) -> Vec<String> {
+ let mut out = HashSet::new();
+ for line in text.lines() {
+ let trimmed = line.trim();
+ if !trimmed.starts_with('#') {
+ continue;
+ }
+ let rest = trimmed.trim_start_matches('#').trim();
+ if rest.is_empty() {
+ continue;
+ }
+ let tag = rest.split_whitespace().next().unwrap_or(rest);
+ let tag = tag.split(':').next().unwrap_or(tag).trim();
+ if tag.is_empty() {
+ continue;
+ }
+ out.insert(format!("#{}", crate::canonical_path::canonicalize_tag(tag)));
+ }
+ let mut tags: Vec<String> = out.into_iter().collect();
+ tags.sort();
+ tags
+}
+
+fn voted_parent_scopes(doc: &dsl::Document) -> Vec<ItemId> {
+ let mut parents = HashSet::new();
+ for stmt in &doc.statements {
+ if let dsl::Stmt::Vote { item1, item2, .. } = stmt {
+ if let (Ok(a), Ok(b)) = (resolve_item(item1), resolve_item(item2)) {
+ if let Some(p) = a.parent() {
+ parents.insert(p);
+ }
+ if let Some(p) = b.parent() {
+ parents.insert(p);
+ }
+ }
+ }
+ }
+ let mut out: Vec<ItemId> = parents.into_iter().collect();
+ out.sort();
+ out
+}
+
+fn rankings_for_simulated(
+ simulated: &ReducerState,
+ scope: &ScopeId,
+ room_wire: &str,
+ doc: &dsl::Document,
+) -> Vec<CheckScopeRanking> {
+ voted_parent_scopes(doc)
+ .iter()
+ .map(|parent| {
+ let scoped_content = simulated
+ .content_for_scope(&scope)
+ .unwrap_or_else(|| simulated.public());
+ let scoped = build_children_rankings(scoped_content, parent);
+ let components: Vec<RankComponent> = scoped
+ .component_rankings
+ .into_iter()
+ .map(|comp| RankComponent {
+ pairs: comp.pairs,
+ ranking: comp
+ .ranked
+ .into_iter()
+ .map(|r| RankRow {
+ item: GardenItemUrl::from_stored(&r.item, room_wire),
+ score: r.score,
+ percent: None,
+ })
+ .collect(),
+ })
+ .collect();
+ CheckScopeRanking {
+ parent: GardenItemUrl::from_stored(parent, room_wire).into_inner(),
+ components,
+ unranked_items: scoped
+ .unranked_items
+ .into_iter()
+ .map(|it| GardenItemUrl::from_stored(&it, room_wire))
+ .collect(),
+ }
+ })
+ .collect()
+}
+
+/// Validate and simulate one `.sorter` document against optional base reducer state.
+pub fn compile_document(
+ base: &ReducerState,
+ room: &str,
+ text: &str,
+) -> Result<CompileResult, CompileError> {
+ let room_key = room.trim();
+ let scope = scope_from_room_wire(room_key);
+ let validated = validate_ingest_document(base, text, &scope).map_err(|(_, message, hint)| {
+ CompileError {
+ ok: false,
+ error: message,
+ hint,
+ }
+ })?;
+
+ let event = Event::Ingest(Ingest {
+ ts: validated.ts,
+ id: uuid::Uuid::new_v4().to_string(),
+ raw: validated.raw_text.clone(),
+ principal: "offline".to_string(),
+ delegate: None,
+ room_id: room_key.to_string(),
+ thread_tag: "offline".to_string(),
+ });
+
+ let mut simulated = base.clone();
+ simulated.apply_event(event);
+
+ Ok(CompileResult {
+ ok: true,
+ threads: threads_in_document(text),
+ rankings: rankings_for_simulated(&simulated, &scope, room_key, &validated.doc),
+ stats: document_stats(&validated.doc),
+ })
+}
+
+fn ingest_parse_error(raw: &str) -> Option<String> {
+ dsl::parse_full(raw).err().map(|e| e.to_string())
+}
+
+fn load_events_from_jsonl(path: &Path) -> Result<(Vec<(usize, Event)>, Vec<BadJsonLine>), std::io::Error> {
+ let text = std::fs::read_to_string(path)?;
+ let mut events = Vec::new();
+ let mut bad_json_lines = Vec::new();
+ for (idx, line) in text.lines().enumerate() {
+ let line_no = idx + 1;
+ let trimmed = line.trim();
+ if trimmed.is_empty() {
+ continue;
+ }
+ match serde_json::from_str::<Event>(trimmed) {
+ Ok(ev) => events.push((line_no, ev)),
+ Err(e) => bad_json_lines.push(BadJsonLine {
+ line: line_no,
+ message: e.to_string(),
+ }),
+ }
+ }
+ Ok((events, bad_json_lines))
+}
+
+/// Replay a JSONL event log into reducer state (same rules as server boot).
+pub fn load_reducer_from_jsonl(path: &Path) -> Result<(ReducerState, Vec<BadJsonLine>), std::io::Error> {
+ let (events, bad_json_lines) = load_events_from_jsonl(path)?;
+ let mut state = ReducerState::default();
+ for (_line_no, ev) in events {
+ state.apply_event(ev);
+ }
+ Ok((state, bad_json_lines))
+}
+
+/// Scan an events.jsonl for corrupt JSON lines and ingests that fail DSL replay.
+pub fn scan_jsonl(path: &Path) -> Result<ScanResult, std::io::Error> {
+ let text = std::fs::read_to_string(path)?;
+ let total_lines = text.lines().count();
+ let (events, bad_json_lines) = load_events_from_jsonl(path)?;
+
+ let mut malformed_ingests = Vec::new();
+ let mut skipped_ingests = 0usize;
+ let mut state = ReducerState::default();
+ let parsed_events = events.len();
+
+ for (line_no, ev) in events {
+ if let Event::Ingest(ref ing) = ev {
+ if let Some(reason) = ingest_parse_error(&ing.raw) {
+ malformed_ingests.push(MalformedIngest {
+ line: line_no,
+ id: ing.id.clone(),
+ room_id: ing.room_id.clone(),
+ thread_tag: ing.thread_tag.clone(),
+ reason,
+ });
+ }
+ let before = state.ingests_by_id.len();
+ state.apply_event(ev);
+ if state.ingests_by_id.len() == before {
+ skipped_ingests += 1;
+ }
+ } else {
+ state.apply_event(ev);
+ }
+ }
+
+ let ok = bad_json_lines.is_empty() && malformed_ingests.is_empty() && skipped_ingests == 0;
+
+ Ok(ScanResult {
+ ok,
+ path: path.display().to_string(),
+ total_lines,
+ parsed_events,
+ bad_json_lines,
+ malformed_ingests,
+ skipped_ingests,
+ })
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ const TUTORIAL: &str = include_str!("../tests/fixtures/tutorial.sorter");
+
+ #[test]
+ fn compile_tutorial_fixture_emits_rankings() {
+ let result = compile_document(&ReducerState::default(), "public", TUTORIAL).unwrap();
+ assert!(result.ok);
+ assert!(!result.threads.is_empty());
+ assert!(result.stats.items >= 6);
+ assert!(result.stats.votes >= 6);
+ assert!(!result.rankings.is_empty());
+ }
+
+ #[test]
+ fn compile_rejects_vote_on_missing_item() {
+ let err = compile_document(
+ &ReducerState::default(),
+ "public",
+ "{ reason }\n~/missing/a 2:1 ~/missing/b",
+ )
+ .unwrap_err();
+ assert!(!err.ok);
+ assert!(err.error.contains("undefined"));
+ }
+
+ #[test]
+ fn scan
… preview truncated; 7,950 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.