Side B fixes a real performance bug (scan no longer does full replay with rank centrality on every ingest), surfaces actual parse error details instead of a vague reason, and adds a genuinely useful new capability (compile --ingest for targeted replay debugging), backed by a new test. Side A is a UI refactor (grouping multiple cli_panel calls into one bordered panel) that's reasonable but purely cosmetic/organizational with no functional bugfix or capability gain of comparable weight.
constitution · epochs · watch · epoch 3
comparison
c_ca72f0995396 (tommy-mor) vs c_6209cd238b3f (tommy-mor)
download prompt · raw event · cmp_f24f795d3ec8c2
council reasoning
B fixes a real performance bottleneck (scan no longer full-replays every ingest through rank centrality), adds compile --ingest for targeted log replay, surfaces full DSL parse errors, and ships tests plus docs—lasting tooling value. A is solid UI polish (grouped click-to-copy CLI rows and JS-safety asserts) but is mostly presentation and call-site churn without comparable core capability.
Side B adds substantial offline tooling improvements: it refactors log replay, makes `scan` a fast parse-only pass, surfaces detailed `parse_error` information, and introduces `compile --ingest` to replay prior state and compile a single event from a log, with corresponding CLI, API, and test updates. Side A mainly improves the HTML CLI panel UX by grouping commands, enabling click-to-copy, and adding assertions that command strings are safe for embedding in single-quoted JavaScript, which is useful but has a narrower, presentation-focused impact.
sides
A — c_ca72f0995396 (tommy-mor)
message
[798c764d] feat(html): grouped cli_panel with hover-to-copy and JS-safe asserts Single bordered panel for multiple commands; rows copy on click without a separate copy control. Assert CLI strings contain no chars that would break single-quoted onclick JS. Made-with: Cursor
diff preview
diff --git a/server/src/html/forum.rs b/server/src/html/forum.rs
index f6e45b05bb92126966e304619f80ef1d45be7a4b..075860914a6ce18bb0dfa73dc5651ef1a6318b67 100644
--- a/server/src/html/forum.rs
+++ b/server/src/html/forum.rs
@@ -718,7 +718,7 @@ pub async fn home(
}
div id="public-new-thread-ui-slot" {}
(render_thread_feed(Some(&nav), "thread-feed", &public_rows, now))
- (cli_panel("npx slugsocial public forum list"))
+ (cli_panel(&["npx slugsocial public forum list"]))
},
None,
theme_from_jar(&jar),
@@ -868,7 +868,7 @@ async fn thread_view_inner(
div id="thread-live-region" {
(compose_form(&nav, &tag, show_compose))
}
- (cli_panel(&cli))
+ (cli_panel(std::slice::from_ref(&cli)))
},
None,
theme_from_jar(&jar),
@@ -984,9 +984,7 @@ pub async fn room_page(
(new_thread_form_for_room(&nav, true, false))
}
}
- (cli_panel(&forum_cli))
- (cli_panel(&garden_cli))
- (cli_panel(&audit_cli))
+ (cli_panel(&[forum_cli, garden_cli, audit_cli]))
},
None,
theme_from_jar(&jar),
@@ -1409,7 +1407,7 @@ pub async fn user_profile_page(
}
}
}
- (cli_panel(&format!("npx slugsocial public forum list")))
+ (cli_panel(&[format!("npx slugsocial public forum list")]))
},
None,
theme_from_jar(&jar),
diff --git a/server/src/html/garden.rs b/server/src/html/garden.rs
index 9b4789a79c3e293cbfc5f033a0eac8650320d94d..c8ce7de450f7d14e04020511e7eb7323bd487deb 100644
--- a/server/src/html/garden.rs
+++ b/server/src/html/garden.rs
@@ -139,7 +139,7 @@ pub async fn garden_index(
}
}
}
- (cli_panel("npx slugsocial garden tree"))
+ (cli_panel(&["npx slugsocial garden tree"]))
},
None,
theme_from_jar(&jar),
@@ -542,7 +542,7 @@ async fn render_scope_view(
ScopeId::Public => format!("npx slugsocial public garden body {}", path.as_str().trim_start_matches("https://slug.social/~/")),
ScopeId::Room(room_id) => format!("npx slugsocial private {room_id} garden body {}", path.as_str().trim_start_matches("https://slug.social/~/")),
};
- (cli_panel(&cli))
+ (cli_panel(std::slice::from_ref(&cli)))
},
None,
theme_from_jar(&jar),
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index e7f5bfb7a4b2dee2adf96447224c58d641092124..6617781a2e8e86c2e2693788ea7cd0eb0e3659a2 100644
--- a/server/src/html/mod.rs
+++ b/server/src/html/mod.rs
@@ -599,18 +599,38 @@ pub(super) fn render_linkified_with_embeds_in_scope(raw: &str, garden_prefix: &s
}
}
-/// Small CLI hint panel showing how to look up this page from the terminal.
-pub(super) fn cli_panel(cmd: &str) -> Markup {
+/// CLI strings are embedded in a single-quoted JS literal; they must never need escaping.
+fn assert_cli_panel_cmd_js_single_quote_safe(s: &str) {
+ assert!(
+ !s.contains('\\')
+ && !s.contains('\'')
+ && !s.contains('\n')
+ && !s.contains('\r'),
+ "cli_panel cmd must not contain `\\`, `'`, or newlines (got {s:?})"
+ );
+}
+
+/// Small CLI hint panel: one border and title; each line is hover-highlighted and copies on click.
+pub(super) fn cli_panel<I: AsRef<str>>(cmds: &[I]) -> Markup {
+ if cmds.is_empty() {
+ return html! {};
+ }
+ for cmd in cmds {
+ assert_cli_panel_cmd_js_single_quote_safe(cmd.as_ref());
+ }
html! {
div class="cli-panel" {
span class="cli-panel-label muted" { "cli" }
- code class="cli-panel-cmd" { (cmd) }
- button
- class="cli-panel-copy"
- title="Copy to clipboard"
- onclick=(format!(r#"navigator.clipboard.writeText('{}'); this.textContent='✓'; setTimeout(() => this.textContent='copy', 2000);"#, cmd.replace("'", "\\'")))
- {
- "copy"
+ div class="cli-panel-cmds" {
+ @for cmd in cmds {
+ @let s = cmd.as_ref();
+ button type="button" class="cli-panel-row" title="Copy command" onclick=(format!(
+ r#"navigator.clipboard.writeText('{}');"#,
+ s
+ )) {
+ code class="cli-panel-cmd" { (s) }
+ }
+ }
}
}
}
diff --git a/server/src/html/search.rs b/server/src/html/search.rs
index 28ceaa53c3fcac6777311535e95fb771b19438f5..43e6ebf36cf0fe72c96f0f9d850bea51ac094c43 100644
--- a/server/src/html/search.rs
+++ b/server/src/html/search.rs
@@ -418,7 +418,7 @@ pub async fn search_page(
value=(query) autocomplete="off" autofocus;
}
(render_search_results(&results, &query))
- (cli_panel("npx slugsocial search <query>"))
+ (cli_panel(&["npx slugsocial search <query>"]))
},
None,
theme_from_jar(&jar),
diff --git a/server/static/theme_default.css b/server/static/theme_default.css
index e024a5c8b74139ae8be37b2a1bd17e4c3abe9324..764b66c8208e91e6138b83c534387cf43c85b8b5 100644
--- a/server/static/theme_default.css
+++ b/server/static/theme_default.css
@@ -610,7 +610,7 @@ code {
CLI PANEL — how to view this page from the terminal
---------------------------------------------------------------- */
div.cli-panel {
- align-items: baseline;
+ align-items: flex-start;
background: var(--g1);
border: var(--bv) solid;
border-color: var(--lo) var(--hi) var(--hi) var(--lo); /* inset */
@@ -621,11 +621,34 @@ div.cli-panel {
width: fit-content;
max-width: 100%;
}
+.cli-panel-cmds {
+ display: flex;
+ flex-direction: column;
+ gap: 4px;
+ flex: 1;
+ min-width: 0;
+}
+button.cli-panel-row {
+ background: transparent;
+ border: none;
+ color: inherit;
+ cursor: pointer;
+ display: block;
+ font: inherit;
+ margin: 0;
+ padding: 2px 4px;
+ text-align: left;
+ width: 100%;
+}
+button.cli-panel-row:hover {
+ background: var(--g3);
+}
.cli-panel-label {
font-size: 11px;
letter-spacing: 0.08em;
text-transform: uppercase;
flex-shrink: 0;
+ padding-top: 2px;
}
.cli-panel-cmd {
background: none;
diff --git a/server/static/theme_retro_craft.css b/server/static/theme_retro_craft.css
index 00714575bfa533d1d9c66653b9089642e2b0a6ca..f89ecbc3a18eb9b2b27d1f7764330f6bd6987552 100644
--- a/server/static/theme_retro_craft.css
+++ b/server/static/theme_retro_craft.css
@@ -306,19 +306,41 @@ a.post-nav-btn:hover {
}
div.cli-panel {
- align-items: baseline;
+ align-items: flex-start;
border: 1px dashed var(--line);
display: flex;
- flex-wrap: wrap;
gap: 0.5rem;
margin: 0.65rem 0;
padding: 0.45rem 0.65rem;
}
+.cli-panel-cmds {
+ display: flex;
+ flex-direction: column;
+ gap: 0.25rem;
+ flex: 1;
+ min-width: 0;
+}
+button.cli-panel-row {
+ background: transparent;
+ border: none;
+ color: inherit;
+ cursor: pointer;
+ display: block;
+ font: inherit;
+ margin: 0;
+ padding: 0.1rem 0.2rem;
+ text-align: left;
+ width: 100%;
+}
+button.cli-panel-row:hover {
+ background: color-mix(in srgb, var(--accent) 14%, transparent);
+}
.cli-panel-label {
color: var(--ink-dim);
font-size: 0.72rem;
letter-spacing: 0.12em;
text-transform: uppercase;
+ padding-top: 0.12rem;
}
.cli-panel-cmd {
color: var(--accent);
B — c_6209cd238b3f (tommy-mor)
message
[6e344666] Improve sorterc scan speed, errors, and ingest compile. Make scan a fast DSL parse pass with human-readable output, surface full parse_error details, and add compile --ingest for single-event replay from a log. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/server/src/offline.rs b/server/src/offline.rs
index 54ad0ded096a305ef8454ab2cdd1c3af71b14f5d..2db6f67e22e00a24ce673d13095644dd9fa9342d 100644
--- a/server/src/offline.rs
+++ b/server/src/offline.rs
@@ -28,6 +28,10 @@ pub struct CompileResult {
pub threads: Vec<String>,
pub rankings: Vec<CheckScopeRanking>,
pub stats: CompileStats,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub ingest_id: Option<String>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub ingest_line: Option<usize>,
}
#[derive(Debug, Clone, Serialize)]
@@ -36,6 +40,8 @@ pub struct CompileError {
pub error: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub hint: Option<String>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub parse_error: Option<String>,
}
#[derive(Debug, Clone, Serialize)]
@@ -50,7 +56,7 @@ pub struct MalformedIngest {
pub id: String,
pub room_id: String,
pub thread_tag: String,
- pub reason: String,
+ pub parse_error: String,
}
#[derive(Debug, Clone, Serialize)]
@@ -59,9 +65,36 @@ pub struct ScanResult {
pub path: String,
pub total_lines: usize,
pub parsed_events: usize,
+ pub ingest_events: usize,
pub bad_json_lines: Vec<BadJsonLine>,
pub malformed_ingests: Vec<MalformedIngest>,
- pub skipped_ingests: usize,
+}
+
+#[derive(Debug)]
+pub enum CompileIngestError {
+ NotFound(String),
+ Io(std::io::Error),
+ Compile(CompileError),
+}
+
+impl CompileIngestError {
+ pub fn into_compile_error(self) -> CompileError {
+ match self {
+ Self::NotFound(id) => CompileError {
+ ok: false,
+ error: format!("ingest not found: {id}"),
+ hint: Some("pass the ingest event id from events.jsonl".into()),
+ parse_error: None,
+ },
+ Self::Io(e) => CompileError {
+ ok: false,
+ error: format!("io error: {e}"),
+ hint: None,
+ parse_error: None,
+ },
+ Self::Compile(e) => e,
+ }
+ }
}
fn document_stats(doc: &dsl::Document) -> CompileStats {
@@ -166,19 +199,22 @@ fn rankings_for_simulated(
.collect()
}
-/// Validate and simulate one `.sorter` document against optional base reducer state.
-pub fn compile_document(
+fn compile_document_inner(
base: &ReducerState,
room: &str,
text: &str,
+ ingest_id: Option<String>,
+ ingest_line: Option<usize>,
) -> Result<CompileResult, CompileError> {
let room_key = room.trim();
let scope = scope_from_room_wire(room_key);
let validated = validate_ingest_document(base, text, &scope).map_err(|(_, message, hint)| {
+ let parse_error = dsl::parse_full(text).err().map(|e| e.to_string());
CompileError {
ok: false,
error: message,
hint,
+ parse_error,
}
})?;
@@ -200,15 +236,27 @@ pub fn compile_document(
threads: threads_in_document(text),
rankings: rankings_for_simulated(&simulated, &scope, room_key, &validated.doc),
stats: document_stats(&validated.doc),
+ ingest_id,
+ ingest_line,
})
}
+/// Validate and simulate one `.sorter` document against optional base reducer state.
+pub fn compile_document(
+ base: &ReducerState,
+ room: &str,
+ text: &str,
+) -> Result<CompileResult, CompileError> {
+ compile_document_inner(base, room, text, None, None)
+}
+
fn ingest_parse_error(raw: &str) -> Option<String> {
dsl::parse_full(raw).err().map(|e| e.to_string())
}
-fn load_events_from_jsonl(path: &Path) -> Result<(Vec<(usize, Event)>, Vec<BadJsonLine>), std::io::Error> {
+fn load_events_from_jsonl(path: &Path) -> Result<(usize, Vec<(usize, Event)>, Vec<BadJsonLine>), std::io::Error> {
let text = std::fs::read_to_string(path)?;
+ let total_lines = text.lines().count();
let mut events = Vec::new();
let mut bad_json_lines = Vec::new();
for (idx, line) in text.lines().enumerate() {
@@ -225,61 +273,90 @@ fn load_events_from_jsonl(path: &Path) -> Result<(Vec<(usize, Event)>, Vec<BadJs
}),
}
}
- Ok((events, bad_json_lines))
+ Ok((total_lines, events, bad_json_lines))
}
-/// Replay a JSONL event log into reducer state (same rules as server boot).
-pub fn load_reducer_from_jsonl(path: &Path) -> Result<(ReducerState, Vec<BadJsonLine>), std::io::Error> {
- let (events, bad_json_lines) = load_events_from_jsonl(path)?;
+fn replay_events(events: &[(usize, Event)]) -> ReducerState {
let mut state = ReducerState::default();
for (_line_no, ev) in events {
- state.apply_event(ev);
+ state.apply_event(ev.clone());
}
- Ok((state, bad_json_lines))
+ state
}
-/// Scan an events.jsonl for corrupt JSON lines and ingests that fail DSL replay.
-pub fn scan_jsonl(path: &Path) -> Result<ScanResult, std::io::Error> {
- let text = std::fs::read_to_string(path)?;
- let total_lines = text.lines().count();
- let (events, bad_json_lines) = load_events_from_jsonl(path)?;
+/// Replay a JSONL event log into reducer state (same rules as server boot).
+pub fn load_reducer_from_jsonl(path: &Path) -> Result<(ReducerState, Vec<BadJsonLine>), std::io::Error> {
+ let (_total_lines, events, bad_json_lines) = load_events_from_jsonl(path)?;
+ Ok((replay_events(&events), bad_json_lines))
+}
- let mut malformed_ingests = Vec::new();
- let mut skipped_ingests = 0usize;
- let mut state = ReducerState::default();
- let parsed_events = events.len();
+/// Find one ingest in a log and compile it against all prior events as base state.
+pub fn compile_ingest_from_log(path: &Path, ingest_id: &str) -> Result<CompileResult, CompileIngestError> {
+ let (_total_lines, events, bad_json_lines) = load_events_from_jsonl(path).map_err(CompileIngestError::Io)?;
+ if !bad_json_lines.is_empty() {
+ return Err(CompileIngestError::Compile(CompileError {
+ ok: false,
+ error: format!("jsonl has {} corrupt line(s)", bad_json_lines.len()),
+ hint: Some("fix the log or use `sorterc scan`".into()),
+ parse_error: None,
+ }));
+ }
+
+ let needle = ingest_id.trim();
+ let mut found: Option<(usize, Ingest)> = None;
+ let mut prior: Vec<(usize, Event)> = Vec::new();
for (line_no, ev) in events {
if let Event::Ingest(ref ing) = ev {
- if let Some(reason) = ingest_parse_error(&ing.raw) {
+ if ing.id == needle {
+ found = Some((line_no, ing.clone()));
+ break;
+ }
+ }
+ prior.push((line_no, ev));
+ }
+
+ let (line_no, ing) = found.ok_or_else(|| CompileIngestError::NotFound(needle.to_string()))?;
+ let base = replay_events(&prior);
+ compile_document_inner(&base, &ing.room_id, &ing.raw, Some(ing.id.clone()), Some(line_no))
+ .map_err(CompileIngestError::Compile)
+}
+
+/// Scan an events.jsonl for corrupt JSON lines and ingests whose DSL fails to parse.
+///
+/// This does not replay the log (which would run rank centrality on every ingest and
+/// can take minutes on real logs). It matches what the server skips on boot: parse failure.
+pub fn scan_jsonl(path: &Path) -> Result<ScanResult, std::io::Error> {
+ let (total_lines, events, bad_json_lines) = load_events_from_jsonl(path)?;
+
+ let mut malformed_ingests = Vec::new();
+ let mut ingest_events = 0usize;
+
+ for (line_no, ev) in &events {
+ if let Event::Ingest(ing) = ev {
+ ingest_events += 1;
+ if let Some(parse_error) = ingest_parse_error(&ing.raw) {
malformed_ingests.push(MalformedIngest {
- line: line_no,
+ line: *line_no,
id: ing.id.clone(),
room_id: ing.room_id.clone(),
thread_tag: ing.thread_tag.clone(),
- reason,
+ parse_error,
});
}
- let before = state.ingests_by_id.len();
- state.apply_event(ev);
- if state.ingests_by_id.len() == before {
- skipped_ingests += 1;
- }
- } else {
- state.apply_event(ev);
}
}
- let ok = bad_json_lines.is_empty() && malformed_ingests.is_empty() && skipped_ingests == 0;
+ let ok = bad_json_lines.is_empty() && malformed_ingests.is_empty();
Ok(ScanResult {
ok,
path: path.display().to_string(),
total_lines,
- parsed_events,
+ parsed_events: events.len(),
+ ingest_events,
bad_json_lines,
malformed_ingests,
- skipped_ingests,
})
}
@@ -330,4 +407,25 @@ mod tests {
assert!(!report.ok);
assert_eq!(report.bad_json_lines.len(), 1);
}
+
+ #[test]
+ fn scan_reports_dsl_parse_error_detail() {
+ let dir = tempfile::tempdir().unwrap();
+ let path = dir.path().join("events.jsonl");
+ let ingest = serde_json::json!({
+ "type": "ingest",
+ "ts": 1,
+ "id": "bad-ingest-id",
+ "raw": "{ no closing brace\n~/a { body }\n~/a 1:0 ~/b",
+ "principal": "test",
+ "room_id": "public",
+ "thread_tag": "t",
+ });
+ std::fs::write(&path, format!("{ingest}\n")).unwrap();
+ let report = scan_jsonl(&path).unwrap();
+ assert!(!report.ok);
+ assert_eq!(report.malformed_ingests.len(), 1);
+ assert_eq!(report.malformed_ingests[0].id, "bad-ingest-id");
+ assert!(report.malformed_ingests[0].parse_error.contains("parse error"));
+ }
}
diff --git a/sorterc/readme.md b/sorterc/readme.md
index 1ebcc3fc935541ea9e47e0458ec67a750fe19fff..e388615c34b40914ca51fc7a79cb738eebc2909b 100644
--- a/sorterc/readme.md
+++ b/sorterc/readme.md
@@ -60,21 +60,33 @@ Rankings use the same structure as the server's dry-run check: parent scope, con
### `scan` — lint an `events.jsonl`
-Reads a JSONL event log and reports problems without starting a server.
+Fast single-pass check. Does **not** replay the log (replay runs rank centrality on every ingest and gets slow fast).
```bash
cargo run -p sorterc -- scan events.jsonl
-cargo run -p sorterc -- scan events.jsonl --pretty
+cargo run -p sorterc -- scan events.jsonl --json
+cargo run -p sorterc -- scan events.jsonl --json --pretty
```
+Default output is human-readable with a blank line between each problem. Use `--json` for machine output.
+
Reports:
- **bad JSON lines** — lines that are not valid JSON
-- **malformed ingests** — ingest events whose `raw` DSL fails to parse
-- **skipped ingests** — ingests dropped during replay (same behavior as server boot)
+- **malformed ingests** — ingest events whose `raw` DSL fails to parse, with full `parse_error` text
Exits 0 when clean, 1 when any issue is found.
+### `compile --ingest` — compile one event from a log
+
+Replay all events **before** the target ingest as base state, then compile that ingest's DSL:
+
+```bash
+cargo run -p sorterc -- compile --ingest cabd8adc-57ae-402d-a940-8e24339ac451 --from events.jsonl --pretty
+```
+
+Output includes `ingest_id`, `ingest_line`, and rankings for that post only. This may take a while for ingests late in a large log (full replay up to that point).
+
## Typical uses
- Iterate on `.sorter` files in an editor and pipe through `compile` to see rankings instantly
diff --git a/sorterc/src/main.rs b/sorterc/src/main.rs
index 71382c180085cb0ad71043c852f8db5d3a48a284..5687c850d16973d28749380b8dc89f3bcebbfc56 100644
--- a/sorterc/src/main.rs
+++ b/sorterc/src/main.rs
@@ -22,9 +22,15 @@ struct Cli {
enum Command {
/// Parse and simulate a .sorter document; emit ranking JSON to stdout.
Compile {
- /// `.sorter` file, or `-` f
… preview truncated; 6,030 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.