Commit B introduces a substantial architectural change around URL canonicalization. It adds a new URL normalization subsystem (`url_rules`), integrates it into `ItemId`, updates parsing, parent/breadcrumb logic, Reddit URL handling, projection application, state validation, and adds a new dependency plus extensive test updates. It also changes the project's canonical identifier format to full HTTPS URLs and touches many core components. By contrast, commit A is a tiny HTML tweak that changes a disabled paginator control from a `<span>` to an anchor with `href="#"`, a localized UI adjustment with minimal impact.
constitution · epochs · watch · epoch 3
comparison
c_bd76779bd7ed (tommy-mor) vs c_77729db919ab (tommy-mor)
download prompt · raw event · cmp_665134c0945078
council reasoning
Commit B introduces a comprehensive URL canonicalization system with new modules, dependency additions, and widespread changes across parsing, storage, tests, and Reddit integration logic. It fundamentally alters how IDs are handled and normalized throughout the codebase. Commit A is a minor HTML tweak changing a disabled span to a link, with negligible impact in comparison.
Commit B introduces a comprehensive URL canonicalization system (new url_rules module with engine and registry), adds a new dependency, refactors ItemId semantics across the codebase to use full canonical URLs, updates parsing, parent/breadcrumb logic, Reddit/YouTube normalization, event application, and adjusts extensive tests and state handling accordingly. It meaningfully changes core identity handling and architecture. Commit A is a trivial HTML change (span to disabled anchor). Therefore B contributes vastly more.
sides
A — c_bd76779bd7ed (tommy-mor)
message
[22b97b71] type
diff preview
diff --git a/server/src/html/forum.rs b/server/src/html/forum.rs
index f1114562c8c0196f78a008028b46d1fda57479f9..493b7596a1d334b174ee9916571446d14507ec97 100644
--- a/server/src/html/forum.rs
+++ b/server/src/html/forum.rs
@@ -665,7 +665,7 @@ fn render_thread_paginator(nav: &ThreadNav, tag: &str, offset: usize, total: usi
@if let Some(o) = newer_offset {
a href=(nav.thread_page_url(tag, o)) class="post-nav-btn" { "newer →" }
} @else {
- span class="post-nav-btn disabled" { "newer →" }
+ a href="#" class="post-nav-btn disabled" { "newer →" }
}
@if !on_latest {
a href=(nav.thread_page_url(tag, latest_offset)) class="post-nav-btn" { "latest" }
B — c_77729db919ab (tommy-mor)
message
[239c074b] url schema stuff
diff preview
diff --git a/AGENTS.md b/AGENTS.md
index 426a88e7c1da54fe0a28c5c76fa4e1f1bc117fcf..e60b9ba6012593361ef10e8fdd9439cd9932e09b 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -58,3 +58,4 @@ Use **tmux** for `cargo run --package sorter2-server` (dev server). Rebuild afte
- First `cargo test` / `cargo build --release` is slow; Clojure smoke test always does a release build.
- `legacy/` and `ideas/` are not part of the workspace build.
+- **ItemId** for web URLs is a canonical full URL (`https://reddit.com/r/rust`). Rules live in [`server/src/url_rules/`](server/src/url_rules/) (composable Rust, not a config DSL). After changing canonicalization rules, rebuild the projection: `cargo run --package sorter2-server -- replay-index`.
diff --git a/Cargo.lock b/Cargo.lock
index 0dd4fce5fb6400ae153cca4e3dbf5a5158e6d8b4..49a908ef935c430dbe63c6a28d8a24e38b489486 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -1951,6 +1951,7 @@ dependencies = [
"tower-http 0.5.2",
"tracing",
"tracing-subscriber",
+ "url",
"urlencoding",
]
diff --git a/REPLAY.sh b/REPLAY.sh
new file mode 100755
index 0000000000000000000000000000000000000000..f2dbd8aea60c02d2feef74805f7ef5c2b7022537
--- /dev/null
+++ b/REPLAY.sh
@@ -0,0 +1,2 @@
+cargo run --package sorter2-server -- replay-index
+
diff --git a/server/Cargo.toml b/server/Cargo.toml
index 27f552c20b97ef28cdde4cb6b1a4980375135111..ad4912791aff59fb1d3293f66ad381ae618cd60b 100644
--- a/server/Cargo.toml
+++ b/server/Cargo.toml
@@ -24,6 +24,7 @@ async-stream = "0.3"
futures-util = { version = "0.3", default-features = false, features = ["std"] }
rand = "0.8"
urlencoding = "2"
+url = "2"
durable = { path = "../durable" }
[dev-dependencies]
diff --git a/server/src/entity_store.rs b/server/src/entity_store.rs
index d5d17c3676e4a8ddec998e9f5a9dbafe9c2d9d0e..d29f39aecca6f12cdcf263cf77c3654eb4ee6cfa 100644
--- a/server/src/entity_store.rs
+++ b/server/src/entity_store.rs
@@ -124,7 +124,7 @@ mod tests {
fn round_trip_payload() {
let tmp = tempfile::tempdir().unwrap();
let store = EntityStore::open(tmp.path()).unwrap();
- let id = ItemId::parse("reddit.com/r/rust").unwrap();
+ let id = ItemId::from_url("https://reddit.com/r/rust").unwrap();
let payload = json!({"kind": "t5", "data": {"display_name": "rust"}});
store.put(&id, &payload).unwrap();
diff --git a/server/src/event_log.rs b/server/src/event_log.rs
index 36f5b406084065b608735987cdb483c236e03081..2c9290b6fdbf2c2ad1c0f1ffd7374b2d9cc97f36 100644
--- a/server/src/event_log.rs
+++ b/server/src/event_log.rs
@@ -199,7 +199,7 @@ mod tests {
log.append(&sample_record(
1,
Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
},
))
.await
@@ -237,7 +237,7 @@ mod tests {
let path = tmp.path().join("events.jsonl");
let log = EventLog::new(&path);
let event = Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
};
log.append(&sample_record(1, event)).await.unwrap();
@@ -255,7 +255,7 @@ mod tests {
let path = tmp.path().join("events.jsonl");
std::fs::write(
&path,
- r#"{"type":"node_ensured","id":"reddit.com/r/rust"}
+ r#"{"type":"node_ensured","id":"https://reddit.com/r/rust"}
{"schema":1,"seq":1,"ts":1,"event":{"type":"vote_recorded","ts":1,"a":"a","b":"b","ratio_left":2,"ratio_right":1,"scope":""}}
"#,
)
@@ -295,7 +295,7 @@ mod tests {
log.append(&sample_record(
1,
Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
},
))
.await
@@ -303,7 +303,7 @@ mod tests {
log.append(&sample_record(
3,
Event::NodeEnsured {
- id: "reddit.com/r/python".into(),
+ id: "https://reddit.com/r/python".into(),
},
))
.await
diff --git a/server/src/journal.rs b/server/src/journal.rs
index 521a108019de1ea870d14c4fafbfe572c20ce0de..50bc89f976edb82b7b0e49e954a8eccbbe82bf87 100644
--- a/server/src/journal.rs
+++ b/server/src/journal.rs
@@ -141,10 +141,10 @@ mod tests {
let j2 = journal.clone();
let (r1, r2) = tokio::join!(
j1.append(Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
}),
j2.append(Event::NodeEnsured {
- id: "reddit.com/r/python".into(),
+ id: "https://reddit.com/r/python".into(),
}),
);
r1.unwrap();
@@ -153,10 +153,10 @@ mod tests {
assert_eq!(projection_store.last_applied_event_count().unwrap(), 2);
let tree = projection_store.load_tree().unwrap();
assert!(tree
- .get(&ItemId::parse("reddit.com/r/rust").unwrap())
+ .get(&ItemId::parse("https://reddit.com/r/rust").unwrap())
.is_some());
assert!(tree
- .get(&ItemId::parse("reddit.com/r/python").unwrap())
+ .get(&ItemId::parse("https://reddit.com/r/python").unwrap())
.is_some());
}
@@ -170,7 +170,7 @@ mod tests {
1,
1,
Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
},
))
.await
@@ -186,7 +186,7 @@ mod tests {
1,
1,
Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
},
)],
)
@@ -202,7 +202,7 @@ mod tests {
);
journal
.append(Event::NodeEnsured {
- id: "reddit.com/r/python".into(),
+ id: "https://reddit.com/r/python".into(),
})
.await
.unwrap();
@@ -227,13 +227,13 @@ mod tests {
journal
.append_many(vec![
Event::NodeEnsured {
- id: "reddit.com/r/rust".into(),
+ id: "https://reddit.com/r/rust".into(),
},
Event::NodeEnsured {
- id: "reddit.com/r/python".into(),
+ id: "https://reddit.com/r/python".into(),
},
Event::NodeEnsured {
- id: "reddit.com/r/clojure".into(),
+ id: "https://reddit.com/r/clojure".into(),
},
])
.await
@@ -245,7 +245,7 @@ mod tests {
assert_eq!(projection_store.last_applied_event_count().unwrap(), 3);
let tree = projection_store.load_tree().unwrap();
assert!(tree
- .get(&ItemId::parse("reddit.com/r/clojure").unwrap())
+ .get(&ItemId::parse("https://reddit.com/r/clojure").unwrap())
.is_some());
}
}
diff --git a/server/src/lib.rs b/server/src/lib.rs
index 9bd5f76fd1406b9b1be4c272f4ba8647edde2678..5c02c8e704e4664453bad75d819df8a067668176 100644
--- a/server/src/lib.rs
+++ b/server/src/lib.rs
@@ -9,6 +9,7 @@ pub mod journal;
pub mod pair;
pub mod parser;
pub mod path_types;
+pub mod url_rules;
pub mod projection_apply;
pub mod projection_store;
pub mod ranking;
diff --git a/server/src/pair.rs b/server/src/pair.rs
index 43f780ba6ea6ce1cdc2e1f4cbb252ba8a10684b9..815a97b80e3e9f348e0937a4f147f2862018edb0 100644
--- a/server/src/pair.rs
+++ b/server/src/pair.rs
@@ -381,42 +381,42 @@ mod tests {
#[test]
fn suggest_prefers_unvoted_pair() {
- let parent = ItemId::parse("reddit.com/r/rust").unwrap();
+ let parent = ItemId::parse("https://reddit.com/r/rust").unwrap();
let mut tree = seed_children(
&parent,
&[
- "reddit.com/r/rust/a",
- "reddit.com/r/rust/b",
- "reddit.com/r/rust/c",
+ "https://reddit.com/r/rust/a",
+ "https://reddit.com/r/rust/b",
+ "https://reddit.com/r/rust/c",
],
);
let vote =
- VoteData::from_recorded(1, "reddit.com/r/rust/a", "reddit.com/r/rust/b", 2, 1).unwrap();
+ VoteData::from_recorded(1, "https://reddit.com/r/rust/a", "https://reddit.com/r/rust/b", 2, 1).unwrap();
tree.apply_vote(&parent, vote);
let group = tree.get(&parent).unwrap().local_ranking.clone();
let pool = children_of(&tree, &parent);
let (l, r) = suggest_next_pair_in_pool(&group, &pool, None).unwrap();
- let voted_ab = (l.as_str() == "reddit.com/r/rust/a" && r.as_str() == "reddit.com/r/rust/b")
- || (l.as_str() == "reddit.com/r/rust/b" && r.as_str() == "reddit.com/r/rust/a");
+ let voted_ab = (l.as_str() == "https://reddit.com/r/rust/a" && r.as_str() == "https://reddit.com/r/rust/b")
+ || (l.as_str() == "https://reddit.com/r/rust/b" && r.as_str() == "https://reddit.com/r/rust/a");
assert!(!voted_ab);
}
#[test]
fn suggest_bridges_separate_components() {
- let parent = ItemId::parse("reddit.com/r/rust").unwrap();
+ let parent = ItemId::parse("https://reddit.com/r/rust").unwrap();
let mut tree = seed_children(
&parent,
&[
- "reddit.com/r/rust/a",
- "reddit.com/r/rust/b",
- "reddit.com/r/rust/c",
- "reddit.com/r/rust/d",
+ "https://reddit.com/r/rust/a",
+ "https://reddit.com/r/rust/b",
+ "https://reddit.com/r/rust/c",
+ "https://reddit.com/r/rust/d",
],
);
let ab =
- VoteData::from_recorded(1, "reddit.com/r/rust/a", "reddit.com/r/rust/b", 2, 1).unwrap();
+ VoteData::from_recorded(1, "https://reddit.com/r/rust/a", "https://reddit.com/r/rust/b", 2, 1).unwrap();
let cd =
- VoteData::from_recorded(2, "reddit.com/r/rust/c", "reddit.com/r/rust/d", 2, 1).unwrap();
+ VoteData::from_recorded(2, "https://reddit.com/r/rust/c", "https://reddit.com/r/rust/d", 2, 1).unwrap();
tree.apply_vote(&parent, ab);
tree.apply_vote(&parent, cd);
let group = tree.get(&parent).unwrap().local_ranking.clone();
@@ -424,37 +424,37 @@ mod tests {
let pair = suggest_next_pair_in_pool(&group, &pool, None).unwrap();
let chosen = pair_set(&pair);
let from_ab =
- chosen.contains("reddit.com/r/rust/a") || chosen.contains("reddit.com/r/rust/b");
+ chosen.contains("https://reddit.com/r/rust/a") || chosen.contains("https://reddit.com/r/rust/b");
let from_cd =
- chosen.contains("reddit.com/r/rust/c") || chosen.contains("reddit.com/r/rust/d");
+ chosen.contains("https://reddit.com/r/rust/c") || chosen.contains("https://reddit.com/r/rust/d");
assert!(from_ab && from_cd, "expected bridge pair, got {:?}", chosen);
}
#[test]
fn suggest_prefers_attach_over_isolate_pair_among_many_unranked() {
- let parent = ItemId::parse("reddit.com/r/rust").unwrap();
+ let parent = ItemId::parse("https://reddit.com/r/rust").unwrap();
let mut tree = seed_children(
&parent,
&[
- "reddit.com/r/rust/a",
- "reddit.com/r/rust/b",
- "reddit.com/r/rust/c",
- "reddit.com/r/rust/d",
- "reddit.com/r/rust/e",
+ "https://reddit.com/r/rust/a",
+ "https://reddit.com/r/rust/b",
+ "https://reddit.com/r/rust/c",
+ "https://reddit.com/r/
… preview truncated; 51,799 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.