Side B fixes a real tokio RwLock re-entrancy deadlock bug in the RPC handler, adds an integration test covering RoomCreate, and hardens babashka test infra (HTTP timeouts, log redirection to avoid pipe deadlocks) plus corrects test assertions to match actual stored username format. Side A is purely a planning document and a thin wrapper struct (RouteContext) that just delegates to existing ThreadNav methods without migrating any call sites, providing no functional change or bugfix.
constitution · epochs · watch · epoch 3
comparison
c_7ec67b9cef2c (tommy-mor) vs c_9608dc0d38ab (tommy-mor)
download prompt · raw event · cmp_57b66ee6ae85fc
council reasoning
B fixes a real Tokio RwLock deadlock by dropping read guards before nested lock acquires in RoomCreate/RoomGrant, plus regression coverage and test harness hang/timeout fixes. A mostly adds a planning doc and a thin RouteContext newtype that still delegates to ThreadNav/CanonicalItemUrl, so it stages work rather than landing lasting behavior.
Side B fixes a real concurrency bug by shortening the lifetime of `tokio::sync::RwLock` read guards before later `read().await`/`write().await` calls in `handle_rpc_batch`, preventing self-deadlock during room creation and grants. It also adds an integration test covering room creation and improves test infrastructure with timeouts and optional log redirection, whereas Side A mainly adds a migration plan document and a thin `RouteContext` wrapper around `ThreadNav` without changing application behavior.
sides
A — c_7ec67b9cef2c (tommy-mor)
message
[1c914c6e] stage set
diff preview
diff --git a/plan.md b/plan.md
new file mode 100644
index 0000000000000000000000000000000000000000..00d6867a1e0ed144a16a020ea037f685ce646c73
--- /dev/null
+++ b/plan.md
@@ -0,0 +1,155 @@
+# Plan: `ItemId` + `RouteContext` (identity vs hrefs)
+
+This document is for **the next agent** to continue the refactor without re-deriving context from chat. It supersedes ad-hoc notes: treat it as the checklist of record until the work lands and this file is deleted or trimmed.
+
+## Goal
+
+- **Identity** (what lives in the reducer graph, votes, indexes) becomes a **structural `ItemId` enum** in `slug-types`, not a canonical `String` / `CanonicalItemUrl` newtype.
+- **Presentation** (tilde / dash display, breadcrumbs) derives from `ItemId` via explicit methods, not string stripping.
+- **Routing** (browser `href`s for public vs room) goes through **`RouteContext`** (started in `server/src/html/routing.rs`) so Maud/handlers do not stitch `/r/…` vs `/~` ad hoc.
+
+**Non-goals for v1 of the migration:** backward-compatible JSONL or dual-read of old canonical strings in the event log (project has accepted breaking changes). If you reintroduce compat, document it here.
+
+## Current state (as of this plan)
+
+- **`CanonicalItemUrl`** (`types/src/paths.rs`): newtype around `String`; `parse` / `parent` / `display_path` / `tilde_tail` / etc. Reducer `ContentState`, `VoteData`, ranking, RPC, search, garden, breadcrumbs all use it or `String` keys derived from it.
+- **`ThreadNav`** (`server/src/html/forum/nav.rs`): encodes scope prefixes for threads and garden URLs; **`RouteContext`** now wraps `ThreadNav` (`server/src/html/routing.rs`, re-exported from `server/src/html/mod.rs`) but **most HTML still takes `&ThreadNav` directly** — migration incomplete.
+- **URL normalization** lives in `types/src/url_normalize.rs` + `canonicalize_item` / `finalize_external_identity_url` in `paths.rs` (YouTube, sorted query params, room path `room_route_segment` in `paths.rs`).
+- **Room HTTP paths** are `/r/{short}{slug}` (fused segment); wire **`room_id`** remains `short/slug` for RPC/events.
+
+## Target architecture
+
+### `ItemId` (types)
+
+Suggested shape (adjust after profiling `Ord` / `Hash` / serde size):
+
+```text
+ItemId::Root — tilde ontology root (today `SLUG_TILDE_ONTOLOGY_ROOT`)
+ItemId::Local { segments } — slug.social ~/… path as Vec<String> (lowercase segments, non-empty for non-root)
+ItemId::External { url: Url } — normalized `url::Url` (crate `url` already in `slug-types`)
+```
+
+**API surface (minimum):**
+
+- `ItemId::parse(&str) -> Option<ItemId>` — single entry from DSL / user input / legacy wire (internally may call `canonicalize_item` + structured split).
+- `ItemId::to_wire_url(&self) -> String` — only for **external** boundaries if needed (HTTP fetch, rare assertions); avoid using as the primary key once maps use `ItemId`.
+- `parent`, `display_path`, `tilde_tail` / `tilde_http_tail`, `tilde_segments`, `last_segment`, `normalized_storage` — port from `CanonicalItemUrl`.
+- **`Ord` + `Hash` + `Eq`** stable for `BTreeSet` / `HashMap` (see `write_actor` scope-rank snapshots).
+- **`Serialize` / `Deserialize`** — decide **tagged JSON** for any persisted or API-carried structs (e.g. `VoteData` in tests). If RPC must stay stringy for clients, use a **DTO layer** that converts `ItemId` ↔ wire at the boundary only.
+
+**Remove:** `CanonicalItemUrl` type and all `path_types::CanonicalItemUrl` / `slug_types::paths::CanonicalItemUrl` exports once call sites are migrated. **`Borrow<str>`** on the old newtype goes away; update `nav!` / any code that assumed map keys borrowed as `str`.
+
+### `RouteContext` (server HTML)
+
+- **File:** `server/src/html/routing.rs` — **`RouteContext(ThreadNav)`** with `item_href`, `item_href_raw`, `thread_url`, `garden_root_url`, `room_url`, `From`/`Into` `ThreadNav`.
+- **Direction:** new code and refactored Maud should take **`&RouteContext`** (or owned where appropriate) instead of `&ThreadNav` when building links. Long term, **`item_href(&ItemId)`** should not parse strings — it should pattern-match `ItemId` and append tilde tail or `/-/…` external tail using the same rules as today’s `ThreadNav::garden_item_url`.
+
+### Axum / garden routes
+
+- **No** single catch-all route (explicit decision): keep the existing router layout in `server/src/lib.rs`.
+- Room routes stay **`/r/:room_key/...`** with `room_key` fused; parsing via `slug_types::room_id_from_route_segment` / `room_route_segment` in `paths.rs`.
+
+## Phased execution (recommended order)
+
+### Phase 0 — Preconditions (quick)
+
+1. Read **`AGENTS.md`** (UI contract, durability matrix, `RpcCommand` vs `HtmlUiAction`).
+2. Run **`cargo test --workspace`** and **`./scripts/clj-test.sh`** on clean `main` before large diffs; repeat after each phase.
+
+### Phase 1 — `ItemId` in `slug-types` (no server yet)
+
+1. Add **`ItemId`** (new file e.g. `types/src/item_id.rs` **or** inline at bottom of `paths.rs` — see **Module cycle** below).
+2. Implement **`ItemId::parse`** using existing **`canonicalize_item`** + normalization; port **`CanonicalItemUrl`** methods to **`ItemId`** with tests ported from `paths.rs` `#[cfg(test)] mod tests`.
+3. **`GardenItemUrl::from_stored(&ItemId, room_wire)`** (and thread helpers) — build absolute hrefs from structure, not from re-parsing a canonical string.
+4. **`TildeHttpPathTail::to_item_id`** (rename from `to_canonical`) / **`tilde_http_path_to_item_id`**.
+5. **`TildeOntologyPath::from_stored(&ItemId)`**.
+6. Export **`ItemId`** from **`types/src/lib.rs`**; update **`server/src/path_types.rs`** re-exports.
+7. **Delete `CanonicalItemUrl`** and fix all **in-crate** references in `types` only until `cargo test` passes for `slug-types`.
+
+**Module cycle trap:** `item_id.rs` must not `use crate::paths::{...}` if `paths.rs` also imports `ItemId` for `GardenItemUrl` in the same module. **Fix one of:**
+
+- **A)** Put `ItemId` **inside `paths.rs`** below `canonicalize_item` / helpers (simplest, large file), or
+- **B)** Split **`canonicalize_item`** (+ dash host helpers + `finalize_external_identity_url`) into **`types/src/item_wire.rs`**, then `paths.rs` + `item_id.rs` both depend on `item_wire` only (cleaner, more files).
+
+### Phase 2 — Reducer + ranking (server core)
+
+1. **`server/src/reducer.rs`**: `ContentState` / `GroupState` / **`VoteData`** — replace **`CanonicalItemUrl`** with **`ItemId`** on all maps, sets, deques, vectors.
+2. **`apply_vote`**: normalize `a`/`b` via **`ItemId::parse`** or **`ItemId`**-aware logic (remove string round-trip).
+3. **`apply_ingest_to_content`**: **`dsl`** still yields strings for item titles in statements; normalize to **`ItemId`** at ingest boundary via **`ItemId::parse`** once per item.
+4. **`server/src/ranking.rs`**, **`server/src/scope_rank.rs`**, **`server/src/api/write_actor.rs`** (including **`BTreeSet`** ordering), **`server/src/api/validate.rs`**, **`server/src/api/helpers.rs`** — propagate **`ItemId`**.
+5. **`server/tests/basic.rs`** and any reducer tests constructing **`VoteData`** — use **`ItemId::parse(...).unwrap()`** or helpers.
+
+### Phase 3 — RPC + search + external resolver
+
+1. **`server/src/api/rpc.rs`**: rank/pair/matchup/search payloads; today many paths use **`GardenItemUrl::from_storage_str(item.as_str(), …)`** — switch to **`ItemId`** + **`GardenItemUrl::from_stored(&item_id, …)`** (or equivalent).
+2. **`server/src/html/search.rs`**: scoring uses item path strings — derive from **`ItemId::display_path`** / **`to_wire_url`** only at the scoring boundary if needed.
+3. **`server/src/external_resolver.rs`**: take **`&ItemId`** or **`ItemId::external_url()`** instead of **`&CanonicalItemUrl`**.
+
+### Phase 4 — HTML / Maud
+
+1. **`ThreadNav::garden_item_url`**: overload or replace with **`garden_item_href(&self, item: &ItemId)`** (no `CanonicalItemUrl::parse` inside).
+2. **`RouteContext`**: extend **`item_href(&ItemId)`**; migrate call sites from **`ThreadNav`** to **`RouteContext`** where only link-building is needed (keep **`ThreadNav`** where scope / auth helpers need the full struct).
+3. **`server/src/html/garden.rs`**, **`breadcrumb_path.rs`**, **`forum/*`**, **`editor.rs`**: replace **`CanonicalItemUrl`** with **`ItemId`**; breadcrumbs should walk **`ItemId::parent`** without string `rsplit`.
+4. **`types` JSON types** (`RankRow`, etc.): decide whether **`GardenItemUrl`** stays string for JSON or becomes a structured field; keep **one** wire format for the public API.
+
+### Phase 5 — Cleanup + docs
+
+1. Remove dead **`canonical_path`** / **`breadcrumb_path`** string logic if fully superseded.
+2. Update **`AGENTS.md`** if durability, `POST /ui`, or command surfaces change.
+3. Delete or shrink **`plan.md`** when done.
+
+## File / symbol checklist (non-exhaustive — grep-driven)
+
+Run periodically:
+
+```bash
+rg "CanonicalItemUrl" -g'*.rs'
+rg "path_types::CanonicalItemUrl" -g'*.rs'
+rg "tilde_http_path_to_canonical" -g'*.rs'
+```
+
+**High-touch files (from prior exploration):**
+
+| Area | Files |
+|------|--------|
+| Types | `types/src/paths.rs`, `types/src/lib.rs`, `types/src/url_normalize.rs`, (optional) `types/src/item_id.rs`, `types/src/item_wire.rs` |
+| Server re-exports | `server/src/path_types.rs`, `server/src/canonical_path.rs` |
+| Reducer / ingest | `server/src/reducer.rs`, `server/src/dsl.rs` (parse output types if changed) |
+| Ranking | `server/src/ranking.rs`, `server/src/scope_rank.rs` |
+| Writer / RPC | `server/src/api/write_actor.rs`, `server/src/api/rpc.rs`, `server/src/api/helpers.rs`, `server/src/api/validate.rs` |
+| HTML | `server/src/html/garden.rs`, `server/src/html/breadcrumb_path.rs`, `server/src/html/forum/nav.rs`, `server/src/html/routing.rs`, `server/src/html/search.rs`, `server/src/html/editor.rs`, `server/src/html/forum/ingest.rs`, … |
+| Tests | `server/tests/basic.rs`, `server/tests/integration.rs`, `types/src/paths.rs` tests, Clojure under `test/` if URLs/assertions mention canonical shapes |
+
+## Events / JSONL
+
+- **`Ingest`** events store **`raw` DSL** only — no change required for item identity inside the event.
+- If any future event type stores item ids as strings, migrate to **structured `ItemId` serde** or accept string only at the event boundary with immediate parse into **`ItemId`** on `apply_event`.
+
+## `nav!` macro (`server/src/paths.rs`)
+
+- Macros use **`keypath($key)`** with **`.clone()`** — **`ItemId`** must be **`Clone`** (already for enums). Remove any reliance on **`Borrow<str>`** for map keys.
+
+## Testing gate
+
+After each phase:
+
+```bash
+cargo test --workspace
+./scripts/clj-test.sh
+```
+
+## Risks / gotchas
+
+1. **`Ord` on `ItemId`**: must match prior **`CanonicalItemUrl`** / `String` ordering wherever **`BTreeSet`** is used (e.g. deterministic scope-rank snapshots in **`write_actor`**).
+2. **External `ItemId`**: **`Url`** equality / hashing — normalization is already centralized in **`url_normalize`**; ensure **`ItemId::parse`** always inserts normalized **`Url`** into **`External`**.
+3. **Fake parent URLs** in garden (e.g. **`https://.`** for external root ranking): find all **`parse("https://.")`** style hacks and express as **`ItemId`** or a dedicated sentinel.
+4. **Serde**: tests and any RPC clients that snapshot JSON may need expectation updates if **`VoteData`** shape changes.
+
+## Optional follow-ups (not blocking `ItemId`)
+
+- More **domain normalizers** in **`url_normalize.rs`** (e.g. `music.youtube.com`, Spotify, etc.).
+- **Room wire** vs **HTTP segment** helpers already in **`paths.rs`** (`ROOM_SHORT_ID_LEN`, `room_route_segment`, `room_id_from_route_segment`).
+
+---
+
+**End state criteria:** `rg CanonicalItemUrl` returns nothing; reducer maps use **`ItemId`**; HTML link generation for items goes through **`RouteContext` + `ItemId`**; tests and Kaocha green.
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index c3dccd03dc6da2a2f6f6fa828657e33a951884b
… preview truncated; 2,831 characters omittedB — c_9608dc0d38ab (tommy-mor)
message
[9ecc4e2e] nice
diff preview
diff --git a/server/src/api/rpc.rs b/server/src/api/rpc.rs
index 5675dcd2e28062acbe3c037b94b77c33adf32474..a18241f3ed7b4a248748fcefc799f9801ca62461 100644
--- a/server/src/api/rpc.rs
+++ b/server/src/api/rpc.rs
@@ -936,7 +936,15 @@ pub async fn handle_rpc_batch(
line_ok(RpcResult::ForumThreads(rpc_list_forum_threads(&reduced, &room)))
}
RpcCommand::RoomCreate { slug, visibility } => {
- match verify_bearer_principal(&headers, &*state.reduced.read().await) {
+ // Scope the first read so its guard drops before any nested `read().await` / `write().await`.
+ // A guard from `match verify(..., &*state.reduced.read().await)` would otherwise live for the
+ // whole `match` and deadlock here (tokio::sync::RwLock is not reentrant).
+ let principal = {
+ let reduced = state.reduced.read().await;
+ verify_bearer_principal(&headers, &*reduced)
+ };
+ match principal {
+ Err((_, m)) => line_err(m, None),
Ok(principal) => {
let slug = slug.trim().to_lowercase();
if slug.is_empty() || slug.len() > 64 {
@@ -994,7 +1002,6 @@ pub async fn handle_rpc_batch(
}
}
}
- Err((_, m)) => line_err(m, None),
}
}
RpcCommand::RoomGrant {
@@ -1002,17 +1009,28 @@ pub async fn handle_rpc_batch(
username,
capability,
} => {
- match verify_bearer_principal(&headers, &*state.reduced.read().await) {
+ let principal = {
+ let reduced = state.reduced.read().await;
+ verify_bearer_principal(&headers, &*reduced)
+ };
+ match principal {
Err((_, m)) => line_err(m, None),
Ok(principal) => {
- let reduced = state.reduced.read().await;
- if !reduced.user_has_cap(&room, &principal, ThreadCapability::Manage) {
+ let can_manage = {
+ let reduced = state.reduced.read().await;
+ reduced.user_has_cap(&room, &principal, ThreadCapability::Manage)
+ };
+ if !can_manage {
line_err("requires Manage capability", None)
} else {
match parse_username(&username) {
Err(msg) => line_err("invalid username", Some(msg)),
Ok(target) => {
- if !reduced.users_by_provider.values().any(|u| u == &target) {
+ let user_exists = {
+ let reduced = state.reduced.read().await;
+ reduced.users_by_provider.values().any(|u| u == &target)
+ };
+ if !user_exists {
line_err(format!("user @{target} not found"), None)
} else {
match parse_capability(&capability) {
diff --git a/server/tests/integration.rs b/server/tests/integration.rs
index 9162bd5bee84035e3908bfb9c8e201b7878ca339..b930120da09fe7d307f0411b84fb639fb8bd0b15 100644
--- a/server/tests/integration.rs
+++ b/server/tests/integration.rs
@@ -95,6 +95,23 @@ async fn test_healthz() {
assert_eq!(response.text().await.unwrap(), "ok");
}
+#[tokio::test]
+async fn test_room_create_private_rpc() {
+ let (addr, _tmp, _log, _handle) = create_test_server().await;
+ let client = reqwest::Client::new();
+ let batch = serde_json::json!([{
+ "RoomCreate": { "slug": "secret-project", "visibility": "private" }
+ }]);
+ let body = rpc_batch(&client, addr, Some(&test_bearer()), batch).await;
+ let line = &body["results"][0];
+ assert_eq!(line["ok"], true, "room create: {:?}", line);
+ let room_id = line["result"]["RoomCreated"]["room_id"].as_str().unwrap();
+ assert!(
+ room_id.contains("/secret-project"),
+ "expected room_id to contain slug, got {room_id}"
+ );
+}
+
#[tokio::test]
async fn test_index_page() {
// HTML routes are offline during the auth-v3 refactor.
diff --git a/test/auth.bb b/test/auth.bb
index 611e04f1806ef81d678a91f499597fe55f691dd7..a67cc1de763434176d2f68e419dd37a8cd328395 100644
--- a/test/auth.bb
+++ b/test/auth.bb
@@ -176,7 +176,7 @@
(assert! (= 200 (:status poll)) "pending-session poll returns 200")
(let [poll-json (json/parse-string (:body poll) true)]
(assert! (:complete poll-json) "pending session complete=true")
- (assert! (= "@bbuser" (:user poll-json)) "poll returns @bbuser")
+ (assert! (= "bbuser" (:user poll-json)) "poll returns stored username bbuser")
(assert! (clojure.string/starts-with? (:token poll-json) "slug_") "poll returns bearer token")
(println "\nwhoami…")
@@ -184,7 +184,7 @@
:headers {"Authorization" (str "Bearer " (:token poll-json))})]
(assert! (= 200 (:status who)) "whoami returns 200")
(let [who-json (json/parse-string (:body who) true)]
- (assert! (= "@bbuser" (:user who-json)) "whoami user is @bbuser"))))))
+ (assert! (= "bbuser" (:user who-json)) "whoami user is bbuser (stored form)"))))))
(println "\nCLI: identity start → OAuth → identity poll → whoami…")
(let [cli-home (str tmp-dir "/cli-home")
@@ -208,7 +208,7 @@
(str "identity poll exits 0 (stderr: " (:err poll-proc) ")"))
(let [poll-cli (json/parse-string (:out poll-proc) true)]
(assert! (= "complete" (:phase poll-cli)) "identity poll --json phase")
- (assert! (= "@cliuser" (:user poll-cli)) "CLI poll user")
+ (assert! (= "cliuser" (:user poll-cli)) "CLI poll user (stored form)")
(assert! (clojure.string/starts-with? (:token poll-cli) "slug_") "CLI poll token")
(let [token-path (str cli-home "/.config/slugsocial/token")]
(assert! (fs/exists? token-path) "token written under isolated HOME")
@@ -219,7 +219,7 @@
(assert! (zero? (:exit who-proc))
(str "whoami exits 0 (stderr: " (:err who-proc) ")"))
(let [who-cli (json/parse-string (:out who-proc) true)]
- (assert! (= "@cliuser" (:user who-cli)) "CLI whoami uses saved token"))))))))
+ (assert! (= "cliuser" (:user who-cli)) "CLI whoami uses saved token"))))))))
(finally
(when-some [s @!server] (common/kill-server s))
diff --git a/test/common.bb b/test/common.bb
index ebb16c615a8b40e8830b5d5765d1e935a45538d0..4ed450377cdbb020f5ff164a812dfbbfc23c0f47 100644
--- a/test/common.bb
+++ b/test/common.bb
@@ -78,9 +78,20 @@
(defn start-server
"Start the slugsocial-server binary with the given env map.
- Returns the babashka.process map."
- [server-bin env-map]
- (p/process [server-bin] {:out :inherit :err :inherit :env env-map}))
+ Returns the babashka.process map.
+
+ When `log-file` (string path) is provided, stdout and stderr are appended there
+ instead of inheriting the parent descriptors. Inheriting shared pipes while the
+ parent blocks on HTTP I/O can fill the pipe buffer and deadlock the server on log writes."
+ ([server-bin env-map]
+ (start-server server-bin env-map nil))
+ ([server-bin env-map log-file]
+ (p/process [server-bin]
+ (if log-file
+ ;; Two string paths (same file): babashka.process can deref the process cleanly.
+ ;; :err :out + ProcessBuilder$Redirect breaks stream copying in deref/kill-server.
+ {:env env-map :out log-file :err log-file}
+ {:out :inherit :err :inherit :env env-map}))))
(defn kill-server
"Forcibly kill a server process (babashka.process map) and wait for it to exit."
diff --git a/test/grants.bb b/test/grants.bb
index 7066c1f2a57f39a362052f8524206dccf37bb7b1..793ba216f2de76a77eb20a76d08403db07ff411b 100644
--- a/test/grants.bb
+++ b/test/grants.bb
@@ -35,13 +35,14 @@
(defn- http-client []
(-> (java.net.http.HttpClient/newBuilder)
(.followRedirects java.net.http.HttpClient$Redirect/ALWAYS)
+ (.connectTimeout (java.time.Duration/ofSeconds 15))
(.build)))
(defn- http-get [url & {:keys [headers]}]
(let [b (java.net.http.HttpRequest/newBuilder (java.net.URI/create url))]
(doseq [[k v] (or headers {})]
(.header b k v))
- (let [req (-> b (.GET) (.build))
+ (let [req (-> b (.timeout (java.time.Duration/ofSeconds 60)) (.GET) (.build))
resp (.send (http-client) req (java.net.http.HttpResponse$BodyHandlers/ofString))]
{:status (.statusCode resp) :body (.body resp)})))
@@ -52,6 +53,7 @@
(doseq [[k v] (or headers {})]
(.header b k v))
(let [req (-> b
+ (.timeout (java.time.Duration/ofSeconds 60))
(.POST (java.net.http.HttpRequest$BodyPublishers/ofString body))
(.build))
resp (.send (http-client) req (java.net.http.HttpResponse$BodyHandlers/ofString))]
@@ -67,6 +69,7 @@
b (java.net.http.HttpRequest/newBuilder (java.net.URI/create url))]
(.header b "Content-Type" "application/x-www-form-urlencoded")
(let [req (-> b
+ (.timeout (java.time.Duration/ofSeconds 60))
(.POST (java.net.http.HttpRequest$BodyPublishers/ofString pairs))
(.build))
resp (.send (http-client) req (java.net.http.HttpResponse$BodyHandlers/ofString))]
Hardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.