{"messages":[{"content":"You are a constitutional council ranking individual git commits for ownership allocation.\n\nCompare these two commits. Decide which contributed more lasting value to the project.\n\nJudge substance, not spectacle:\n- Prefer correct, lasting design and real bugfixes over churn, formatting, renames, or generated noise.\n- Prefer clarity and necessity over sheer line count. A small precise change can beat a large diffuse one.\n- Do not favor a side merely because its patch is longer or noisier.\n- Weight what the change does for the project, not the contributor's name.\n\nReturn ONLY a JSON object: {\"winner\": \"A\" or \"B\", \"ratio\": \"N:M\", \"explanation\": \"...\"}\nThe explanation must cite concrete differences in the patches (1-3 sentences).\n\nSide A — contributor: tommy-mor\nSide A — commit message:\n[bf118bdf] Sanitize Reddit entity body HTML before rendering.\n\nUse ammonia at render time so untrusted selftext_html cannot execute scripts in our origin.\n\nCo-authored-by: Cursor \n\nSide A — unified diff (full patch):\ndiff --git a/Cargo.lock b/Cargo.lock\nindex 3dec7cb72a182dc654a37dca8ba0b49d77504daa..0dd4fce5fb6400ae153cca4e3dbf5a5158e6d8b4 100644\n--- a/Cargo.lock\n+++ b/Cargo.lock\n@@ -11,6 +11,19 @@ dependencies = [\n \"memchr\",\n ]\n \n+[[package]]\n+name = \"ammonia\"\n+version = \"4.1.2\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"17e913097e1a2124b46746c980134e8c954bc17a6a59bb3fde96f088d126dde6\"\n+dependencies = [\n+ \"cssparser\",\n+ \"html5ever\",\n+ \"maplit\",\n+ \"tendril\",\n+ \"url\",\n+]\n+\n [[package]]\n name = \"anyhow\"\n version = \"1.0.102\"\n@@ -355,6 +368,29 @@ version = \"0.2.4\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5\"\n \n+[[package]]\n+name = \"cssparser\"\n+version = \"0.35.0\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"4e901edd733a1472f944a45116df3f846f54d37e67e68640ac8bb69689aca2aa\"\n+dependencies = [\n+ \"cssparser-macros\",\n+ \"dtoa-short\",\n+ \"itoa\",\n+ \"phf\",\n+ \"smallvec\",\n+]\n+\n+[[package]]\n+name = \"cssparser-macros\"\n+version = \"0.6.1\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"13b588ba4ac1a99f7f2964d24b3d896ddc6bf847ee3855dbd4366f058cfcd331\"\n+dependencies = [\n+ \"quote\",\n+ \"syn\",\n+]\n+\n [[package]]\n name = \"deranged\"\n version = \"0.5.8\"\n@@ -381,6 +417,21 @@ version = \"0.15.7\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b\"\n \n+[[package]]\n+name = \"dtoa\"\n+version = \"1.0.11\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590\"\n+\n+[[package]]\n+name = \"dtoa-short\"\n+version = \"0.3.5\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87\"\n+dependencies = [\n+ \"dtoa\",\n+]\n+\n [[package]]\n name = \"durable\"\n version = \"0.2.0\"\n@@ -482,6 +533,16 @@ dependencies = [\n \"percent-encoding\",\n ]\n \n+[[package]]\n+name = \"futf\"\n+version = \"0.1.5\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"df420e2e84819663797d1ec6544b13c5be84629e7bb00dc960d6917db2987843\"\n+dependencies = [\n+ \"mac\",\n+ \"new_debug_unreachable\",\n+]\n+\n [[package]]\n name = \"futures-channel\"\n version = \"0.3.32\"\n@@ -614,6 +675,17 @@ version = \"0.5.0\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea\"\n \n+[[package]]\n+name = \"html5ever\"\n+version = \"0.35.0\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"55d958c2f74b664487a2035fe1dadb032c48718a03b63f3ab0b8537db8549ed4\"\n+dependencies = [\n+ \"log\",\n+ \"markup5ever\",\n+ \"match_token\",\n+]\n+\n [[package]]\n name = \"http\"\n version = \"1.4.1\"\n@@ -974,6 +1046,15 @@ version = \"0.8.2\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0\"\n \n+[[package]]\n+name = \"lock_api\"\n+version = \"0.4.14\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965\"\n+dependencies = [\n+ \"scopeguard\",\n+]\n+\n [[package]]\n name = \"log\"\n version = \"0.4.30\"\n@@ -990,6 +1071,40 @@ dependencies = [\n \"libc\",\n ]\n \n+[[package]]\n+name = \"mac\"\n+version = \"0.1.1\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4\"\n+\n+[[package]]\n+name = \"maplit\"\n+version = \"1.0.2\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d\"\n+\n+[[package]]\n+name = \"markup5ever\"\n+version = \"0.35.0\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"311fe69c934650f8f19652b3946075f0fc41ad8757dbb68f1ca14e7900ecc1c3\"\n+dependencies = [\n+ \"log\",\n+ \"tendril\",\n+ \"web_atoms\",\n+]\n+\n+[[package]]\n+name = \"match_token\"\n+version = \"0.35.0\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"ac84fd3f360fcc43dc5f5d186f02a94192761a080e8bc58621ad4d12296a58cf\"\n+dependencies = [\n+ \"proc-macro2\",\n+ \"quote\",\n+ \"syn\",\n+]\n+\n [[package]]\n name = \"matchers\"\n version = \"0.2.0\"\n@@ -1092,6 +1207,12 @@ dependencies = [\n \"tempfile\",\n ]\n \n+[[package]]\n+name = \"new_debug_unreachable\"\n+version = \"1.0.6\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086\"\n+\n [[package]]\n name = \"nom\"\n version = \"7.1.3\"\n@@ -1175,6 +1296,29 @@ dependencies = [\n \"vcpkg\",\n ]\n \n+[[package]]\n+name = \"parking_lot\"\n+version = \"0.12.5\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a\"\n+dependencies = [\n+ \"lock_api\",\n+ \"parking_lot_core\",\n+]\n+\n+[[package]]\n+name = \"parking_lot_core\"\n+version = \"0.9.12\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1\"\n+dependencies = [\n+ \"cfg-if\",\n+ \"libc\",\n+ \"redox_syscall\",\n+ \"smallvec\",\n+ \"windows-link\",\n+]\n+\n [[package]]\n name = \"peeking_take_while\"\n version = \"0.1.2\"\n@@ -1187,6 +1331,58 @@ version = \"2.3.2\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220\"\n \n+[[package]]\n+name = \"phf\"\n+version = \"0.11.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078\"\n+dependencies = [\n+ \"phf_macros\",\n+ \"phf_shared\",\n+]\n+\n+[[package]]\n+name = \"phf_codegen\"\n+version = \"0.11.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a\"\n+dependencies = [\n+ \"phf_generator\",\n+ \"phf_shared\",\n+]\n+\n+[[package]]\n+name = \"phf_generator\"\n+version = \"0.11.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d\"\n+dependencies = [\n+ \"phf_shared\",\n+ \"rand 0.8.6\",\n+]\n+\n+[[package]]\n+name = \"phf_macros\"\n+version = \"0.11.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"f84ac04429c13a7ff43785d75ad27569f2951ce0ffd30a3321230db2fc727216\"\n+dependencies = [\n+ \"phf_generator\",\n+ \"phf_shared\",\n+ \"proc-macro2\",\n+ \"quote\",\n+ \"syn\",\n+]\n+\n+[[package]]\n+name = \"phf_shared\"\n+version = \"0.11.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5\"\n+dependencies = [\n+ \"siphasher\",\n+]\n+\n [[package]]\n name = \"pin-project-lite\"\n version = \"0.2.17\"\n@@ -1223,6 +1419,12 @@ dependencies = [\n \"zerocopy\",\n ]\n \n+[[package]]\n+name = \"precomputed-hash\"\n+version = \"0.1.1\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c\"\n+\n [[package]]\n name = \"prettyplease\"\n version = \"0.2.37\"\n@@ -1379,6 +1581,15 @@ dependencies = [\n \"rand_core 0.9.5\",\n ]\n \n+[[package]]\n+name = \"redox_syscall\"\n+version = \"0.5.18\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d\"\n+dependencies = [\n+ \"bitflags 2.11.1\",\n+]\n+\n [[package]]\n name = \"regex\"\n version = \"1.12.3\"\n@@ -1563,6 +1774,12 @@ dependencies = [\n \"windows-sys 0.61.2\",\n ]\n \n+[[package]]\n+name = \"scopeguard\"\n+version = \"1.2.0\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49\"\n+\n [[package]]\n name = \"security-framework\"\n version = \"3.7.0\"\n@@ -1683,6 +1900,12 @@ dependencies = [\n \"libc\",\n ]\n \n+[[package]]\n+name = \"siphasher\"\n+version = \"1.0.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649\"\n+\n [[package]]\n name = \"slab\"\n version = \"0.4.12\"\n@@ -1709,6 +1932,7 @@ dependencies = [\n name = \"sorter2-server\"\n version = \"0.0.1\"\n dependencies = [\n+ \"ammonia\",\n \"async-stream\",\n \"axum\",\n \"axum-extra\",\n@@ -1742,6 +1966,31 @@ version = \"1.2.1\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596\"\n \n+[[package]]\n+name = \"string_cache\"\n+version = \"0.8.9\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"bf776ba3fa74f83bf4b63c3dcbbf82173db2632ed8452cb2d891d33f459de70f\"\n+dependencies = [\n+ \"new_debug_unreachable\",\n+ \"parking_lot\",\n+ \"phf_shared\",\n+ \"precomputed-hash\",\n+ \"serde\",\n+]\n+\n+[[package]]\n+name = \"string_cache_codegen\"\n+version = \"0.5.4\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"c711928715f1fe0fe509c53b43e993a9a557babc2d0a3567d0a3006f1ac931a0\"\n+dependencies = [\n+ \"phf_generator\",\n+ \"phf_shared\",\n+ \"proc-macro2\",\n+ \"quote\",\n+]\n+\n [[package]]\n name = \"subtle\"\n version = \"2.6.1\"\n@@ -1813,6 +2062,17 @@ dependencies = [\n \"windows-sys 0.61.2\",\n ]\n \n+[[package]]\n+name = \"tendril\"\n+version = \"0.4.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"d24a120c5fc464a3458240ee02c299ebcb9d67b5249c8848b09d639dca8d7bb0\"\n+dependencies = [\n+ \"futf\",\n+ \"mac\",\n+ \"utf-8\",\n+]\n+\n [[package]]\n name = \"thiserror\"\n version = \"1.0.69\"\n@@ -2116,6 +2376,12 @@ version = \"2.1.3\"\n source = \"registry+https://github.com/rust-lang/crates.io-index\"\n checksum = \"daf8dba3b7eb870caf1ddeed7bc9d2a049f3cfdfae7cb521b087cc33ae4c49da\"\n \n+[[package]]\n+name = \"utf-8\"\n+version = \"0.7.6\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9\"\n+\n [[package]]\n name = \"utf8_iter\"\n version = \"1.0.4\"\n@@ -2281,6 +2547,18 @@ dependencies = [\n \"wasm-bindgen\",\n ]\n \n+[[package]]\n+name = \"web_atoms\"\n+version = \"0.1.3\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"57ffde1dc01240bdf9992e3205668b235e59421fd085e8a317ed98da0178d414\"\n+dependencies = [\n+ \"phf\",\n+ \"phf_codegen\",\n+ \"string_cache\",\n+ \"string_cache_codegen\",\n+]\n+\n [[package]]\n name = \"windows-link\"\n version = \"0.2.1\"\ndiff --git a/server/Cargo.toml b/server/Cargo.toml\nindex 47659ff82fbfb50972eb2b87575e80f66e572ba4..27f552c20b97ef28cdde4cb6b1a4980375135111 100644\n--- a/server/Cargo.toml\n+++ b/server/Cargo.toml\n@@ -13,6 +13,7 @@ serde = { version = \"1\", features = [\"derive\"] }\n serde_json = \"1\"\n thiserror = \"1\"\n maud = { version = \"0.26\", features = [\"axum\"] }\n+ammonia = \"4.1\"\n tower = \"0.5\"\n tower-http = { version = \"0.5\", features = [\"trace\"] }\n tracing = \"0.1\"\ndiff --git a/server/src/fetch/html.rs b/server/src/fetch/html.rs\nindex dadf050515f0473943dad97df5d318032c8cb385..5b160c6b8bd216dfaf80149854aec0566cd00460 100644\n--- a/server/src/fetch/html.rs\n+++ b/server/src/fetch/html.rs\n@@ -4,6 +4,7 @@ use maud::{html, Markup};\n \n use crate::{\n form_template::template_json_compact,\n+ html::sanitize::entity_body_html,\n path_types::ItemId,\n reddit::{is_children_fetchable, is_fetchable},\n reducer::NodeState,\n@@ -27,7 +28,7 @@ pub fn entity_panel(node: &NodeState) -> Markup {\n p class=\"muted small\" { \"by \" (author) }\n }\n @if let Some(body) = &data.body_html {\n- div class=\"entity-body\" { (maud::PreEscaped(body)) }\n+ div class=\"entity-body\" { (maud::PreEscaped(entity_body_html(body))) }\n }\n }\n }\ndiff --git a/server/src/html/mod.rs b/server/src/html/mod.rs\nindex e180a0ca542a33e2300c0a4809e6b9cfee07ecfe..a58cbbee3490a08a625cb06df06848c59a615d65 100644\n--- a/server/src/html/mod.rs\n+++ b/server/src/html/mod.rs\n@@ -20,6 +20,7 @@ use crate::{\n ui_action::UI_RPC_FIELD,\n };\n \n+pub mod sanitize;\n pub mod vote;\n \n const SORTER_CSS: &str = include_str!(\"../../static/sorter.css\");\ndiff --git a/server/src/html/sanitize.rs b/server/src/html/sanitize.rs\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..6685ed2535281b9fd5d65b042cbc5a11c5d4df37\n--- /dev/null\n+++ b/server/src/html/sanitize.rs\n@@ -0,0 +1,39 @@\n+//! Whitelist sanitization for untrusted HTML fragments (e.g. Reddit `selftext_html`).\n+\n+use std::sync::LazyLock;\n+\n+use ammonia::Builder;\n+\n+static ENTITY_BODY: LazyLock> = LazyLock::new(|| {\n+ let mut b = Builder::default();\n+ b.strip_comments(true);\n+ b.link_rel(Some(\"noopener noreferrer\"));\n+ b\n+});\n+\n+/// Sanitize HTML safe for embedding in our pages via [`maud::PreEscaped`].\n+pub fn entity_body_html(raw: &str) -> String {\n+ ENTITY_BODY.clean(raw).to_string()\n+}\n+\n+#[cfg(test)]\n+mod tests {\n+ use super::entity_body_html;\n+\n+ #[test]\n+ fn keeps_benign_markup() {\n+ assert_eq!(\n+ entity_body_html(\"

release notes

\"),\n+ \"

release notes

\"\n+ );\n+ }\n+\n+ #[test]\n+ fn strips_scripts_and_event_handlers() {\n+ let raw = \"

ok

\";\n+ let clean = entity_body_html(raw);\n+ assert!(!clean.contains(\"ok

\"));\n+ }\n+}\ndiff --git a/server/src/render/reddit.rs b/server/src/render/reddit.rs\nindex c4f98fc760c32ce1b41a91bd41e97908bd198937..7f840aa33b734a31d8cf3341a0581c8bcb9bbcf3 100644\n--- a/server/src/render/reddit.rs\n+++ b/server/src/render/reddit.rs\n@@ -3,6 +3,7 @@\n use maud::{html, Markup};\n \n use crate::{\n+ html::sanitize::entity_body_html,\n path_types::ItemId,\n reducer::{EntityData, GlobalTree, NodeState},\n };\n@@ -57,7 +58,7 @@ fn post_entity_card(data: &EntityData) -> Markup {\n }\n }\n @if let Some(body) = &data.body_html {\n- div class=\"entity-body\" { (maud::PreEscaped(body)) }\n+ div class=\"entity-body\" { (maud::PreEscaped(entity_body_html(body))) }\n }\n }\n }\n\n\nSide B — contributor: tommy-mor\nSide B — commit message:\n[8d8230d1] reddit\n\nSide B — unified diff (full patch):\ndiff --git a/.gitignore b/.gitignore\nindex 4c7073f9fac0c30fd2050d79a60ef447af58ebeb..ada462e900d24a3a6d08165d158c80f79c35a5a5 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -7,3 +7,4 @@\n data/\n repomix-output.xml\n dev-data/\n+.env\ndiff --git a/server/Cargo.toml b/server/Cargo.toml\nindex 7906a8547d56b8e6a48ef59c37aa82a8510fdee9..4677fedcb45292eebebe7e9cf6ce2f5738f18ddf 100644\n--- a/server/Cargo.toml\n+++ b/server/Cargo.toml\n@@ -16,6 +16,7 @@ tower = \"0.5\"\n tower-http = { version = \"0.5\", features = [\"trace\"] }\n tracing = \"0.1\"\n tracing-subscriber = { version = \"0.3\", features = [\"env-filter\"] }\n+reqwest = { version = \"0.12\", features = [\"json\"] }\n \n [dev-dependencies]\n reqwest = { version = \"0.12\", features = [\"json\"] }\ndiff --git a/server/src/html/mod.rs b/server/src/html/mod.rs\nindex 2864407ed6e8ec284a1dc663acf1805534566b62..df6505021d9f446c2b453e20e3eb3cf696a111f9 100644\n--- a/server/src/html/mod.rs\n+++ b/server/src/html/mod.rs\n@@ -272,5 +272,16 @@ pub async fn home(State(state): State, uri: Uri) -> impl IntoResponse\n \n pub async fn browse(State(state): State, uri: Uri) -> impl IntoResponse {\n let item = ItemId::from_browse_uri(uri.path()).unwrap_or(ItemId::root());\n+ if item.as_str().starts_with(\"reddit.com\") {\n+ let needs_fetch = {\n+ let tree = state.tree.read().await;\n+ tree.get(&item)\n+ .map(|n| n.data.is_none())\n+ .unwrap_or(true)\n+ };\n+ if needs_fetch {\n+ state.reddit.request_fetch(item.clone());\n+ }\n+ }\n item_page(state, uri, item).await\n }\ndiff --git a/server/src/reddit.rs b/server/src/reddit.rs\nindex d203dca09245daf869b3aa942898447700ae69fb..90053ad03b1d7c8e94f325dd4ee64c2b4f7da900 100644\n--- a/server/src/reddit.rs\n+++ b/server/src/reddit.rs\n@@ -1,4 +1,12 @@\n-//! Reddit API import (async, decoupled from UI request path).\n+//! Reddit API import via a single background worker (rate limits, dedup, backoff).\n+\n+use std::collections::{HashMap, HashSet};\n+use std::sync::Arc;\n+use std::time::{Duration, Instant};\n+\n+use reqwest::{header, Client, StatusCode};\n+use serde::Deserialize;\n+use tokio::sync::{mpsc, RwLock};\n \n use crate::{\n path_types::ItemId,\n@@ -10,12 +18,401 @@ pub fn ensure_partial_tree(tree: &mut GlobalTree, id: &ItemId) {\n tree.ensure_path(id);\n }\n \n-/// Placeholder for Reddit JSON import. Returns entity data when implemented.\n-pub async fn fetch_reddit_entity(_id: &ItemId) -> Option {\n- None\n+pub struct RedditCommand {\n+ pub id: ItemId,\n+}\n+\n+#[derive(Clone)]\n+pub struct RedditBroker {\n+ tx: mpsc::Sender,\n+}\n+\n+#[derive(Clone)]\n+struct RedditCredentials {\n+ client_id: String,\n+ client_secret: String,\n+}\n+\n+struct OAuthToken {\n+ access_token: String,\n+ expires_at: Instant,\n+}\n+\n+impl RedditBroker {\n+ pub fn spawn(tree: Arc>, user_agent: &str) -> Self {\n+ let (tx, rx) = mpsc::channel(100);\n+\n+ let mut headers = header::HeaderMap::new();\n+ headers.insert(\n+ header::USER_AGENT,\n+ header::HeaderValue::from_str(user_agent).expect(\"valid user agent\"),\n+ );\n+\n+ let client = Client::builder()\n+ .default_headers(headers)\n+ .timeout(Duration::from_secs(15))\n+ .build()\n+ .expect(\"reqwest client\");\n+\n+ let creds = RedditCredentials::from_env();\n+ tokio::spawn(reddit_worker(rx, tree, client, creds));\n+\n+ Self { tx }\n+ }\n+\n+ /// Fire-and-forget: queue a fetch; worker updates the tree when done.\n+ pub fn request_fetch(&self, id: ItemId) {\n+ let _ = self.tx.try_send(RedditCommand { id });\n+ }\n+}\n+\n+impl RedditCredentials {\n+ fn from_env() -> Option {\n+ let client_id = std::env::var(\"REDDIT_CLIENT_ID\").ok()?;\n+ let client_secret = std::env::var(\"REDDIT_CLIENT_SECRET\").ok()?;\n+ if client_id.is_empty() || client_secret.is_empty() {\n+ return None;\n+ }\n+ Some(Self {\n+ client_id,\n+ client_secret,\n+ })\n+ }\n+}\n+\n+pub fn default_user_agent() -> String {\n+ std::env::var(\"REDDIT_USER_AGENT\").unwrap_or_else(|_| {\n+ \"web:sorter2.social:v0.0.1 (by /u/sorter2)\".to_string()\n+ })\n }\n \n-/// Apply fetched entity data to a node (called from async worker).\n-pub fn apply_entity(tree: &mut GlobalTree, id: &ItemId, data: EntityData) {\n- tree.set_entity_data(id, data);\n+async fn reddit_worker(\n+ mut rx: mpsc::Receiver,\n+ tree: Arc>,\n+ client: Client,\n+ creds: Option,\n+) {\n+ let mut in_flight = HashSet::new();\n+ let mut recently_fetched: HashMap = HashMap::new();\n+ let mut current_delay = Duration::from_secs(1);\n+ let mut oauth: Option = None;\n+ let cache_ttl = Duration::from_secs(300);\n+\n+ while let Some(cmd) = rx.recv().await {\n+ let now = Instant::now();\n+ recently_fetched.retain(|_, t| now.duration_since(*t) < cache_ttl);\n+\n+ if in_flight.contains(&cmd.id) || recently_fetched.contains_key(&cmd.id) {\n+ continue;\n+ }\n+\n+ in_flight.insert(cmd.id.clone());\n+ let fetch_id = cmd.id.clone();\n+\n+ tokio::time::sleep(current_delay).await;\n+\n+ if let Some(c) = &creds {\n+ oauth = ensure_oauth_token(&client, c, oauth.take()).await;\n+ }\n+\n+ let token = oauth.as_ref().map(|t| t.access_token.as_str());\n+ let use_oauth = token.is_some();\n+\n+ match do_fetch(&client, &fetch_id, use_oauth, token).await {\n+ Ok(FetchOutcome::Entity(data)) => {\n+ let mut w = tree.write().await;\n+ w.set_entity_data(&fetch_id, data);\n+ recently_fetched.insert(fetch_id.clone(), Instant::now());\n+ current_delay = Duration::from_millis(600);\n+ }\n+ Ok(FetchOutcome::NotFound) => {\n+ recently_fetched.insert(fetch_id.clone(), Instant::now());\n+ }\n+ Ok(FetchOutcome::RateLimited { reset_secs }) => {\n+ let wait = Duration::from_secs(reset_secs.max(1));\n+ tracing::warn!(\n+ \"Reddit rate limit for {}; sleeping {}s\",\n+ fetch_id,\n+ wait.as_secs()\n+ );\n+ tokio::time::sleep(wait).await;\n+ current_delay = (current_delay * 2).min(Duration::from_secs(60));\n+ }\n+ Err(e) => {\n+ tracing::warn!(\"Reddit fetch failed for {}: {}\", fetch_id, e);\n+ current_delay = (current_delay * 2).min(Duration::from_secs(60));\n+ }\n+ }\n+\n+ in_flight.remove(&fetch_id);\n+ }\n+}\n+\n+enum FetchOutcome {\n+ Entity(EntityData),\n+ NotFound,\n+ RateLimited { reset_secs: u64 },\n+}\n+\n+async fn ensure_oauth_token(\n+ client: &Client,\n+ creds: &RedditCredentials,\n+ existing: Option,\n+) -> Option {\n+ if let Some(t) = existing {\n+ if Instant::now() < t.expires_at - Duration::from_secs(60) {\n+ return Some(t);\n+ }\n+ }\n+\n+ let resp = client\n+ .post(\"https://www.reddit.com/api/v1/access_token\")\n+ .basic_auth(&creds.client_id, Some(&creds.client_secret))\n+ .form(&[(\"grant_type\", \"client_credentials\")])\n+ .send()\n+ .await;\n+\n+ let resp = match resp {\n+ Ok(r) => r,\n+ Err(e) => {\n+ tracing::warn!(\"Reddit OAuth token request failed: {e}\");\n+ return None;\n+ }\n+ };\n+\n+ if !resp.status().is_success() {\n+ tracing::warn!(\"Reddit OAuth token HTTP {}\", resp.status());\n+ return None;\n+ }\n+\n+ #[derive(Deserialize)]\n+ struct TokenResponse {\n+ access_token: String,\n+ expires_in: u64,\n+ }\n+\n+ let body: TokenResponse = match resp.json().await {\n+ Ok(b) => b,\n+ Err(e) => {\n+ tracing::warn!(\"Reddit OAuth token parse failed: {e}\");\n+ return None;\n+ }\n+ };\n+\n+ Some(OAuthToken {\n+ access_token: body.access_token,\n+ expires_at: Instant::now() + Duration::from_secs(body.expires_in),\n+ })\n+}\n+\n+async fn do_fetch(\n+ client: &Client,\n+ id: &ItemId,\n+ use_oauth: bool,\n+ bearer: Option<&str>,\n+) -> Result {\n+ let url = map_item_to_reddit_api(id, use_oauth);\n+ if url.is_empty() {\n+ return Ok(FetchOutcome::NotFound);\n+ }\n+\n+ let mut req = client.get(&url);\n+ if let Some(token) = bearer {\n+ req = req.bearer_auth(token);\n+ }\n+\n+ let resp = req.send().await.map_err(|e| e.to_string())?;\n+\n+ if resp.status() == StatusCode::TOO_MANY_REQUESTS {\n+ let reset = rate_limit_reset_secs(&resp);\n+ return Ok(FetchOutcome::RateLimited { reset_secs: reset });\n+ }\n+\n+ if resp.status() == StatusCode::SERVICE_UNAVAILABLE {\n+ return Err(\"Reddit unavailable (503)\".to_string());\n+ }\n+\n+ if !resp.status().is_success() {\n+ return Ok(FetchOutcome::NotFound);\n+ }\n+\n+ if rate_limit_remaining(&resp) == Some(0) {\n+ let reset = rate_limit_reset_secs(&resp);\n+ return Ok(FetchOutcome::RateLimited { reset_secs: reset });\n+ }\n+\n+ let bytes = resp.bytes().await.map_err(|e| e.to_string())?;\n+ Ok(parse_reddit_json(id, &bytes)\n+ .map(FetchOutcome::Entity)\n+ .unwrap_or(FetchOutcome::NotFound))\n+}\n+\n+fn rate_limit_remaining(resp: &reqwest::Response) -> Option {\n+ resp.headers()\n+ .get(\"x-ratelimit-remaining\")\n+ .and_then(|v| v.to_str().ok())\n+ .and_then(|s| s.parse::().ok())\n+ .map(|f| f.floor() as u64)\n+}\n+\n+fn rate_limit_reset_secs(resp: &reqwest::Response) -> u64 {\n+ resp.headers()\n+ .get(\"x-ratelimit-reset\")\n+ .and_then(|v| v.to_str().ok())\n+ .and_then(|s| s.parse::().ok())\n+ .map(|f| f.ceil() as u64)\n+ .unwrap_or(5)\n+}\n+\n+/// Map canonical item id to Reddit JSON API URL.\n+pub fn map_item_to_reddit_api(id: &ItemId, oauth: bool) -> String {\n+ let path = id.as_str();\n+ if !path.starts_with(\"reddit.com/\") && path != \"reddit.com\" {\n+ return String::new();\n+ }\n+\n+ let base = if oauth {\n+ \"https://oauth.reddit.com\"\n+ } else {\n+ \"https://www.reddit.com\"\n+ };\n+\n+ let segments: Vec<&str> = path.split('/').collect();\n+\n+ if let Some(i) = segments.iter().position(|&p| p == \"comments\") {\n+ if segments.len() > i + 1 {\n+ let api_path = segments[1..=i + 1].join(\"/\");\n+ return format!(\"{base}/{api_path}.json?raw_json=1\");\n+ }\n+ }\n+\n+ if segments.len() == 3 && segments[1] == \"r\" {\n+ return format!(\"{base}/r/{}/about.json?raw_json=1\", segments[2]);\n+ }\n+\n+ String::new()\n+}\n+\n+fn parse_reddit_json(id: &ItemId, bytes: &[u8]) -> Option {\n+ let v: serde_json::Value = serde_json::from_slice(bytes).ok()?;\n+ let segments: Vec<&str> = id.as_str().split('/').collect();\n+\n+ if segments.iter().any(|&p| p == \"comments\") {\n+ parse_post_listing(&v)\n+ } else {\n+ parse_subreddit_about(&v)\n+ }\n+}\n+\n+fn parse_subreddit_about(v: &serde_json::Value) -> Option {\n+ let data = v.get(\"data\")?;\n+ let title = data\n+ .get(\"title\")\n+ .or_else(|| data.get(\"display_name\"))\n+ .and_then(|t| t.as_str())?\n+ .to_string();\n+ let body_html = data\n+ .get(\"public_description_html\")\n+ .or_else(|| data.get(\"public_description\"))\n+ .and_then(|t| t.as_str())\n+ .map(|s| s.to_string());\n+ let thumb_url = data\n+ .get(\"icon_img\")\n+ .or_else(|| data.get(\"community_icon\"))\n+ .and_then(|t| t.as_str())\n+ .filter(|s| !s.is_empty())\n+ .map(|s| s.to_string());\n+\n+ Some(EntityData {\n+ title,\n+ author: None,\n+ body_html,\n+ thumb_url,\n+ })\n+}\n+\n+fn parse_post_listing(v: &serde_json::Value) -> Option {\n+ let listing = v.as_array()?.first()?;\n+ let child = listing\n+ .pointer(\"/data/children/0/data\")?;\n+ let title = child.get(\"title\")?.as_str()?.to_string();\n+ let author = child\n+ .get(\"author\")\n+ .and_then(|a| a.as_str())\n+ .filter(|a| *a != \"[deleted]\")\n+ .map(|s| s.to_string());\n+ let body_html = child\n+ .get(\"selftext_html\")\n+ .and_then(|t| t.as_str())\n+ .filter(|s| !s.is_empty())\n+ .map(|s| s.to_string());\n+ let thumb_url = child\n+ .get(\"thumbnail\")\n+ .and_then(|t| t.as_str())\n+ .filter(|s| s.starts_with(\"http\"))\n+ .map(|s| s.to_string());\n+\n+ Some(EntityData {\n+ title,\n+ author,\n+ body_html,\n+ thumb_url,\n+ })\n+}\n+\n+#[cfg(test)]\n+mod tests {\n+ use super::*;\n+\n+ #[test]\n+ fn map_subreddit_about_url() {\n+ let id = ItemId::parse(\"reddit.com/r/rust\").unwrap();\n+ assert_eq!(\n+ map_item_to_reddit_api(&id, false),\n+ \"https://www.reddit.com/r/rust/about.json?raw_json=1\"\n+ );\n+ assert_eq!(\n+ map_item_to_reddit_api(&id, true),\n+ \"https://oauth.reddit.com/r/rust/about.json?raw_json=1\"\n+ );\n+ }\n+\n+ #[test]\n+ fn map_post_url() {\n+ let id =\n+ ItemId::parse(\"reddit.com/r/amitheasshole/comments/1trnvdl\").unwrap();\n+ assert_eq!(\n+ map_item_to_reddit_api(&id, false),\n+ \"https://www.reddit.com/r/amitheasshole/comments/1trnvdl.json?raw_json=1\"\n+ );\n+ }\n+\n+ #[test]\n+ fn map_non_reddit_empty() {\n+ let id = ItemId::opaque(\"example.com/foo\");\n+ assert!(map_item_to_reddit_api(&id, false).is_empty());\n+ }\n+\n+ #[test]\n+ fn parse_subreddit_fixture() {\n+ let json = r#\"{\"kind\":\"t5\",\"data\":{\"title\":\"Rust\",\"display_name\":\"rust\",\"public_description\":\"systems\"}}\"#;\n+ let entity = parse_reddit_json(\n+ &ItemId::parse(\"reddit.com/r/rust\").unwrap(),\n+ json.as_bytes(),\n+ )\n+ .unwrap();\n+ assert_eq!(entity.title, \"Rust\");\n+ }\n+\n+ #[test]\n+ fn parse_post_fixture() {\n+ let json = r#\"[{\"kind\":\"Listing\",\"data\":{\"children\":[{\"kind\":\"t3\",\"data\":{\"title\":\"AITA\",\"author\":\"op\",\"selftext_html\":\"<p>hi</p>\",\"thumbnail\":\"https://b.thumbs.redditmedia.com/x.jpg\"}}]}}]\"#;\n+ let entity = parse_reddit_json(\n+ &ItemId::parse(\"reddit.com/r/x/comments/abc\").unwrap(),\n+ json.as_bytes(),\n+ )\n+ .unwrap();\n+ assert_eq!(entity.title, \"AITA\");\n+ assert_eq!(entity.author.as_deref(), Some(\"op\"));\n+ }\n }\ndiff --git a/server/src/state.rs b/server/src/state.rs\nindex cc1722f5a5bf4d415f2327ea585c488a15a75592..8c03aa60c15aee803a534439400b69935b1a3d84 100644\n--- a/server/src/state.rs\n+++ b/server/src/state.rs\n@@ -5,9 +5,10 @@ use tokio::sync::RwLock;\n use crate::{\n event_log::EventLog,\n events::Event,\n+ journal::JournalClient,\n path_types::ItemId,\n+ reddit::{default_user_agent, RedditBroker},\n reducer::{GlobalTree, VoteData},\n- journal::JournalClient,\n views::ViewStore,\n };\n \n@@ -73,6 +74,7 @@ pub struct AppState {\n pub views: ViewStore,\n pub tree: Arc>,\n journal: JournalClient,\n+ pub reddit: RedditBroker,\n }\n \n impl AppState {\n@@ -112,6 +114,7 @@ impl AppState {\n \n let tree = Arc::new(RwLock::new(tree));\n let journal = JournalClient::spawn(tree.clone(), event_log.clone());\n+ let reddit = RedditBroker::spawn(tree.clone(), &default_user_agent());\n \n Self {\n cfg: Arc::new(cfg),\n@@ -119,6 +122,7 @@ impl AppState {\n views,\n tree,\n journal,\n+ reddit,\n }\n }\n \n@@ -127,8 +131,11 @@ impl AppState {\n id: id.as_str().to_string(),\n };\n self.event_log.append(&event).await.map_err(|e| e.to_string())?;\n- let mut w = self.tree.write().await;\n- w.ensure_path(id);\n+ {\n+ let mut w = self.tree.write().await;\n+ w.ensure_path(id);\n+ }\n+ self.reddit.request_fetch(id.clone());\n Ok(())\n }\n \ndiff --git a/todo b/todo\nnew file mode 100644\nindex 0000000000000000000000000000000000000000..d196e8cb4cc80ccb95eeff01c73607d520e13212\n--- /dev/null\n+++ b/todo\n@@ -0,0 +1,7 @@\n+reddit import (only on explicit request)\n+reddit rendering\n+vote redering\n+pair chosing\n+nsfw gate\n+\n+logins (uuid user, two sides, oauths, and pseudonyms)\n","role":"user"}],"model":"~anthropic/claude-sonnet-latest"}