Side A fixes a real, scoped security bug (XSS via unsanitized Reddit HTML) with a small, well-tested, focused change (sanitize.rs + two call sites + unit tests). Side B is a large, unfocused grab-bag (dotenv loading, explicit fetch UI, event-log payload storage/replay, reddit config refactor, integration test harness) that adds significant surface area and complexity without a single clear, well-isolated purpose, making it harder to review and more risky despite adding some legitimate architecture (raw payload persistence, explicit fetch).
constitution · epochs · watch · epoch 3
comparison
c_c0df72aee6da (tommy-mor) vs c_e57094c6229a (tommy-mor)
download prompt · raw event · cmp_8503a769b8ab46
council reasoning
B lands lasting architecture: explicit user-initiated fetch, full API payloads in EntityImported/event-log replay, entity_raw + derived views, configurable Reddit bases, and a mock-backed integration test. A is a correct, focused XSS fix (ammonia at PreEscaped render sites with unit tests) but is shorter-scope hardening on top of entity HTML, not a comparable product/design step.
Side A fixes a concrete security issue by sanitizing untrusted Reddit `body_html` with the `ammonia` library before every `maud::PreEscaped` render, adds a dedicated sanitization module, and includes tests verifying scripts and event handlers are stripped while benign markup is preserved. Side B introduces a broad Reddit import/event-log refactor, explicit fetch UI, dotenv support, and related plumbing, but it is a much larger architectural change whose core value is feature expansion rather than addressing an immediate correctness or security problem.
sides
A — c_c0df72aee6da (tommy-mor)
message
[bf118bdf] Sanitize Reddit entity body HTML before rendering. Use ammonia at render time so untrusted selftext_html cannot execute scripts in our origin. Co-authored-by: Cursor <cursoragent@cursor.com>
diff preview
diff --git a/Cargo.lock b/Cargo.lock
index 3dec7cb72a182dc654a37dca8ba0b49d77504daa..0dd4fce5fb6400ae153cca4e3dbf5a5158e6d8b4 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -11,6 +11,19 @@ dependencies = [
"memchr",
]
+[[package]]
+name = "ammonia"
+version = "4.1.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "17e913097e1a2124b46746c980134e8c954bc17a6a59bb3fde96f088d126dde6"
+dependencies = [
+ "cssparser",
+ "html5ever",
+ "maplit",
+ "tendril",
+ "url",
+]
+
[[package]]
name = "anyhow"
version = "1.0.102"
@@ -355,6 +368,29 @@ version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5"
+[[package]]
+name = "cssparser"
+version = "0.35.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4e901edd733a1472f944a45116df3f846f54d37e67e68640ac8bb69689aca2aa"
+dependencies = [
+ "cssparser-macros",
+ "dtoa-short",
+ "itoa",
+ "phf",
+ "smallvec",
+]
+
+[[package]]
+name = "cssparser-macros"
+version = "0.6.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "13b588ba4ac1a99f7f2964d24b3d896ddc6bf847ee3855dbd4366f058cfcd331"
+dependencies = [
+ "quote",
+ "syn",
+]
+
[[package]]
name = "deranged"
version = "0.5.8"
@@ -381,6 +417,21 @@ version = "0.15.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b"
+[[package]]
+name = "dtoa"
+version = "1.0.11"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590"
+
+[[package]]
+name = "dtoa-short"
+version = "0.3.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87"
+dependencies = [
+ "dtoa",
+]
+
[[package]]
name = "durable"
version = "0.2.0"
@@ -482,6 +533,16 @@ dependencies = [
"percent-encoding",
]
+[[package]]
+name = "futf"
+version = "0.1.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "df420e2e84819663797d1ec6544b13c5be84629e7bb00dc960d6917db2987843"
+dependencies = [
+ "mac",
+ "new_debug_unreachable",
+]
+
[[package]]
name = "futures-channel"
version = "0.3.32"
@@ -614,6 +675,17 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
+[[package]]
+name = "html5ever"
+version = "0.35.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "55d958c2f74b664487a2035fe1dadb032c48718a03b63f3ab0b8537db8549ed4"
+dependencies = [
+ "log",
+ "markup5ever",
+ "match_token",
+]
+
[[package]]
name = "http"
version = "1.4.1"
@@ -974,6 +1046,15 @@ version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
+[[package]]
+name = "lock_api"
+version = "0.4.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965"
+dependencies = [
+ "scopeguard",
+]
+
[[package]]
name = "log"
version = "0.4.30"
@@ -990,6 +1071,40 @@ dependencies = [
"libc",
]
+[[package]]
+name = "mac"
+version = "0.1.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
+
+[[package]]
+name = "maplit"
+version = "1.0.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3e2e65a1a2e43cfcb47a895c4c8b10d1f4a61097f9f254f183aee60cad9c651d"
+
+[[package]]
+name = "markup5ever"
+version = "0.35.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "311fe69c934650f8f19652b3946075f0fc41ad8757dbb68f1ca14e7900ecc1c3"
+dependencies = [
+ "log",
+ "tendril",
+ "web_atoms",
+]
+
+[[package]]
+name = "match_token"
+version = "0.35.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ac84fd3f360fcc43dc5f5d186f02a94192761a080e8bc58621ad4d12296a58cf"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
[[package]]
name = "matchers"
version = "0.2.0"
@@ -1092,6 +1207,12 @@ dependencies = [
"tempfile",
]
+[[package]]
+name = "new_debug_unreachable"
+version = "1.0.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
+
[[package]]
name = "nom"
version = "7.1.3"
@@ -1175,6 +1296,29 @@ dependencies = [
"vcpkg",
]
+[[package]]
+name = "parking_lot"
+version = "0.12.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a"
+dependencies = [
+ "lock_api",
+ "parking_lot_core",
+]
+
+[[package]]
+name = "parking_lot_core"
+version = "0.9.12"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1"
+dependencies = [
+ "cfg-if",
+ "libc",
+ "redox_syscall",
+ "smallvec",
+ "windows-link",
+]
+
[[package]]
name = "peeking_take_while"
version = "0.1.2"
@@ -1187,6 +1331,58 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
+[[package]]
+name = "phf"
+version = "0.11.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078"
+dependencies = [
+ "phf_macros",
+ "phf_shared",
+]
+
+[[package]]
+name = "phf_codegen"
+version = "0.11.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a"
+dependencies = [
+ "phf_generator",
+ "phf_shared",
+]
+
+[[package]]
+name = "phf_generator"
+version = "0.11.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d"
+dependencies = [
+ "phf_shared",
+ "rand 0.8.6",
+]
+
+[[package]]
+name = "phf_macros"
+version = "0.11.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f84ac04429c13a7ff43785d75ad27569f2951ce0ffd30a3321230db2fc727216"
+dependencies = [
+ "phf_generator",
+ "phf_shared",
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "phf_shared"
+version = "0.11.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5"
+dependencies = [
+ "siphasher",
+]
+
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -1223,6 +1419,12 @@ dependencies = [
"zerocopy",
]
+[[package]]
+name = "precomputed-hash"
+version = "0.1.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
+
[[package]]
name = "prettyplease"
version = "0.2.37"
@@ -1379,6 +1581,15 @@ dependencies = [
"rand_core 0.9.5",
]
+[[package]]
+name = "redox_syscall"
+version = "0.5.18"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
+dependencies = [
+ "bitflags 2.11.1",
+]
+
[[package]]
name = "regex"
version = "1.12.3"
@@ -1563,6 +1774,12 @@ dependencies = [
"windows-sys 0.61.2",
]
+[[package]]
+name = "scopeguard"
+version = "1.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
+
[[package]]
name = "security-framework"
version = "3.7.0"
@@ -1683,6 +1900,12 @@ dependencies = [
"libc",
]
+[[package]]
+name = "siphasher"
+version = "1.0.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
+
[[package]]
name = "slab"
version = "0.4.12"
@@ -1709,6 +1932,7 @@ dependencies = [
name = "sorter2-server"
version = "0.0.1"
dependencies = [
+ "ammonia",
"async-stream",
"axum",
"axum-extra",
@@ -1742,6 +1966,31 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
+[[package]]
+name = "string_cache"
+version = "0.8.9"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "bf776ba3fa74f83bf4b63c3dcbbf82173db2632ed8452cb2d891d33f459de70f"
+dependencies = [
+ "new_debug_unreachable",
+ "parking_lot",
+ "phf_shared",
+ "precomputed-hash",
+ "serde",
+]
+
+[[package]]
+name = "string_cache_codegen"
+version = "0.5.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c711928715f1fe0fe509c53b43e993a9a557babc2d0a3567d0a3006f1ac931a0"
+dependencies = [
+ "phf_generator",
+ "phf_shared",
+ "proc-macro2",
+ "quote",
+]
+
[[package]]
name = "subtle"
version = "2.6.1"
@@ -1813,6 +2062,17 @@ dependencies = [
"windows-sys 0.61.2",
]
+[[package]]
+name = "tendril"
+version = "0.4.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d24a120c5fc464a3458240ee02c299ebcb9d67b5249c8848b09d639dca8d7bb0"
+dependencies = [
+ "futf",
+ "mac",
+ "utf-8",
+]
+
[[package]]
name = "thiserror"
version = "1.0.69"
@@ -2116,6 +2376,12 @@ version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "daf8dba3b7eb870caf1ddeed7bc9d2a049f3cfdfae7cb521b087cc33ae4c49da"
+[[package]]
+name = "utf-8"
+version = "0.7.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9"
+
[[package]]
name = "utf8_iter"
version = "1.0.4"
@@ -2281,6 +2547,18 @@ dependencies = [
"wasm-bindgen",
]
+[[package]]
+name = "web_atoms"
+version = "0.1.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57ffde1dc01240bdf9992e3205668b235e59421fd085e8a317ed98da0178d414"
+dependencies = [
+ "phf",
+ "phf_codegen",
+ "string_cache",
+ "string_cache_codegen",
+]
+
[[package]]
name = "windows-link"
version = "0.2.1"
diff --git a/server/Cargo.toml b/server/Cargo.toml
index 47659ff82fbfb50972eb2b87575e80f66e572ba4..27f552c20b97ef28cdde4cb6b1a4980375135111 100644
--- a/server/Cargo.toml
+++ b/server/Cargo.toml
@@ -13,6 +13,7 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "1"
maud = { version = "0.26", features = ["axum"] }
+ammonia = "4.1"
tower = "0.5"
tower-http = { version = "0.5", features = ["trace"] }
tracing = "0.1"
diff --git a/server/src/fetch/html.rs b/server/src/fetch/html.rs
index dadf050515f0473943dad97df5d318032c8cb385..5b160c6b8bd216dfaf80149854aec0566cd00460 100644
--- a/server/src/fetch/html.rs
+++ b/server/src/fetch/html.rs
@@ -4,6 +4,7 @@ use maud::{html, Markup};
use crate::{
form_template::template_json_compact,
+ html::sanitize::entity_body_html,
path_types::ItemId,
reddit::{is_children_fetchable, is_fetchable},
reducer::NodeState,
@@ -27,7 +28,7 @@ pub fn entity_panel(node: &NodeState) -> Markup {
p class="muted small" { "by " (author) }
}
@if let Some(body) = &data.body_html {
- div class="entity-body" { (maud::PreEscaped(body)) }
+ div class="entity-body" { (maud::PreEscaped(entity_body_html(body))) }
}
}
}
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index e180a0ca542a33e2300c0a4809e6b9cfee07ecfe..a58cbbee3490a08a625cb06df06848c59a615d65 100644
--- a/server/src/html/mod.rs
+++ b/server/src/html/mod.rs
@@ -20,6 +20,7 @@ use crate::{
ui
… preview truncated; 2,223 characters omittedB — c_e57094c6229a (tommy-mor)
message
[40b975bf] nice
diff preview
diff --git a/Cargo.lock b/Cargo.lock
index 266e876bb7ccbe788beb1d5bd53ad5b45ee5825b..2cea973082716e761ef6f5dd5886acc08ff9aac0 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -222,6 +222,12 @@ dependencies = [
"syn",
]
+[[package]]
+name = "dotenvy"
+version = "0.15.7"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b"
+
[[package]]
name = "encoding_rs"
version = "0.8.35"
@@ -1238,6 +1244,7 @@ version = "0.0.1"
dependencies = [
"axum",
"axum-extra",
+ "dotenvy",
"maud",
"reqwest",
"serde",
diff --git a/server/Cargo.toml b/server/Cargo.toml
index 4677fedcb45292eebebe7e9cf6ce2f5738f18ddf..bd600138b613bd0f546bdec217a5334cdcb20aa5 100644
--- a/server/Cargo.toml
+++ b/server/Cargo.toml
@@ -17,6 +17,7 @@ tower-http = { version = "0.5", features = ["trace"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
reqwest = { version = "0.12", features = ["json"] }
+dotenvy = "0.15"
[dev-dependencies]
reqwest = { version = "0.12", features = ["json"] }
diff --git a/server/src/api/ui_html.rs b/server/src/api/ui_html.rs
index d2024bd4582bcc8482b461b2ba4fedbd8bff7c66..b33a84e8bb5e817b26592868d88090e6d664d950 100644
--- a/server/src/api/ui_html.rs
+++ b/server/src/api/ui_html.rs
@@ -6,7 +6,7 @@ use axum::{
use std::collections::HashMap;
use crate::{
- html::{input_panel, js_string_literal, ranking_panel, JsBuilder},
+ html::{entity_section, input_panel, js_string_literal, ranking_panel, JsBuilder},
parser::parse_reddit_url,
path_types::ItemId,
reddit::ensure_partial_tree,
@@ -87,6 +87,20 @@ pub async fn post_ui_html(
.into_response()
}
},
+ HtmlUiAction::FetchEntity { item } => {
+ let id = parse_item_param(&item);
+ if id.is_root() {
+ return ui_js_warn("nothing to fetch for the root").into_response();
+ }
+ state.queue_entity_fetch(id.clone());
+ let tree = state.tree.read().await;
+ let empty = crate::reducer::NodeState::default();
+ let node = tree.get(&id).unwrap_or(&empty);
+ let panel = entity_section(&id, node, true);
+ JsBuilder::new()
+ .morph_selector("#entity-section", panel)
+ .into_response()
+ },
}
}
diff --git a/server/src/events.rs b/server/src/events.rs
index ed5be6b13b9d46e838831d6ce0f96f569b401730..07ce24b5e56cf72b0b442c3c3241efbf6c3b006a 100644
--- a/server/src/events.rs
+++ b/server/src/events.rs
@@ -1,4 +1,5 @@
use serde::{Deserialize, Serialize};
+use serde_json::Value;
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
@@ -18,4 +19,10 @@ pub enum Event {
},
/// Register a node path in the fractal tree (no external fetch).
NodeEnsured { id: String },
+ /// Full upstream API payload for a node (domain-specific view derived at replay/render time).
+ EntityImported {
+ id: String,
+ ts: i64,
+ payload: Value,
+ },
}
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index df6505021d9f446c2b453e20e3eb3cf696a111f9..caf1309c8d93b47104499c57f9cc35ee7631fbb9 100644
--- a/server/src/html/mod.rs
+++ b/server/src/html/mod.rs
@@ -10,6 +10,7 @@ use crate::{
form_template::template_json_compact,
path_types::ItemId,
ranking::{top_bottom, RankedItem},
+ reddit::is_fetchable,
reducer::{GroupState, NodeState},
state::AppState,
ui_action::UI_RPC_FIELD,
@@ -151,7 +152,7 @@ pub fn breadcrumb_path(item: &ItemId) -> Markup {
fn entity_panel(node: &NodeState) -> Markup {
html! {
@if let Some(data) = &node.data {
- section id="entity-panel" class="demo-panel entity-card" {
+ div id="entity-panel" class="entity-card" {
h2 { (data.title) }
@if let Some(author) = &data.author {
p class="muted small" { "by " (author) }
@@ -164,6 +165,42 @@ fn entity_panel(node: &NodeState) -> Markup {
}
}
+/// Reddit/API import control — only shown on fetchable pages; never auto-fires.
+pub fn fetch_entity_panel(item: &ItemId, has_data: bool, fetching: bool) -> Markup {
+ if !is_fetchable(item) {
+ return html! {};
+ }
+ let label = if fetching {
+ "Fetching…"
+ } else if has_data {
+ "Fetch more"
+ } else {
+ "Fetch from Reddit"
+ };
+ let rpc = template_json_compact(&serde_json::json!({
+ "action": "fetch_entity",
+ "item": item.as_str(),
+ }))
+ .expect("fetch_entity rpc template");
+ html! {
+ form method="post" action="/ui" id="fetch-entity-form" class="fetch-entity-form" {
+ input type="hidden" name=(UI_RPC_FIELD) value=(rpc);
+ button type="submit" class="btn-secondary" disabled=(fetching) { (label) }
+ }
+ }
+}
+
+/// Entity card + explicit fetch control (morphed as `#entity-section`).
+pub fn entity_section(item: &ItemId, node: &NodeState, fetching: bool) -> Markup {
+ let has_data = node.data.is_some();
+ html! {
+ section id="entity-section" class="demo-panel" {
+ (entity_panel(node))
+ (fetch_entity_panel(item, has_data, fetching))
+ }
+ }
+}
+
fn rank_list(label: &str, items: &[RankedItem], start_rank: usize) -> Markup {
html! {
@if !items.is_empty() {
@@ -260,7 +297,7 @@ async fn item_page(state: AppState, uri: Uri, item: ItemId) -> Markup {
h1 { "sorter" }
(input_panel("", None))
(breadcrumb_path(&item))
- (entity_panel(node))
+ (entity_section(&item, node, false))
(ranking_panel(&item, group))
};
layout("sorter2", body, views)
@@ -272,16 +309,5 @@ pub async fn home(State(state): State<AppState>, uri: Uri) -> impl IntoResponse
pub async fn browse(State(state): State<AppState>, uri: Uri) -> impl IntoResponse {
let item = ItemId::from_browse_uri(uri.path()).unwrap_or(ItemId::root());
- if item.as_str().starts_with("reddit.com") {
- let needs_fetch = {
- let tree = state.tree.read().await;
- tree.get(&item)
- .map(|n| n.data.is_none())
- .unwrap_or(true)
- };
- if needs_fetch {
- state.reddit.request_fetch(item.clone());
- }
- }
item_page(state, uri, item).await
}
diff --git a/server/src/main.rs b/server/src/main.rs
index c22ec6c9f5358e5ec99fb83210dc351938505a93..1f0cddc39302b35b0cd6a6219f44c9d59202facf 100644
--- a/server/src/main.rs
+++ b/server/src/main.rs
@@ -2,6 +2,10 @@ use sorter2_server::state::AppConfig;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+ if std::env::var("SORTER2_SKIP_DOTENV").is_err() {
+ let _ = dotenvy::dotenv();
+ }
+
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
diff --git a/server/src/reddit.rs b/server/src/reddit.rs
index 90053ad03b1d7c8e94f325dd4ee64c2b4f7da900..ff0f01e57b18af878eb5be3efc47204a7673589d 100644
--- a/server/src/reddit.rs
+++ b/server/src/reddit.rs
@@ -6,11 +6,15 @@ use std::time::{Duration, Instant};
use reqwest::{header, Client, StatusCode};
use serde::Deserialize;
+use serde_json::Value;
use tokio::sync::{mpsc, RwLock};
use crate::{
+ event_log::EventLog,
+ events::Event,
+ html::now_ms,
path_types::ItemId,
- reducer::{EntityData, GlobalTree},
+ reducer::GlobalTree,
};
/// Bootstrap blank nodes along a URL path so breadcrumbs and voting work before fetch.
@@ -20,6 +24,8 @@ pub fn ensure_partial_tree(tree: &mut GlobalTree, id: &ItemId) {
pub struct RedditCommand {
pub id: ItemId,
+ /// User-initiated fetch bypasses the in-memory "recently fetched" cache.
+ pub force: bool,
}
#[derive(Clone)]
@@ -33,19 +39,31 @@ struct RedditCredentials {
client_secret: String,
}
+#[derive(Clone)]
+pub struct RedditApiConfig {
+ pub api_base: String,
+ pub oauth_base: String,
+ pub user_agent: String,
+ creds: Option<RedditCredentials>,
+}
+
struct OAuthToken {
access_token: String,
expires_at: Instant,
}
impl RedditBroker {
- pub fn spawn(tree: Arc<RwLock<GlobalTree>>, user_agent: &str) -> Self {
+ pub fn spawn(
+ tree: Arc<RwLock<GlobalTree>>,
+ event_log: Arc<EventLog>,
+ config: RedditApiConfig,
+ ) -> Self {
let (tx, rx) = mpsc::channel(100);
let mut headers = header::HeaderMap::new();
headers.insert(
header::USER_AGENT,
- header::HeaderValue::from_str(user_agent).expect("valid user agent"),
+ header::HeaderValue::from_str(&config.user_agent).expect("valid user agent"),
);
let client = Client::builder()
@@ -54,22 +72,38 @@ impl RedditBroker {
.build()
.expect("reqwest client");
- let creds = RedditCredentials::from_env();
- tokio::spawn(reddit_worker(rx, tree, client, creds));
+ tokio::spawn(reddit_worker(rx, tree, event_log, client, config));
Self { tx }
}
- /// Fire-and-forget: queue a fetch; worker updates the tree when done.
- pub fn request_fetch(&self, id: ItemId) {
- let _ = self.tx.try_send(RedditCommand { id });
+ /// Queue a fetch; drops when the channel is full (backpressure).
+ pub fn request_fetch(&self, id: ItemId, force: bool) {
+ let _ = self.tx.try_send(RedditCommand { id, force });
+ }
+}
+
+impl RedditApiConfig {
+ pub fn from_env() -> Self {
+ Self {
+ api_base: reddit_api_base(),
+ oauth_base: reddit_oauth_base(),
+ user_agent: default_user_agent(),
+ creds: RedditCredentials::from_env(),
+ }
}
}
impl RedditCredentials {
+ /// Reddit's OAuth docs call these "client id" and "client secret"; the app
+ /// registration UI often labels them "app id" / "app secret" — same values.
fn from_env() -> Option<Self> {
- let client_id = std::env::var("REDDIT_CLIENT_ID").ok()?;
- let client_secret = std::env::var("REDDIT_CLIENT_SECRET").ok()?;
+ let client_id = std::env::var("REDDIT_CLIENT_ID")
+ .or_else(|_| std::env::var("REDDIT_APP_ID"))
+ .ok()?;
+ let client_secret = std::env::var("REDDIT_CLIENT_SECRET")
+ .or_else(|_| std::env::var("REDDIT_APP_SECRET"))
+ .ok()?;
if client_id.is_empty() || client_secret.is_empty() {
return None;
}
@@ -80,29 +114,63 @@ impl RedditCredentials {
}
}
+pub fn reddit_api_base() -> String {
+ std::env::var("REDDIT_API_BASE").unwrap_or_else(|_| "https://www.reddit.com".into())
+}
+
+pub fn reddit_oauth_base() -> String {
+ std::env::var("REDDIT_OAUTH_BASE").unwrap_or_else(|_| "https://www.reddit.com".into())
+}
+
pub fn default_user_agent() -> String {
std::env::var("REDDIT_USER_AGENT").unwrap_or_else(|_| {
"web:sorter2.social:v0.0.1 (by /u/sorter2)".to_string()
})
}
+/// True when this node can be loaded from the Reddit JSON API.
+pub fn is_fetchable(id: &ItemId) -> bool {
+ !map_item_to_reddit_api(id, "https://example.com").is_empty()
+}
+
+/// Derive UI-facing fields from a stored payload (Reddit-specific when under reddit.com).
+pub fn entity_view_from_payload(id: &ItemId, payload: &Value) -> Option<crate::reducer::EntityData> {
+ if id.as_str().starts_with("reddit.com") {
+ return parse_reddit_view(id, payload);
+ }
+ None
+}
+
+/// Apply a full API payload to the in-memory tree (view derived for known domains).
+pub fn apply_entity_import(tree: &mut GlobalTree, id: &ItemId, payload: Value) {
+ let view = entity_view_from_payload(id, &payload);
+ tree.apply_entity_raw(id, payload, view);
+}
+
async fn red
… preview truncated; 22,673 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.