Side A delivers a coherent, tested feature (server-rendered theme cookie handling plus room-scoped canonical URL helpers) with unit tests and a genuine bugfix (theme cookie dropped on login). Side B's Reddit importer, while more elaborate, ships with rough edges—a leftover 'sorter2' user-agent string, a stray 'todo' file admitting the feature is unfinished, a duplicated reqwest dependency, and an auto-fetch-on-view behavior that contradicts its own todo note about fetching only on explicit request—making it less polished and less clearly finished than Side A's focused, tested changes.
constitution · epochs · watch · epoch 3
comparison
c_3f420a1f5aa1 (tommy-mor) vs c_c124c217f89c (tommy-mor)
download prompt · raw event · cmp_7629f7dccb9eb7
council reasoning
A lands durable product correctness: cookie/SSR theme switching with auth cookie re-issue, plus room-aware wire URLs (`item_path_for_api_in_room`, `forum_thread_web_url`) wired through RPC with tests—fixing private-room API links and CLI hints. B adds a solid Reddit fetch worker (OAuth, rate limits, parse/map tests), but it is a narrower external-import foundation still marked incomplete versus A’s core API/UX paths.
Side A delivers two lasting functional improvements across the codebase: a complete server-backed theme system (cookie persistence, theme selection, propagation through layouts, preserving the theme across authentication redirects) and correct room-aware URL generation for RPC/web responses via new helper functions with accompanying tests. Side B adds substantial Reddit background-fetch infrastructure with OAuth, rate limiting, and parsing, but it is only partially integrated (triggering fetches while rendering/importing) and leaves rendering and broader functionality unfinished, reducing its immediate long-term project value.
sides
A — c_3f420a1f5aa1 (tommy-mor)
message
[2fe70b0e] themes
diff preview
diff --git a/server/src/api/auth.rs b/server/src/api/auth.rs
index 559db65de14c6157690ffbf18eca0cf65b0a5202..b3631b06153d52f88348fef927e7a024b7b85ad6 100644
--- a/server/src/api/auth.rs
+++ b/server/src/api/auth.rs
@@ -1,7 +1,7 @@
use axum::{
body::Body,
extract::{Path, Query, State},
- http::{header, HeaderMap, HeaderValue, StatusCode},
+ http::{header, HeaderMap, HeaderValue, StatusCode, Uri},
response::{IntoResponse, Redirect, Response},
Form, Json,
};
@@ -17,7 +17,7 @@ use crate::{
events::{Event, GrantAdded, TokenIssued, UserRegistered},
html::{
auth_complete_page, auth_signed_in_fragment, choose_username_error_fragment,
- choose_username_page, JsBuilder,
+ choose_username_page, theme_cookie_header_from_jar, theme_from_jar, theme_next_from_uri, JsBuilder,
},
identity::{parse_agent, parse_username},
reducer::ReducerState,
@@ -48,15 +48,17 @@ fn js_form_error_fragment(session: &str, error: &str) -> Response {
.into_response()
}
-fn js_signed_in_fragment(bearer: &str) -> Response {
+fn js_signed_in_fragment(bearer: &str, jar: &CookieJar) -> Response {
let mut response = JsBuilder::new()
.id("choose-username-form")
.morph_inner(auth_signed_in_fragment())
.redirect("/auth/complete")
.into_response();
- response
- .headers_mut()
- .insert(header::SET_COOKIE, session_cookie_header_value(bearer));
+ let headers = response.headers_mut();
+ headers.append(header::SET_COOKIE, session_cookie_header_value(bearer));
+ if let Some(theme) = theme_cookie_header_from_jar(jar) {
+ headers.append(header::SET_COOKIE, theme);
+ }
response
}
@@ -69,13 +71,18 @@ pub fn optional_principal(headers: &HeaderMap, jar: &CookieJar, reduced: &Reduce
verify_token(reduced, c.value()).ok()
}
-fn redirect_with_session_cookie(public_url: &str, path_and_query: &str, bearer: &str) -> Response {
- Response::builder()
+fn redirect_with_session_cookie(public_url: &str, path_and_query: &str, bearer: &str, jar: &CookieJar) -> Response {
+ let mut res = Response::builder()
.status(StatusCode::TEMPORARY_REDIRECT)
.header(header::LOCATION, format!("{public_url}{path_and_query}"))
- .header(header::SET_COOKIE, session_cookie_header_value(bearer))
.body(Body::empty())
- .unwrap()
+ .unwrap();
+ let headers = res.headers_mut();
+ headers.append(header::SET_COOKIE, session_cookie_header_value(bearer));
+ if let Some(theme) = theme_cookie_header_from_jar(jar) {
+ headers.append(header::SET_COOKIE, theme);
+ }
+ res
}
async fn apply_invite_redemption(state: &AppState, invite_token: &str, grantee_username: &str) -> Result<(), String> {
@@ -286,7 +293,11 @@ pub struct AuthCallbackQuery {
pub state: String,
}
-pub async fn get_auth_callback(Query(q): Query<AuthCallbackQuery>, State(state): State<AppState>) -> impl IntoResponse {
+pub async fn get_auth_callback(
+ Query(q): Query<AuthCallbackQuery>,
+ State(state): State<AppState>,
+ jar: CookieJar,
+) -> impl IntoResponse {
let sessions = pending_sessions(&state);
{
let sessions_read = sessions.read().await;
@@ -365,7 +376,7 @@ pub async fn get_auth_callback(Query(q): Query<AuthCallbackQuery>, State(state):
}
let cookie_bearer = bearer.clone();
s.complete = Some((username, bearer));
- return redirect_with_session_cookie(&public_url, "/", &cookie_bearer).into_response();
+ return redirect_with_session_cookie(&public_url, "/", &cookie_bearer, &jar).into_response();
}
}
@@ -378,14 +389,20 @@ pub struct ChooseUsernameQuery {
pub error: Option<String>,
}
-pub async fn get_choose_username(Query(q): Query<ChooseUsernameQuery>, State(state): State<AppState>) -> impl IntoResponse {
+pub async fn get_choose_username(
+ Query(q): Query<ChooseUsernameQuery>,
+ State(state): State<AppState>,
+ jar: CookieJar,
+ uri: Uri,
+) -> impl IntoResponse {
let sessions = pending_sessions(&state);
let sessions_read = sessions.read().await;
if !sessions_read.contains_key(&q.session) {
return api_error(StatusCode::NOT_FOUND, "unknown session", None).into_response();
}
drop(sessions_read);
- choose_username_page(&q.session, q.error.as_deref()).into_response()
+ let next = theme_next_from_uri(&uri);
+ choose_username_page(&q.session, q.error.as_deref(), theme_from_jar(&jar), &next).into_response()
}
#[derive(Debug, Deserialize)]
@@ -396,6 +413,7 @@ pub struct ChooseUsernameForm {
pub async fn post_choose_username(
State(state): State<AppState>,
+ jar: CookieJar,
Form(form): Form<ChooseUsernameForm>,
) -> impl IntoResponse {
let canon_user = match parse_username(&form.username) {
@@ -477,7 +495,7 @@ pub async fn post_choose_username(
s.complete = Some((canon_user.clone(), bearer.clone()));
}
- js_signed_in_fragment(&bearer).into_response()
+ js_signed_in_fragment(&bearer, &jar).into_response()
}
/// Start a browser-only OAuth flow (no CLI polling). Sets session cookie on success.
@@ -569,8 +587,9 @@ pub async fn get_pending_session(
.into_response()
}
-pub async fn get_auth_complete() -> impl IntoResponse {
- auth_complete_page()
+pub async fn get_auth_complete(jar: CookieJar, uri: Uri) -> impl IntoResponse {
+ let next = theme_next_from_uri(&uri);
+ auth_complete_page(theme_from_jar(&jar), &next).into_response()
}
pub async fn get_whoami(State(state): State<AppState>, headers: HeaderMap) -> impl IntoResponse {
diff --git a/server/src/api/helpers.rs b/server/src/api/helpers.rs
index 81e2a55fa3abb8609b4099f91a989480336e11eb..9b71491e9f9efc44a2a4beba09be8f64bd2ff2ee 100644
--- a/server/src/api/helpers.rs
+++ b/server/src/api/helpers.rs
@@ -39,6 +39,55 @@ pub fn item_path_for_api(item: &str) -> String {
}
}
+/// Same as [`item_path_for_api`], but for private rooms ontology items are prefixed with
+/// `/r/{short}/{slug}` so the URL matches the web app (`/r/…/~/…` routes).
+pub fn item_path_for_api_in_room(item: &str, room_wire: &str) -> String {
+ let room = room_wire.trim();
+ if room.is_empty() || room == "public" {
+ return item_path_for_api(item);
+ }
+ let Some((short, slug)) = room.split_once('/') else {
+ return item_path_for_api(item);
+ };
+ if short.is_empty() || slug.is_empty() {
+ return item_path_for_api(item);
+ }
+ let Some(c) = CanonicalItemUrl::parse(item) else {
+ return item_path_for_api(item);
+ };
+ let root = CanonicalItemUrl::ontology_root();
+ let item_norm = c.as_str().trim_end_matches('/');
+ let root_norm = root.as_str().trim_end_matches('/');
+ if let Some(tail) = c.tilde_tail() {
+ return if tail.is_empty() {
+ format!("https://slug.social/r/{short}/{slug}/~")
+ } else {
+ format!("https://slug.social/r/{short}/{slug}/~/{}", tail)
+ };
+ }
+ if item_norm == root_norm {
+ return format!("https://slug.social/r/{short}/{slug}/~");
+ }
+ item_path_for_api(item)
+}
+
+/// Absolute thread URL for forum JSON (`/t/…` vs `/r/…/t/…`).
+pub fn forum_thread_web_url(room_wire: &str, thread_tag: &str) -> String {
+ let room = room_wire.trim();
+ let tag = thread_tag.trim().trim_start_matches('#');
+ if room.is_empty() || room == "public" {
+ format!("https://slug.social/t/{tag}")
+ } else if let Some((short, slug)) = room.split_once('/') {
+ if short.is_empty() || slug.is_empty() {
+ format!("https://slug.social/t/{tag}")
+ } else {
+ format!("https://slug.social/r/{short}/{slug}/t/{tag}")
+ }
+ } else {
+ format!("https://slug.social/t/{tag}")
+ }
+}
+
/// Resolve an item path as a first-class canonical path.
pub fn resolve_item(item: &str) -> Result<String, String> {
let canonical = canonicalize_item(item);
@@ -188,3 +237,52 @@ pub fn vote_touches_path(a: &str, b: &str, parent_canon: &str) -> bool {
let under = |item: &str| item == parent_canon || item.starts_with(&format!("{}/", parent_canon));
under(a) || under(b)
}
+
+#[cfg(test)]
+mod wire_url_tests {
+ use super::{forum_thread_web_url, item_path_for_api_in_room};
+
+ #[test]
+ fn public_room_unchanged() {
+ let u = "https://slug.social/~/a/b";
+ assert_eq!(item_path_for_api_in_room(u, "public"), u);
+ }
+
+ #[test]
+ fn private_room_prefixes_ontology() {
+ assert_eq!(
+ item_path_for_api_in_room("https://slug.social/~/topic/x", "9ab12cd/my-room"),
+ "https://slug.social/r/9ab12cd/my-room/~/topic/x"
+ );
+ }
+
+ #[test]
+ fn private_room_ontology_root() {
+ assert_eq!(
+ item_path_for_api_in_room("https://slug.social/~", "9ab12cd/my-room"),
+ "https://slug.social/r/9ab12cd/my-room/~"
+ );
+ assert_eq!(
+ item_path_for_api_in_room("https://slug.social/~/", "9ab12cd/my-room"),
+ "https://slug.social/r/9ab12cd/my-room/~"
+ );
+ }
+
+ #[test]
+ fn external_url_untouched_in_private_room() {
+ let u = "https://example.com/z";
+ assert_eq!(item_path_for_api_in_room(u, "9ab12cd/my-room"), u);
+ }
+
+ #[test]
+ fn forum_web_public_vs_room() {
+ assert_eq!(
+ forum_thread_web_url("public", "debate"),
+ "https://slug.social/t/debate"
+ );
+ assert_eq!(
+ forum_thread_web_url("9ab12cd/my-room", "#debate"),
+ "https://slug.social/r/9ab12cd/my-room/t/debate"
+ );
+ }
+}
diff --git a/server/src/api/rpc.rs b/server/src/api/rpc.rs
index 31f5fcfb4eaf4df0a9cbac532dd3dedfe3611810..5b91f5836625eedbb1cd9423168046e3fb576c17 100644
--- a/server/src/api/rpc.rs
+++ b/server/src/api/rpc.rs
@@ -27,8 +27,9 @@ use crate::{
use super::auth::verify_bearer_principal;
use super::helpers::{
- compute_connectivity_stats, is_pair_voted, item_path_for_api, now_ms, paginate_rankings,
- parse_parent_specs, pick_random_distinct, resolve_item, vote_touches_path,
+ compute_connectivity_stats, forum_thread_web_url, is_pair_voted, item_path_for_api,
+ item_path_for_api_in_room, now_ms, paginate_rankings, parse_parent_specs, pick_random_distinct,
+ resolve_item, vote_touches_path,
};
use super::validate::{normalize_room_and_thread, validate_ingest_document};
@@ -148,6 +149,7 @@ fn compute_scope_rank_changes(
parent: &str,
before: &crate::scope_rank::ChildrenRankings,
after: &crate::scope_rank::ChildrenRankings,
+ room_wire: &str,
) -> Option<ScopeRankChanges> {
fn build_positions(rankings: &crate::scope_rank::ChildrenRankings) -> HashMap<String, Option<RankPosition>> {
let mut map = HashMap::new();
@@ -182,7 +184,7 @@ fn compute_scope_rank_changes(
};
if changed {
changes.push(RankChange {
- item: item_path_for_api(&item),
+ item: item_path_for_api_in_room(&item, room_wire),
before: b,
after: a,
});
@@ -204,7 +206,7 @@ fn compute_scope_rank_changes(
parent: if parent.is_empty() {
"/".to_string()
} else {
- item_path_for_api(parent)
+ item_path_for_api_in_room(parent, room_wire)
},
changes,
})
@@ -256,6 +258,7 @@ fn build_rank_response_for_content(
offset: usize,
limit: Option<usize>,
want_percent: bool,
+ room_wire: &str,
) -> Result<RankResponse, RpcErr> {
let parent_owned = parent.map(|s| s.to_string());
let specs = parse_parent_specs(parent_owned.as_ref());
@@ -299,7 +302,7 @@ fn build_rank_response_for_content(
.ranked
.into_iter()
.map(|r| RankRow {
-
… preview truncated; 42,928 characters omittedB — c_c124c217f89c (tommy-mor)
message
[8d8230d1] reddit
diff preview
diff --git a/.gitignore b/.gitignore
index 4c7073f9fac0c30fd2050d79a60ef447af58ebeb..ada462e900d24a3a6d08165d158c80f79c35a5a5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@
data/
repomix-output.xml
dev-data/
+.env
diff --git a/server/Cargo.toml b/server/Cargo.toml
index 7906a8547d56b8e6a48ef59c37aa82a8510fdee9..4677fedcb45292eebebe7e9cf6ce2f5738f18ddf 100644
--- a/server/Cargo.toml
+++ b/server/Cargo.toml
@@ -16,6 +16,7 @@ tower = "0.5"
tower-http = { version = "0.5", features = ["trace"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+reqwest = { version = "0.12", features = ["json"] }
[dev-dependencies]
reqwest = { version = "0.12", features = ["json"] }
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index 2864407ed6e8ec284a1dc663acf1805534566b62..df6505021d9f446c2b453e20e3eb3cf696a111f9 100644
--- a/server/src/html/mod.rs
+++ b/server/src/html/mod.rs
@@ -272,5 +272,16 @@ pub async fn home(State(state): State<AppState>, uri: Uri) -> impl IntoResponse
pub async fn browse(State(state): State<AppState>, uri: Uri) -> impl IntoResponse {
let item = ItemId::from_browse_uri(uri.path()).unwrap_or(ItemId::root());
+ if item.as_str().starts_with("reddit.com") {
+ let needs_fetch = {
+ let tree = state.tree.read().await;
+ tree.get(&item)
+ .map(|n| n.data.is_none())
+ .unwrap_or(true)
+ };
+ if needs_fetch {
+ state.reddit.request_fetch(item.clone());
+ }
+ }
item_page(state, uri, item).await
}
diff --git a/server/src/reddit.rs b/server/src/reddit.rs
index d203dca09245daf869b3aa942898447700ae69fb..90053ad03b1d7c8e94f325dd4ee64c2b4f7da900 100644
--- a/server/src/reddit.rs
+++ b/server/src/reddit.rs
@@ -1,4 +1,12 @@
-//! Reddit API import (async, decoupled from UI request path).
+//! Reddit API import via a single background worker (rate limits, dedup, backoff).
+
+use std::collections::{HashMap, HashSet};
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+use reqwest::{header, Client, StatusCode};
+use serde::Deserialize;
+use tokio::sync::{mpsc, RwLock};
use crate::{
path_types::ItemId,
@@ -10,12 +18,401 @@ pub fn ensure_partial_tree(tree: &mut GlobalTree, id: &ItemId) {
tree.ensure_path(id);
}
-/// Placeholder for Reddit JSON import. Returns entity data when implemented.
-pub async fn fetch_reddit_entity(_id: &ItemId) -> Option<EntityData> {
- None
+pub struct RedditCommand {
+ pub id: ItemId,
+}
+
+#[derive(Clone)]
+pub struct RedditBroker {
+ tx: mpsc::Sender<RedditCommand>,
+}
+
+#[derive(Clone)]
+struct RedditCredentials {
+ client_id: String,
+ client_secret: String,
+}
+
+struct OAuthToken {
+ access_token: String,
+ expires_at: Instant,
+}
+
+impl RedditBroker {
+ pub fn spawn(tree: Arc<RwLock<GlobalTree>>, user_agent: &str) -> Self {
+ let (tx, rx) = mpsc::channel(100);
+
+ let mut headers = header::HeaderMap::new();
+ headers.insert(
+ header::USER_AGENT,
+ header::HeaderValue::from_str(user_agent).expect("valid user agent"),
+ );
+
+ let client = Client::builder()
+ .default_headers(headers)
+ .timeout(Duration::from_secs(15))
+ .build()
+ .expect("reqwest client");
+
+ let creds = RedditCredentials::from_env();
+ tokio::spawn(reddit_worker(rx, tree, client, creds));
+
+ Self { tx }
+ }
+
+ /// Fire-and-forget: queue a fetch; worker updates the tree when done.
+ pub fn request_fetch(&self, id: ItemId) {
+ let _ = self.tx.try_send(RedditCommand { id });
+ }
+}
+
+impl RedditCredentials {
+ fn from_env() -> Option<Self> {
+ let client_id = std::env::var("REDDIT_CLIENT_ID").ok()?;
+ let client_secret = std::env::var("REDDIT_CLIENT_SECRET").ok()?;
+ if client_id.is_empty() || client_secret.is_empty() {
+ return None;
+ }
+ Some(Self {
+ client_id,
+ client_secret,
+ })
+ }
+}
+
+pub fn default_user_agent() -> String {
+ std::env::var("REDDIT_USER_AGENT").unwrap_or_else(|_| {
+ "web:sorter2.social:v0.0.1 (by /u/sorter2)".to_string()
+ })
}
-/// Apply fetched entity data to a node (called from async worker).
-pub fn apply_entity(tree: &mut GlobalTree, id: &ItemId, data: EntityData) {
- tree.set_entity_data(id, data);
+async fn reddit_worker(
+ mut rx: mpsc::Receiver<RedditCommand>,
+ tree: Arc<RwLock<GlobalTree>>,
+ client: Client,
+ creds: Option<RedditCredentials>,
+) {
+ let mut in_flight = HashSet::new();
+ let mut recently_fetched: HashMap<ItemId, Instant> = HashMap::new();
+ let mut current_delay = Duration::from_secs(1);
+ let mut oauth: Option<OAuthToken> = None;
+ let cache_ttl = Duration::from_secs(300);
+
+ while let Some(cmd) = rx.recv().await {
+ let now = Instant::now();
+ recently_fetched.retain(|_, t| now.duration_since(*t) < cache_ttl);
+
+ if in_flight.contains(&cmd.id) || recently_fetched.contains_key(&cmd.id) {
+ continue;
+ }
+
+ in_flight.insert(cmd.id.clone());
+ let fetch_id = cmd.id.clone();
+
+ tokio::time::sleep(current_delay).await;
+
+ if let Some(c) = &creds {
+ oauth = ensure_oauth_token(&client, c, oauth.take()).await;
+ }
+
+ let token = oauth.as_ref().map(|t| t.access_token.as_str());
+ let use_oauth = token.is_some();
+
+ match do_fetch(&client, &fetch_id, use_oauth, token).await {
+ Ok(FetchOutcome::Entity(data)) => {
+ let mut w = tree.write().await;
+ w.set_entity_data(&fetch_id, data);
+ recently_fetched.insert(fetch_id.clone(), Instant::now());
+ current_delay = Duration::from_millis(600);
+ }
+ Ok(FetchOutcome::NotFound) => {
+ recently_fetched.insert(fetch_id.clone(), Instant::now());
+ }
+ Ok(FetchOutcome::RateLimited { reset_secs }) => {
+ let wait = Duration::from_secs(reset_secs.max(1));
+ tracing::warn!(
+ "Reddit rate limit for {}; sleeping {}s",
+ fetch_id,
+ wait.as_secs()
+ );
+ tokio::time::sleep(wait).await;
+ current_delay = (current_delay * 2).min(Duration::from_secs(60));
+ }
+ Err(e) => {
+ tracing::warn!("Reddit fetch failed for {}: {}", fetch_id, e);
+ current_delay = (current_delay * 2).min(Duration::from_secs(60));
+ }
+ }
+
+ in_flight.remove(&fetch_id);
+ }
+}
+
+enum FetchOutcome {
+ Entity(EntityData),
+ NotFound,
+ RateLimited { reset_secs: u64 },
+}
+
+async fn ensure_oauth_token(
+ client: &Client,
+ creds: &RedditCredentials,
+ existing: Option<OAuthToken>,
+) -> Option<OAuthToken> {
+ if let Some(t) = existing {
+ if Instant::now() < t.expires_at - Duration::from_secs(60) {
+ return Some(t);
+ }
+ }
+
+ let resp = client
+ .post("https://www.reddit.com/api/v1/access_token")
+ .basic_auth(&creds.client_id, Some(&creds.client_secret))
+ .form(&[("grant_type", "client_credentials")])
+ .send()
+ .await;
+
+ let resp = match resp {
+ Ok(r) => r,
+ Err(e) => {
+ tracing::warn!("Reddit OAuth token request failed: {e}");
+ return None;
+ }
+ };
+
+ if !resp.status().is_success() {
+ tracing::warn!("Reddit OAuth token HTTP {}", resp.status());
+ return None;
+ }
+
+ #[derive(Deserialize)]
+ struct TokenResponse {
+ access_token: String,
+ expires_in: u64,
+ }
+
+ let body: TokenResponse = match resp.json().await {
+ Ok(b) => b,
+ Err(e) => {
+ tracing::warn!("Reddit OAuth token parse failed: {e}");
+ return None;
+ }
+ };
+
+ Some(OAuthToken {
+ access_token: body.access_token,
+ expires_at: Instant::now() + Duration::from_secs(body.expires_in),
+ })
+}
+
+async fn do_fetch(
+ client: &Client,
+ id: &ItemId,
+ use_oauth: bool,
+ bearer: Option<&str>,
+) -> Result<FetchOutcome, String> {
+ let url = map_item_to_reddit_api(id, use_oauth);
+ if url.is_empty() {
+ return Ok(FetchOutcome::NotFound);
+ }
+
+ let mut req = client.get(&url);
+ if let Some(token) = bearer {
+ req = req.bearer_auth(token);
+ }
+
+ let resp = req.send().await.map_err(|e| e.to_string())?;
+
+ if resp.status() == StatusCode::TOO_MANY_REQUESTS {
+ let reset = rate_limit_reset_secs(&resp);
+ return Ok(FetchOutcome::RateLimited { reset_secs: reset });
+ }
+
+ if resp.status() == StatusCode::SERVICE_UNAVAILABLE {
+ return Err("Reddit unavailable (503)".to_string());
+ }
+
+ if !resp.status().is_success() {
+ return Ok(FetchOutcome::NotFound);
+ }
+
+ if rate_limit_remaining(&resp) == Some(0) {
+ let reset = rate_limit_reset_secs(&resp);
+ return Ok(FetchOutcome::RateLimited { reset_secs: reset });
+ }
+
+ let bytes = resp.bytes().await.map_err(|e| e.to_string())?;
+ Ok(parse_reddit_json(id, &bytes)
+ .map(FetchOutcome::Entity)
+ .unwrap_or(FetchOutcome::NotFound))
+}
+
+fn rate_limit_remaining(resp: &reqwest::Response) -> Option<u64> {
+ resp.headers()
+ .get("x-ratelimit-remaining")
+ .and_then(|v| v.to_str().ok())
+ .and_then(|s| s.parse::<f64>().ok())
+ .map(|f| f.floor() as u64)
+}
+
+fn rate_limit_reset_secs(resp: &reqwest::Response) -> u64 {
+ resp.headers()
+ .get("x-ratelimit-reset")
+ .and_then(|v| v.to_str().ok())
+ .and_then(|s| s.parse::<f64>().ok())
+ .map(|f| f.ceil() as u64)
+ .unwrap_or(5)
+}
+
+/// Map canonical item id to Reddit JSON API URL.
+pub fn map_item_to_reddit_api(id: &ItemId, oauth: bool) -> String {
+ let path = id.as_str();
+ if !path.starts_with("reddit.com/") && path != "reddit.com" {
+ return String::new();
+ }
+
+ let base = if oauth {
+ "https://oauth.reddit.com"
+ } else {
+ "https://www.reddit.com"
+ };
+
+ let segments: Vec<&str> = path.split('/').collect();
+
+ if let Some(i) = segments.iter().position(|&p| p == "comments") {
+ if segments.len() > i + 1 {
+ let api_path = segments[1..=i + 1].join("/");
+ return format!("{base}/{api_path}.json?raw_json=1");
+ }
+ }
+
+ if segments.len() == 3 && segments[1] == "r" {
+ return format!("{base}/r/{}/about.json?raw_json=1", segments[2]);
+ }
+
+ String::new()
+}
+
+fn parse_reddit_json(id: &ItemId, bytes: &[u8]) -> Option<EntityData> {
+ let v: serde_json::Value = serde_json::from_slice(bytes).ok()?;
+ let segments: Vec<&str> = id.as_str().split('/').collect();
+
+ if segments.iter().any(|&p| p == "comments") {
+ parse_post_listing(&v)
+ } else {
+ parse_subreddit_about(&v)
+ }
+}
+
+fn parse_subreddit_about(v: &serde_json::Value) -> Option<EntityData> {
+ let data = v.get("data")?;
+ let title = data
+ .get("title")
+ .or_else(|| data.get("display_name"))
+ .and_then(|t| t.as_str())?
+ .to_string();
+ let body_html = data
+ .get("public_description_html")
+ .or_else(|| data.get("public_description"))
+ .and_then(|t| t.as_str())
+ .map(|s| s.to_string());
+ let thumb_url = data
+ .get("icon_img")
+ .or_else(|| data.get("community_icon"))
+ .and_then(|t| t.as_str())
+ .filter(|s| !s.is_empty())
+ .map(|s| s.to_string());
+
+ Some(EntityData {
+ title,
+ author: None,
+ body_html,
+ thumb_url,
+ })
+}
+
+fn parse_post_listing(v:
… preview truncated; 4,570 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.