Side A is a disciplined, well-motivated refactor that fixes real correctness issues (strict wire-identity validation, optional delegate, removing ad-hoc reducer-side identity rewriting) and updates the entire call graph and test suite consistently across CLI, server, and types crates. Side B adds a genuinely useful Reddit-fetch feature with reasonable async/rate-limit design and tests, but the commit is poorly scoped/labeled ('reddit'), bundles unrelated changes (.gitignore, duplicate reqwest dependency, a stray 'todo' file), and is less clearly integrated with lasting architectural clarity.
constitution · epochs · watch · epoch 3
comparison
c_e2ee16c7ada5 (tommy-mor) vs c_c124c217f89c (tommy-mor)
download prompt · raw event · cmp_8c0860836a8036
council reasoning
A permanently hardens core identity/API design: splits path vs identity modules, makes wire form naked (no @), optional delegate, and stops the reducer from rewriting principals/agents—contract and correctness across auth, ingest, feed, and types. B replaces a reddit stub with a real broker/OAuth/rate-limit fetch path, but that is an additive integration on browse/tree rather than a foundational domain fix.
Side A introduces a substantial architectural refactor by separating path normalization into `canonical_path.rs` and identity parsing into `identity.rs`, removing identity rewriting from the reducer, making delegate identities optional, and consistently enforcing stored-form usernames/agent IDs across APIs with validation. Side B adds a useful Reddit background fetch worker with OAuth, rate limiting, and lazy fetch triggering, but it is a narrower feature addition compared with A's broad cleanup of identity semantics and API/storage invariants that affects core project design.
sides
A — c_e2ee16c7ada5 (tommy-mor)
message
[80ad7753] refactor: split canonical_path and identity; strict wire identity without @ - Add canonical_path.rs (tag + item URL normalization) and identity.rs (parse_username/parse_agent; reject @ in API input). - Slim events.rs to event types only; reducer applies no identity rewriting. - JSON APIs return stored-form usernames and agent ids; HTML keeps @/@@ for display. - Optional delegate on ingest; CLI and tests use naked uuid:rig:model. Made-with: Cursor
diff preview
diff --git a/cli/src/main.rs b/cli/src/main.rs
index 5ac8e289f2b7a366b5959d9338d02f9b546f408a..630c5dea1f78c0ec9bc53e6b96234a0dc75bb705 100644
--- a/cli/src/main.rs
+++ b/cli/src/main.rs
@@ -61,8 +61,8 @@ enum Command {
/// Example: --before 2026-06-01
#[arg(long, value_name = "DATE_OR_MS")]
before: Option<String>,
- /// Filter to posts from this actor (UUID prefix match).
- /// Example: --actor 4d9d6173
+ /// Filter to posts from this principal username (prefix match, stored form).
+ /// Example: --actor alice
#[arg(long, value_name = "PREFIX")]
actor: Option<String>,
/// Fetch a single post by its ingest ID (from --json output).
@@ -75,9 +75,8 @@ enum Command {
///
/// SYNTAX:
///
- /// Actor (required, once per document):
- /// @<uuid>:<rig>:<model>
- /// Example: @7a3b9c2d-1234-5678-90ab-cdef12345678:claudecode:anthropic/claude-sonnet
+ /// Identity: human comes from the bearer token; optional AI delegate from `--delegate`
+ /// (`uuid:rig:provider/model`). The document body is DSL only (items, votes, prose) — no `@` lines.
///
/// Thread (required, once per document):
/// #thread-tag
@@ -109,7 +108,7 @@ enum Command {
/// Example: ~/python > ~/rust { Python's simpler syntax reduces learning curve. }
///
/// Prose (optional, anywhere):
- /// Any line that doesn't start with @, #, or ~ is prose.
+ /// Any line that doesn't start with # or ~ (or `http`) is prose.
/// Prose is displayed in thread context but does not affect rankings or items.
/// Use prose to write blog posts, reasoning, or notes within your ingest.
///
@@ -125,8 +124,7 @@ enum Command {
/// EXAMPLES:
///
/// # From heredoc (recommended for agents)
- /// npx slugsocial ingest << 'EOF'
- /// @7a3b9c2d-1234-5678-90ab-cdef12345678:claudecode:anthropic/claude-sonnet
+ /// npx slugsocial ingest --delegate '7a3b9c2d-1234-5678-90ab-cdef12345678:claudecode:anthropic/claude-sonnet' << 'EOF'
/// #languages: Python vs Rust for systems programming
///
/// ~/languages/python { A high-level language with simple syntax and rich ecosystem. }
@@ -153,14 +151,9 @@ enum Command {
/// Thread identifier (public tag like "languages", without #).
#[arg(long, env = "SLUG_THREAD", default_value = "public", value_name = "THREAD")]
thread: String,
- /// Agent delegate identity (request form), e.g. @@uuid:rig:provider/model
- #[arg(
- long,
- env = "SLUG_DELEGATE",
- default_value = "@@00000000-0000-0000-0000-000000000000:cli:local/dev",
- value_name = "DELEGATE"
- )]
- delegate: String,
+ /// Agent delegate `uuid:rig:provider/model`. Omit for human-only ingests.
+ #[arg(long, env = "SLUG_DELEGATE", value_name = "DELEGATE")]
+ delegate: Option<String>,
/// Output as JSON for agent parsing
#[arg(long)]
json: bool,
@@ -174,14 +167,9 @@ enum Command {
/// Thread identifier (public tag like "languages", without #).
#[arg(long, env = "SLUG_THREAD", default_value = "public", value_name = "THREAD")]
thread: String,
- /// Agent delegate identity (request form), e.g. @@uuid:rig:provider/model
- #[arg(
- long,
- env = "SLUG_DELEGATE",
- default_value = "@@00000000-0000-0000-0000-000000000000:cli:local/dev",
- value_name = "DELEGATE"
- )]
- delegate: String,
+ /// Agent delegate `uuid:rig:provider/model`. Omit for human-only ingests.
+ #[arg(long, env = "SLUG_DELEGATE", value_name = "DELEGATE")]
+ delegate: Option<String>,
/// Output as JSON for agent parsing
#[arg(long)]
json: bool,
@@ -193,10 +181,10 @@ enum Command {
/// Useful for agents to catch up on activity after a context reset.
///
/// Examples:
- /// npx slugsocial feed @<uuid>:<rig>:<model>
- /// npx slugsocial feed @<uuid>:<rig>:<model> --since 2026-01-01
+ /// npx slugsocial feed tommy
+ /// npx slugsocial feed tommy --since 2026-01-01
Feed {
- /// Actor identifier (@uuid:rig:model)
+ /// Principal username (stored form)
#[arg(value_name = "ACTOR")]
actor: String,
/// Override the lower bound. Accepts Unix ms or YYYY-MM-DD.
@@ -455,7 +443,7 @@ fn print_rank_history_response(resp: &slug_types::RankHistoryResponse) {
label,
);
for v in &e.caused_by {
- println!(" {} {} {} {}", v.a, v.ratio, v.b, v.actor.as_deref().map(|a| format!(" (@{})", a)).unwrap_or_default());
+ println!(" {} {} {} {}", v.a, v.ratio, v.b, v.actor.as_deref().map(|a| format!(" ({})", a)).unwrap_or_default());
if !v.body.is_empty() {
println!(" {}", v.body.lines().next().unwrap_or(&v.body).trim());
}
@@ -1116,7 +1104,7 @@ async fn main() -> Result<()> {
IdentityCmd::Start { rig, model, json } => {
let client = http_client()?;
let uuid = uuid::Uuid::new_v4().to_string();
- let delegate = format!("@@{}:{}:{}", uuid, rig, model);
+ let delegate = format!("{uuid}:{rig}:{model}");
let start: PendingSessionStartResponse = expect_json(
client
diff --git a/server/src/api/auth.rs b/server/src/api/auth.rs
index cb0faa29b834931e2c2b2f5c174c875e2e2e9346..995ce4a61d29b024c399c656134f541ecfd880cf 100644
--- a/server/src/api/auth.rs
+++ b/server/src/api/auth.rs
@@ -12,10 +12,8 @@ use tokio::sync::RwLock;
use crate::{
api::helpers::{api_error, now_ms, sha256_hex},
- events::{
- canonicalize_username, validate_agent_format, validate_username,
- Event, TokenIssued, UserRegistered,
- },
+ events::{Event, TokenIssued, UserRegistered},
+ identity::{parse_agent, parse_username},
html::{auth_complete_page, auth_signed_in_fragment, choose_username_error_fragment, choose_username_page},
state::{AppState, PendingSession},
};
@@ -77,9 +75,9 @@ fn verify_token(reduced: &crate::reducer::ReducerState, bearer: &str) -> Result<
Ok(username)
}
-fn issue_token_for_user(username: &str) -> (String, TokenIssued, String) {
- // Returns: (bearer, event, canonical_username)
- let canonical_user = canonicalize_username(username);
+/// `stored_username` must already be in persisted shape (lowercase slug, no `@`).
+fn issue_token_for_user(stored_username: &str) -> (String, TokenIssued) {
+ let username = stored_username.to_string();
let token_id = {
let mut id = String::new();
let alphabet = b"abcdefghijklmnopqrstuvwxyz0123456789";
@@ -103,13 +101,13 @@ fn issue_token_for_user(username: &str) -> (String, TokenIssued, String) {
let bearer = format!("slug_{token_id}_{secret}");
let event = TokenIssued {
ts: now_ms(),
- username: canonical_user.clone(),
+ username: username.clone(),
token_id,
token_hash,
salt,
issued_via: "oauth".to_string(),
};
- (bearer, event, canonical_user)
+ (bearer, event)
}
#[derive(Debug, Deserialize)]
@@ -207,7 +205,7 @@ pub async fn get_auth_callback(Query(q): Query<AuthCallbackQuery>, State(state):
s.provider = Some("google".to_string());
s.provider_id = Some(sub.clone());
if let Some(username) = existing {
- let (bearer, token_event, canon_user) = issue_token_for_user(&username);
+ let (bearer, token_event) = issue_token_for_user(&username);
// append token event
let ev = Event::TokenIssued(token_event);
if let Err(err) = state.event_log.append(&ev).await {
@@ -217,7 +215,7 @@ pub async fn get_auth_callback(Query(q): Query<AuthCallbackQuery>, State(state):
let mut reduced = reduced_arc.write().await;
reduced.apply_event(ev);
}
- s.complete = Some((canon_user, bearer));
+ s.complete = Some((username, bearer));
return Redirect::temporary(&format!("{public_url}/auth/complete")).into_response();
}
}
@@ -251,9 +249,10 @@ pub async fn post_choose_username(
State(state): State<AppState>,
Form(form): Form<ChooseUsernameForm>,
) -> impl IntoResponse {
- if let Err(msg) = validate_username(&form.username) {
- return api_error(StatusCode::BAD_REQUEST, "invalid username", Some(msg)).into_response();
- }
+ let canon_user = match parse_username(&form.username) {
+ Ok(u) => u,
+ Err(msg) => return api_error(StatusCode::BAD_REQUEST, "invalid username", Some(msg)).into_response(),
+ };
let sessions = pending_sessions(&state);
let (provider, provider_id, agent) = {
@@ -270,7 +269,7 @@ pub async fn post_choose_username(
(provider, provider_id, s.agent.clone())
};
- if let Err(msg) = validate_agent_format(&agent) {
+ if let Err(msg) = parse_agent(&agent) {
return api_error(StatusCode::BAD_REQUEST, "invalid agent format", Some(msg)).into_response();
}
@@ -280,7 +279,7 @@ pub async fn post_choose_username(
if reduced.users_by_provider.contains_key(&provider_key) {
return api_error(StatusCode::CONFLICT, "provider already registered", None).into_response();
}
- if reduced.users_by_provider.values().any(|u| u == &canonicalize_username(&form.username)) {
+ if reduced.users_by_provider.values().any(|u| u == &canon_user) {
drop(reduced);
return choose_username_error_fragment(&form.session, "that username is taken — try another").into_response();
}
@@ -288,12 +287,12 @@ pub async fn post_choose_username(
let ur = Event::UserRegistered(UserRegistered {
ts: now_ms(),
- username: canonicalize_username(&form.username),
+ username: canon_user.clone(),
provider: provider.to_lowercase(),
provider_id: provider_id.clone(),
});
- let (bearer, ti, canon_user) = issue_token_for_user(&form.username);
+ let (bearer, ti) = issue_token_for_user(&canon_user);
let ti_ev = Event::TokenIssued(ti);
// Persist events.
@@ -325,15 +324,18 @@ pub async fn post_pending_session(
State(state): State<AppState>,
Json(req): Json<PendingSessionStartRequest>,
) -> impl IntoResponse {
- if let Err(msg) = validate_agent_format(&req.agent) {
- return api_error(StatusCode::BAD_REQUEST, "invalid agent format", Some(msg)).into_response();
- }
+ let agent_naked = match parse_agent(&req.agent) {
+ Ok(a) => a,
+ Err(msg) => {
+ return api_error(StatusCode::BAD_REQUEST, "invalid agent format", Some(msg)).into_response();
+ }
+ };
let session = format!("p_{}", uuid::Uuid::new_v4().simple());
let public_url = std::env::var("SLUG_PUBLIC_URL").unwrap_or_else(|_| "http://127.0.0.1:8080".to_string());
let login_url = format!("{public_url}/auth/login?session={}", urlencoding::encode(&session));
let poll_url = format!("/api/v0/pending-session/{}", session);
let s = PendingSession {
- agent: req.agent.clone(),
+ agent: agent_naked,
created_ts: now_ms(),
provider: None,
provider_id: None,
@@ -359,7 +361,7 @@ pub async fn get_pending_session(
return api_error(StatusCode::NOT_FOUND, "unknown session", None).into_response();
};
let (complete, user, token) = match &s.complete {
- Some((u, t)) => (true, Some(format!("@{}", u)), Some(t.clone())),
+ Some((u, t)) => (true, Some(u.clone()), Some(t.clone())),
None => (false, None, None),
};
Json(PendingSessionPollResponse {
@@ -388,7 +390,7 @@ pub async fn get_whoami(State(state): State<AppState>, headers: HeaderMap)
… preview truncated; 62,772 characters omittedB — c_c124c217f89c (tommy-mor)
message
[8d8230d1] reddit
diff preview
diff --git a/.gitignore b/.gitignore
index 4c7073f9fac0c30fd2050d79a60ef447af58ebeb..ada462e900d24a3a6d08165d158c80f79c35a5a5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@
data/
repomix-output.xml
dev-data/
+.env
diff --git a/server/Cargo.toml b/server/Cargo.toml
index 7906a8547d56b8e6a48ef59c37aa82a8510fdee9..4677fedcb45292eebebe7e9cf6ce2f5738f18ddf 100644
--- a/server/Cargo.toml
+++ b/server/Cargo.toml
@@ -16,6 +16,7 @@ tower = "0.5"
tower-http = { version = "0.5", features = ["trace"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
+reqwest = { version = "0.12", features = ["json"] }
[dev-dependencies]
reqwest = { version = "0.12", features = ["json"] }
diff --git a/server/src/html/mod.rs b/server/src/html/mod.rs
index 2864407ed6e8ec284a1dc663acf1805534566b62..df6505021d9f446c2b453e20e3eb3cf696a111f9 100644
--- a/server/src/html/mod.rs
+++ b/server/src/html/mod.rs
@@ -272,5 +272,16 @@ pub async fn home(State(state): State<AppState>, uri: Uri) -> impl IntoResponse
pub async fn browse(State(state): State<AppState>, uri: Uri) -> impl IntoResponse {
let item = ItemId::from_browse_uri(uri.path()).unwrap_or(ItemId::root());
+ if item.as_str().starts_with("reddit.com") {
+ let needs_fetch = {
+ let tree = state.tree.read().await;
+ tree.get(&item)
+ .map(|n| n.data.is_none())
+ .unwrap_or(true)
+ };
+ if needs_fetch {
+ state.reddit.request_fetch(item.clone());
+ }
+ }
item_page(state, uri, item).await
}
diff --git a/server/src/reddit.rs b/server/src/reddit.rs
index d203dca09245daf869b3aa942898447700ae69fb..90053ad03b1d7c8e94f325dd4ee64c2b4f7da900 100644
--- a/server/src/reddit.rs
+++ b/server/src/reddit.rs
@@ -1,4 +1,12 @@
-//! Reddit API import (async, decoupled from UI request path).
+//! Reddit API import via a single background worker (rate limits, dedup, backoff).
+
+use std::collections::{HashMap, HashSet};
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+use reqwest::{header, Client, StatusCode};
+use serde::Deserialize;
+use tokio::sync::{mpsc, RwLock};
use crate::{
path_types::ItemId,
@@ -10,12 +18,401 @@ pub fn ensure_partial_tree(tree: &mut GlobalTree, id: &ItemId) {
tree.ensure_path(id);
}
-/// Placeholder for Reddit JSON import. Returns entity data when implemented.
-pub async fn fetch_reddit_entity(_id: &ItemId) -> Option<EntityData> {
- None
+pub struct RedditCommand {
+ pub id: ItemId,
+}
+
+#[derive(Clone)]
+pub struct RedditBroker {
+ tx: mpsc::Sender<RedditCommand>,
+}
+
+#[derive(Clone)]
+struct RedditCredentials {
+ client_id: String,
+ client_secret: String,
+}
+
+struct OAuthToken {
+ access_token: String,
+ expires_at: Instant,
+}
+
+impl RedditBroker {
+ pub fn spawn(tree: Arc<RwLock<GlobalTree>>, user_agent: &str) -> Self {
+ let (tx, rx) = mpsc::channel(100);
+
+ let mut headers = header::HeaderMap::new();
+ headers.insert(
+ header::USER_AGENT,
+ header::HeaderValue::from_str(user_agent).expect("valid user agent"),
+ );
+
+ let client = Client::builder()
+ .default_headers(headers)
+ .timeout(Duration::from_secs(15))
+ .build()
+ .expect("reqwest client");
+
+ let creds = RedditCredentials::from_env();
+ tokio::spawn(reddit_worker(rx, tree, client, creds));
+
+ Self { tx }
+ }
+
+ /// Fire-and-forget: queue a fetch; worker updates the tree when done.
+ pub fn request_fetch(&self, id: ItemId) {
+ let _ = self.tx.try_send(RedditCommand { id });
+ }
+}
+
+impl RedditCredentials {
+ fn from_env() -> Option<Self> {
+ let client_id = std::env::var("REDDIT_CLIENT_ID").ok()?;
+ let client_secret = std::env::var("REDDIT_CLIENT_SECRET").ok()?;
+ if client_id.is_empty() || client_secret.is_empty() {
+ return None;
+ }
+ Some(Self {
+ client_id,
+ client_secret,
+ })
+ }
+}
+
+pub fn default_user_agent() -> String {
+ std::env::var("REDDIT_USER_AGENT").unwrap_or_else(|_| {
+ "web:sorter2.social:v0.0.1 (by /u/sorter2)".to_string()
+ })
}
-/// Apply fetched entity data to a node (called from async worker).
-pub fn apply_entity(tree: &mut GlobalTree, id: &ItemId, data: EntityData) {
- tree.set_entity_data(id, data);
+async fn reddit_worker(
+ mut rx: mpsc::Receiver<RedditCommand>,
+ tree: Arc<RwLock<GlobalTree>>,
+ client: Client,
+ creds: Option<RedditCredentials>,
+) {
+ let mut in_flight = HashSet::new();
+ let mut recently_fetched: HashMap<ItemId, Instant> = HashMap::new();
+ let mut current_delay = Duration::from_secs(1);
+ let mut oauth: Option<OAuthToken> = None;
+ let cache_ttl = Duration::from_secs(300);
+
+ while let Some(cmd) = rx.recv().await {
+ let now = Instant::now();
+ recently_fetched.retain(|_, t| now.duration_since(*t) < cache_ttl);
+
+ if in_flight.contains(&cmd.id) || recently_fetched.contains_key(&cmd.id) {
+ continue;
+ }
+
+ in_flight.insert(cmd.id.clone());
+ let fetch_id = cmd.id.clone();
+
+ tokio::time::sleep(current_delay).await;
+
+ if let Some(c) = &creds {
+ oauth = ensure_oauth_token(&client, c, oauth.take()).await;
+ }
+
+ let token = oauth.as_ref().map(|t| t.access_token.as_str());
+ let use_oauth = token.is_some();
+
+ match do_fetch(&client, &fetch_id, use_oauth, token).await {
+ Ok(FetchOutcome::Entity(data)) => {
+ let mut w = tree.write().await;
+ w.set_entity_data(&fetch_id, data);
+ recently_fetched.insert(fetch_id.clone(), Instant::now());
+ current_delay = Duration::from_millis(600);
+ }
+ Ok(FetchOutcome::NotFound) => {
+ recently_fetched.insert(fetch_id.clone(), Instant::now());
+ }
+ Ok(FetchOutcome::RateLimited { reset_secs }) => {
+ let wait = Duration::from_secs(reset_secs.max(1));
+ tracing::warn!(
+ "Reddit rate limit for {}; sleeping {}s",
+ fetch_id,
+ wait.as_secs()
+ );
+ tokio::time::sleep(wait).await;
+ current_delay = (current_delay * 2).min(Duration::from_secs(60));
+ }
+ Err(e) => {
+ tracing::warn!("Reddit fetch failed for {}: {}", fetch_id, e);
+ current_delay = (current_delay * 2).min(Duration::from_secs(60));
+ }
+ }
+
+ in_flight.remove(&fetch_id);
+ }
+}
+
+enum FetchOutcome {
+ Entity(EntityData),
+ NotFound,
+ RateLimited { reset_secs: u64 },
+}
+
+async fn ensure_oauth_token(
+ client: &Client,
+ creds: &RedditCredentials,
+ existing: Option<OAuthToken>,
+) -> Option<OAuthToken> {
+ if let Some(t) = existing {
+ if Instant::now() < t.expires_at - Duration::from_secs(60) {
+ return Some(t);
+ }
+ }
+
+ let resp = client
+ .post("https://www.reddit.com/api/v1/access_token")
+ .basic_auth(&creds.client_id, Some(&creds.client_secret))
+ .form(&[("grant_type", "client_credentials")])
+ .send()
+ .await;
+
+ let resp = match resp {
+ Ok(r) => r,
+ Err(e) => {
+ tracing::warn!("Reddit OAuth token request failed: {e}");
+ return None;
+ }
+ };
+
+ if !resp.status().is_success() {
+ tracing::warn!("Reddit OAuth token HTTP {}", resp.status());
+ return None;
+ }
+
+ #[derive(Deserialize)]
+ struct TokenResponse {
+ access_token: String,
+ expires_in: u64,
+ }
+
+ let body: TokenResponse = match resp.json().await {
+ Ok(b) => b,
+ Err(e) => {
+ tracing::warn!("Reddit OAuth token parse failed: {e}");
+ return None;
+ }
+ };
+
+ Some(OAuthToken {
+ access_token: body.access_token,
+ expires_at: Instant::now() + Duration::from_secs(body.expires_in),
+ })
+}
+
+async fn do_fetch(
+ client: &Client,
+ id: &ItemId,
+ use_oauth: bool,
+ bearer: Option<&str>,
+) -> Result<FetchOutcome, String> {
+ let url = map_item_to_reddit_api(id, use_oauth);
+ if url.is_empty() {
+ return Ok(FetchOutcome::NotFound);
+ }
+
+ let mut req = client.get(&url);
+ if let Some(token) = bearer {
+ req = req.bearer_auth(token);
+ }
+
+ let resp = req.send().await.map_err(|e| e.to_string())?;
+
+ if resp.status() == StatusCode::TOO_MANY_REQUESTS {
+ let reset = rate_limit_reset_secs(&resp);
+ return Ok(FetchOutcome::RateLimited { reset_secs: reset });
+ }
+
+ if resp.status() == StatusCode::SERVICE_UNAVAILABLE {
+ return Err("Reddit unavailable (503)".to_string());
+ }
+
+ if !resp.status().is_success() {
+ return Ok(FetchOutcome::NotFound);
+ }
+
+ if rate_limit_remaining(&resp) == Some(0) {
+ let reset = rate_limit_reset_secs(&resp);
+ return Ok(FetchOutcome::RateLimited { reset_secs: reset });
+ }
+
+ let bytes = resp.bytes().await.map_err(|e| e.to_string())?;
+ Ok(parse_reddit_json(id, &bytes)
+ .map(FetchOutcome::Entity)
+ .unwrap_or(FetchOutcome::NotFound))
+}
+
+fn rate_limit_remaining(resp: &reqwest::Response) -> Option<u64> {
+ resp.headers()
+ .get("x-ratelimit-remaining")
+ .and_then(|v| v.to_str().ok())
+ .and_then(|s| s.parse::<f64>().ok())
+ .map(|f| f.floor() as u64)
+}
+
+fn rate_limit_reset_secs(resp: &reqwest::Response) -> u64 {
+ resp.headers()
+ .get("x-ratelimit-reset")
+ .and_then(|v| v.to_str().ok())
+ .and_then(|s| s.parse::<f64>().ok())
+ .map(|f| f.ceil() as u64)
+ .unwrap_or(5)
+}
+
+/// Map canonical item id to Reddit JSON API URL.
+pub fn map_item_to_reddit_api(id: &ItemId, oauth: bool) -> String {
+ let path = id.as_str();
+ if !path.starts_with("reddit.com/") && path != "reddit.com" {
+ return String::new();
+ }
+
+ let base = if oauth {
+ "https://oauth.reddit.com"
+ } else {
+ "https://www.reddit.com"
+ };
+
+ let segments: Vec<&str> = path.split('/').collect();
+
+ if let Some(i) = segments.iter().position(|&p| p == "comments") {
+ if segments.len() > i + 1 {
+ let api_path = segments[1..=i + 1].join("/");
+ return format!("{base}/{api_path}.json?raw_json=1");
+ }
+ }
+
+ if segments.len() == 3 && segments[1] == "r" {
+ return format!("{base}/r/{}/about.json?raw_json=1", segments[2]);
+ }
+
+ String::new()
+}
+
+fn parse_reddit_json(id: &ItemId, bytes: &[u8]) -> Option<EntityData> {
+ let v: serde_json::Value = serde_json::from_slice(bytes).ok()?;
+ let segments: Vec<&str> = id.as_str().split('/').collect();
+
+ if segments.iter().any(|&p| p == "comments") {
+ parse_post_listing(&v)
+ } else {
+ parse_subreddit_about(&v)
+ }
+}
+
+fn parse_subreddit_about(v: &serde_json::Value) -> Option<EntityData> {
+ let data = v.get("data")?;
+ let title = data
+ .get("title")
+ .or_else(|| data.get("display_name"))
+ .and_then(|t| t.as_str())?
+ .to_string();
+ let body_html = data
+ .get("public_description_html")
+ .or_else(|| data.get("public_description"))
+ .and_then(|t| t.as_str())
+ .map(|s| s.to_string());
+ let thumb_url = data
+ .get("icon_img")
+ .or_else(|| data.get("community_icon"))
+ .and_then(|t| t.as_str())
+ .filter(|s| !s.is_empty())
+ .map(|s| s.to_string());
+
+ Some(EntityData {
+ title,
+ author: None,
+ body_html,
+ thumb_url,
+ })
+}
+
+fn parse_post_listing(v:
… preview truncated; 4,570 characters omittedHardlinks — judgments / attempts / prompt
judgments
attempts
Prompt text is loaded only by the download route.