Mirror of a627388 but for the forum image path. The same image is often
re-uploaded under different filenames across pages/posts, so existsSync
on the target name can't catch content-duplicates. After fetching the
buffer, hash the first 64KB and compare against existing same-size files
in the target folder (same md5+size signature as gallery's duplicate
scanner). Confirmed against a known dani-speegle-2 pair:
skip IMG_79695f8914f20ce38b07.jpg — same content as
72759c89-7e53-4976-839a-7d952c444579.jpg
buildSizeIndex is built once per job in runForumScrape and threaded
through scrapeForumPage → downloadImage; the hash cache amortizes across
all pages in the job.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same video gets uploaded to turbo.cr under different IDs and resolves to
different filenames, so the existsSync(filename) check can't catch
content-duplicates. Switched to the same signature the gallery scanner
uses — md5 of the first 64KB plus exact byte-size match — and apply it
during the download stream so we abort once a same-content existing file
is detected. Avoids re-downloading content the user already has (or has
deliberately deleted via the duplicate scanner).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Newer versions of jsonwebtoken reject undefined for expiresIn with
"expiresIn should be a number of seconds or string representing a timespan".
Omit the option entirely when no expiration is desired — cookie maxAge
already controls session lifetime.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous fallback scanned for any anchor whose text was a number, which
matched widget elements (online count, trending threads, etc.) and inflated
the page count — a sidebar showing "26" caused detectMaxPage to report 26
pages on threads that were actually 12 and 8 pages long.
Now we derive the thread's URL prefix from the input baseUrl and only count
page-N references in hrefs that match that thread, ignoring sidebar
references to unrelated threads. The bare numeric-anchor scan is dropped.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DDoS-Guard now binds session cookies to the issuing browser's fingerprint, so
direct Node fetch returns 403 even with valid cookies. Page HTML for any
forum_site with stored cookies is now fetched via a FlareSolverr browser
session opened once per scrape job.
- Hybrid cookie refresh: FlareSolverr clears the DDoS-Guard captcha, those
cookies seed undetected_chromedriver, Turnstile auto-solves in the real
browser, login form submits, final cookies + browser UA persist to forum_sites
- Per-site user_agent column so subsequent scraper requests match the UA the
cookies were issued for (DDoS-Guard rejects UA mismatches)
- XenForo search rewritten as proper CSRF POST /search/search → results page
parse, replacing the broken ?q=... GET that only returned the search form
- Pagination regex fallback in detectMaxPage catches XenForo pages that
cheerio's class-based selectors miss
- New scrapers/turbo.js handles turbo.cr /embed/ and /a/ URLs by rendering
the page via FlareSolverr and grabbing the signed mp4 from the resolved
<video src> attribute (gallery-dl can't extract these — obfuscated WASM)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- JWT-based app authentication with user roles, folder/route access control
- Dashboard with storage stats, health checks, and recent activity
- Auto-download/scrape scheduler (12h interval) with per-user and per-job configs
- Video upload, tagging, HLS transcoding, and detail pages
- New scrapers: LeakGallery, Mega (megajs), yt-dlp
- FlareSolverr integration for Cloudflare-protected sites
- Gallery: advanced filtering (date, size, search), sort modes, equal-mix shuffle
- Forum sites management with stored cookies/auth
- GridWall/GridCell components for responsive media grid
- Media API with folder-access permissions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Converts desktop sidebar to hidden on mobile, adds bottom tab bar with
5 primary items and a "More" overflow menu. All pages get responsive
spacing, smaller touch targets, and tighter grids on small screens.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Safari/iOS browsers lack Widevine EME support, so DRM videos couldn't play.
This adds a server-side decrypt-and-stream pipeline that reuses the existing
downloadDrmMedia() code to decrypt videos on demand, cache them, and serve
plain MP4s with Range support for native <video> playback.
- server/drm-stream.js: new router with decrypt, status, and serve endpoints
- client/src/components/ServerDrmVideo.jsx: decrypt→poll→play lifecycle component
- DrmVideo.jsx: Widevine detection, falls back to ServerDrmVideo for Safari
- MediaGrid.jsx: passes raw mpdUrl through to DrmVideo
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DRM video download pipeline with pywidevine subprocess for Widevine key acquisition
- Scraper system: forum threads, Coomer/Kemono API, and MediaLink (Fapello) scrapers
- SQLite-backed media index for instant gallery loads with startup scan
- Duplicate detection and gallery filtering/sorting
- HLS video component, log viewer, and scrape management UI
- Dockerfile updated for Python/pywidevine, docker-compose volume for CDM
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Trey T