mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
* fix: remove all automatic third-party egress (OSM tiles, Scalar fonts, editor Google Fonts, AI model download fallbacks) Phone-home audit follow-up. The product no longer makes any automatic third-party request; user-initiated click-outs stay, and production now fails closed on missing AI models. 1. GPS leak via OSM tiles: the strip-metadata panel auto-loaded tile.openstreetmap.org tiles encoding the photo's GPS position. The Leaflet mini-map is gone; coordinates render as text plus an explicit View on map link (openstreetmap.org, opens on click only). Removed tile.openstreetmap.org from the CSP img-src, dropped the leaflet dependency, added the viewOnMap i18n key to all 21 locales. 2. Scalar docs fonts: /api/docs loaded Inter and JetBrains Mono from fonts.scalar.com. Scalar now renders with withDefaultFonts: false and both --scalar-font and --scalar-font-code pinned to system stacks; fonts.scalar.com removed from the docs CSP font-src. Verified by injecting GET /api/docs/: config carries withDefaultFonts false and the served page has no fonts.scalar.com reference. 3. Editor Google Fonts: the editor font picker built fonts.googleapis.com stylesheet URLs for 25 web fonts the served CSP already blocked. The remote loading path is deleted; the picker now offers system fonts only, with a SELF_HOSTED_FONTS seam (FontFace API, same origin) for bundling fonts later. Unknown families saved in old documents fall back to the browser default. 4. Python sidecar fails closed on model downloads: new packages/ai/python/offline_guard.py gates every runtime download fallback (inpaint, outpaint, restore, noise_removal, detect_faces, enhance_faces, face_landmarks, red_eye_removal, remove_bg, ocr, transcribe, upscale) behind SNAPOTTER_ALLOW_MODEL_DOWNLOAD=1 with an actionable error. Bundled models keep working untouched. 5. OCR and transcription library-internal downloads: unbundled PaddleOCR language and detection fallbacks now raise the guard error naming the language instead of resolving models over the network; faster-whisper gets local_files_only when downloads are off. 6. GFPGAN and CodeFormer cwd-relative weights: facexlib and codeformer-pip resolve helper weights relative to the process cwd and fetch them from GitHub when absent. They are now symlinked from the installed bundle files under MODELS_PATH/gfpgan/facelib before the libraries load, failing closed when unresolvable. Defense in depth: HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 are set in the runtime image and in the sidecar spawn env; install_feature.py lifts them for user-initiated bundle installs and restores them afterwards (it can run in-process inside the dispatcher). SNAPOTTER_ALLOW_MODEL_DOWNLOAD is documented in .env.example, default off. Validation: typecheck 9/9 workspaces, Biome clean on touched files, 5178 unit tests pass, py_compile on all touched scripts, guard behavior exercised in both dispatcher exec and per-request import modes, zero remaining runtime references to the three hosts. Docker build and live AI inference need post-merge verification on the GPU host. Claude-Session: https://claude.ai/code/session_01XGB4pGvTvb7sUX4JN745U7 * fix: allow AI model downloads by default, make strict offline mode opt-in Product call: ease of use first. The download gating from the previous commit inverts its default: runtime model fetches (public model weights only, never user data) are allowed out of the box so AI tools self-heal, and SNAPOTTER_ALLOW_MODEL_DOWNLOAD=0 becomes the explicit strict offline mode for airgapped deployments, where every fallback raises the actionable error instead of fetching. Changes: offline_guard blocks only on an explicit 0/false; the unconditional HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE image ENV is removed and bridge.ts sets those flags for the sidecar only in strict mode; .env.example documents the new default; install_feature's lift/restore stays. All bundled-path preferences, pre-existence checks, and symlink pre-placement remain, so installed bundles never trigger a download. The OSM, Scalar font, and editor font fixes are unchanged. Validation rerun: typecheck 9/9, Biome clean on touched files, 5178 unit tests pass, py_compile on touched scripts, guard behavior verified for unset/1 (allowed) and 0/false (blocked with the new message). Claude-Session: https://claude.ai/code/session_01XGB4pGvTvb7sUX4JN745U7
109 lines
4.6 KiB
Python
109 lines
4.6 KiB
Python
"""Gate for runtime model downloads, with an optional strict offline mode.
|
|
|
|
Models normally arrive through user-initiated feature bundle installs
|
|
(install_feature.py), and the resolvers in the AI scripts always prefer those
|
|
bundled files. When a model is missing, scripts may fetch the public model
|
|
weights as a fallback so tools work out of the box; that fallback only ever
|
|
downloads public model files, never user data.
|
|
|
|
Setting SNAPOTTER_ALLOW_MODEL_DOWNLOAD=0 enables strict offline mode for
|
|
airgapped or locked-down deployments: every script calls
|
|
ensure_download_allowed() immediately before any download fallback, so a
|
|
missing file then surfaces as an actionable error instead of an outbound
|
|
fetch.
|
|
"""
|
|
import os
|
|
|
|
|
|
def downloads_allowed():
|
|
"""True unless strict offline mode is explicitly enabled.
|
|
|
|
Runtime model downloads are allowed by default; only an explicit
|
|
SNAPOTTER_ALLOW_MODEL_DOWNLOAD=0 (or "false") blocks them.
|
|
"""
|
|
return os.environ.get("SNAPOTTER_ALLOW_MODEL_DOWNLOAD", "1").lower() not in ("0", "false")
|
|
|
|
|
|
def ensure_download_allowed(what):
|
|
"""Raise a clear, actionable error when strict offline mode blocks a fetch."""
|
|
if downloads_allowed():
|
|
return
|
|
raise RuntimeError(
|
|
f"{what} is missing and automatic downloads are disabled by "
|
|
"SNAPOTTER_ALLOW_MODEL_DOWNLOAD=0. Reinstall the feature bundle from "
|
|
"Settings, or unset SNAPOTTER_ALLOW_MODEL_DOWNLOAD to permit downloads."
|
|
)
|
|
|
|
|
|
def link_bundled_weight(link_path, target_path):
|
|
"""Best-effort: make link_path resolve to an installed bundle file.
|
|
|
|
gfpgan and codeformer-pip hardcode weight paths relative to the process
|
|
cwd, while the feature bundles install those weights under MODELS_PATH.
|
|
Symlinking the expected path to the bundled file lets the libraries find
|
|
the weight without downloading. Returns True when link_path exists
|
|
afterwards (already present, or successfully linked).
|
|
"""
|
|
if os.path.exists(link_path):
|
|
return True
|
|
if not os.path.exists(target_path):
|
|
return False
|
|
try:
|
|
parent = os.path.dirname(link_path)
|
|
if parent:
|
|
os.makedirs(parent, exist_ok=True)
|
|
os.symlink(target_path, link_path)
|
|
except OSError:
|
|
return os.path.exists(link_path)
|
|
return True
|
|
|
|
|
|
GFPGAN_HELPER_WEIGHTS = ("detection_Resnet50_Final.pth", "parsing_parsenet.pth")
|
|
|
|
|
|
def prepare_gfpgan_helper_weights(models_base):
|
|
"""Resolve GFPGAN's cwd-relative facexlib helper weights offline.
|
|
|
|
gfpgan 1.3.x hardcodes FaceRestoreHelper(model_rootpath="gfpgan/weights"),
|
|
a path relative to the process cwd, and facexlib downloads any file
|
|
missing from it (GitHub release URLs). The feature bundles install those
|
|
weights under <models>/gfpgan/facelib, so link them into the expected
|
|
location; when a weight cannot be resolved locally, strict offline mode
|
|
errors instead of downloading.
|
|
"""
|
|
for fname in GFPGAN_HELPER_WEIGHTS:
|
|
link = os.path.join("gfpgan", "weights", fname)
|
|
target = os.path.join(models_base, "gfpgan", "facelib", fname)
|
|
if not link_bundled_weight(link, target):
|
|
ensure_download_allowed(f"GFPGAN helper weight {fname}")
|
|
|
|
|
|
def prepare_codeformer_weights(models_base):
|
|
"""Resolve codeformer-pip's cwd-relative weights offline.
|
|
|
|
codeformer-pip 0.0.4 downloads four weights into a cwd-relative
|
|
CodeFormer/weights/ tree at import time of codeformer.app. Three of them
|
|
ship in the feature bundles and are linked here so they never re-download;
|
|
RealESRGAN_x2plus.pth (a background-upscale helper this app never invokes)
|
|
is not bundled, so it downloads once on first use unless strict offline
|
|
mode blocks it.
|
|
"""
|
|
expected = {
|
|
os.path.join("CodeFormer", "weights", "CodeFormer", "codeformer.pth"): os.path.join(
|
|
models_base, "codeformer", "codeformer.pth"
|
|
),
|
|
os.path.join("CodeFormer", "weights", "facelib", "detection_Resnet50_Final.pth"): os.path.join(
|
|
models_base, "gfpgan", "facelib", "detection_Resnet50_Final.pth"
|
|
),
|
|
os.path.join("CodeFormer", "weights", "facelib", "parsing_parsenet.pth"): os.path.join(
|
|
models_base, "gfpgan", "facelib", "parsing_parsenet.pth"
|
|
),
|
|
}
|
|
for link, target in expected.items():
|
|
if not link_bundled_weight(link, target):
|
|
ensure_download_allowed(f"CodeFormer weight {os.path.basename(link)}")
|
|
|
|
x2plus = os.path.join("CodeFormer", "weights", "realesrgan", "RealESRGAN_x2plus.pth")
|
|
if not os.path.exists(x2plus):
|
|
ensure_download_allowed("CodeFormer helper weight RealESRGAN_x2plus.pth")
|