mirror of
https://github.com/guillaumemeyer/watermarks-remover.git
synced 2026-08-22 13:11:57 +02:00
a2e72ed019
* feat: add vendor text-watermark detection and SynthID image scorer sidecar Adds Layer B watermark detection as a first-class service capability: - text_detectors.py: a registry of text-watermark detectors behind one interface — Google's official SynthID-text detector via the Gemini API (taskType DETECT_TEXT_WATERMARK), a Claude placeholder ready for Anthropic's announced detection API, and the MarkLLM research harness (KGW / SynthID, same-config-only). Fail-soft: unconfigured or errored detectors never block cleaning. - server.py: new POST /detect endpoint, detect_before / detect_after options on /clean (before/after scoring for text and images), an opt-in /inspect "detect" flag, and /capabilities gains text_detectors and scorers.synthid_http. - synthid_score_server.py: a stdlib HTTP sidecar for the reverse-SynthID scorer, so the published core image never bundles the non-commercial upstream code; wired via WATERMARKS_SYNTHID_SCORER_URL. - score_synthid.py: extract score_file() so the CLI and the sidecar share one implementation. - compose.yaml / Dockerfile.synthid / .env.example: wr-synthid-score sidecar service and env wiring. - README + skill docs, plus tests for the detectors, the /detect endpoint, and the image sidecar. * feat: per-candidate watermark detection for Layer B rewrite candidates When --candidates N (N > 1) is combined with --markllm-scheme or WATERMARKS_GEMINI_API_KEY, run every configured text detector from the text_detectors.py registry on each candidate and report per-candidate measurements in --json-stats as candidate_scores entries carrying lexical_divergence, selection_score, selected, and per-detector reports (is_watermarked, score, threshold where the detector provides one). Candidate selection stays purely lexical; the detections are observability for correlating lexical divergence with watermark removal (issue #106). Converges rewrite_text.py onto the shared detector registry: - MarkLLMTextDetector gains constructor overrides (scheme, upstream_dir, model, timeout) plus the checkout-venv interpreter preference and the WATERMARKS_MARKLLM_RLIMIT_AS preexec guard ported from rewrite_text.py; the old _markllm_detect / _venv_python / _markllm_preexec helpers are gone. - run_all_text_detectors() accepts an injected MarkLLM instance and an include_markllm switch so CLI flag gating stays intact. - before/after/cleared semantics unchanged; detection remains fail-soft. * docs: pin Watermarks in the Sand reference to arXiv v5 * fix: mark only one rewrite candidate as selected (#110) --------- Co-authored-by: Zhenxin Ai <142008897+ai-kunkun@users.noreply.github.com>
60 lines
3.1 KiB
Bash
60 lines
3.1 KiB
Bash
# Copy to .env for `docker compose` (docker compose auto-loads .env from the
|
|
# repo root). Everything here is optional — the core service works with no
|
|
# configuration at all.
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Core HTTP service (used by wr-core)
|
|
# ---------------------------------------------------------------------------
|
|
# Optional bearer token for the HTTP API. When set, every request must send
|
|
# `Authorization: Bearer <key>`.
|
|
WATERMARKS_SERVER_API_KEY=
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Vendor text-watermark detection (wr-core)
|
|
# ---------------------------------------------------------------------------
|
|
# Optional Google Gemini API key. When set, the service can run Google's
|
|
# official SynthID-text watermark detector via POST /detect and the
|
|
# detect_before / detect_after clean options. Env only — never on argv.
|
|
# Privacy: text is sent to Google only when this key is configured.
|
|
# WATERMARKS_GEMINI_API_KEY=
|
|
# WATERMARKS_GEMINI_MODEL=gemini-2.5-flash
|
|
# WATERMARKS_GEMINI_TIMEOUT=30
|
|
# WATERMARKS_GEMINI_MAX_CHARS=1000000
|
|
|
|
# Optional MarkLLM research harness (host checkouts only; not in the core
|
|
# image). Same-config-only detection — not a vendor oracle.
|
|
# WATERMARKS_MARKLLM_DIR=~/MarkLLM
|
|
# WATERMARKS_MARKLLM_SCHEME=kgw # kgw | synthid
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Harness / heavy backends (only used by the harness/heavy profiles)
|
|
# ---------------------------------------------------------------------------
|
|
# Optional Hugging Face token for gated models (CtrlRegen, MarkLLM,
|
|
# MarkDiffusion score models). Env only — never on argv.
|
|
HF_TOKEN=
|
|
|
|
# SynthID image scoring over HTTP (heavy profile): point wr-core at the
|
|
# wr-synthid-score sidecar and share the same bearer key on both sides.
|
|
# With the heavy profile up, uncomment:
|
|
# WATERMARKS_SYNTHID_SCORER_URL=http://wr-synthid-score:8766
|
|
# WATERMARKS_SYNTHID_SCORER_API_KEY=
|
|
# WATERMARKS_SYNTHID_SCORER_TIMEOUT=60
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Client-side (used by the skill or curl, NOT by compose)
|
|
# ---------------------------------------------------------------------------
|
|
# Where to reach the service. Defaults to http://127.0.0.1:8765.
|
|
# WATERMARKS_SERVICE_URL=http://127.0.0.1:8765
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Layer B statistical-watermark rewrite (only for the rewrite_text.py hook,
|
|
# which runs inside the core image or a local checkout; the agent skill does
|
|
# Layer B itself with its own model and does not need these)
|
|
# ---------------------------------------------------------------------------
|
|
# WATERMARKS_REWRITE_BACKEND=ollama # or: openai-compatible
|
|
# WATERMARKS_REWRITE_MODEL=llama3.2
|
|
# WATERMARKS_REWRITE_BASE_URL=http://127.0.0.1:11434
|
|
# WATERMARKS_REWRITE_API_KEY= # env only, never on argv
|
|
# WATERMARKS_REWRITE_ALLOW_REMOTE=1 # only for non-loopback endpoints
|
|
# WATERMARKS_REWRITE_REASONING_EFFORT=none # none/low/medium/high, or off to omit
|