mirror of
https://github.com/guillaumemeyer/watermarks-remover.git
synced 2026-08-22 13:11:57 +02:00
a2e72ed019
* feat: add vendor text-watermark detection and SynthID image scorer sidecar Adds Layer B watermark detection as a first-class service capability: - text_detectors.py: a registry of text-watermark detectors behind one interface — Google's official SynthID-text detector via the Gemini API (taskType DETECT_TEXT_WATERMARK), a Claude placeholder ready for Anthropic's announced detection API, and the MarkLLM research harness (KGW / SynthID, same-config-only). Fail-soft: unconfigured or errored detectors never block cleaning. - server.py: new POST /detect endpoint, detect_before / detect_after options on /clean (before/after scoring for text and images), an opt-in /inspect "detect" flag, and /capabilities gains text_detectors and scorers.synthid_http. - synthid_score_server.py: a stdlib HTTP sidecar for the reverse-SynthID scorer, so the published core image never bundles the non-commercial upstream code; wired via WATERMARKS_SYNTHID_SCORER_URL. - score_synthid.py: extract score_file() so the CLI and the sidecar share one implementation. - compose.yaml / Dockerfile.synthid / .env.example: wr-synthid-score sidecar service and env wiring. - README + skill docs, plus tests for the detectors, the /detect endpoint, and the image sidecar. * feat: per-candidate watermark detection for Layer B rewrite candidates When --candidates N (N > 1) is combined with --markllm-scheme or WATERMARKS_GEMINI_API_KEY, run every configured text detector from the text_detectors.py registry on each candidate and report per-candidate measurements in --json-stats as candidate_scores entries carrying lexical_divergence, selection_score, selected, and per-detector reports (is_watermarked, score, threshold where the detector provides one). Candidate selection stays purely lexical; the detections are observability for correlating lexical divergence with watermark removal (issue #106). Converges rewrite_text.py onto the shared detector registry: - MarkLLMTextDetector gains constructor overrides (scheme, upstream_dir, model, timeout) plus the checkout-venv interpreter preference and the WATERMARKS_MARKLLM_RLIMIT_AS preexec guard ported from rewrite_text.py; the old _markllm_detect / _venv_python / _markllm_preexec helpers are gone. - run_all_text_detectors() accepts an injected MarkLLM instance and an include_markllm switch so CLI flag gating stays intact. - before/after/cleared semantics unchanged; detection remains fail-soft. * docs: pin Watermarks in the Sand reference to arXiv v5 * fix: mark only one rewrite candidate as selected (#110) --------- Co-authored-by: Zhenxin Ai <142008897+ai-kunkun@users.noreply.github.com>
65 lines
2.5 KiB
Docker
65 lines
2.5 KiB
Docker
# Optional local Docker image for the reverse-SynthID pixel scorer.
|
|
#
|
|
# Build from the repository root (build context = service/):
|
|
# docker build -f service/Dockerfile.synthid -t watermarks-remover-synthid-scorer service/
|
|
#
|
|
# The upstream code is fetched from source at build time and is NOT
|
|
# redistributed by this repository. Users must comply with the upstream
|
|
# project's non-commercial Research License.
|
|
#
|
|
# Vendored fork hardening:
|
|
# - base image pinned by digest (no moving tag drift)
|
|
# - upstream checkout pinned to a commit SHA (no moving branch)
|
|
# - deps pinned exactly in requirements-synthid-scorer.txt
|
|
# - pip itself pinned (no unpinned bootstrap step)
|
|
# - runs as an unprivileged user (a parser bug in a crafted image can no
|
|
# longer write files as root inside the container)
|
|
|
|
# Pinned upstream commit (2026-07-17). Keep in sync with setup_synthid.sh.
|
|
ARG REVERSE_SYNTHID_REF=b11083676fd3ee3ff97ce9d03c0e409e46905902
|
|
|
|
# python:3.14-slim linux/amd64 digest.
|
|
FROM python:3.14-slim@sha256:ce40764625a4ff50df3548277632e7f96c4e77fe75fa848aae9885476e7df5a4
|
|
|
|
ARG REVERSE_SYNTHID_REF
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
git \
|
|
libgl1 \
|
|
libglib2.0-0 \
|
|
passwd \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN git clone --depth 1 --filter=blob:none --sparse \
|
|
https://github.com/aloshdenny/reverse-SynthID.git /opt/reverse-synthid \
|
|
&& cd /opt/reverse-synthid \
|
|
&& git fetch --depth 1 origin "${REVERSE_SYNTHID_REF}" \
|
|
&& git checkout --detach "${REVERSE_SYNTHID_REF}" \
|
|
&& git sparse-checkout set --no-cone \
|
|
'/src/' \
|
|
'/artifacts/spectral_codebook_v4.npz' \
|
|
'/requirements.txt' \
|
|
'/LICENSE' \
|
|
'/README.md'
|
|
|
|
COPY scripts/requirements-synthid-scorer.txt /app/requirements-synthid-scorer.txt
|
|
COPY scripts/score_synthid.py /app/score_synthid.py
|
|
COPY scripts/synthid_score_server.py /app/synthid_score_server.py
|
|
|
|
RUN python3 -m pip install --no-cache-dir "pip==26.2.1" \
|
|
&& python3 -m pip install --no-cache-dir -r /app/requirements-synthid-scorer.txt
|
|
|
|
# Unprivileged runtime user. The scorer only reads input files and writes to
|
|
# stdout, so nothing under /opt, /app, or the mounted data dir needs root.
|
|
RUN useradd --create-home --uid 10001 --shell /usr/sbin/nologin scorer
|
|
USER scorer
|
|
|
|
ENV REVERSE_SYNTHID_DIR=/opt/reverse-synthid \
|
|
HOME=/home/scorer \
|
|
PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1
|
|
|
|
WORKDIR /app
|
|
ENTRYPOINT ["python3", "/app/score_synthid.py"]
|