mirror of
https://github.com/guillaumemeyer/watermarks-remover.git
synced 2026-08-22 13:11:57 +02:00
Guillaume Meyer (The Opinionated Man)andGitHub
09e64c4ded
* feat: optional MarkLLM text-watermark verification harness Add an optional external backend wrapping THU-BPM/MarkLLM (Apache-2.0) so a specific statistical text-watermark scheme (KGW / SynthID-Text) can be verified before/after a Layer B rewrite. - detect_text_watermark.py: detect/watermark subcommands, external checkout at a pinned commit, exit codes 0/1/2/3, --json - rewrite_text.py --markllm-scheme: before/after detection around the rewrite, reports a `cleared` flag; never fails the rewrite when the backend is unavailable - setup_markllm.sh + requirements-markllm.txt (pinned deps) + Dockerfile.markllm + Makefile bootstrap/smoke/docker targets - tests/test_markllm_detect.py: 16 mock-based cases (no torch in CI) - Docs: verification-harness caveat (same-config-only, not a vendor-detector oracle) in README, SKILL.md, removal-matrix, vendor-notes * chore: tidy merged Unreleased changelog list * security: harden the MarkLLM harness (offline, caps, supply-chain) Addresses the PR security review: - detect_text_watermark.py: --offline loads the scoring model from the HF cache only (local_files_only + HF_HUB_OFFLINE, no remote code), and the algorithm config is capped at 1 MiB so a crafted huge file is refused before either this script or upstream reads it into memory - rewrite_text.py: WATERMARKS_MARKLLM_RLIMIT_AS (env, POSIX) optionally applies RLIMIT_AS to the MarkLLM subprocess; off by default because torch/CUDA needs large address spaces - Dockerfile.markllm: drop the unpinned torch install (it is pinned in requirements-markllm.txt) and verify the cloned upstream commit SHA - tests: offline flag, config-too-large, and preexec/rlimit cases - docs: hardening knobs in README + SKILL.md; changelog updated
70 lines
2.6 KiB
Docker
70 lines
2.6 KiB
Docker
# Optional local Docker image for the MarkLLM text-watermark harness.
|
|
#
|
|
# Build from the repository root:
|
|
# docker build -f Dockerfile.markllm -t watermarks-remover-markllm .
|
|
#
|
|
# The upstream code is fetched from source at build time and is NOT
|
|
# redistributed by this repository. Upstream is Apache-2.0.
|
|
#
|
|
# Vendored fork hardening:
|
|
# - base image pinned by digest (no moving tag drift)
|
|
# - upstream checkout pinned to a commit SHA (no moving branch)
|
|
# - deps pinned exactly in requirements-markllm.txt
|
|
# - pip itself pinned (no unpinned bootstrap step)
|
|
# - runs as an unprivileged user (a parser bug in a crafted file can no
|
|
# longer write files as root inside the container)
|
|
|
|
# Pinned upstream commit (2026-07-10). Keep in sync with setup_markllm.sh.
|
|
ARG MARKLLM_REF=c45ddc40f7b761beabe55a1b8dc4690e531d1c6d
|
|
|
|
# python:3.14-slim linux/amd64 digest.
|
|
FROM python:3.14-slim@sha256:ce40764625a4ff50df3548277632e7f96c4e77fe75fa848aae9885476e7df5a4
|
|
|
|
ARG MARKLLM_REF
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
git \
|
|
libgl1 \
|
|
libglib2.0-0 \
|
|
passwd \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN git clone --depth 1 --filter=blob:none --sparse \
|
|
https://github.com/THU-BPM/MarkLLM.git /opt/markllm \
|
|
&& cd /opt/markllm \
|
|
&& git fetch --depth 1 origin "${MARKLLM_REF}" \
|
|
&& git checkout --detach "${MARKLLM_REF}" \
|
|
&& git sparse-checkout set --no-cone \
|
|
'/watermark/' \
|
|
'/config/' \
|
|
'/utils/' \
|
|
'/exceptions/' \
|
|
'/evaluation/dataset.py' \
|
|
'/LICENSE' \
|
|
'/README.md' \
|
|
&& test "$(git -C /opt/markllm rev-parse HEAD)" = "${MARKLLM_REF}"
|
|
|
|
COPY skills/remove-ai-marks/scripts/requirements-markllm.txt /app/requirements-markllm.txt
|
|
COPY skills/remove-ai-marks/scripts/detect_text_watermark.py /app/detect_text_watermark.py
|
|
|
|
# torch is pinned (with everything else) in requirements-markllm.txt; there is
|
|
# no separate unpinned install step. No GPU wheel index inside the image —
|
|
# CUDA users should run setup_markllm.sh on the host instead.
|
|
RUN python3 -m pip install --no-cache-dir "pip==26.2.1" \
|
|
&& python3 -m pip install --no-cache-dir -r /app/requirements-markllm.txt
|
|
|
|
# Unprivileged runtime user. The harness only reads input files and writes to
|
|
# stdout, so nothing under /opt, /app, or the mounted data dir needs root.
|
|
RUN useradd --create-home --uid 10001 --shell /usr/sbin/nologin markllm
|
|
USER markllm
|
|
|
|
ENV MARKLLM_DIR=/opt/markllm \
|
|
HOME=/home/markllm \
|
|
PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
HF_HOME=/home/markllm/.cache/huggingface
|
|
|
|
WORKDIR /app
|
|
ENTRYPOINT ["python3", "/app/detect_text_watermark.py"]
|