mirror of
https://github.com/guillaumemeyer/watermarks-remover.git
synced 2026-08-22 13:11:57 +02:00
feat: optional MarkLLM text-watermark verification harness (#53)
* feat: optional MarkLLM text-watermark verification harness Add an optional external backend wrapping THU-BPM/MarkLLM (Apache-2.0) so a specific statistical text-watermark scheme (KGW / SynthID-Text) can be verified before/after a Layer B rewrite. - detect_text_watermark.py: detect/watermark subcommands, external checkout at a pinned commit, exit codes 0/1/2/3, --json - rewrite_text.py --markllm-scheme: before/after detection around the rewrite, reports a `cleared` flag; never fails the rewrite when the backend is unavailable - setup_markllm.sh + requirements-markllm.txt (pinned deps) + Dockerfile.markllm + Makefile bootstrap/smoke/docker targets - tests/test_markllm_detect.py: 16 mock-based cases (no torch in CI) - Docs: verification-harness caveat (same-config-only, not a vendor-detector oracle) in README, SKILL.md, removal-matrix, vendor-notes * chore: tidy merged Unreleased changelog list * security: harden the MarkLLM harness (offline, caps, supply-chain) Addresses the PR security review: - detect_text_watermark.py: --offline loads the scoring model from the HF cache only (local_files_only + HF_HUB_OFFLINE, no remote code), and the algorithm config is capped at 1 MiB so a crafted huge file is refused before either this script or upstream reads it into memory - rewrite_text.py: WATERMARKS_MARKLLM_RLIMIT_AS (env, POSIX) optionally applies RLIMIT_AS to the MarkLLM subprocess; off by default because torch/CUDA needs large address spaces - Dockerfile.markllm: drop the unpinned torch install (it is pinned in requirements-markllm.txt) and verify the cloned upstream commit SHA - tests: offline flag, config-too-large, and preexec/rlimit cases - docs: hardening knobs in README + SKILL.md; changelog updated
This commit is contained in:
parent
1b9b94ac5c
commit
09e64c4ded
@@ -241,6 +241,72 @@ docker run --rm -e HF_TOKEN="$HF_TOKEN" \
|
||||
watermarks-remover-ctrlregen /data/shot.png -o /data/shot.ctrlregen.png
|
||||
```
|
||||
|
||||
## Optional MarkLLM text-watermark verification
|
||||
|
||||
For **controlled experiments**, an optional external harness wraps
|
||||
[`THU-BPM/MarkLLM`](https://github.com/THU-BPM/MarkLLM) (Apache-2.0) to
|
||||
watermark test text and re-detect it after a Layer B rewrite — e.g. prove that
|
||||
a KGW (Kirchenbauer, your "open-LLM" row) or SynthID-Text (Gemini row) mark
|
||||
disappears under your rewrite. It is a **verification harness, not an oracle**:
|
||||
MarkLLM detection is only valid against the *same* scheme config + keys used at
|
||||
generation, and it cannot certify a vendor detector will fail.
|
||||
|
||||
The backend is **not bundled**. `setup_markllm.sh` clones upstream at a pinned
|
||||
commit, creates a venv, and installs pinned deps (torch + transformers); the
|
||||
scoring model (default `facebook/opt-1.3b`, Apache-2.0) downloads from Hugging
|
||||
Face on first run.
|
||||
|
||||
```bash
|
||||
SCRIPTS=skills/remove-ai-marks/scripts
|
||||
|
||||
# Bootstrap (clones upstream, creates ~/MarkLLM/.venv, installs deps).
|
||||
"$SCRIPTS/setup_markllm.sh"
|
||||
|
||||
# Generate watermarked + unwatermarked sample text under the KGW scheme.
|
||||
MARKLLM_DIR=~/MarkLLM \
|
||||
~/MarkLLM/.venv/bin/python "$SCRIPTS/detect_text_watermark.py" watermark prompt.txt \
|
||||
--scheme kgw -o wm.txt -o2 plain.txt
|
||||
|
||||
# Detect the scheme mark in a text file.
|
||||
MARKLLM_DIR=~/MarkLLM \
|
||||
~/MarkLLM/.venv/bin/python "$SCRIPTS/detect_text_watermark.py" detect wm.txt --scheme kgw --json
|
||||
```
|
||||
|
||||
**Verification around a Layer B rewrite:** pass `--markllm-scheme` to
|
||||
`rewrite_text.py` (with `--markllm-dir`), and it records the MarkLLM detection
|
||||
before/after plus a `cleared` flag:
|
||||
|
||||
```bash
|
||||
export WATERMARKS_REWRITE_BACKEND=ollama WATERMARKS_REWRITE_MODEL=llama3.2
|
||||
MARKLLM_DIR=~/MarkLLM \
|
||||
python3 "$SCRIPTS/rewrite_text.py" wm.txt -o wm.rewritten.txt \
|
||||
--markllm-scheme kgw --markllm-dir "$HOME/MarkLLM" --json-stats
|
||||
```
|
||||
|
||||
If the backend is unconfigured or its deps are missing, the rewrite proceeds
|
||||
and the report notes verification was unavailable. A GPU is recommended; CPU
|
||||
runs work but are slow, and the model download is a few GB.
|
||||
|
||||
Hardening knobs:
|
||||
|
||||
- `--offline` on the adapter (or any MarkLLM run) loads the scoring model from
|
||||
the Hugging Face cache only — zero network egress; fails fast if not cached.
|
||||
Custom remote code is never executed (transformers `trust_remote_code` is
|
||||
never enabled).
|
||||
- `WATERMARKS_MARKLLM_RLIMIT_AS=<bytes>` (env, POSIX) applies an address-space
|
||||
limit to the MarkLLM subprocess spawned by `rewrite_text.py`. Off by default
|
||||
because torch/CUDA usually needs large address spaces.
|
||||
- Config files are capped at 1 MiB; the upstream checkout and the base image
|
||||
are pinned by SHA/digest.
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
make docker-markllm-build
|
||||
docker run --rm --user "$(id -u):$(id -g)" -v "$(pwd):/data" \
|
||||
watermarks-remover-markllm detect /data/wm.txt --scheme kgw --json
|
||||
```
|
||||
|
||||
## Coverage matrix
|
||||
|
||||
| Channel | Claude | Gemini/SynthID | OpenAI | Open-LLM |
|
||||
@@ -369,6 +435,12 @@ make smoke # quick CLI smoke on fixtures
|
||||
### Unreleased
|
||||
|
||||
- Add stdlib-only WebP inspection and metadata cleaning for RIFF `C2PA`, XMP, EXIF, and ICC profile chunks
|
||||
- New optional MarkLLM harness (external `THU-BPM/MarkLLM` checkout, Apache-2.0): `detect_text_watermark.py` with `detect` / `watermark` subcommands for KGW and SynthID schemes
|
||||
- `rewrite_text.py --markllm-scheme` runs before/after detection around a Layer B rewrite (env-gated; reports `cleared`)
|
||||
- `setup_markllm.sh` bootstrap + `requirements-markllm.txt` (pinned deps) + `Dockerfile.markllm` and Makefile `bootstrap-markllm` / `smoke-markllm` / `docker-markllm-build` / `docker-markllm-help`
|
||||
- Mock-based tests (`tests/test_markllm_detect.py`, 21 cases) — no torch in CI
|
||||
- Docs: verification-harness caveat (same-config-only, not a vendor-detector oracle) in README, SKILL.md, `removal-matrix.md`, `vendor-notes.md`
|
||||
- Hardening: `--offline` cache-only model loading (no HF egress, no remote code), 1 MiB config cap, optional `WATERMARKS_MARKLLM_RLIMIT_AS` on the rewrite subprocess, pinned torch in the Dockerfile, and clone-SHA verification in `Dockerfile.markllm`
|
||||
|
||||
### [v0.4.0](https://github.com/guillaumemeyer/watermarks-remover/releases/tag/v0.4.0) — pixel removal, finding confidence, Windows & false-positive fixes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user