Implements the replay test of ARBI's keyed-Gumbel technical report
(u = PRF(Hash(key, window), token); exact Gamma-tail p-value; repeated-window
masking) as a stdlib-only detector (detect_gumbel.py) — no GPU, model, or
logits — wired into the TextDetector registry, /capabilities and the
iterative Layer B rewrite loop (--gumbel-key; priority gumbel > markllm >
lexical divergence) with a gumbel.before/after/cleared report.
Also cites the ARBI article in the README bibliography (renamed from
References) and names EXP/Gumbel in the open-LLM vendor row. Same-key-only:
valid against the same key, tokenizer, and PRF layout used at generation;
not a vendor oracle. Key is never logged.
Co-authored-by: guillaumemeyer <guillaumemeyer@users.noreply.github.com>
* feat: add reproducible SynthID-text removal benchmark
bench_synthid_text.py orchestrates the existing Layer B machinery into a
controlled, shareable experiment: generate watermarked + unwatermarked
samples with the MarkLLM SynthID scheme, run removal variants (strength x
candidates) plus controls (no-removal, Layer-A-only, optional re-stamp),
and report clear rate, score suppression, quality, and cost (tokens,
wall time, USD) with a clears-per-MTok efficiency ratio.
Emits report.md / results.json / results.csv with the exact reproduction
command and pinned commits; optional Gemini official-detector tier when
WATERMARKS_GEMINI_API_KEY is set. Mock-based tests, no torch in CI.
* docs: add README section on running the SynthID-text benchmark
Explains what LLM performs the Layer B rewrite (an external model configured
via WATERMARKS_REWRITE_* env vars or --rewrite-* flags; MarkLLM's opt-1.3b is
only the watermark generator/detector) and how to run a benchmark with Ollama
or an OpenAI-compatible endpoint, plus the non-origin-model re-stamp caveat.
* fix: honor WATERMARKS_REWRITE_ALLOW_REMOTE in the SynthID-text benchmark
The --rewrite-allow-remote flag now defaults from the env var (matching
rewrite_text.py and the other WATERMARKS_REWRITE_* settings), so a
non-loopback rewrite endpoint works after sourcing .env without an extra
flag.
* fix: MarkLLM sparse checkout and deps for the SynthID harness
- setup_markllm.sh sparse-checkout omitted '/visualize/', which
watermark/base.py imports at module load — every scheme (incl. SynthID)
failed with 'No module named visualize' during generation/detection.
- requirements-markllm.txt omitted scikit-learn, imported by the SynthID
detector (watermark/synthid/detector_bayesian_torch.py).
Both broke the MarkLLM harness at runtime; the benchmark's sanity gate then
excluded every sample, producing empty per-variant results.
* fix: drop 4 GiB RLIMIT_AS on benchmark subprocesses
_run_cmd applied the common child RLIMIT_AS (default 4 GiB) via
subprocess_preexec_fn to every MarkLLM/rewrite child. torch needs a much
larger address space: CUDA init failed with 'out of memory' at
cudaGetDeviceCount and the 5.2 GB fp32 opt-1.3b could not load, so every
sample was excluded at generation. text_detectors.py already applies no
address-space cap to MarkLLM by default; the benchmark now matches.
* perf: keep MarkLLM resident via a serve worker (624 cold starts -> 1)
The benchmark spawned a fresh torch + opt-1.3b process per operation
(~60-90s each); a full run needs ~624 of them. detect_text_watermark.py
gains a 'serve' mode (JSON-lines over stdin/stdout, ready handshake) that
loads the model once; bench_synthid_text.py uses it via MarkLLMWorker with
automatic fallback to one-shot subprocesses (--no-worker to force).
Turns ~8h runs into ~40-60min.
* perf: skip per-candidate Gemini detections in rewrite subprocess
* feat: run the SynthID-text benchmark from the wr-markllm compose service
- Dockerfile.markllm: add '/visualize/' to the sparse checkout (same fix as
setup_markllm.sh) and COPY the benchmark + rewrite scripts (stdlib-only).
- compose.yaml: wr-markllm gets the WATERMARKS_REWRITE_* and
WATERMARKS_GEMINI_* env wiring, a bench-out volume for --out-dir, and a
read-only mount of the bundled corpus (build context is service/, so the
corpus cannot be COPY'd).
- docs: docker compose run example.
Note: the image ships CPU torch by design, so the container path is for
portability/CI; GPU runs use the host setup_markllm.sh venv.
* feat: per-sample progress logging in the benchmark
The persistent worker returns samples in-memory, so nothing is written
until the end of a run — runs looked stuck. eprint a [gen i/N] line per
generated sample and a [removal] summary per sample.
* chore: migrate Gemini config to gemini-3.6-flash; document SynthID-text retirement
Google retired SynthID text watermarking on the Generative Language API
(Aug 2026): text output is no longer watermarked and DETECT_TEXT_WATERMARK
is rejected on current 3.x models (confirmed by Google AI staff). Migrate
the default detection model to gemini-3.6-flash, document the retirement
in vendor-notes.md and the benchmark report caveat, and keep the detector
seam fail-soft until a vendor endpoint (e.g. Vertex AI) returns.
* feat: remove gemini-synthid-text detector (Google retired text watermarking)
Google removed SynthID text watermarking from the Generative Language API
(Aug 2026): text output is no longer watermarked and DETECT_TEXT_WATERMARK
is rejected on current 3.x models, so the vendor detector had nothing to
detect. Remove GeminiSynthIDTextDetector and its wiring:
- text_detectors.py: drop the Gemini class, HTTP helpers, and constants;
keep MarkLLM + Claude seams (registry now markllm + claude-text).
- server.py / rewrite_text.py: per-candidate detection now triggers on
--markllm-scheme only.
- bench_synthid_text.py: remove the Gemini tier (before/after, report
table, --no-gemini flag); report caveat notes the retirement.
- configs/docs: drop WATERMARKS_GEMINI_* from .env.example / compose /
README / SKILL.md / vendor-notes.md; keep the retirement note.
- tests: gemini tests removed or converted to MarkLLM (mocked subprocess).
- Dockerfile.markllm: parameterize BASE_IMAGE + TORCH_INDEX_URL so a GPU/
arm64 image can be built (used for the --gpus all benchmark run).
* feat: iterative detection-guided Layer B rewriting (default 3 attempts)
Layer B (rewrite_text.py) now rewrites iteratively and stops as soon as an
attempt passes watermark evaluation:
- --candidates defaults to 3 (WATERMARKS_REWRITE_CANDIDATES); each attempt is
one rewrite + one evaluation, and the loop exits on the first attempt the
evaluator reports as not watermarked.
- Evaluator priority: MarkLLM same-config detection (when --markllm-scheme is
passed) > bigram-Jaccard lexical divergence (fallback; no verdict, all
attempts generated, most diverged selected). A vendor-detector seam is
reserved ahead of MarkLLM for a future SynthID-text endpoint (Google retired
text watermarking on its API in Aug 2026).
- Best-effort fallback when the max is exhausted: the lowest-score attempt is
returned with a note; detector errors are fail-soft and never fail the
rewrite.
- --json-stats now reports evaluator / attempts_made / passed and per-attempt
candidate_scores records (passed, evaluation); markllm before/after/cleared
is unchanged and the selected attempt's verdict is reused (no duplicate
MarkLLM detection).
Benchmark (bench_synthid_text.py):
- --variants default becomes paraphrase:3 (candidates = max attempts).
- Rows/report/CSV carry attempts per document (mean_attempts, att column;
attempts / evaluator / passed columns).
Tests, README, docs/synthid-text-benchmark.md and .env.example updated; 490
tests pass, ruff clean.
* feat: split rewrite attempts into --candidates x --max-loops (defaults 1 x 1)
Follow-up to the iterative Layer B rewrite: separate "variants per round"
from "evaluation rounds", so the retry loop is explicit and defaults stay
conservative.
- rewrite_text.py: --candidates (WATERMARKS_REWRITE_CANDIDATES) is now the
number of variants generated per loop iteration (default 1); new
--max-loops (WATERMARKS_REWRITE_LOOPS) caps the evaluation rounds (default
1) -- each round generates --candidates variants and stops as soon as one
passes, so raising --max-loops retries new variants until an evaluation
passes. Stats now report max_loops and per-attempt records carry the loop
index.
- bench_synthid_text.py: new --rewrite-loops flag (default 1) passed through
to --max-loops.
- README / docs / .env.example updated; tests cover the 1x1 defaults, loop
retry until pass, and cross-loop exhaustion.
* feat: MarkLLM serve worker over loopback TCP (WATERMARKS_MARKLLM_PORT)
detect_text_watermark.py serve can now also listen on a loopback TCP port,
and MarkLLMTextDetector reuses a resident worker when
WATERMARKS_MARKLLM_PORT is set (falls back to a one-shot subprocess when the
worker is unreachable). This avoids a ~20s torch+model cold start per detect
for callers that run a worker out-of-band.
Tests: loopback worker protocol + detector worker-port routing (mock-based).
* ci: add macOS runner to the test matrix
* feat: add reproducible SynthID-text removal benchmark
bench_synthid_text.py orchestrates the existing Layer B machinery into a
controlled, shareable experiment: generate watermarked + unwatermarked
samples with the MarkLLM SynthID scheme, run removal variants (strength x
candidates) plus controls (no-removal, Layer-A-only, optional re-stamp),
and report clear rate, score suppression, quality, and cost (tokens,
wall time, USD) with a clears-per-MTok efficiency ratio.
Emits report.md / results.json / results.csv with the exact reproduction
command and pinned commits; optional Gemini official-detector tier when
WATERMARKS_GEMINI_API_KEY is set. Mock-based tests, no torch in CI.
* docs: add README section on running the SynthID-text benchmark
Explains what LLM performs the Layer B rewrite (an external model configured
via WATERMARKS_REWRITE_* env vars or --rewrite-* flags; MarkLLM's opt-1.3b is
only the watermark generator/detector) and how to run a benchmark with Ollama
or an OpenAI-compatible endpoint, plus the non-origin-model re-stamp caveat.
* fix: honor WATERMARKS_REWRITE_ALLOW_REMOTE in the SynthID-text benchmark
The --rewrite-allow-remote flag now defaults from the env var (matching
rewrite_text.py and the other WATERMARKS_REWRITE_* settings), so a
non-loopback rewrite endpoint works after sourcing .env without an extra
flag.
* fix: MarkLLM sparse checkout and deps for the SynthID harness
- setup_markllm.sh sparse-checkout omitted '/visualize/', which
watermark/base.py imports at module load — every scheme (incl. SynthID)
failed with 'No module named visualize' during generation/detection.
- requirements-markllm.txt omitted scikit-learn, imported by the SynthID
detector (watermark/synthid/detector_bayesian_torch.py).
Both broke the MarkLLM harness at runtime; the benchmark's sanity gate then
excluded every sample, producing empty per-variant results.
* fix: drop 4 GiB RLIMIT_AS on benchmark subprocesses
_run_cmd applied the common child RLIMIT_AS (default 4 GiB) via
subprocess_preexec_fn to every MarkLLM/rewrite child. torch needs a much
larger address space: CUDA init failed with 'out of memory' at
cudaGetDeviceCount and the 5.2 GB fp32 opt-1.3b could not load, so every
sample was excluded at generation. text_detectors.py already applies no
address-space cap to MarkLLM by default; the benchmark now matches.
* perf: keep MarkLLM resident via a serve worker (624 cold starts -> 1)
The benchmark spawned a fresh torch + opt-1.3b process per operation
(~60-90s each); a full run needs ~624 of them. detect_text_watermark.py
gains a 'serve' mode (JSON-lines over stdin/stdout, ready handshake) that
loads the model once; bench_synthid_text.py uses it via MarkLLMWorker with
automatic fallback to one-shot subprocesses (--no-worker to force).
Turns ~8h runs into ~40-60min.
* perf: skip per-candidate Gemini detections in rewrite subprocess
* feat: run the SynthID-text benchmark from the wr-markllm compose service
- Dockerfile.markllm: add '/visualize/' to the sparse checkout (same fix as
setup_markllm.sh) and COPY the benchmark + rewrite scripts (stdlib-only).
- compose.yaml: wr-markllm gets the WATERMARKS_REWRITE_* and
WATERMARKS_GEMINI_* env wiring, a bench-out volume for --out-dir, and a
read-only mount of the bundled corpus (build context is service/, so the
corpus cannot be COPY'd).
- docs: docker compose run example.
Note: the image ships CPU torch by design, so the container path is for
portability/CI; GPU runs use the host setup_markllm.sh venv.
* feat: per-sample progress logging in the benchmark
The persistent worker returns samples in-memory, so nothing is written
until the end of a run — runs looked stuck. eprint a [gen i/N] line per
generated sample and a [removal] summary per sample.
* chore: migrate Gemini config to gemini-3.6-flash; document SynthID-text retirement
Google retired SynthID text watermarking on the Generative Language API
(Aug 2026): text output is no longer watermarked and DETECT_TEXT_WATERMARK
is rejected on current 3.x models (confirmed by Google AI staff). Migrate
the default detection model to gemini-3.6-flash, document the retirement
in vendor-notes.md and the benchmark report caveat, and keep the detector
seam fail-soft until a vendor endpoint (e.g. Vertex AI) returns.
* feat: remove gemini-synthid-text detector (Google retired text watermarking)
Google removed SynthID text watermarking from the Generative Language API
(Aug 2026): text output is no longer watermarked and DETECT_TEXT_WATERMARK
is rejected on current 3.x models, so the vendor detector had nothing to
detect. Remove GeminiSynthIDTextDetector and its wiring:
- text_detectors.py: drop the Gemini class, HTTP helpers, and constants;
keep MarkLLM + Claude seams (registry now markllm + claude-text).
- server.py / rewrite_text.py: per-candidate detection now triggers on
--markllm-scheme only.
- bench_synthid_text.py: remove the Gemini tier (before/after, report
table, --no-gemini flag); report caveat notes the retirement.
- configs/docs: drop WATERMARKS_GEMINI_* from .env.example / compose /
README / SKILL.md / vendor-notes.md; keep the retirement note.
- tests: gemini tests removed or converted to MarkLLM (mocked subprocess).
- Dockerfile.markllm: parameterize BASE_IMAGE + TORCH_INDEX_URL so a GPU/
arm64 image can be built (used for the --gpus all benchmark run).
* fix: harden notes aggregation against non-string notes
A run completed all samples but crashed at the final aggregate step with
'cannot use list as a set element' when a row's notes contained a
non-string value. Filter notes to strings (aggregate + CSV) and add a
regression test.
* perf: let the rewrite subprocess reuse the resident MarkLLM worker
The rewrite subprocess (rewrite_text.py) ran its own before/after MarkLLM
detects, each a ~20s torch+model cold start (~12 per sample = ~5min of the
~6min/sample runtime). Now:
- detect_text_watermark.py serve gains --port N: a loopback TCP JSON-lines
listener (default -1 = off) sharing the resident model, with a lock so
stdin and socket requests never run the model concurrently.
- text_detectors.MarkLLMTextDetector checks WATERMARKS_MARKLLM_PORT and
does a fast loopback detect when a worker is up, falling back to the
one-shot subprocess otherwise.
- The benchmark worker publishes its port via that env var, so the rewrite
subprocess inherits it and its detects hit the resident model.
Turns ~6 min/sample into ~1-2 min; a full run drops from ~2h to ~40-50min.
Tests: loopback-client + fallback + env-publish coverage.
* chore: add benchmark-smoke.sh / benchmark-full.sh wrappers
Simple host wrappers: source .env, default MARKLLM_DIR to ~/MarkLLM, use a
repo-local HF cache by default, and run bench_synthid_text.py with a quick
(2 docs, 1 seed, paraphrase:1) or full (8 docs x 3 seeds, three variants,
re-stamp control) configuration. OUT_DIR overrides the output location.
clean_svg/clean_odt stripped metadata with lazy dot-matches-all regexes; on many unclosed opening tags the engine rescans to end-of-input from every candidate start (O(n^2)), and the GIL stalls the whole single-process service. A ~1.4 KB ODT request pinned a core for ~99 s. Replace every lazy .*?</close> block scan in container_meta.py (SVG metadata/xmpmeta/comments, ODT meta:generator/dc:creator, HTML JSON-LD, OOXML/ODF text-run scrubs, docProps fields, EPUB OPF meta/dc, PDF xpacket/stream) with a linear scan pairing opening tags with a forward pointer over closing tags - identical match semantics, O(n). Add regression tests asserting the advisory PoC completes in <5 s and that stripping behavior is unchanged.
* feat: add vendor text-watermark detection and SynthID image scorer sidecar
Adds Layer B watermark detection as a first-class service capability:
- text_detectors.py: a registry of text-watermark detectors behind one
interface — Google's official SynthID-text detector via the Gemini API
(taskType DETECT_TEXT_WATERMARK), a Claude placeholder ready for
Anthropic's announced detection API, and the MarkLLM research harness
(KGW / SynthID, same-config-only). Fail-soft: unconfigured or errored
detectors never block cleaning.
- server.py: new POST /detect endpoint, detect_before / detect_after options
on /clean (before/after scoring for text and images), an opt-in
/inspect "detect" flag, and /capabilities gains text_detectors and
scorers.synthid_http.
- synthid_score_server.py: a stdlib HTTP sidecar for the reverse-SynthID
scorer, so the published core image never bundles the non-commercial
upstream code; wired via WATERMARKS_SYNTHID_SCORER_URL.
- score_synthid.py: extract score_file() so the CLI and the sidecar share
one implementation.
- compose.yaml / Dockerfile.synthid / .env.example: wr-synthid-score sidecar
service and env wiring.
- README + skill docs, plus tests for the detectors, the /detect endpoint,
and the image sidecar.
* feat: per-candidate watermark detection for Layer B rewrite candidates
When --candidates N (N > 1) is combined with --markllm-scheme or
WATERMARKS_GEMINI_API_KEY, run every configured text detector from the
text_detectors.py registry on each candidate and report per-candidate
measurements in --json-stats as candidate_scores entries carrying
lexical_divergence, selection_score, selected, and per-detector reports
(is_watermarked, score, threshold where the detector provides one).
Candidate selection stays purely lexical; the detections are observability
for correlating lexical divergence with watermark removal (issue #106).
Converges rewrite_text.py onto the shared detector registry:
- MarkLLMTextDetector gains constructor overrides (scheme, upstream_dir,
model, timeout) plus the checkout-venv interpreter preference and the
WATERMARKS_MARKLLM_RLIMIT_AS preexec guard ported from rewrite_text.py;
the old _markllm_detect / _venv_python / _markllm_preexec helpers are gone.
- run_all_text_detectors() accepts an injected MarkLLM instance and an
include_markllm switch so CLI flag gating stays intact.
- before/after/cleared semantics unchanged; detection remains fail-soft.
* docs: pin Watermarks in the Sand reference to arXiv v5
* fix: mark only one rewrite candidate as selected (#110)
---------
Co-authored-by: Zhenxin Ai <142008897+ai-kunkun@users.noreply.github.com>
* fix: rewrite ODT/EPUB manifests and measure real zip bytes (#122)
Two container correctness/security fixes from issue #122:
- clean_odt dropped marker-bearing parts while leaving their entries in
META-INF/manifest.xml, so readers flagged the package as damaged. It is
now two-pass: compute the dropped set, then rewrite the manifest
attribute-order-independently, and write each part exactly once. The same
bug class in clean_epub (dropped parts left in the OPF manifest, plus
dangling spine itemrefs) gets the same two-pass treatment.
- The zip budget trusted ZipInfo.file_size from the archive's own central
directory, so a crafted DOCX/ODT could declare a tiny size and still
expand via zf.read. Budgets are now charged on actual decompressed bytes
via _read_zip_member (streaming, cap enforced mid-read), with the declared
size kept only as a fast-path pre-reject.
* fix: classify unrecognized bytes as "unknown", not text (#122)
Two classification defects from issue #122:
- format_dispatch.classify_bytes fell back to "text" for any unrecognized
file, so a binary with valid UTF-8 runs could be decoded and written back
mangled (corrupted with --in-place) in clean_file auto mode. Unrecognized
bytes now classify as "unknown"; clean_file refuses them in auto mode
(exit 2, no write, router advice) and --as text / --force-text are the
explicit opt-ins. inspect_file reports kind "unknown" (exit 0), audit_lib
records a non-actionable item, and the HTTP server answers /inspect with
kind "unknown" but rejects /clean of unknown formats (400).
- classify(path) read the whole file to sniff a header, and only a full read
could detect zip containers. It now routes known extensions without
reading, sniffs a 4096-byte header once for images and prefix-based
containers, and reads the whole file only when the header is a zip local
header (PK), where the container signature lives in the central directory.
* feat: distinct exit code for partial audits (#122)
audit_dir and audit_website reported success (0) even when some files or
URLs could not be scanned; the exit status was computed only over the items
that succeeded. A scan that is missing items is not a clean scan.
- common.EXIT_PARTIAL = 3, with precedence: partial (3) > actionable (1)
> clean (0) — an incomplete audit is the more important CI signal.
- audit_dir returns 3 when any file was skipped/failed; audit_website
returns 3 when any URL failed to fetch or inspect. Both are independent
of the output format (human/json/sarif already share one return).
* fix: verify the pinned upstream ref on existing checkouts (#122)
setup_ctrlregen.sh/setup_synthid.sh (and their .ps1 twins) only verified
the pinned commit in the fresh-clone branch; an existing checkout at an
unknown or drifted revision was silently reused, defeating the commit pin.
All four scripts now check HEAD against the pinned ref in the
existing-checkout branch too, and repair by fetch + detach checkout
(re-applying the sparse-checkout set), failing hard if the ref cannot be
reached or the re-pin does not land on it.
* docs: unknown-format behavior, audit exit codes, backend isolation (#122)
- README: clean_file no longer auto-cleans unrecognized formats (--as text
/ --force-text are the opt-ins), and the CtrlRegen bootstrap documents the
isolation expectation for its research-era dependency pins plus the new
re-pin check on existing checkouts.
- SKILL.md: audit exit codes (0/1/2/3, partial=3) and a note that /clean
requires a name with a known extension.
- audit_website: document why stdlib ElementTree is used (stdlib-first) and
that defusedxml is the fallback if that policy changes (DTD rejection stays).
- requirements-ctrlregen.txt: advisory/isolation note for the pinned research
dependencies.
* test: ODT manifest and EPUB OPF dangling-ref regressions (#122)
- clean_odt: dropped marker-bearing parts remove their META-INF/manifest.xml
file-entry (attribute-order-independent), exactly one manifest entry, root
and surviving entries kept, and the manifest is byte-identical when nothing
is dropped.
- clean_epub: dropped non-content parts lose their <item> entry in the OPF
manifest, so the book no longer references removed members.
Three independent failure modes from #117:
- $ErrorActionPreference 'Stop' + 2>$null on a native command aborts the
script on torch's harmless stderr warnings (e.g. "Failed to initialize
NumPy" when torch is installed before numpy). Run the probes through a
new Invoke-NativeQuiet helper that lowers EAP to 'Continue' for the
block and restores it afterwards.
- The wheel index tag was derived from the driver's CUDA version, e.g.
cu131 for a 13.1 driver, which does not exist (HTTP 403) and silently
fell back to the default index, i.e. the CPU build on Windows. Probe
the published indices and pick the highest one <= driver that answers
HTTP 200; cu126 is still forced below compute capability 7.5.
- Installing torch alone let requirements-ctrlregen.txt resolve torchvision
from PyPI, and torchvision pins an exact torch, so pip replaced the +cu
build with a +cpu one while the script still exited 0. Install torch AND
torchvision together from the chosen index, and verify after the
requirements install that torch.cuda.is_available() is true - if a GPU
was detected but torch ends up CPU-only, warn loudly and exit non-zero.
Also add a CI step (windows-latest, pwsh) that parses the setup .ps1
scripts and asserts the post-install CUDA verification survives.
Fixes#117
Add stdlib-only detection, inspection, and cleaning for four more formats.
- BMP: locate the pixel payload via the DIB header and strip trailing
non-image metadata (the only place non-standard BMP metadata can live),
rewriting the file-size field.
- GIF: drop comment and XMP application extensions while preserving
NETSCAPE2.0 looping, ICC, graphic-control, and image blocks.
- TIFF (classic + BigTIFF): walk IFD chains and drop XMP/EXIF/GPS/IPTC/
Photoshop/MakerNote tags, zeroing orphaned payloads while keeping
strip/tile offsets valid.
- EPUB: scrub OPF package metadata and XHTML meta/JSON-LD, clean embedded
raster/SVG media, apply Layer A to XHTML body text, and pass OCF-encrypted
parts through untouched.
All four route through format_dispatch, so the unified CLIs, the HTTP
service, and the audits pick them up automatically.
Introduce Ruff (pinned at 0.16.3) as the project linter + formatter and
enforce it in CI:
- requirements-dev.txt: pin ruff==0.16.3 (exact pins, no drift)
- ruff.toml: line-length 100, target py312; rule set E/F/W/I/UP/B/SIM/RUF/PLW/S
with deliberate ignores (E501 for content strings, S603 for safe_arg
subprocess calls, S101 asserts in tests) and per-file test ignores
- Makefile: add lint / format / lint-fix targets
- .github/workflows/ci.yml: add lint job (ruff check + format --check)
- .gitignore: whitelist ruff.toml
Also fix every finding the new gate surfaced so CI is green:
- 109+ auto-fixes from ruff --fix (import sorting, simplifications,
unused vars, re.I aliases, etc.)
- explicit check=False on all subprocess.run calls (PLW1510)
- harden sitemap XML parsing: reject DTD/entity declarations (S314)
- replace hardcoded /tmp paths in tests with tmp_path (S108)
- narrow/annotate intentional bare excepts (S110/S112), bind loop vars
in closures (B023), raise ... from None (B904), strict= for zip (B905)
- ruff format applied across service/ and tests/
Verified: ruff check + ruff format --check pass; 287 tests pass, 1 skip.
Bump accelerate to 1.14.0, controlnet-aux to 0.0.10, and safetensors to
0.8.0 in requirements-ctrlregen.txt. Verified end-to-end against the pinned
noai-watermark commit b642ae45 (import + CUDA inference) and via Docker build.
Pin huggingface_hub to 0.24.0: 0.26.0+ removed the cached_download symbol
that diffusers 0.27.2 still imports, so unpinned installs resolve 0.36.2 and
the backend fails to import. Refresh the Dockerfile.ctrlregen base-image
comment to drop the now-stale safetensors 0.4.3 mention.
* feat: split skill from service, add HTTP API and Docker distribution
The agent skill (skills/remove-ai-marks/) is now a code-free remote client:
all implementation moved to service/scripts/ and runs behind a stdlib HTTP
service (server.py) with /health, /capabilities, /inspect, /clean and a
dynamically generated OpenAPI 3.0.3 spec at /openapi.json.
- Move scripts/ and the backend Dockerfiles under service/
- server.py: JSON/base64 HTTP entrypoint with size caps, binary guard,
atomic writes, loopback default, optional bearer auth
- Core Dockerfile (exiftool/qpdf/c2patool preinstalled) and a GHCR publish
workflow for the core/markllm/markdiffusion images
- compose.yaml (wr-* services, harness/heavy profiles) + compose-check.sh
to validate the running stack (exit code only)
- Fix markllm image build (tokenizers 0.22.2, CPU-only torch) and ctrlregen
build (python:3.11 base for the 2023-era research pins)
- Fix markllm/markdiffusion harness images missing common.py at runtime
* docs: add .env.example and service configuration guide
* fix: disable chain-of-thought for openai-compatible Layer B rewrites
deepseek-v4-flash is a reasoning model: a one-line paraphrase burned 9,894
reasoning tokens (~100s) and hit the default timeout. Send
reasoning_effort=none by default for the openai-compatible backend
(--reasoning-effort / WATERMARKS_REWRITE_REASONING_EFFORT; 'off' omits the
parameter), cutting the same rewrite to ~1s / 12 tokens. Tested end-to-end
against api.deepseek.com.
* fix: sanitize client-supplied filename in HTTP service
CodeQL 'uncontrolled data in path expression' (server.py): a name like
'../../x' flowed into Path(tmpdir) / name, letting an upload escape the
request temp dir on write. Sanitize name to its basename in _decode_input
(_safe_name) and refuse any joined path whose parent is not the tmpdir at
the write sites (_tmp_path). Tests cover traversal names.
* chore: gitignore .env (contains local rewrite credentials)
* chore: deny-by-default gitignore and dockerignore; document compose env config
.gitignore and service/.dockerignore now exclude everything by default and
explicitly allow only what is publishable/needed: tracked source, docs,
tests, .github, and (for images) the service/scripts/ tree that every
Dockerfile COPYs. Root .dockerignore documents that all builds use service/
as context. README Configuration section now covers .env setup for docker
compose, host-side export for CLI runs, and the full variable table.
Unify the file-to-pipeline classification that was duplicated across
inspect_file.py, clean_file.py and audit_lib.py into one format_dispatch
module. Adding a format now means touching a single extension table and
sniff path; audit_lib no longer imports classify from a CLI entry point.
* feat: optional MarkLLM text-watermark verification harness
Add an optional external backend wrapping THU-BPM/MarkLLM (Apache-2.0)
so a specific statistical text-watermark scheme (KGW / SynthID-Text)
can be verified before/after a Layer B rewrite.
- detect_text_watermark.py: detect/watermark subcommands, external
checkout at a pinned commit, exit codes 0/1/2/3, --json
- rewrite_text.py --markllm-scheme: before/after detection around the
rewrite, reports a `cleared` flag; never fails the rewrite when the
backend is unavailable
- setup_markllm.sh + requirements-markllm.txt (pinned deps) +
Dockerfile.markllm + Makefile bootstrap/smoke/docker targets
- tests/test_markllm_detect.py: 16 mock-based cases (no torch in CI)
- Docs: verification-harness caveat (same-config-only, not a
vendor-detector oracle) in README, SKILL.md, removal-matrix, vendor-notes
* chore: tidy merged Unreleased changelog list
* security: harden the MarkLLM harness (offline, caps, supply-chain)
Addresses the PR security review:
- detect_text_watermark.py: --offline loads the scoring model from the HF
cache only (local_files_only + HF_HUB_OFFLINE, no remote code), and the
algorithm config is capped at 1 MiB so a crafted huge file is refused
before either this script or upstream reads it into memory
- rewrite_text.py: WATERMARKS_MARKLLM_RLIMIT_AS (env, POSIX) optionally
applies RLIMIT_AS to the MarkLLM subprocess; off by default because
torch/CUDA needs large address spaces
- Dockerfile.markllm: drop the unpinned torch install (it is pinned in
requirements-markllm.txt) and verify the cloned upstream commit SHA
- tests: offline flag, config-too-large, and preexec/rlimit cases
- docs: hardening knobs in README + SKILL.md; changelog updated
* fix: SECURITY.md only references the private advisories channel
* fix: preserve load-bearing script invisibles, strip PUA in Layer A (#38)
Mongolian FVS (U+180B-180D), Khmer inherent vowels (U+17B4/17B5) and
Hangul fillers (U+115F/1160) are orthography, not hidden marks, when
they follow a base from their own script. STRIP_CODEPOINTS removed them
unconditionally, corrupting real text (FVS glyph selection, Khmer
phonemic vowels, Hangul partial-syllable composition).
Mirror the existing #28 glue machinery: keep them after a same-script
base, still strip them when floating between unrelated characters, and
honor --strip-emoji-glue for paranoid mode. Add them to _is_glue so
chained FVS stay bound to their base.
Private-use codepoints (U+E000-F8FF and the supplementary PUA planes)
render as nothing portable and carry no agreed meaning, making them a
serviceable hiding place; they are Co, so they fell outside the Cf
strip path. Strip them by default with a new 'private_use' inspect kind.
Refs #38.
Mongolian FVS (U+180B-180D), Khmer inherent vowels (U+17B4/17B5) and
Hangul fillers (U+115F/1160) are orthography, not hidden marks, when
they follow a base from their own script. STRIP_CODEPOINTS removed them
unconditionally, corrupting real text (FVS glyph selection, Khmer
phonemic vowels, Hangul partial-syllable composition).
Mirror the existing #28 glue machinery: keep them after a same-script
base, still strip them when floating between unrelated characters, and
honor --strip-emoji-glue for paranoid mode. Add them to _is_glue so
chained FVS stay bound to their base.
Private-use codepoints (U+E000-F8FF and the supplementary PUA planes)
render as nothing portable and carry no agreed meaning, making them a
serviceable hiding place; they are Co, so they fell outside the Cf
strip path. Strip them by default with a new 'private_use' inspect kind.
Refs #38.
Add the resolved path at the top of every human report (File: line) and
include path in JSON output for all kinds, so batch inspection via find
can attribute hits to a file. Closes#31.
The runner prefers .venv/Scripts/python.exe on Windows, but the test
hardcoded the POSIX .venv/bin/python layout, so it fell back to
sys.executable and failed the Windows CI leg.
Wires a standalone clean_ctrlregen.py adapter plus setup_ctrlregen.sh bootstrap, Dockerfile, Makefile targets, and clean_image.py --remove-pixel ctrlregen. The backend is cloned at a pinned commit and never bundled (noai-watermark ships no LICENSE file). Includes mock-based tests and docs with research references.
- classify findings as confirmed/probable/informational/likely_false_positive
- expose confidence in text/image/container JSON and human reports
- add audit_dir.py and audit_website.py for aggregate reports
- document the confidence taxonomy and audit commands in SKILL.md
Closes#13
- DOCX: scan only docProps/customXml parts, not the visible body (#14)
- Text Layer A: preserve emoji VS16/ZWJ after an emoji base; add paranoid flag (#22)
- HTML: treat CMS generator tags as informational, not AI metadata (#13)
- PDF: exclude stream payloads from AI-marker byte scan (#13)
- Add unsupported/best-effort notes to inspect reports
* fix: make subprocess and symlink tests Windows-compatible
- Gate preexec_fn (POSIX-only) so optional tools can run on Windows
- Skip symlink tests when the platform denies the privilege
- Add a Windows CI leg and a Windows CLI smoke run
* fix: use UTF-8 stdio and fail Windows smoke on first error
- Reconfigure stdin/stdout/stderr to UTF-8 so Windows redirected streams
stop raising UnicodeEncodeError on invisible Unicode characters
- Chain the Windows smoke commands so a failing script fails the step
- dependabot.yml: weekly updates for GitHub Actions (rotates the SHA pins),
root pip deps (requirements-dev.txt), and the scorer requirements
(compatibility-coupled to the pinned upstream commit — human re-verify
noted), monthly for the digest-pinned Docker base image.
- CODEOWNERS: keep the default maintainer ownership and make ownership of
/.github and SECURITY.md explicit.
