* fix: parse single-quoted attributes in relationship/manifest pruning
XML allows attribute values in single or double quotes. The Target
extraction in _prune_dangling_relationships and the full-path
extraction in _prune_odt_manifest_entries matched only double quotes,
so a valid single-quoted Relationship parsed as an empty target,
resolved to the package base directory, and was deleted outright —
corrupting DOCX/XLSX/PPTX and ODT packages produced by tools that
emit single quotes (#130).
Both extractions now use a backreference pattern
((["'])(.*?)\1) so either quote style resolves the real target, and
pruning still only drops relationships whose parts were genuinely
removed. Regression tests cover the single-quoted kept relationship
(the reported corruption) and the double-quoted dropped part.
* fix: satisfy ruff lint and format checks
---------
Co-authored-by: yzxcj797 <yzxcj797@users.noreply.github.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
clean_svg/clean_odt stripped metadata with lazy dot-matches-all regexes; on many unclosed opening tags the engine rescans to end-of-input from every candidate start (O(n^2)), and the GIL stalls the whole single-process service. A ~1.4 KB ODT request pinned a core for ~99 s. Replace every lazy .*?</close> block scan in container_meta.py (SVG metadata/xmpmeta/comments, ODT meta:generator/dc:creator, HTML JSON-LD, OOXML/ODF text-run scrubs, docProps fields, EPUB OPF meta/dc, PDF xpacket/stream) with a linear scan pairing opening tags with a forward pointer over closing tags - identical match semantics, O(n). Add regression tests asserting the advisory PoC completes in <5 s and that stripping behavior is unchanged.
The file-cleaners layer covered 15 formats -- all image, document, or
text -- and zero audio/video. That gap gets more expensive every month:
Sora, Veo, ElevenLabs, and Suno all embed provenance through the same
mechanisms image generators do, just in different containers.
New av_meta.py adds inspect/clean for:
- MP4/MOV/M4A/M4V: top-level C2PA (jumb/c2pa box) and XMP (uuid box)
detection/stripping reuse inspect_isobmff()/strip_isobmff() from
image_meta.py unchanged -- that's exactly the mechanism the C2PA spec
defines for ISOBMFF-family containers, already proven for AVIF/HEIC.
moov/udta (where generator/tool tags live) is handled separately since
it's MP4-specific.
- WAV: RIFF LIST INFO chunk + embedded id3 chunk.
- MP3: ID3v2 frames, per-frame for v2.3/v2.4, whole-tag fallback for
v2.2 (3-byte frame IDs are detected but not decomposed, so a partial
rewrite is never attempted there).
Every box/chunk/frame is either kept byte-identical or dropped whole --
nothing does a partial in-place rewrite of a payload, so a container can
never come out semantically mangled. Default strip_all_metadata=True
matches this project's existing default (privacy-first: drop everything,
--keep-non-ai-metadata narrows to only AI-flagged content), same as the
image cleaners.
Wired through the full dispatch stack so the feature isn't a half
integration: format_dispatch.py (new "av" Kind), inspect_file.py /
clean_file.py (--as av), audit_lib.py (so audit_dir.py's CI/SARIF path
and the pre-commit hooks from #135 both cover audio/video too), and
server.py (HTTP /inspect and /clean).
Closes#134
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
CI gating for AI provenance marks already exists (audit_dir.py's -j
concurrency + SARIF export from #101), but that only runs after a
marked file has already been committed and pushed. Catch it at commit
time instead, using git's own hook point.
Adds two hooks via .pre-commit-hooks.yaml:
- watermarks-remover-check: fails the commit and lists findings when
staged files carry AI/C2PA marks. Wraps audit_lib.scan_file() /
is_actionable() -- the exact per-file logic audit_dir.py already
uses for CI, so the pre-commit gate and the CI gate agree on what
counts as actionable.
- watermarks-remover-clean (opt-in): rewrites staged files in place by
shelling out to clean_file.py --in-place per file (no duplicated
cleaning logic), then exits 1 so the developer reviews the diff and
re-stages -- the same convention as auto-fixing hooks like ruff --fix.
.pre-commit-hooks.yaml needed an explicit allow-rule in the deny-by-
default .gitignore, same as every other root-level config file already
listed there.
Closes#135
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
Directory-scale cleaning already exists in the CLI (audit_dir.py, -j
concurrency, SARIF export from #101), but the HTTP service handled one
file per request. Any web app or CI step talking to the service over
HTTP instead of the CLI paid N full round trips to clean N files.
Extract the single-file /inspect and /clean logic into _inspect_payload
and _clean_payload so both the existing single-file endpoints and the
new batch endpoints run the identical pipeline — no duplicated cleaning
logic. A malformed entry in a batch (bad base64, unknown option,
unrecognized format) surfaces as that entry's "ok": false with an
"error" string instead of aborting the rest of the batch.
Capped at WATERMARKS_MAX_BATCH_FILES per request (default 50) as
defense-in-depth against a request packing many tiny files into one
call; the existing MAX_BODY_BYTES envelope cap already bounds total
payload size the same as a single-file request.
/openapi.json picks up both routes automatically since the spec is
generated from the route table.
Closes#136
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
* feat: add vendor text-watermark detection and SynthID image scorer sidecar
Adds Layer B watermark detection as a first-class service capability:
- text_detectors.py: a registry of text-watermark detectors behind one
interface — Google's official SynthID-text detector via the Gemini API
(taskType DETECT_TEXT_WATERMARK), a Claude placeholder ready for
Anthropic's announced detection API, and the MarkLLM research harness
(KGW / SynthID, same-config-only). Fail-soft: unconfigured or errored
detectors never block cleaning.
- server.py: new POST /detect endpoint, detect_before / detect_after options
on /clean (before/after scoring for text and images), an opt-in
/inspect "detect" flag, and /capabilities gains text_detectors and
scorers.synthid_http.
- synthid_score_server.py: a stdlib HTTP sidecar for the reverse-SynthID
scorer, so the published core image never bundles the non-commercial
upstream code; wired via WATERMARKS_SYNTHID_SCORER_URL.
- score_synthid.py: extract score_file() so the CLI and the sidecar share
one implementation.
- compose.yaml / Dockerfile.synthid / .env.example: wr-synthid-score sidecar
service and env wiring.
- README + skill docs, plus tests for the detectors, the /detect endpoint,
and the image sidecar.
* feat: per-candidate watermark detection for Layer B rewrite candidates
When --candidates N (N > 1) is combined with --markllm-scheme or
WATERMARKS_GEMINI_API_KEY, run every configured text detector from the
text_detectors.py registry on each candidate and report per-candidate
measurements in --json-stats as candidate_scores entries carrying
lexical_divergence, selection_score, selected, and per-detector reports
(is_watermarked, score, threshold where the detector provides one).
Candidate selection stays purely lexical; the detections are observability
for correlating lexical divergence with watermark removal (issue #106).
Converges rewrite_text.py onto the shared detector registry:
- MarkLLMTextDetector gains constructor overrides (scheme, upstream_dir,
model, timeout) plus the checkout-venv interpreter preference and the
WATERMARKS_MARKLLM_RLIMIT_AS preexec guard ported from rewrite_text.py;
the old _markllm_detect / _venv_python / _markllm_preexec helpers are gone.
- run_all_text_detectors() accepts an injected MarkLLM instance and an
include_markllm switch so CLI flag gating stays intact.
- before/after/cleared semantics unchanged; detection remains fail-soft.
* docs: pin Watermarks in the Sand reference to arXiv v5
* fix: mark only one rewrite candidate as selected (#110)
---------
Co-authored-by: Zhenxin Ai <142008897+ai-kunkun@users.noreply.github.com>
* fix: rewrite ODT/EPUB manifests and measure real zip bytes (#122)
Two container correctness/security fixes from issue #122:
- clean_odt dropped marker-bearing parts while leaving their entries in
META-INF/manifest.xml, so readers flagged the package as damaged. It is
now two-pass: compute the dropped set, then rewrite the manifest
attribute-order-independently, and write each part exactly once. The same
bug class in clean_epub (dropped parts left in the OPF manifest, plus
dangling spine itemrefs) gets the same two-pass treatment.
- The zip budget trusted ZipInfo.file_size from the archive's own central
directory, so a crafted DOCX/ODT could declare a tiny size and still
expand via zf.read. Budgets are now charged on actual decompressed bytes
via _read_zip_member (streaming, cap enforced mid-read), with the declared
size kept only as a fast-path pre-reject.
* fix: classify unrecognized bytes as "unknown", not text (#122)
Two classification defects from issue #122:
- format_dispatch.classify_bytes fell back to "text" for any unrecognized
file, so a binary with valid UTF-8 runs could be decoded and written back
mangled (corrupted with --in-place) in clean_file auto mode. Unrecognized
bytes now classify as "unknown"; clean_file refuses them in auto mode
(exit 2, no write, router advice) and --as text / --force-text are the
explicit opt-ins. inspect_file reports kind "unknown" (exit 0), audit_lib
records a non-actionable item, and the HTTP server answers /inspect with
kind "unknown" but rejects /clean of unknown formats (400).
- classify(path) read the whole file to sniff a header, and only a full read
could detect zip containers. It now routes known extensions without
reading, sniffs a 4096-byte header once for images and prefix-based
containers, and reads the whole file only when the header is a zip local
header (PK), where the container signature lives in the central directory.
* feat: distinct exit code for partial audits (#122)
audit_dir and audit_website reported success (0) even when some files or
URLs could not be scanned; the exit status was computed only over the items
that succeeded. A scan that is missing items is not a clean scan.
- common.EXIT_PARTIAL = 3, with precedence: partial (3) > actionable (1)
> clean (0) — an incomplete audit is the more important CI signal.
- audit_dir returns 3 when any file was skipped/failed; audit_website
returns 3 when any URL failed to fetch or inspect. Both are independent
of the output format (human/json/sarif already share one return).
* fix: verify the pinned upstream ref on existing checkouts (#122)
setup_ctrlregen.sh/setup_synthid.sh (and their .ps1 twins) only verified
the pinned commit in the fresh-clone branch; an existing checkout at an
unknown or drifted revision was silently reused, defeating the commit pin.
All four scripts now check HEAD against the pinned ref in the
existing-checkout branch too, and repair by fetch + detach checkout
(re-applying the sparse-checkout set), failing hard if the ref cannot be
reached or the re-pin does not land on it.
* docs: unknown-format behavior, audit exit codes, backend isolation (#122)
- README: clean_file no longer auto-cleans unrecognized formats (--as text
/ --force-text are the opt-ins), and the CtrlRegen bootstrap documents the
isolation expectation for its research-era dependency pins plus the new
re-pin check on existing checkouts.
- SKILL.md: audit exit codes (0/1/2/3, partial=3) and a note that /clean
requires a name with a known extension.
- audit_website: document why stdlib ElementTree is used (stdlib-first) and
that defusedxml is the fallback if that policy changes (DTD rejection stays).
- requirements-ctrlregen.txt: advisory/isolation note for the pinned research
dependencies.
* test: ODT manifest and EPUB OPF dangling-ref regressions (#122)
- clean_odt: dropped marker-bearing parts remove their META-INF/manifest.xml
file-entry (attribute-order-independent), exactly one manifest entry, root
and surviving entries kept, and the manifest is byte-identical when nothing
is dropped.
- clean_epub: dropped non-content parts lose their <item> entry in the OPF
manifest, so the book no longer references removed members.
Three independent failure modes from #117:
- $ErrorActionPreference 'Stop' + 2>$null on a native command aborts the
script on torch's harmless stderr warnings (e.g. "Failed to initialize
NumPy" when torch is installed before numpy). Run the probes through a
new Invoke-NativeQuiet helper that lowers EAP to 'Continue' for the
block and restores it afterwards.
- The wheel index tag was derived from the driver's CUDA version, e.g.
cu131 for a 13.1 driver, which does not exist (HTTP 403) and silently
fell back to the default index, i.e. the CPU build on Windows. Probe
the published indices and pick the highest one <= driver that answers
HTTP 200; cu126 is still forced below compute capability 7.5.
- Installing torch alone let requirements-ctrlregen.txt resolve torchvision
from PyPI, and torchvision pins an exact torch, so pip replaced the +cu
build with a +cpu one while the script still exited 0. Install torch AND
torchvision together from the chosen index, and verify after the
requirements install that torch.cuda.is_available() is true - if a GPU
was detected but torch ends up CPU-only, warn loudly and exit non-zero.
Also add a CI step (windows-latest, pwsh) that parses the setup .ps1
scripts and asserts the post-install CUDA verification survives.
Fixes#117
* Handle malformed DOCX/ODT beyond BadZipFile in container inspectors
A corrupt, truncated, encrypted, or unsupported-compression container
raises more than BadZipFile (NotImplementedError, RuntimeError, EOFError,
OSError, ValueError, zlib.error), so inspect_docx / inspect_odt and
detect_container_format now catch the whole family and degrade to a clear
finding.
The zip-bomb rejection in _check_zip_budget now raises a dedicated
ZipBudgetExceeded so it keeps propagating out of the inspectors, matching
clean_docx / clean_odt, instead of being reported as an unparseable zip.
* fix: satisfy ruff lint and format checks
---------
Co-authored-by: eeshsaxena <chinmaymd72@gmail.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
`base64 -w0 FILE` is a GNU coreutils form. On macOS/BSD it fails with
`base64: invalid argument FILE` and prints usage, so every documented
`/inspect` and `/clean` example is unrunnable on a Mac host.
This matters more than a normal docs typo because SKILL.md instructs the
agent to call the service via these exact commands and explicitly forbids
falling back to local cleaning when the service call fails. A macOS agent
therefore stops at its first request instead of degrading gracefully.
Replace all 7 occurrences (2 in README.md, 5 in SKILL.md) with the POSIX
form, which reads from stdin and unwraps any line breaks:
base64 < FILE | tr -d '\n'
Verified byte-identical output on both implementations:
- BSD/macOS base64 (macOS 15, Darwin 24.6.0)
- GNU coreutils base64 (`gbase64`)
Also verified the rewritten commands verbatim against a live
`server.py` on 127.0.0.1:8765 — `/inspect` and `/clean` both return
`ok: true` and correctly strip U+200B / U+00AD.
Docs-only; no code paths changed, so no unit tests added.
`python3 -m pytest -q` and `ruff check`/`format --check` pass.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
* docs: add Windows auto-start guide via Task Scheduler
* docs: fix markdown escaping, remove admin requirement, use path placeholder, link from README
* docs: add watermarks-remover-web to Ecosystem (#77)
* docs: note AVIF and HEIC in the watermarks-remover-web entry
The client now ports the ISOBMFF paths from image_meta.py as well, so the
entry's format list was out of date.
* docs: rename the web UI entry to unmark-web and keep it higher-level
Renames the project per the maintainer's request, and drops the
references to internal module names, the client's proxy script and the
format list, which were the parts most likely to drift.
---------
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
Add stdlib-only detection, inspection, and cleaning for four more formats.
- BMP: locate the pixel payload via the DIB header and strip trailing
non-image metadata (the only place non-standard BMP metadata can live),
rewriting the file-size field.
- GIF: drop comment and XMP application extensions while preserving
NETSCAPE2.0 looping, ICC, graphic-control, and image blocks.
- TIFF (classic + BigTIFF): walk IFD chains and drop XMP/EXIF/GPS/IPTC/
Photoshop/MakerNote tags, zeroing orphaned payloads while keeping
strip/tile offsets valid.
- EPUB: scrub OPF package metadata and XHTML meta/JSON-LD, clean embedded
raster/SVG media, apply Layer A to XHTML body text, and pass OCF-encrypted
parts through untouched.
All four route through format_dispatch, so the unified CLIs, the HTTP
service, and the audits pick them up automatically.
* fix: pin vendored Cursor-skill text engine to the service copy
The engine vendored into skills/clean-user-facing-text/ had silently
drifted behind service/scripts/text_unicode.py: it still blanket-
stripped legitimate RTL directional marks and isolates (corrupting
mixed RTL/LTR prose) and stripped emoji variation selectors after
arrow and symbol bases, because it never received the preservable-bidi
and emoji-base updates. Nothing in the suite compared the two copies.
Replace the vendored engine with an exact copy of the service one and
add a byte-equality test so any future engine change must land in both
files in the same commit, plus a behavioural regression test for the
RTL and arrow cases through the vendored CLI. The CLI wrappers remain
deliberately different (text-only skill: no stylometry, no
--strip-bidi).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: re-pin vendored engine after merging main and correct bidi comment
Merging main into the PR branch kept main's ruff-formatted service
engine (#103) but the PR's vendored copy pinned in 519c6db, so the two
drifted apart again and test_vendored_text_unicode_is_identical_to_service_engine
failed. Re-sync the vendored copy to the current service engine so the
byte-for-byte pin holds.
Also correct the _PRESERVABLE_BIDI_CPS comment: paired LRE/RLE embeddings
are preserved via _valid_bidi_embedding_indices, so only overrides and
unpaired embeddings remain destructive by default.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
The bullet 'Best-effort: prose was rewritten...' uses a full sentence with
passive verb while adjacent bullets use noun phrases. Removed 'was' to
match the parallel structure of the other bullets.
Audit metadata:
- Category: grammar
- Severity: minor
- File: skills/clean-user-facing-text/SKILL.md, line 61
- Fingerprint: a9c80f5f5afbfbe7
Co-authored-by: pi <pi@m2.local>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
resolve_device() auto-selects 'mps' whenever torch reports it available.
Both schemes this harness can load build a device-bound RNG in their
constructor:
KGW.__init__ -> torch.Generator(device=config.device) (kgw.py)
SynthID.__init__ -> torch.Generator(device=config.device) (synthid.py)
torch.Generator supports only the 'cpu' and 'cuda' device types, so on
Apple Silicon a resolved 'mps' device raises
RuntimeError: Device type MPS is not supported for torch.Generator() api
and both the `detect` and `watermark` subcommands crash out of the box on
any Mac where torch reports MPS available.
Drop the mps branch from the auto path so `--device auto` resolves to cpu
on CUDA-less machines. `--device mps` is still honored as an explicit
override (resolve_device returns any non-'auto' value unchanged), so this
changes only the automatic default, not the ability to opt in.
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
Introduce Ruff (pinned at 0.16.3) as the project linter + formatter and
enforce it in CI:
- requirements-dev.txt: pin ruff==0.16.3 (exact pins, no drift)
- ruff.toml: line-length 100, target py312; rule set E/F/W/I/UP/B/SIM/RUF/PLW/S
with deliberate ignores (E501 for content strings, S603 for safe_arg
subprocess calls, S101 asserts in tests) and per-file test ignores
- Makefile: add lint / format / lint-fix targets
- .github/workflows/ci.yml: add lint job (ruff check + format --check)
- .gitignore: whitelist ruff.toml
Also fix every finding the new gate surfaced so CI is green:
- 109+ auto-fixes from ruff --fix (import sorting, simplifications,
unused vars, re.I aliases, etc.)
- explicit check=False on all subprocess.run calls (PLW1510)
- harden sitemap XML parsing: reject DTD/entity declarations (S314)
- replace hardcoded /tmp paths in tests with tmp_path (S108)
- narrow/annotate intentional bare excepts (S110/S112), bind loop vars
in closures (B023), raise ... from None (B904), strict= for zip (B905)
- ruff format applied across service/ and tests/
Verified: ruff check + ruff format --check pass; 287 tests pass, 1 skip.
* fix(inspect): run Layer A scan on markdown/html containers
inspect_container() never scanned the text body, so a .md or .html file
carrying invisible Unicode was reported suspicious=false while
clean_container() went on to strip it. Identical bytes gave opposite
verdicts depending on the file extension.
Scan Layer A for exactly the formats clean_container() scrubs (markdown,
html) so inspect predicts clean. Decode with surrogateescape to match
clean's decoding. Expose the count as suspicious_total, the same key
TextInspectReport uses, so the HTTP server's suspicious flag and the
inspect_file CLI exit code pick it up without special-casing.
* docs: changelog entry for the container Layer A inspect fix
* fix(audit): drop duplicate Layer A scan for markdown/html containers
inspect_container() now scans the body for markdown/html, so
audit_lib.scan_file's own Layer A scan produced the same findings twice
(once as 'layer-a:' from the container report, once as 'layer-a [kind]')
and double-counted them in the aggregate. Keep the stylometry check,
which still needs the decoded body text.
---------
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
* fix: keep the SynthID scorer's --json stdout pure
The reverse-SynthID upstream prints progress ('CodebookV4 loaded: ...')
straight to stdout. image_meta.py parses the scorer's stdout with
json.loads, so the leak corrupts every score payload into
{'available': False, 'error': 'bad scorer JSON: ...'}.
Redirect stdout to stderr around the upstream calls so --json owns
stdout. Regression test drives the real script against a deliberately
noisy stub upstream (with a stub cv2, so it runs without OpenCV).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: probe realpath -m support instead of realpath presence
macOS ships BSD realpath, which exists but has no -m flag, so
'command -v realpath' takes the GNU branch and both setup bootstraps
abort on the first path they normalize. Probe the flag itself; BSD
systems fall through to the portable pwd fallback already in place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Adriel <adriel@Adriels-MacBook-Pro-2026.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
Keep valid RTL controls, script joiners, variation sequences, and emoji structure while still removing malformed carriers, with regression coverage for each case.
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
* feat: add lightweight Cursor and Codex text skill
Package the text-only workflow with safe cross-platform installation, optional persistent instructions, and focused tests so users can adopt it without the media tooling.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: keep the lightweight skill independently reviewable
Avoid coupling the packaging PR to the separate Unicode safety change so either pull request can merge on its own.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: force UTF-8 for subprocess stdin in Windows CI
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
Bump accelerate to 1.14.0, controlnet-aux to 0.0.10, and safetensors to
0.8.0 in requirements-ctrlregen.txt. Verified end-to-end against the pinned
noai-watermark commit b642ae45 (import + CUDA inference) and via Docker build.
Pin huggingface_hub to 0.24.0: 0.26.0+ removed the cached_download symbol
that diffusers 0.27.2 still imports, so unpinned installs resolve 0.36.2 and
the backend fails to import. Refresh the Dockerfile.ctrlregen base-image
comment to drop the now-stale safetensors 0.4.3 mention.
* feat: split skill from service, add HTTP API and Docker distribution
The agent skill (skills/remove-ai-marks/) is now a code-free remote client:
all implementation moved to service/scripts/ and runs behind a stdlib HTTP
service (server.py) with /health, /capabilities, /inspect, /clean and a
dynamically generated OpenAPI 3.0.3 spec at /openapi.json.
- Move scripts/ and the backend Dockerfiles under service/
- server.py: JSON/base64 HTTP entrypoint with size caps, binary guard,
atomic writes, loopback default, optional bearer auth
- Core Dockerfile (exiftool/qpdf/c2patool preinstalled) and a GHCR publish
workflow for the core/markllm/markdiffusion images
- compose.yaml (wr-* services, harness/heavy profiles) + compose-check.sh
to validate the running stack (exit code only)
- Fix markllm image build (tokenizers 0.22.2, CPU-only torch) and ctrlregen
build (python:3.11 base for the 2023-era research pins)
- Fix markllm/markdiffusion harness images missing common.py at runtime
* docs: add .env.example and service configuration guide
* fix: disable chain-of-thought for openai-compatible Layer B rewrites
deepseek-v4-flash is a reasoning model: a one-line paraphrase burned 9,894
reasoning tokens (~100s) and hit the default timeout. Send
reasoning_effort=none by default for the openai-compatible backend
(--reasoning-effort / WATERMARKS_REWRITE_REASONING_EFFORT; 'off' omits the
parameter), cutting the same rewrite to ~1s / 12 tokens. Tested end-to-end
against api.deepseek.com.
* fix: sanitize client-supplied filename in HTTP service
CodeQL 'uncontrolled data in path expression' (server.py): a name like
'../../x' flowed into Path(tmpdir) / name, letting an upload escape the
request temp dir on write. Sanitize name to its basename in _decode_input
(_safe_name) and refuse any joined path whose parent is not the tmpdir at
the write sites (_tmp_path). Tests cover traversal names.
* chore: gitignore .env (contains local rewrite credentials)
* chore: deny-by-default gitignore and dockerignore; document compose env config
.gitignore and service/.dockerignore now exclude everything by default and
explicitly allow only what is publishable/needed: tracked source, docs,
tests, .github, and (for images) the service/scripts/ tree that every
Dockerfile COPYs. Root .dockerignore documents that all builds use service/
as context. README Configuration section now covers .env setup for docker
compose, host-side export for CLI runs, and the full variable table.
