mirror of
https://github.com/guillaumemeyer/watermarks-remover.git
synced 2026-08-22 13:11:57 +02:00
feat: split skill from service, add HTTP API and Docker distribution (#60)
* feat: split skill from service, add HTTP API and Docker distribution The agent skill (skills/remove-ai-marks/) is now a code-free remote client: all implementation moved to service/scripts/ and runs behind a stdlib HTTP service (server.py) with /health, /capabilities, /inspect, /clean and a dynamically generated OpenAPI 3.0.3 spec at /openapi.json. - Move scripts/ and the backend Dockerfiles under service/ - server.py: JSON/base64 HTTP entrypoint with size caps, binary guard, atomic writes, loopback default, optional bearer auth - Core Dockerfile (exiftool/qpdf/c2patool preinstalled) and a GHCR publish workflow for the core/markllm/markdiffusion images - compose.yaml (wr-* services, harness/heavy profiles) + compose-check.sh to validate the running stack (exit code only) - Fix markllm image build (tokenizers 0.22.2, CPU-only torch) and ctrlregen build (python:3.11 base for the 2023-era research pins) - Fix markllm/markdiffusion harness images missing common.py at runtime * docs: add .env.example and service configuration guide * fix: disable chain-of-thought for openai-compatible Layer B rewrites deepseek-v4-flash is a reasoning model: a one-line paraphrase burned 9,894 reasoning tokens (~100s) and hit the default timeout. Send reasoning_effort=none by default for the openai-compatible backend (--reasoning-effort / WATERMARKS_REWRITE_REASONING_EFFORT; 'off' omits the parameter), cutting the same rewrite to ~1s / 12 tokens. Tested end-to-end against api.deepseek.com. * fix: sanitize client-supplied filename in HTTP service CodeQL 'uncontrolled data in path expression' (server.py): a name like '../../x' flowed into Path(tmpdir) / name, letting an upload escape the request temp dir on write. Sanitize name to its basename in _decode_input (_safe_name) and refuse any joined path whose parent is not the tmpdir at the write sites (_tmp_path). Tests cover traversal names. * chore: gitignore .env (contains local rewrite credentials) * chore: deny-by-default gitignore and dockerignore; document compose env config .gitignore and service/.dockerignore now exclude everything by default and explicitly allow only what is publishable/needed: tracked source, docs, tests, .github, and (for images) the service/scripts/ tree that every Dockerfile COPYs. Root .dockerignore documents that all builds use service/ as context. README Configuration section now covers .env setup for docker compose, host-side export for CLI runs, and the full variable table.
This commit is contained in:
parent
61440192af
commit
55d4bdc9fc
+116
@@ -0,0 +1,116 @@
|
||||
# Whole-infra bring-up for watermarks-remover.
|
||||
#
|
||||
# docker compose up --build -d # core HTTP service only
|
||||
# docker compose --profile harness up --build -d # + markllm / markdiffusion harnesses
|
||||
# docker compose --profile heavy up --build -d # + ctrlregen / synthid (local builds)
|
||||
#
|
||||
# The skill and any web app talk to the wr-core service at http://127.0.0.1:8765
|
||||
# (loopback-only host mapping). The heavy/harness images are one-shot CLIs:
|
||||
# `up` starts them with `--help` to confirm the image, and real jobs run via
|
||||
# `docker compose run`, e.g.:
|
||||
# docker compose run --rm wr-ctrlregen /data/shot.png -o /data/out.png
|
||||
#
|
||||
# ctrlregen and synthid bake in upstream code that is not publicly
|
||||
# redistributable (all-rights-reserved / non-commercial Research License), so
|
||||
# they build from source locally and are never pushed to GHCR.
|
||||
|
||||
name: watermarks-remover
|
||||
|
||||
services:
|
||||
wr-core:
|
||||
build:
|
||||
context: service
|
||||
dockerfile: Dockerfile
|
||||
image: ghcr.io/guillaumemeyer/watermarks-remover:latest
|
||||
ports:
|
||||
- "127.0.0.1:8765:8765"
|
||||
environment:
|
||||
# Empty by default (no auth). Set to require `Authorization: Bearer <key>`.
|
||||
WATERMARKS_SERVER_API_KEY: ${WATERMARKS_SERVER_API_KEY:-}
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
init: true
|
||||
user: "10001:10001"
|
||||
restart: unless-stopped
|
||||
|
||||
wr-markllm:
|
||||
profiles: [harness]
|
||||
build:
|
||||
context: service
|
||||
dockerfile: Dockerfile.markllm
|
||||
image: ghcr.io/guillaumemeyer/watermarks-remover:markllm-latest
|
||||
# One-shot CLI: `up` just confirms the image; run real jobs with
|
||||
# `docker compose run --rm wr-markllm detect /data/wm.txt --scheme kgw`.
|
||||
command: ["--help"]
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
init: true
|
||||
user: "10001:10001"
|
||||
environment:
|
||||
HF_TOKEN: ${HF_TOKEN:-}
|
||||
HF_HOME: /home/markllm/.cache/huggingface
|
||||
volumes:
|
||||
- markllm-cache:/home/markllm/.cache/huggingface
|
||||
|
||||
wr-markdiffusion:
|
||||
profiles: [harness]
|
||||
build:
|
||||
context: service
|
||||
dockerfile: Dockerfile.markdiffusion
|
||||
image: ghcr.io/guillaumemeyer/watermarks-remover:markdiffusion-latest
|
||||
command: ["--help"]
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
init: true
|
||||
user: "10001:10001"
|
||||
environment:
|
||||
HF_TOKEN: ${HF_TOKEN:-}
|
||||
HF_HOME: /home/markdiffusion/.cache/huggingface
|
||||
volumes:
|
||||
- markdiffusion-cache:/home/markdiffusion/.cache/huggingface
|
||||
|
||||
wr-ctrlregen:
|
||||
profiles: [heavy]
|
||||
build:
|
||||
context: service
|
||||
dockerfile: Dockerfile.ctrlregen
|
||||
# Local-only tag: upstream `noai-watermark` ships no LICENSE (all-rights-
|
||||
# reserved), so this image is never published.
|
||||
image: watermarks-remover-ctrlregen:local
|
||||
command: ["--help"]
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
init: true
|
||||
user: "10001:10001"
|
||||
environment:
|
||||
HF_TOKEN: ${HF_TOKEN:-}
|
||||
NOAI_WATERMARK_DIR: /opt/noai-watermark
|
||||
volumes:
|
||||
- ctrlregen-cache:/home/remover/.cache/huggingface
|
||||
|
||||
wr-synthid:
|
||||
profiles: [heavy]
|
||||
build:
|
||||
context: service
|
||||
dockerfile: Dockerfile.synthid
|
||||
# Local-only tag: upstream `reverse-SynthID` is under a non-commercial
|
||||
# Research License, so this image is never published.
|
||||
image: watermarks-remover-synthid-scorer:local
|
||||
command: ["--help"]
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
init: true
|
||||
user: "10001:10001"
|
||||
volumes:
|
||||
- synthid-cache:/home/scorer/.cache/huggingface
|
||||
|
||||
volumes:
|
||||
markllm-cache:
|
||||
markdiffusion-cache:
|
||||
ctrlregen-cache:
|
||||
synthid-cache:
|
||||
Reference in New Issue
Block a user