External/watermarks-remover
1
0
Fork 0
mirror of https://github.com/guillaumemeyer/watermarks-remover.git synced 2026-08-22 13:11:57 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: vendor text-watermark detection (Gemini SynthID, Claude seam, MarkLLM) + SynthID image scorer sidecar (#109)

Browse Source 
* feat: add vendor text-watermark detection and SynthID image scorer sidecar

Adds Layer B watermark detection as a first-class service capability:

- text_detectors.py: a registry of text-watermark detectors behind one
  interface — Google's official SynthID-text detector via the Gemini API
  (taskType DETECT_TEXT_WATERMARK), a Claude placeholder ready for
  Anthropic's announced detection API, and the MarkLLM research harness
  (KGW / SynthID, same-config-only). Fail-soft: unconfigured or errored
  detectors never block cleaning.
- server.py: new POST /detect endpoint, detect_before / detect_after options
  on /clean (before/after scoring for text and images), an opt-in
  /inspect "detect" flag, and /capabilities gains text_detectors and
  scorers.synthid_http.
- synthid_score_server.py: a stdlib HTTP sidecar for the reverse-SynthID
  scorer, so the published core image never bundles the non-commercial
  upstream code; wired via WATERMARKS_SYNTHID_SCORER_URL.
- score_synthid.py: extract score_file() so the CLI and the sidecar share
  one implementation.
- compose.yaml / Dockerfile.synthid / .env.example: wr-synthid-score sidecar
  service and env wiring.
- README + skill docs, plus tests for the detectors, the /detect endpoint,
  and the image sidecar.

* feat: per-candidate watermark detection for Layer B rewrite candidates

When --candidates N (N > 1) is combined with --markllm-scheme or
WATERMARKS_GEMINI_API_KEY, run every configured text detector from the
text_detectors.py registry on each candidate and report per-candidate
measurements in --json-stats as candidate_scores entries carrying
lexical_divergence, selection_score, selected, and per-detector reports
(is_watermarked, score, threshold where the detector provides one).

Candidate selection stays purely lexical; the detections are observability
for correlating lexical divergence with watermark removal (issue #106).

Converges rewrite_text.py onto the shared detector registry:
- MarkLLMTextDetector gains constructor overrides (scheme, upstream_dir,
  model, timeout) plus the checkout-venv interpreter preference and the
  WATERMARKS_MARKLLM_RLIMIT_AS preexec guard ported from rewrite_text.py;
  the old _markllm_detect / _venv_python / _markllm_preexec helpers are gone.
- run_all_text_detectors() accepts an injected MarkLLM instance and an
  include_markllm switch so CLI flag gating stays intact.
- before/after/cleared semantics unchanged; detection remains fail-soft.

* docs: pin Watermarks in the Sand reference to arXiv v5

* fix: mark only one rewrite candidate as selected (#110)

---------

Co-authored-by: Zhenxin Ai <142008897+ai-kunkun@users.noreply.github.com>
This commit is contained in:
Guillaume Meyer (The Opinionated Man)
2026-08-17 18:31:22 -07:00
committed by GitHub
co-authored by Zhenxin Ai
parent 6df80e77a4
commit a2e72ed019
14 changed files with 1963 additions and 328 deletions
Download Patch File Download Diff File Expand all files Collapse all files
service/Dockerfile.synthid
+1
View File
@@ -45,6 +45,7 @@ RUN git clone --depth 1 --filter=blob:none --sparse \
COPY scripts/requirements-synthid-scorer.txt /app/requirements-synthid-scorer.txt
COPY scripts/score_synthid.py /app/score_synthid.py
COPY scripts/synthid_score_server.py /app/synthid_score_server.py
RUN python3 -m pip install --no-cache-dir "pip==26.2.1" \
&& python3 -m pip install --no-cache-dir -r /app/requirements-synthid-scorer.txt
service/scripts/rewrite_text.py
+57 -109
View File
@@ -27,17 +27,16 @@ import itertools
import json
import os
import re
import subprocess
import sys
import urllib.error
import urllib.request
from collections.abc import Callable
from pathlib import Path
from urllib.parse import urlparse
sys.path.insert(0, str(Path(__file__).resolve().parent))
from common import cleaned_path, eprint, read_text_input, write_text_output
from text_detectors import MarkLLMTextDetector, run_all_text_detectors
from text_unicode import clean_text
DEFAULT_MARKLLM_MODEL = "facebook/opt-1.3b"
@@ -174,100 +173,31 @@ class _NoRedirect(urllib.request.HTTPRedirectHandler):
raise urllib.error.HTTPError(req.full_url, code, msg, headers, fp)
SCRIPTS_DIR = Path(__file__).resolve().parent
def _per_candidate_detections(
candidates: list[str],
markllm_detector: MarkLLMTextDetector | None,
) -> list[list[dict]]:
"""Run every configured text detector on each rewrite candidate.
def _venv_python(upstream: Path) -> Path | None:
"""Locate the MarkLLM checkout's venv interpreter, if it exists."""
if os.name == "nt":
candidate = upstream / ".venv" / "Scripts" / "python.exe"
else:
candidate = upstream / ".venv" / "bin" / "python"
return candidate if candidate.is_file() else None
def _markllm_preexec() -> Callable[[], None] | None:
"""Optional RLIMIT_AS guard for the MarkLLM child; None means "no limit".
torch/CUDA usually needs large address spaces, so unlike the
exiftool/c2patool/SynthID children (common.subprocess_rlimits) this is
opt-in via WATERMARKS_MARKLLM_RLIMIT_AS (byte count, hex/octal allowed).
POSIX only; on Windows preexec_fn must stay None.
Fail-soft: a detector that is unconfigured, times out, or errors yields
an ``available: False`` entry and never fails the rewrite. The MarkLLM
harness is only included when ``markllm_detector`` is given (i.e. the
caller passed --markllm-scheme); other detectors (e.g.
gemini-synthid-text) are key-gated by their own environment.
"""
raw = os.environ.get("WATERMARKS_MARKLLM_RLIMIT_AS")
if not raw or os.name != "posix":
return None
try:
limit = int(raw, 0)
except ValueError:
return None
def _apply() -> None:
import resource
resource.setrlimit(resource.RLIMIT_AS, (limit, limit))
return _apply
def _markllm_detect(
text: str,
*,
scheme: str,
upstream_dir: str,
model: str,
timeout: float,
) -> dict:
"""Run the MarkLLM adapter on *text*; never fails the rewrite.
Returns the adapter's JSON payload, or an ``available: False`` dict with
an ``error`` string when the backend is unconfigured or broken. The Layer B
rewrite proceeds regardless; MarkLLM verification is best-effort.
"""
if not upstream_dir:
return {"available": False, "error": "no MARKLLM_DIR set"}
upstream = Path(upstream_dir).expanduser().resolve()
if not upstream.is_dir() or not (upstream / "watermark").is_dir():
return {"available": False, "error": f"MarkLLM checkout missing: {upstream}"}
venv_python = _venv_python(upstream)
if venv_python is None:
return {"available": False, "error": f"MarkLLM venv missing: {upstream}"}
cmd = [
str(venv_python),
str(SCRIPTS_DIR / "detect_text_watermark.py"),
"detect",
"-",
"--scheme",
scheme,
"--upstream-dir",
str(upstream),
"--model",
model,
"--json",
]
try:
r = subprocess.run(
cmd,
input=text,
capture_output=True,
text=True,
timeout=timeout,
preexec_fn=_markllm_preexec(),
check=False,
)
except (OSError, subprocess.SubprocessError, TimeoutError) as e:
return {"available": False, "error": f"MarkLLM adapter error: {e}"}
if r.returncode != 0:
return {
"available": False,
"error": (r.stderr or "").strip() or f"adapter exited {r.returncode}",
}
try:
return json.loads(r.stdout)
except ValueError as e:
return {"available": False, "error": f"adapter JSON parse error: {e}"}
detections: list[list[dict]] = []
for cand in candidates:
try:
detections.append(
run_all_text_detectors(
cand,
markllm=markllm_detector,
include_markllm=markllm_detector is not None,
)
)
except Exception as e: # defensive: the registry contract is fail-soft
detections.append([{"available": False, "error": f"candidate detection failed: {e}"}])
return detections
def build_prompt(strength: str, text: str, *, lang: str, original_lang: str) -> str:
@@ -400,16 +330,17 @@ def rewrite(
info["reasoning_effort"] = reasoning_effort
markllm: dict | None = None
markllm_detector: MarkLLMTextDetector | None = None
if markllm_scheme:
markllm_detector = MarkLLMTextDetector(
scheme=markllm_scheme,
upstream_dir=markllm_dir,
model=markllm_model or DEFAULT_MARKLLM_MODEL,
timeout=markllm_timeout,
)
markllm = {
"scheme": markllm_scheme,
"before": _markllm_detect(
text,
scheme=markllm_scheme,
upstream_dir=markllm_dir or "",
model=markllm_model or DEFAULT_MARKLLM_MODEL,
timeout=markllm_timeout,
),
"before": markllm_detector.detect(text),
}
if not markllm["before"]["available"]:
eprint(f"markllm verification unavailable: {markllm['before']['error']}")
@@ -447,7 +378,29 @@ def rewrite(
else:
info["candidates"] = n
out, scores = _select_candidate(text, outs)
info["candidate_scores"] = scores
selected_idx = max(range(len(outs)), key=lambda i: scores[i])
trigger = markllm_scheme is not None or bool(
os.environ.get("WATERMARKS_GEMINI_API_KEY", "").strip()
)
detections = _per_candidate_detections(outs, markllm_detector) if trigger else []
info["candidate_scores"] = []
for i, cand in enumerate(outs):
info["candidate_scores"].append(
{
"lexical_divergence": _lexical_divergence(text, cand),
"selection_score": scores[i],
"selected": i == selected_idx,
"detections": detections[i] if trigger else [],
}
)
if trigger and detections:
names = sorted(
{d.get("detector", "?") for dets in detections for d in dets if d.get("available")}
)
eprint(
f"note: running per-candidate watermark detection on {n} candidates"
+ (f" ({', '.join(names)})" if names else "")
)
if layer_a_after:
out, stats = clean_text(out)
@@ -461,13 +414,8 @@ def rewrite(
)
if markllm:
after = _markllm_detect(
out,
scheme=markllm["scheme"],
upstream_dir=markllm_dir or "",
model=markllm_model or DEFAULT_MARKLLM_MODEL,
timeout=markllm_timeout,
)
assert markllm_detector is not None # set together with markllm above
after = markllm_detector.detect(out)
markllm["after"] = after
before = markllm["before"]
if before.get("available") and after.get("available"):
service/scripts/score_synthid.py
+101 -72
View File
@@ -11,6 +11,9 @@ Exit codes:
1 scorer runtime error
2 bad input (missing/unreadable image, bad args)
3 scorer unavailable (not configured / missing deps / missing codebook)
The scoring logic lives in :func:score_file so the CLI and the HTTP
sidecar (synthid_score_server.py) share one implementation.
"""
from __future__ import annotations
@@ -21,6 +24,7 @@ import json
import os
import sys
from pathlib import Path
from typing import Any
def resolve_upstream(raw: str | None) -> Path | None:
@@ -32,6 +36,90 @@ def resolve_upstream(raw: str | None) -> Path | None:
return upstream
def score_file(
path: Path,
*,
upstream_dir: str | None = None,
codebook: Path | None = None,
model: str | None = None,
) -> tuple[int, dict[str, Any] | None]:
"""Score *path* with the reverse-SynthID extractor.
Returns (exit_code, payload) matching the CLI exit-code contract:
0 = scored (payload present), 2 = bad input (payload None),
3 = scorer unavailable (payload None). Errors are printed to stderr so
callers parsing stdout JSON are never corrupted.
"""
if not path.is_file():
print(f"not a file: {path}", file=sys.stderr)
return 2, None
raw_upstream = upstream_dir or os.environ.get("REVERSE_SYNTHID_DIR")
upstream = resolve_upstream(str(raw_upstream) if raw_upstream else None)
if upstream is None:
print(
"SynthID scorer not configured: set REVERSE_SYNTHID_DIR or pass --upstream-dir",
file=sys.stderr,
)
return 3, None
extraction = upstream / "src" / "extraction"
if not extraction.is_dir():
print(f"upstream extraction dir not found: {extraction}", file=sys.stderr)
return 3, None
codebook_path = codebook or upstream / "artifacts" / "spectral_codebook_v4.npz"
codebook_path = Path(codebook_path).expanduser().resolve()
if not codebook_path.is_file():
print(f"codebook not found: {codebook_path}", file=sys.stderr)
return 3, None
sys.path.insert(0, str(extraction))
try:
import cv2
from robust_extractor import RobustSynthIDExtractor
from synthid_bypass_v4 import SpectralCodebookV4
except ImportError as e:
print(f"optional scorer dependencies missing: {e}", file=sys.stderr)
return 3, None
try:
img = cv2.imread(str(path))
if img is None:
print(f"could not load image: {path}", file=sys.stderr)
return 2, None
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# Upstream prints progress ("CodebookV4 loaded: ...") straight to
# stdout, which corrupts --json for any caller that parses us
# (image_meta.py json.loads our stdout). Keep stdout ours alone.
with contextlib.redirect_stdout(sys.stderr):
codebook_v4 = SpectralCodebookV4()
codebook_v4.load(str(codebook_path))
extractor = RobustSynthIDExtractor()
result = extractor.detect_from_v4_codebook(rgb, codebook_v4, model=model)
except Exception as e:
print(f"scorer error: {e}", file=sys.stderr)
return 1, None
payload = {
"available": True,
"upstream_dir": str(upstream),
"codebook": str(codebook_path),
"model": model,
"profile_key": result.details.get("profile_key"),
"exact_match": result.details.get("exact_match"),
"is_watermarked": result.is_watermarked,
"confidence": result.confidence,
"phase_match": result.phase_match,
"per_channel_scores": result.details.get("per_channel_scores"),
"per_channel_n": result.details.get("per_channel_n"),
"multi_scale_consistency": result.multi_scale_consistency,
}
return 0, payload
def main() -> int:
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("path", type=Path, help="Image to score (PNG/JPEG/etc.)")
@@ -51,83 +139,24 @@ def main() -> int:
p.add_argument("--json", action="store_true", help="Emit JSON on stdout")
args = p.parse_args()
if not args.path.is_file():
print(f"not a file: {args.path}", file=sys.stderr)
return 2
raw_upstream = args.upstream_dir or os.environ.get("REVERSE_SYNTHID_DIR")
upstream = resolve_upstream(str(raw_upstream) if raw_upstream else None)
if upstream is None:
print(
"SynthID scorer not configured: set REVERSE_SYNTHID_DIR or pass --upstream-dir",
file=sys.stderr,
)
return 3
extraction = upstream / "src" / "extraction"
if not extraction.is_dir():
print(f"upstream extraction dir not found: {extraction}", file=sys.stderr)
return 3
codebook = args.codebook or upstream / "artifacts" / "spectral_codebook_v4.npz"
codebook = Path(codebook).expanduser().resolve()
if not codebook.is_file():
print(f"codebook not found: {codebook}", file=sys.stderr)
return 3
sys.path.insert(0, str(extraction))
try:
import cv2
from robust_extractor import RobustSynthIDExtractor
from synthid_bypass_v4 import SpectralCodebookV4
except ImportError as e:
print(f"optional scorer dependencies missing: {e}", file=sys.stderr)
return 3
try:
img = cv2.imread(str(args.path))
if img is None:
print(f"could not load image: {args.path}", file=sys.stderr)
return 2
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# Upstream prints progress ("CodebookV4 loaded: ...") straight to
# stdout, which corrupts --json for any caller that parses us
# (image_meta.py json.loads our stdout). Keep stdout ours alone.
with contextlib.redirect_stdout(sys.stderr):
codebook_v4 = SpectralCodebookV4()
codebook_v4.load(str(codebook))
extractor = RobustSynthIDExtractor()
result = extractor.detect_from_v4_codebook(rgb, codebook_v4, model=args.model)
except Exception as e:
print(f"scorer error: {e}", file=sys.stderr)
return 1
payload = {
"available": True,
"upstream_dir": str(upstream),
"codebook": str(codebook),
"model": args.model,
"profile_key": result.details.get("profile_key"),
"exact_match": result.details.get("exact_match"),
"is_watermarked": result.is_watermarked,
"confidence": result.confidence,
"phase_match": result.phase_match,
"per_channel_scores": result.details.get("per_channel_scores"),
"per_channel_n": result.details.get("per_channel_n"),
"multi_scale_consistency": result.multi_scale_consistency,
}
code, payload = score_file(
args.path,
upstream_dir=str(args.upstream_dir) if args.upstream_dir else None,
codebook=args.codebook,
model=args.model,
)
if code != 0:
return code
if args.json:
json.dump(payload, sys.stdout, indent=2)
sys.stdout.write("\n")
else:
label = "yes" if result.is_watermarked else "no"
print(f"SynthID score: confidence {result.confidence:.3f} (watermarked: {label})")
print(f" phase_match: {result.phase_match:.3f}")
if result.details.get("profile_key"):
print(f" profile: {result.details['profile_key']}")
label = "yes" if payload["is_watermarked"] else "no"
print(f"SynthID score: confidence {payload['confidence']:.3f} (watermarked: {label})")
print(f" phase_match: {payload['phase_match']:.3f}")
if payload.get("profile_key"):
print(f" profile: {payload['profile_key']}")
return 0
service/scripts/server.py
+119 -5
View File
@@ -9,6 +9,7 @@ Endpoints:
GET /capabilities -> which optional tools / pixel backends are present
GET /openapi.json -> dynamically generated OpenAPI 3.0.3 spec
POST /inspect -> {"file": <base64>, "name": "x.png"} -> findings JSON
POST /detect -> {"file": <base64>, "name": "x.txt"} -> watermark detector reports
POST /clean -> {"file": <base64>, "name": "x.png", "options": {...}}
-> {"cleaned": <base64>, "report": {...}}
@@ -43,8 +44,9 @@ from common import (
)
from container_meta import clean_container, inspect_container
from format_dispatch import classify_bytes
from image_meta import clean_image, inspect_image
from image_meta import clean_image, inspect_image, run_synthid_score
from score_stylometry import score_text_stylometry
from text_detectors import detector_status, run_all_text_detectors, run_text_detectors
from text_unicode import clean_text, inspect_text
VERSION = os.environ.get("WATERMARKS_SERVER_VERSION", "dev")
@@ -64,6 +66,8 @@ ALLOWED_CLEAN_OPTIONS = {
"also_layer_a_text": bool,
"remove_pixel": str,
"strip_all_metadata": bool,
"detect_before": bool,
"detect_after": bool,
}
@@ -85,8 +89,10 @@ def capabilities() -> dict[str, Any]:
},
"scorers": {
"synthid": bool(os.environ.get("REVERSE_SYNTHID_DIR")),
"synthid_http": bool(os.environ.get("WATERMARKS_SYNTHID_SCORER_URL")),
"stylometry": True,
},
"text_detectors": detector_status(),
"harnesses": {
"markllm": bool(os.environ.get("MARKLLM_DIR")),
},
@@ -178,12 +184,17 @@ _OPENAPI_PATHS: dict[str, dict[str, Any]] = {
type="object",
properties={
"synthid": _schema(type="boolean"),
"synthid_http": _schema(type="boolean"),
"stylometry": _schema(type="boolean"),
},
),
"harnesses": _schema(
type="object", properties={"markllm": _schema(type="boolean")}
),
"text_detectors": _schema(
type="object",
additionalProperties=_schema(type="boolean"),
),
},
)
},
@@ -202,7 +213,25 @@ _OPENAPI_PATHS: dict[str, dict[str, Any]] = {
"summary": "Inspect a file for AI provenance marks (text / image / container auto-routed)",
"requestBody": _schema(
required=True,
content={"application/json": _schema(schema=_file_request())},
content={
"application/json": _schema(
schema=_file_request(
{
"properties": {
"detect": _schema(
type="boolean",
description=(
"Also run configured text watermark detectors "
"(opt-in; may call vendor APIs and send text "
"to them)"
),
)
},
"required": [],
}
)
)
},
),
"responses": {
"200": _schema(
@@ -239,6 +268,25 @@ _OPENAPI_PATHS: dict[str, dict[str, Any]] = {
},
}
},
"/detect": {
"post": {
"summary": "Run watermark detectors on a file (text: vendor/statistical; image: SynthID score)",
"requestBody": _schema(
required=True,
content={"application/json": _schema(schema=_file_request())},
),
"responses": {
"200": _schema(
type="object",
properties={
"ok": _schema(type="boolean"),
"kind": _schema(type="string", enum=["text", "image", "container"]),
"detections": _schema(type="array", items=_schema(type="object")),
},
)
},
}
},
}
_ERROR_SCHEMA = _schema(
@@ -398,7 +446,7 @@ class Handler(BaseHTTPRequestHandler):
if not self._authorized():
self._respond(HTTPStatus.UNAUTHORIZED, {"ok": False, "error": "unauthorized"})
return
if path not in ("/inspect", "/clean"):
if path not in ("/inspect", "/clean", "/detect"):
self._respond(HTTPStatus.NOT_FOUND, {"ok": False, "error": "not found"})
return
body = self._read_json()
@@ -417,7 +465,9 @@ class Handler(BaseHTTPRequestHandler):
return
try:
if path == "/inspect":
self._handle_inspect(data, name)
self._handle_inspect(data, name, body)
elif path == "/detect":
self._handle_detect(data, name)
else:
self._handle_clean(data, name, body)
except ValueError as e:
@@ -428,7 +478,7 @@ class Handler(BaseHTTPRequestHandler):
HTTPStatus.INTERNAL_SERVER_ERROR, {"ok": False, "error": "internal error"}
)
def _handle_inspect(self, data: bytes, name: str) -> None:
def _handle_inspect(self, data: bytes, name: str, body: dict[str, Any]) -> None:
kind = classify_bytes(data, Path(name).suffix)
if kind == "unknown":
self._respond(
@@ -443,6 +493,7 @@ class Handler(BaseHTTPRequestHandler):
},
)
return
run_detect = body.get("detect") is True
with tempfile.TemporaryDirectory(prefix="wm-inspect-") as tmp:
path = _tmp_path(Path(tmp), name or "input")
path.write_bytes(data)
@@ -455,19 +506,69 @@ class Handler(BaseHTTPRequestHandler):
report = inspect_text(raw_text).to_dict()
s_rep = score_text_stylometry(raw_text, path=name or "<text>")
report["stylometry"] = s_rep.to_dict()
if run_detect:
report["text_detectors"] = run_all_text_detectors(raw_text)
elif kind == "image":
report = inspect_image(path).to_dict()
else:
report = inspect_container(path).to_dict()
detected_wm = any(
entry.get("available") and entry.get("is_watermarked")
for entry in report.get("text_detectors") or []
)
suspicious = (
bool(report.get("suspicious_total"))
or bool(report.get("has_c2pa") or report.get("has_ai_metadata"))
or bool(report.get("stylometry", {}).get("score", 0.0) >= 0.65)
or detected_wm
)
self._respond(
HTTPStatus.OK, {"ok": True, "kind": kind, "report": report, "suspicious": suspicious}
)
def _handle_detect(self, data: bytes, name: str) -> None:
kind = classify_bytes(data, Path(name).suffix)
with tempfile.TemporaryDirectory(prefix="wm-detect-") as tmp:
path = _tmp_path(Path(tmp), name or "input")
path.write_bytes(data)
if kind == "text":
if looks_binary(data):
raise ValueError(
"refusing to detect bytes that look like a binary container as text"
)
raw_text = data.decode("utf-8", errors="surrogateescape")
detections: list[dict[str, Any]] = run_all_text_detectors(raw_text)
s_rep = score_text_stylometry(raw_text, path=name or "<text>")
detections.append({"detector": "stylometry", "available": True, **s_rep.to_dict()})
elif kind == "image":
score = run_synthid_score(path)
if score is None:
score = {
"detector": "synthid",
"available": False,
"error": (
"no SynthID scorer configured (set "
"WATERMARKS_SYNTHID_SCORER_URL or REVERSE_SYNTHID_DIR)"
),
}
else:
score.setdefault("detector", "synthid")
detections = [score]
else:
detections = []
report = inspect_container(path).to_dict()
self._respond(
HTTPStatus.OK,
{
"ok": True,
"kind": kind,
"detections": detections,
"report": report,
},
)
return
self._respond(HTTPStatus.OK, {"ok": True, "kind": kind, "detections": detections})
def _handle_clean(self, data: bytes, name: str, body: dict[str, Any]) -> None:
kind = classify_bytes(data, Path(name).suffix)
if kind == "unknown":
@@ -498,13 +599,22 @@ class Handler(BaseHTTPRequestHandler):
"refusing to clean bytes that look like a binary container as text"
)
text = data.decode("utf-8", errors="surrogateescape")
detect_before = bool(options.get("detect_before"))
detect_after = bool(options.get("detect_after"))
detector_reports: dict[str, Any] = {}
if detect_before:
detector_reports["before"] = run_text_detectors(text)
cleaned, stats = clean_text(
text,
nfkc=bool(options.get("nfkc")),
aggressive_homoglyphs=bool(options.get("aggressive_homoglyphs")),
)
if detect_after:
detector_reports["after"] = run_text_detectors(cleaned)
cleaned_bytes = cleaned.encode("utf-8", errors="surrogateescape")
report: dict[str, Any] = {"kind": "text", "stats": stats, "length": len(cleaned)}
if detector_reports:
report["text_detectors"] = detector_reports
elif kind == "image":
dest = tmpdir / "out.png"
strip_all = not bool(options.get("keep_non_ai_metadata"))
@@ -519,6 +629,10 @@ class Handler(BaseHTTPRequestHandler):
strip_all_metadata=strip_all,
remove_pixel=remove_pixel,
)
if bool(options.get("detect_before")) and result.get("synthid_before") is None:
result["synthid_before"] = run_synthid_score(src)
if bool(options.get("detect_after")) and result.get("synthid_after") is None:
result["synthid_after"] = run_synthid_score(dest)
cleaned_bytes = dest.read_bytes()
report = {"kind": "image", **result}
else:
service/scripts/synthid_score_server.py
+176
View File
@@ -0,0 +1,176 @@
#!/usr/bin/env python3
"""Tiny stdlib HTTP sidecar exposing the reverse-SynthID pixel scorer.
Runs inside the local-only wr-synthid heavy image so the published core
image never bundles the non-commercial reverse-SynthID code. The core
service calls this sidecar for SynthID image scoring when
WATERMARKS_SYNTHID_SCORER_URL is set (see compose.yaml / .env.example).
Endpoints:
GET /health -> {"ok": true, "version": ...}
POST /score -> {"file": <base64>} -> score_synthid payload
Hardening mirrors server.py: optional bearer key, input size caps,
unprivileged user, read-only rootfs with a /tmp tmpfs. Intended for the
compose network or a trusted network only.
"""
from __future__ import annotations
import argparse
import base64
import binascii
import json
import os
import sys
import tempfile
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
sys.path.insert(0, str(Path(__file__).resolve().parent))
from score_synthid import score_file
VERSION = os.environ.get("WATERMARKS_SYNTHID_SERVER_VERSION", "dev")
# Mirror common.MAX_INPUT_BYTES (env-overridable) with the base64 envelope
# headroom. Read at import; the sidecar image does not copy common.py, so the
# default is repeated here.
MAX_INPUT_BYTES = int(os.environ.get("WATERMARKS_MAX_INPUT_BYTES", str(256 << 20)))
MAX_BODY_BYTES = MAX_INPUT_BYTES + (MAX_INPUT_BYTES >> 1)
API_KEY = os.environ.get("WATERMARKS_SYNTHID_SCORER_API_KEY", "").strip()
MODEL = os.environ.get("WATERMARKS_SYNTHID_MODEL", "").strip() or None
def _json_ok(payload: dict[str, Any]) -> bytes:
return json.dumps(payload, ensure_ascii=False, indent=2).encode("utf-8")
class Handler(BaseHTTPRequestHandler):
server_version = f"watermarks-remover-synthid/{VERSION}"
def log_message(self, fmt: str, *args: object) -> None:
print(f"{self.address_string()} - {fmt % args}", file=sys.stderr)
def _authorized(self) -> bool:
if not API_KEY:
return True
return self.headers.get("Authorization", "") == f"Bearer {API_KEY}"
def _read_json(self) -> dict[str, Any] | None:
raw = self.headers.get("Content-Length")
if raw is None or not raw.isdigit():
return None
length = int(raw)
if length > MAX_BODY_BYTES:
return None
try:
body = json.loads(self.rfile.read(length).decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError, OSError):
return None
return body if isinstance(body, dict) else None
def _respond(self, status: int, payload: dict[str, Any]) -> None:
data = _json_ok(payload)
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(data)))
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(data)
def do_GET(self) -> None:
if not self._authorized():
self._respond(HTTPStatus.UNAUTHORIZED, {"ok": False, "error": "unauthorized"})
return
if urlparse(self.path).path == "/health":
self._respond(HTTPStatus.OK, {"ok": True, "version": VERSION})
else:
self._respond(HTTPStatus.NOT_FOUND, {"ok": False, "error": "not found"})
def do_POST(self) -> None:
if not self._authorized():
self._respond(HTTPStatus.UNAUTHORIZED, {"ok": False, "error": "unauthorized"})
return
if urlparse(self.path).path != "/score":
self._respond(HTTPStatus.NOT_FOUND, {"ok": False, "error": "not found"})
return
body = self._read_json()
if body is None:
raw_len = self.headers.get("Content-Length")
oversized = raw_len is not None and raw_len.isdigit() and int(raw_len) > MAX_BODY_BYTES
self._respond(
HTTPStatus.REQUEST_ENTITY_TOO_LARGE if oversized else HTTPStatus.BAD_REQUEST,
{"ok": False, "error": "invalid request body"},
)
return
raw = body.get("file")
if not isinstance(raw, str):
self._respond(HTTPStatus.BAD_REQUEST, {"ok": False, "error": "missing 'file' field"})
return
try:
data = base64.b64decode(raw, validate=True)
except (binascii.Error, ValueError):
self._respond(
HTTPStatus.BAD_REQUEST, {"ok": False, "error": "'file' is not valid base64"}
)
return
if len(data) > MAX_INPUT_BYTES:
self._respond(
HTTPStatus.REQUEST_ENTITY_TOO_LARGE, {"ok": False, "error": "file too large"}
)
return
with tempfile.TemporaryDirectory(prefix="wm-synthid-") as tmp:
path = Path(tmp) / "input.png"
try:
path.write_bytes(data)
except OSError as e:
self._respond(HTTPStatus.INTERNAL_SERVER_ERROR, {"ok": False, "error": str(e)})
return
code, payload = score_file(path, model=MODEL)
if code == 0 and payload is not None:
self._respond(HTTPStatus.OK, payload)
elif code == 2:
self._respond(HTTPStatus.BAD_REQUEST, {"ok": False, "error": "could not load image"})
else:
# exit 1 (runtime error) or 3 (unavailable) -> fail-soft payload,
# matching the shape image_meta.run_synthid_score expects.
self._respond(
HTTPStatus.OK,
{"available": False, "error": "scorer unavailable (see sidecar stderr)"},
)
def main() -> int:
global API_KEY # noqa: PLW0603 — CLI overrides env
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--host", default=os.environ.get("WATERMARKS_SYNTHID_SERVER_HOST", "127.0.0.1"))
p.add_argument(
"--port", type=int, default=int(os.environ.get("WATERMARKS_SYNTHID_SERVER_PORT", "8766"))
)
p.add_argument("--api-key", default=API_KEY, help="require this bearer token (default: none)")
args = p.parse_args()
if args.host not in ("127.0.0.1", "localhost", "::1"):
print(
f"warning: binding {args.host} — intended for a trusted network only", file=sys.stderr
)
API_KEY = args.api_key
print(f"synthid scorer sidecar {VERSION} on http://{args.host}:{args.port}", file=sys.stderr)
server = ThreadingHTTPServer((args.host, args.port), Handler)
try:
server.serve_forever()
except KeyboardInterrupt:
server.shutdown()
return 0
if __name__ == "__main__":
raise SystemExit(main())
service/scripts/text_detectors.py
+487
View File
@@ -0,0 +1,487 @@
#!/usr/bin/env python3
"""Vendor and research text-watermark detectors behind one interface.
Detects statistical (Layer B) text watermarks using vendor-provided or
research detectors. Every detector implements the same small protocol:
name: str stable identifier (surfaced in /capabilities)
available() -> bool configured and usable right now
detect(text) -> dict JSON-safe report; never raises
Reports follow the fail-soft contract: a detector that is unconfigured,
times out, or errors returns {"available": False, "error": ...} and can
never block cleaning.
Detectors:
- gemini-synthid-text — Google's official SynthID-text detector, called
through the Gemini API (taskType DETECT_TEXT_WATERMARK). Activated by
WATERMARKS_GEMINI_API_KEY. User text is sent to Google only when the
operator sets that key.
- markllm — optional research harness (KGW / SynthID schemes) via
detect_text_watermark.py, activated by MARKLLM_DIR. Same-config-only
detection; not a vendor oracle.
- claude-text — placeholder for Anthropic's announced text-watermark
detection API. Reports unavailable until a public endpoint exists; the
interface it must implement is already defined here.
"""
from __future__ import annotations
import contextlib
import json
import os
import subprocess
import sys
import tempfile
import time
import urllib.error
import urllib.request
from collections.abc import Callable
from pathlib import Path
from typing import Any, Protocol
from urllib.parse import urlparse
GEMINI_DETECT_URL = (
"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent"
)
DEFAULT_GEMINI_MODEL = "gemini-2.5-flash"
DEFAULT_GEMINI_TIMEOUT = 30.0
DEFAULT_GEMINI_MAX_CHARS = 1_000_000
DEFAULT_MARKLLM_SCHEME = "kgw"
DEFAULT_MARKLLM_TIMEOUT = 600.0
class DetectorError(RuntimeError):
"""A detector call failed (network, HTTP error, timeout)."""
class TextDetector(Protocol):
name: str
def available(self) -> bool: ...
def detect(self, text: str) -> dict[str, Any]: ...
def _env_float(name: str, default: float) -> float:
try:
return float(os.environ.get(name, str(default)))
except ValueError:
return default
def _env_int(name: str, default: int) -> int:
try:
return int(os.environ.get(name, str(default)))
except ValueError:
return default
# ---------------------------------------------------------------------------
# Gemini (Google's official SynthID-text detector)
# ---------------------------------------------------------------------------
_WATERMARKED_VERDICTS = ("watermarked", "ai-generated", "ai generated", "likely ai")
def _verdict_is_watermarked(verdict: str | None) -> bool | None:
"""Map the detector model's free-text verdict to a boolean, or None."""
if not verdict:
return None
low = verdict.strip().lower()
if low.startswith(("unlikely", "no", "not")):
return False
return any(marker in low for marker in _WATERMARKED_VERDICTS)
def _extract_numeric_score(candidate: dict[str, Any], top: dict[str, Any]) -> float | None:
"""Pull a numeric watermark score from any of the known response shapes."""
for container in (candidate, top):
for key in (
"syntheticTextScore",
"synthetic_text_score",
"watermarkScore",
"watermark_score",
"score",
):
value = container.get(key)
if isinstance(value, (int, float)):
return float(value)
attribution = candidate.get("attributionMetadata") or {}
if isinstance(attribution, dict):
for key in ("syntheticTextScore", "synthetic_text_score", "score"):
value = attribution.get(key)
if isinstance(value, (int, float)):
return float(value)
st = attribution.get("syntheticText")
if isinstance(st, dict):
for key in ("score", "confidence"):
value = st.get(key)
if isinstance(value, (int, float)):
return float(value)
return None
def parse_gemini_detect_response(data: dict[str, Any]) -> dict[str, Any]:
"""Parse a generateContent response from a DETECT_TEXT_WATERMARK call.
The endpoint can answer with either a free-text verdict
("Likely AI-generated") or a structured score; both shapes are handled
defensively so upstream schema changes degrade to an error report
instead of a crash.
"""
candidates = data.get("candidates") or []
candidate = candidates[0] if candidates else {}
if not isinstance(candidate, dict):
candidate = {}
if not candidate:
feedback = data.get("promptFeedback") or {}
block = feedback.get("blockReason")
if block:
raise DetectorError(f"Gemini blocked the request: {block}")
raise DetectorError("Gemini returned no candidates")
verdict: str | None = None
content = candidate.get("content") or {}
parts = content.get("parts") or []
if parts and isinstance(parts[0], dict):
verdict = parts[0].get("text")
score = _extract_numeric_score(candidate, data)
is_watermarked = _verdict_is_watermarked(verdict)
if is_watermarked is None and score is not None:
is_watermarked = score >= 0.5
if verdict is None and score is None:
raise DetectorError(
f"unexpected Gemini response (no verdict or score): {json.dumps(data)[:400]}"
)
raw = {
key: candidate[key]
for key in ("attributionMetadata", "finishReason", "index")
if candidate.get(key) is not None
}
return {
"is_watermarked": is_watermarked,
"score": score,
"verdict": verdict,
"raw": raw,
}
def _post_json(url: str, body: dict[str, Any], api_key: str, timeout: float) -> dict[str, Any]:
"""POST *body* to *url*, retrying once on transient failures."""
if urlparse(url).scheme not in ("http", "https"):
raise DetectorError(f"refusing non-http(s) Gemini endpoint: {url}")
# S310: URL scheme is restricted to http/https just above.
req = urllib.request.Request( # noqa: S310
url,
data=json.dumps(body).encode("utf-8"),
headers={"Content-Type": "application/json", "x-goog-api-key": api_key},
method="POST",
)
last_err = "Gemini API call failed"
for attempt in range(2):
try:
with urllib.request.urlopen(req, timeout=timeout) as resp: # noqa: S310
payload = json.loads(resp.read().decode("utf-8"))
if not isinstance(payload, dict):
raise DetectorError("non-object Gemini response")
return payload
except urllib.error.HTTPError as e:
last_err = f"Gemini API HTTP {e.code}: {e.read().decode('utf-8', 'replace')[:300]}"
if e.code not in (429, 500, 502, 503, 504):
raise DetectorError(last_err) from e
except (urllib.error.URLError, TimeoutError, OSError) as e:
last_err = f"Gemini API unreachable: {e}"
if attempt == 0:
time.sleep(1.0)
raise DetectorError(last_err)
class GeminiSynthIDTextDetector:
"""Google's official SynthID-text detector via the Gemini API."""
name = "gemini-synthid-text"
vendor = "google"
def available(self) -> bool:
return bool(os.environ.get("WATERMARKS_GEMINI_API_KEY", "").strip())
def detect(self, text: str) -> dict[str, Any]:
api_key = os.environ.get("WATERMARKS_GEMINI_API_KEY", "").strip()
if not api_key:
return {
"detector": self.name,
"vendor": self.vendor,
"available": False,
"error": "WATERMARKS_GEMINI_API_KEY not set",
}
max_chars = _env_int("WATERMARKS_GEMINI_MAX_CHARS", DEFAULT_GEMINI_MAX_CHARS)
if len(text) > max_chars:
return {
"detector": self.name,
"vendor": self.vendor,
"available": True,
"skipped": True,
"reason": f"text longer than {max_chars} chars",
"is_watermarked": None,
}
model = (
os.environ.get("WATERMARKS_GEMINI_MODEL", DEFAULT_GEMINI_MODEL) or DEFAULT_GEMINI_MODEL
)
timeout = _env_float("WATERMARKS_GEMINI_TIMEOUT", DEFAULT_GEMINI_TIMEOUT)
url = GEMINI_DETECT_URL.format(model=model)
body = {
"contents": [{"role": "user", "parts": [{"text": text}]}],
"generationConfig": {"taskType": "DETECT_TEXT_WATERMARK"},
}
report: dict[str, Any] = {
"detector": self.name,
"vendor": self.vendor,
"model": model,
"available": True,
}
try:
data = _post_json(url, body, api_key, timeout)
except DetectorError as e:
report["available"] = False
report["error"] = str(e)
return report
try:
parsed = parse_gemini_detect_response(data)
except DetectorError as e:
report["available"] = False
report["error"] = str(e)
return report
report.update(parsed)
return report
# ---------------------------------------------------------------------------
# MarkLLM (open-source research harness: KGW / SynthID schemes)
# ---------------------------------------------------------------------------
def _venv_python(upstream: Path) -> Path | None:
"""Prefer the MarkLLM checkout's venv interpreter, if it exists."""
if os.name == "nt":
candidate = upstream / ".venv" / "Scripts" / "python.exe"
else:
candidate = upstream / ".venv" / "bin" / "python"
return candidate if candidate.is_file() else None
def _markllm_preexec() -> Callable[[], None] | None:
"""Optional RLIMIT_AS guard for the MarkLLM child; None means "no limit".
torch/CUDA usually needs large address spaces, so this is opt-in via
WATERMARKS_MARKLLM_RLIMIT_AS (byte count, hex/octal allowed). POSIX only;
on Windows preexec_fn must stay None.
"""
raw = os.environ.get("WATERMARKS_MARKLLM_RLIMIT_AS")
if not raw or os.name != "posix":
return None
try:
limit = int(raw, 0)
except ValueError:
return None
def _apply() -> None:
import resource
resource.setrlimit(resource.RLIMIT_AS, (limit, limit))
return _apply
class MarkLLMTextDetector:
"""Same-config-only research detection via detect_text_watermark.py.
Constructor overrides (scheme, upstream_dir, model, timeout) take
precedence over the environment, so callers such as rewrite_text.py can
keep CLI flags driving the harness. When the MarkLLM checkout has a
venv, its interpreter runs the child process; otherwise the current
interpreter is used (the service image bundles the harness deps).
"""
name = "markllm"
def __init__(
self,
*,
scheme: str | None = None,
upstream_dir: str | None = None,
model: str | None = None,
timeout: float | None = None,
) -> None:
self._scheme = scheme
self._upstream_dir = upstream_dir
self._model = model
self._timeout = timeout
def available(self) -> bool:
upstream = self._upstream_dir or os.environ.get("MARKLLM_DIR", "").strip()
return bool(upstream)
def detect(self, text: str) -> dict[str, Any]:
upstream = self._upstream_dir or os.environ.get("MARKLLM_DIR", "").strip()
scheme = (
self._scheme
or os.environ.get("WATERMARKS_MARKLLM_SCHEME", "")
or DEFAULT_MARKLLM_SCHEME
)
report: dict[str, Any] = {
"detector": self.name,
"scheme": scheme,
"vendor": "open-llm",
"available": False,
}
if not upstream:
report["error"] = "MARKLLM_DIR not set"
return report
script = Path(__file__).resolve().parent / "detect_text_watermark.py"
timeout = (
self._timeout
if self._timeout is not None
else _env_float("WATERMARKS_MARKLLM_TIMEOUT", DEFAULT_MARKLLM_TIMEOUT)
)
venv_python = _venv_python(Path(upstream).expanduser().resolve())
python = str(venv_python) if venv_python is not None else sys.executable
with tempfile.NamedTemporaryFile("w", suffix=".txt", encoding="utf-8", delete=False) as f:
f.write(text)
tmp = f.name
cmd = [python, str(script), "detect", tmp, "--scheme", scheme, "--json"]
if self._model:
cmd += ["--model", self._model]
if self._upstream_dir:
cmd += ["--upstream-dir", str(Path(upstream).expanduser().resolve())]
try:
try:
r = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=timeout,
preexec_fn=_markllm_preexec(),
check=False,
)
except subprocess.TimeoutExpired:
report["error"] = "MarkLLM detection timed out"
return report
if r.returncode == 3:
report["error"] = (r.stderr or "").strip()[:400] or "MarkLLM unavailable"
return report
if r.returncode != 0:
report["error"] = (r.stderr or "").strip()[:400] or f"MarkLLM exit {r.returncode}"
return report
try:
payload = json.loads(r.stdout or "{}")
except json.JSONDecodeError as e:
report["error"] = f"bad MarkLLM JSON: {e}"
return report
finally:
with contextlib.suppress(OSError):
Path(tmp).unlink()
if not isinstance(payload, dict):
report["error"] = "bad MarkLLM response"
return report
payload["available"] = True
payload["detector"] = self.name
payload["note"] = (
"MarkLLM is a research harness: detection is only valid against the "
"same scheme config and keys used at generation; not a vendor detector."
)
return payload
# ---------------------------------------------------------------------------
# Claude (Anthropic) — announced detector API, not yet public
# ---------------------------------------------------------------------------
class ClaudeTextDetector:
"""Placeholder for Anthropic's announced text-watermark detection API.
Anthropic has announced a watermark detection API for Claude-generated
text; no public endpoint exists yet. When it ships, set
WATERMARKS_CLAUDE_API_KEY, flip available() to check it, and fill in
detect() against the documented endpoint.
"""
name = "claude-text"
vendor = "anthropic"
def available(self) -> bool:
return False
def detect(self, text: str) -> dict[str, Any]:
return {
"detector": self.name,
"vendor": self.vendor,
"available": False,
"error": (
"Anthropic has announced a text-watermark detection API for "
"Claude; no public endpoint is available yet. When it ships, "
"set WATERMARKS_CLAUDE_API_KEY and implement ClaudeTextDetector."
),
}
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
def all_detectors(
markllm: MarkLLMTextDetector | None = None, *, include_markllm: bool = True
) -> list[TextDetector]:
detectors: list[TextDetector] = [GeminiSynthIDTextDetector()]
if include_markllm:
detectors.append(markllm or MarkLLMTextDetector())
detectors.append(ClaudeTextDetector())
return detectors
def detector_status() -> dict[str, bool]:
"""Configured/usable status per detector (for /capabilities)."""
return {d.name: d.available() for d in all_detectors()}
def run_all_text_detectors(
text: str,
*,
markllm: MarkLLMTextDetector | None = None,
include_markllm: bool = True,
) -> list[dict[str, Any]]:
"""Run every detector (including unavailable ones, with reasons).
markllm injects a caller-parameterized MarkLLM detector (e.g. one
driven by rewrite_text.py CLI flags); pass include_markllm=False to
exclude the MarkLLM harness entirely.
"""
return [d.detect(text) for d in all_detectors(markllm, include_markllm=include_markllm)]
def run_text_detectors(
text: str,
*,
markllm: MarkLLMTextDetector | None = None,
include_markllm: bool = True,
) -> list[dict[str, Any]]:
"""Run only the detectors that are configured and usable."""
return [
d.detect(text)
for d in all_detectors(markllm, include_markllm=include_markllm)
if d.available()
]