Add optional reverse-SynthID scorer and install paths

- Add score_synthid.py adapter for external reverse-SynthID scoring
- Surface optional SynthID confidence in inspect_image/clean_image
- Add one-command bootstrap script and scorer-only requirements
- Add local Dockerfile for scorer runtime
- Update README, SKILL, vendor notes, Makefile, and tests
This commit is contained in:
guillaume
2026-08-12 10:37:41 -07:00
parent c49592113e
commit 1693606268
13 changed files with 611 additions and 8 deletions
+6
View File
@@ -0,0 +1,6 @@
.git
.venv
__pycache__
.pytest_cache
*.pyc
tests
+40
View File
@@ -0,0 +1,40 @@
# Optional local Docker image for the reverse-SynthID pixel scorer.
#
# Build from the repository root:
# docker build -f Dockerfile.synthid -t watermarks-remover-synthid-scorer .
#
# The upstream code is fetched from source at build time and is NOT
# redistributed by this repository. Users must comply with the upstream
# project's non-commercial Research License.
FROM python:3.11-slim
ARG REVERSE_SYNTHID_REF=main
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git \
libgl1 \
libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
RUN git clone --depth 1 --filter=blob:none --sparse \
--branch "${REVERSE_SYNTHID_REF}" \
https://github.com/aloshdenny/reverse-SynthID.git /opt/reverse-synthid \
&& cd /opt/reverse-synthid \
&& git sparse-checkout set --no-cone \
'/src/' \
'/artifacts/spectral_codebook_v4.npz' \
'/requirements.txt' \
'/LICENSE' \
'/README.md'
COPY skills/remove-ai-marks/scripts/requirements-synthid-scorer.txt /app/requirements-synthid-scorer.txt
COPY skills/remove-ai-marks/scripts/score_synthid.py /app/score_synthid.py
RUN pip install --no-cache-dir -r /app/requirements-synthid-scorer.txt
ENV REVERSE_SYNTHID_DIR=/opt/reverse-synthid
WORKDIR /app
ENTRYPOINT ["python3", "/app/score_synthid.py"]
+17 -1
View File
@@ -1,4 +1,4 @@
.PHONY: test smoke install-skill clean
.PHONY: test smoke smoke-synthid bootstrap-synthid docker-synthid-build docker-synthid-help install-skill clean
SCRIPTS := skills/remove-ai-marks/scripts
PYTHON ?= $(shell if [ -x .venv/bin/python ]; then echo .venv/bin/python; else echo python3; fi)
@@ -16,6 +16,22 @@ smoke:
python3 $(SCRIPTS)/clean_file.py tests/fixtures/sample_meta.svg -o /tmp/sample_meta.cleaned.svg
@echo "smoke ok"
smoke-synthid:
@if [ -z "$(REVERSE_SYNTHID_DIR)" ]; then \
echo "smoke-synthid skipped (set REVERSE_SYNTHID_DIR)"; \
else \
$(PYTHON) $(SCRIPTS)/score_synthid.py --help >/dev/null && echo "score_synthid adapter present"; \
fi
bootstrap-synthid:
./skills/remove-ai-marks/scripts/setup_synthid.sh
docker-synthid-build:
docker build -f Dockerfile.synthid -t watermarks-remover-synthid-scorer .
docker-synthid-help:
docker run --rm watermarks-remover-synthid-scorer --help
install-skill:
mkdir -p $(HOME)/.grok/skills
ln -sfn $(CURDIR)/skills/remove-ai-marks $(HOME)/.grok/skills/remove-ai-marks
+47 -3
View File
@@ -75,6 +75,50 @@ python3 "$SCRIPTS/inspect_image.py" shot.png
python3 "$SCRIPTS/clean_image.py" shot.png -o shot.cleaned.png
```
## Optional SynthID pixel scoring
`inspect_image.py` and `clean_image.py` can report a pixel-domain SynthID
confidence score when an external checkout of
[`aloshdenny/reverse-SynthID`](https://github.com/aloshdenny/reverse-SynthID)
is available. The scorer is **not bundled**: it is loaded at runtime from your
checkout, and its code remains under the upstream project's non-commercial
Research License.
### Option 1: one-command bootstrap (no Docker)
```bash
SCRIPTS=skills/remove-ai-marks/scripts
# Clones upstream, creates a venv, and installs scorer-only dependencies.
"$SCRIPTS/setup_synthid.sh"
# Score an image (default checkout: ~/reverse-SynthID).
REVERSE_SYNTHID_DIR=~/reverse-SynthID \
~/reverse-SynthID/.venv/bin/python "$SCRIPTS/score_synthid.py" shot.png
# Or surface the score from inspect / clean (same venv Python).
REVERSE_SYNTHID_DIR=~/reverse-SynthID \
~/reverse-SynthID/.venv/bin/python "$SCRIPTS/inspect_image.py" shot.png
```
`setup_synthid.sh` accepts `--dir PATH`, `--ref REF`, and `--full` (install the
full upstream `requirements.txt`, which adds `torch`/`diffusers` for the
upstream VAE bypass this project does not use).
### Option 2: local Docker build
```bash
make docker-synthid-build
docker run --rm -v "$(pwd):/data" watermarks-remover-synthid-scorer /data/shot.png
```
The image is built locally from the upstream source at build time. It is not
published, so it does not redistribute the upstream code.
V4 scoring uses `artifacts/spectral_codebook_v4.npz` from the upstream checkout
(~220 MB). This is **detection/scoring only** — it does not remove pixel
watermarks.
## Coverage matrix
| Channel | Claude | Gemini/SynthID | OpenAI | Open-LLM |
@@ -82,7 +126,7 @@ python3 "$SCRIPTS/clean_image.py" shot.png -o shot.cleaned.png
| Unicode / edit-based text | Layer A | Layer A | Layer A | Layer A |
| Statistical sampling text | Layer B best-effort | Layer B best-effort | Layer B if present | Layer B best-effort |
| C2PA / file metadata | Yes (listed formats) | Yes when present | Yes when present | Yes when present |
| Pixel image marks | Out of scope | Out of scope | Out of scope | Out of scope |
| Pixel image marks | Out of scope | Optional SynthID score (external); removal out of scope | Out of scope | Out of scope |
| Training backdoors | Out of scope | Out of scope | Out of scope | Out of scope |
Details: [`skills/remove-ai-marks/references/vendor-notes.md`](skills/remove-ai-marks/references/vendor-notes.md), [`mark-classes.md`](skills/remove-ai-marks/references/mark-classes.md).
@@ -136,7 +180,7 @@ Layer B makes sense when you specifically want the premium model's **thinking an
| HTML | meta, JSON-LD, data-ai* | Strip tags/attrs |
| Markdown | YAML frontmatter AI keys | Drop keys + Layer A body |
Pixel-domain watermarks and **C2PA soft binding** (in-content watermark that can re-link a remote Content Credentials manifest after metadata is stripped) remain **out of scope**. Stripping hard-bound C2PA does **not** clear those channels.
Pixel-domain watermark **removal** and **C2PA soft binding** (in-content watermark that can re-link a remote Content Credentials manifest after metadata is stripped) remain **out of scope**. Stripping hard-bound C2PA does **not** clear those channels. An optional local SynthID scorer is available for detection only (see above).
### Residual risk after a clean
@@ -147,7 +191,7 @@ To check residual signals yourself (optional, external):
| Channel | What we remove | What may remain | External check (examples) |
| --- | --- | --- | --- |
| Hard-bound C2PA / EXIF / XMP | Yes | Soft-bound / pixel marks | [c2patool](https://github.com/contentauth/c2pa-rs/tree/main/cli), [Content Credentials verify](https://contentcredentials.org/verify) |
| SynthID-class media | No | Pixel/audio/video watermark | Provider tools (e.g. [Google SynthID](https://deepmind.google/science/synthid/) / Vertex detector where offered) |
| SynthID-class media | No (optional local score only) | Pixel/audio/video watermark | Provider tools (e.g. [Google SynthID](https://deepmind.google/science/synthid/) / Vertex detector where offered); optional local [reverse-SynthID](https://github.com/aloshdenny/reverse-SynthID) scorer |
| Statistical text | Best-effort rewrite | Strong marks after light edit | No public universal detector; vendor tools when available |
Industry two-layer context (C2PA + imperceptible watermark): [Institute of AI PM guide](https://www.institutepm.com/knowledge-hub/ai-content-provenance-watermarking).
+8 -1
View File
@@ -64,6 +64,12 @@ python3 "$SCRIPTS/inspect_image.py" --json image.png
Show a short summary (suspicious codepoints; C2PA/AI flags).
Optional: when `REVERSE_SYNTHID_DIR` is set, `inspect_image.py` and
`clean_image.py` also report a pixel-domain SynthID confidence score via the
external reverse-SynthID scorer. That is **detection only**, not removal.
Bootstrap the external checkout with `scripts/setup_synthid.sh`, or build a
local image with `make docker-synthid-build`.
### 3. Deterministic clean (always for matching inputs)
**Text — Layer A:**
@@ -166,7 +172,8 @@ Always state:
- Layer A does **not** remove token-sampling watermarks.
- Layer B cannot be gold-verified without vendor detectors / keys.
- PDF strip is best-effort without `exiftool`.
- Pixel-domain image/audio/video watermarks (SynthID-media, etc.) are out of scope.
- Pixel-domain image/audio/video watermarks (SynthID-media, etc.) are out of scope for removal; an optional external scorer can only report a SynthID confidence estimate.
- The reverse-SynthID scorer is external, best-effort, and under a non-commercial Research License; it is not bundled and is not an official Google detector.
- **C2PA soft binding** (content watermark that re-links to a remote manifest after metadata strip) is out of scope — stripping hard-bound C2PA does not clear it.
- Data-driven / backdoor model marks (trigger phrases) are out of scope.
@@ -32,6 +32,7 @@ Source: [How Claude marks AI-generated content](https://support.claude.com/en/ar
- **Data-driven / backdoor** (trigger phrases) → **out of scope**
- **Generative** (sampling) → Layer B best-effort
- Productionized in Gemini-scale systems; open research code exists, but **production keys are not public** — this skill does **not** ship a SynthID detector.
- Optional external reference: [`aloshdenny/reverse-SynthID`](https://github.com/aloshdenny/reverse-SynthID) provides a reverse-engineered pixel-domain scorer. It is **not bundled** here, is best-effort, and is under a non-commercial Research License; it is not the official Google detector.
**Skill mapping:** same Layer B rewrite attacks (paraphrase / back-translate / structural) used in the literature against sampling watermarks.
@@ -29,6 +29,12 @@ def main() -> int:
help="Only drop segments/chunks that look like C2PA/AI (less aggressive)",
)
p.add_argument("--json", action="store_true", help="JSON result on stdout")
p.add_argument(
"--synthid-dir",
type=str,
default=None,
help="reverse-SynthID checkout root for optional pixel SynthID scoring",
)
args = p.parse_args()
if not args.path.is_file():
@@ -49,6 +55,7 @@ def main() -> int:
src,
dest,
strip_all_metadata=not args.keep_non_ai_metadata,
synthid_dir=args.synthid_dir,
)
except Exception as e:
eprint(f"error: {e}")
@@ -60,6 +67,20 @@ def main() -> int:
eprint(f"wrote {result['output']} ({result['bytes_in']} -> {result['bytes_out']})")
for a in result["actions"]:
eprint(f" - {a}")
if result.get("synthid_before") and result["synthid_before"].get("available"):
label = "yes" if result["synthid_before"].get("is_watermarked") else "no"
eprint(
"SynthID before: "
f"confidence {result['synthid_before'].get('confidence', 0.0):.3f} "
f"(watermarked: {label})"
)
if result.get("synthid_after") and result["synthid_after"].get("available"):
label = "yes" if result["synthid_after"].get("is_watermarked") else "no"
eprint(
"SynthID after: "
f"confidence {result['synthid_after'].get('confidence', 0.0):.3f} "
f"(watermarked: {label})"
)
if result["still_has_c2pa"] or result["still_has_ai_metadata"]:
eprint("warning: residual C2PA/AI signals may remain")
for f in result.get("post_findings") or []:
+55 -2
View File
@@ -2,9 +2,12 @@
from __future__ import annotations
import json
import os
import re
import struct
import subprocess
import sys
import zlib
from dataclasses import dataclass, field
from pathlib import Path
@@ -12,6 +15,8 @@ from typing import Any
from common import which
SCRIPTS_DIR = Path(__file__).resolve().parent
PNG_SIG = b"\x89PNG\r\n\x1a\n"
JPEG_SOI = b"\xff\xd8"
@@ -57,6 +62,7 @@ class ImageInspectReport:
has_ai_metadata: bool
findings: list[str] = field(default_factory=list)
tools: dict[str, Any] = field(default_factory=dict)
synthid: dict[str, Any] | None = None
def to_dict(self) -> dict:
return {
@@ -66,6 +72,7 @@ class ImageInspectReport:
"has_ai_metadata": self.has_ai_metadata,
"findings": self.findings,
"tools": self.tools,
"synthid": self.synthid,
}
@@ -246,7 +253,48 @@ def run_optional_tools(path: Path) -> dict[str, Any]:
return tools
def inspect_image(path: Path) -> ImageInspectReport:
def run_synthid_score(
path: Path,
upstream_dir: str | None = None,
) -> dict[str, Any] | None:
"""Run the optional reverse-SynthID scorer in a subprocess.
Returns None when the scorer is not configured or unavailable (exit 3),
so callers can keep the default "no SynthID score" behavior.
"""
if upstream_dir is None:
upstream_dir = os.environ.get("REVERSE_SYNTHID_DIR")
if not upstream_dir:
return None
script = SCRIPTS_DIR / "score_synthid.py"
cmd = [
sys.executable,
str(script),
str(path),
"--upstream-dir",
str(upstream_dir),
"--json",
]
try:
r = subprocess.run(cmd, capture_output=True, text=True, timeout=180)
except Exception as e:
return {"available": False, "error": str(e)}
if r.returncode == 3:
return None
if r.returncode != 0:
return {"available": False, "error": (r.stderr or "").strip()[:2000]}
try:
return json.loads(r.stdout or "{}")
except json.JSONDecodeError as e:
return {"available": False, "error": f"bad scorer JSON: {e}"}
def inspect_image(
path: Path,
synthid_dir: str | None = None,
) -> ImageInspectReport:
data = path.read_bytes()
fmt = detect_format(data)
if fmt == "png":
@@ -270,6 +318,7 @@ def inspect_image(path: Path) -> ImageInspectReport:
has_ai_metadata=has_ai,
findings=findings,
tools=tools,
synthid=run_synthid_score(path, synthid_dir),
)
@@ -418,7 +467,9 @@ def clean_image(
dest: Path,
*,
strip_all_metadata: bool = True,
synthid_dir: str | None = None,
) -> dict[str, Any]:
synthid_before = run_synthid_score(path, synthid_dir)
data = path.read_bytes()
fmt = detect_format(data)
if fmt == "png":
@@ -450,7 +501,7 @@ def clean_image(
except Exception as e:
actions.append(f"exiftool failed: {e}")
after = inspect_image(dest)
after = inspect_image(dest, synthid_dir=synthid_dir)
return {
"input": str(path),
"output": str(dest),
@@ -461,4 +512,6 @@ def clean_image(
"still_has_c2pa": after.has_c2pa,
"still_has_ai_metadata": after.has_ai_metadata,
"post_findings": after.findings,
"synthid_before": synthid_before,
"synthid_after": after.synthid,
}
@@ -17,13 +17,19 @@ def main() -> int:
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("path", type=Path, help="Image path (PNG or JPEG)")
p.add_argument("--json", action="store_true")
p.add_argument(
"--synthid-dir",
type=str,
default=None,
help="reverse-SynthID checkout root for optional pixel SynthID scoring",
)
args = p.parse_args()
if not args.path.is_file():
print(f"not a file: {args.path}", file=sys.stderr)
return 2
report = inspect_image(args.path)
report = inspect_image(args.path, synthid_dir=args.synthid_dir)
if args.json:
emit_json(report.to_dict())
else:
@@ -43,6 +49,15 @@ def main() -> int:
print("exiftool highlights:")
for line in et["interesting_lines"][:20]:
print(f" {line}")
if report.synthid and report.synthid.get("available"):
label = "yes" if report.synthid.get("is_watermarked") else "no"
print(
"SynthID score: "
f"confidence {report.synthid.get('confidence', 0.0):.3f} "
f"(watermarked: {label})"
)
elif report.synthid and report.synthid.get("error"):
print(f"SynthID score: error: {report.synthid['error']}")
return 0 if not (report.has_c2pa or report.has_ai_metadata) else 1
@@ -0,0 +1,9 @@
# Minimal dependencies for the optional reverse-SynthID pixel scorer.
# The upstream repo's full requirements.txt adds torch/diffusers and other
# packages only needed for its VAE/bypass pipeline, which this project does not use.
numpy>=1.21.0
scipy>=1.7.0
opencv-python>=4.5.0
PyWavelets>=1.1.1
scikit-learn>=0.24.0
Pillow>=8.0.0
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env python3
"""Optional SynthID pixel-domain scorer backed by an external reverse-SynthID checkout.
This script does NOT vendor upstream code. It imports the scorer from a
user-provided checkout (https://github.com/aloshdenny/reverse-SynthID) at
runtime, using that environment's optional dependencies (numpy, opencv,
scipy, PyWavelets, scikit-learn, Pillow).
Exit codes:
0 scored successfully
1 scorer runtime error
2 bad input (missing/unreadable image, bad args)
3 scorer unavailable (not configured / missing deps / missing codebook)
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
def resolve_upstream(raw: str | None) -> Path | None:
if not raw:
return None
upstream = Path(raw).expanduser().resolve()
if not upstream.is_dir():
return None
return upstream
def main() -> int:
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("path", type=Path, help="Image to score (PNG/JPEG/etc.)")
p.add_argument(
"--upstream-dir",
type=Path,
default=None,
help="reverse-SynthID checkout root (default: $REVERSE_SYNTHID_DIR)",
)
p.add_argument(
"--codebook",
type=Path,
default=None,
help="spectral_codebook_v4.npz path (default: <upstream>/artifacts/)",
)
p.add_argument("--model", type=str, default=None, help="Optional model hint")
p.add_argument("--json", action="store_true", help="Emit JSON on stdout")
args = p.parse_args()
if not args.path.is_file():
print(f"not a file: {args.path}", file=sys.stderr)
return 2
raw_upstream = args.upstream_dir or os.environ.get("REVERSE_SYNTHID_DIR")
upstream = resolve_upstream(str(raw_upstream) if raw_upstream else None)
if upstream is None:
print(
"SynthID scorer not configured: set REVERSE_SYNTHID_DIR or pass --upstream-dir",
file=sys.stderr,
)
return 3
extraction = upstream / "src" / "extraction"
if not extraction.is_dir():
print(f"upstream extraction dir not found: {extraction}", file=sys.stderr)
return 3
codebook = args.codebook or upstream / "artifacts" / "spectral_codebook_v4.npz"
codebook = Path(codebook).expanduser().resolve()
if not codebook.is_file():
print(f"codebook not found: {codebook}", file=sys.stderr)
return 3
sys.path.insert(0, str(extraction))
try:
import cv2 # noqa: E402
from robust_extractor import RobustSynthIDExtractor # noqa: E402
from synthid_bypass_v4 import SpectralCodebookV4 # noqa: E402
except ImportError as e:
print(f"optional scorer dependencies missing: {e}", file=sys.stderr)
return 3
try:
img = cv2.imread(str(args.path))
if img is None:
print(f"could not load image: {args.path}", file=sys.stderr)
return 2
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
codebook_v4 = SpectralCodebookV4()
codebook_v4.load(str(codebook))
extractor = RobustSynthIDExtractor()
result = extractor.detect_from_v4_codebook(
rgb, codebook_v4, model=args.model
)
except Exception as e:
print(f"scorer error: {e}", file=sys.stderr)
return 1
payload = {
"available": True,
"upstream_dir": str(upstream),
"codebook": str(codebook),
"model": args.model,
"profile_key": result.details.get("profile_key"),
"exact_match": result.details.get("exact_match"),
"is_watermarked": result.is_watermarked,
"confidence": result.confidence,
"phase_match": result.phase_match,
"per_channel_scores": result.details.get("per_channel_scores"),
"per_channel_n": result.details.get("per_channel_n"),
"multi_scale_consistency": result.multi_scale_consistency,
}
if args.json:
json.dump(payload, sys.stdout, indent=2)
sys.stdout.write("\n")
else:
label = "yes" if result.is_watermarked else "no"
print(f"SynthID score: confidence {result.confidence:.3f} (watermarked: {label})")
print(f" phase_match: {result.phase_match:.3f}")
if result.details.get("profile_key"):
print(f" profile: {result.details['profile_key']}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env bash
set -euo pipefail
# Bootstrap an external reverse-SynthID checkout for the optional pixel scorer.
#
# The upstream project (https://github.com/aloshdenny/reverse-SynthID) is
# licensed under a non-commercial Research License and is NOT bundled in this
# repository. This script clones it locally and installs only the dependencies
# needed by score_synthid.py.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEFAULT_DIR="${REVERSE_SYNTHID_DIR:-$HOME/reverse-SynthID}"
DIR=""
REF="main"
PYTHON="${PYTHON:-python3}"
FULL=0
usage() {
cat <<'EOF'
Usage: setup_synthid.sh [--dir PATH] [--ref REF] [--full] [--python PYTHON]
Clones (if needed) aloshdenny/reverse-SynthID, creates a venv, and installs
the Python dependencies required by score_synthid.py.
Options:
--dir PATH checkout directory (default: $REVERSE_SYNTHID_DIR or ~/reverse-SynthID)
--ref REF git ref to clone (default: main)
--full install upstream requirements.txt (adds torch/diffusers for VAE bypass)
--python PY Python interpreter used to create the venv (default: python3)
EOF
}
while [[ $# -gt 0 ]]; do
case "$1" in
--dir)
DIR="${2:?--dir requires a value}"
shift 2
;;
--ref)
REF="${2:?--ref requires a value}"
shift 2
;;
--full)
FULL=1
shift
;;
--python)
PYTHON="${2:?--python requires a value}"
shift 2
;;
-h|--help)
usage
exit 0
;;
*)
echo "unknown option: $1" >&2
usage >&2
exit 2
;;
esac
done
DIR="${DIR:-$DEFAULT_DIR}"
mkdir -p "$(dirname "$DIR")"
if command -v realpath >/dev/null 2>&1; then
DIR="$(realpath -m "$DIR")"
else
DIR="$(cd "$(dirname "$DIR")" && pwd)/$(basename "$DIR")"
fi
if [[ ! -d "$DIR/.git" ]]; then
echo "Cloning reverse-SynthID into $DIR"
git clone --depth 1 --filter=blob:none --sparse --branch "$REF" \
https://github.com/aloshdenny/reverse-SynthID.git "$DIR"
git -C "$DIR" sparse-checkout set --no-cone \
'/src/' \
'/artifacts/spectral_codebook_v4.npz' \
'/requirements.txt' \
'/LICENSE' \
'/README.md'
else
echo "Using existing checkout: $DIR"
fi
if [[ ! -x "$DIR/.venv/bin/python" ]]; then
echo "Creating venv at $DIR/.venv"
"$PYTHON" -m venv "$DIR/.venv"
fi
echo "Installing Python dependencies"
"$DIR/.venv/bin/python" -m pip install --upgrade pip
if [[ "$FULL" -eq 1 ]]; then
echo "Installing full upstream requirements.txt (includes torch/diffusers)"
"$DIR/.venv/bin/python" -m pip install -r "$DIR/requirements.txt"
else
echo "Installing scorer-only dependencies"
"$DIR/.venv/bin/python" -m pip install -r "$SCRIPT_DIR/requirements-synthid-scorer.txt"
fi
codebook="$DIR/artifacts/spectral_codebook_v4.npz"
if [[ ! -f "$codebook" ]]; then
echo "warning: codebook not found at $codebook" >&2
echo "run: git -C '$DIR' sparse-checkout add '/artifacts/spectral_codebook_v4.npz'" >&2
fi
cat <<EOF
Done. Score an image with:
export REVERSE_SYNTHID_DIR="$DIR"
"$DIR/.venv/bin/python" "\$REPO/skills/remove-ai-marks/scripts/score_synthid.py" IMAGE
EOF
+146
View File
@@ -0,0 +1,146 @@
"""Tests for the optional reverse-SynthID scorer adapter."""
from __future__ import annotations
import json
import subprocess
import sys
from pathlib import Path
from types import SimpleNamespace
import pytest
ROOT = Path(__file__).resolve().parents[1]
SCRIPTS = ROOT / "skills" / "remove-ai-marks" / "scripts"
sys.path.insert(0, str(SCRIPTS))
import image_meta # noqa: E402
from image_meta import ImageInspectReport, run_synthid_score # noqa: E402
SCORE_SCRIPT = SCRIPTS / "score_synthid.py"
def test_score_synthid_cli_unavailable_without_upstream(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
monkeypatch.delenv("REVERSE_SYNTHID_DIR", raising=False)
dummy = tmp_path / "img.png"
dummy.write_bytes(b"not really an image")
r = subprocess.run(
[sys.executable, str(SCORE_SCRIPT), str(dummy)],
capture_output=True,
text=True,
)
assert r.returncode == 3
assert "REVERSE_SYNTHID_DIR" in (r.stderr or "")
def test_run_synthid_score_unconfigured_returns_none(
monkeypatch: pytest.MonkeyPatch,
):
monkeypatch.delenv("REVERSE_SYNTHID_DIR", raising=False)
assert run_synthid_score(Path("x.png")) is None
def test_run_synthid_score_unavailable_returns_none(
monkeypatch: pytest.MonkeyPatch,
):
def fake_run(*args, **kwargs):
return SimpleNamespace(returncode=3, stdout="", stderr="unavailable")
monkeypatch.setattr(image_meta.subprocess, "run", fake_run)
assert run_synthid_score(Path("x.png"), upstream_dir="/tmp/upstream") is None
def test_run_synthid_score_parses_json(
monkeypatch: pytest.MonkeyPatch,
):
payload = {
"available": True,
"is_watermarked": True,
"confidence": 0.91,
"phase_match": 0.65,
}
captured: dict = {}
def fake_run(cmd, **kwargs):
captured["cmd"] = cmd
return SimpleNamespace(returncode=0, stdout=json.dumps(payload), stderr="")
monkeypatch.setattr(image_meta.subprocess, "run", fake_run)
result = run_synthid_score(Path("img.png"), upstream_dir="/tmp/upstream")
assert result == payload
assert "--json" in captured["cmd"]
assert "--upstream-dir" in captured["cmd"]
assert "/tmp/upstream" in captured["cmd"]
def test_run_synthid_score_runtime_error_is_reported(
monkeypatch: pytest.MonkeyPatch,
):
def fake_run(*args, **kwargs):
return SimpleNamespace(returncode=1, stdout="", stderr="boom")
monkeypatch.setattr(image_meta.subprocess, "run", fake_run)
result = run_synthid_score(Path("img.png"), upstream_dir="/tmp/upstream")
assert result is not None
assert result.get("available") is False
assert "boom" in result.get("error", "")
def test_inspect_image_cli_prints_synthid_score(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str],
):
import importlib.util
spec = importlib.util.spec_from_file_location(
"inspect_image_cli", str(SCRIPTS / "inspect_image.py")
)
assert spec and spec.loader
cli = importlib.util.module_from_spec(spec)
spec.loader.exec_module(cli)
report = ImageInspectReport(
path="shot.png",
format="png",
has_c2pa=False,
has_ai_metadata=False,
synthid={
"available": True,
"is_watermarked": True,
"confidence": 0.91,
},
)
img = tmp_path / "shot.png"
img.write_bytes(b"not really an image")
monkeypatch.setattr(cli, "inspect_image", lambda path, synthid_dir=None: report)
monkeypatch.setattr(sys, "argv", ["inspect_image.py", str(img)])
assert cli.main() == 0
out = capsys.readouterr().out
assert "SynthID score: confidence 0.910 (watermarked: yes)" in out
def test_inspect_report_to_dict_includes_synthid():
report = ImageInspectReport(
path="x.png",
format="png",
has_c2pa=False,
has_ai_metadata=False,
synthid={"available": True, "confidence": 0.8},
)
assert report.to_dict()["synthid"]["confidence"] == 0.8
empty = ImageInspectReport(
path="x.png",
format="png",
has_c2pa=False,
has_ai_metadata=False,
)
assert empty.to_dict()["synthid"] is None