* fix: decode XML entities before Layer A scrub in OOXML/ODF
Watermark carriers encoded as XML character references (e.g.
​) reached Layer A as nine ASCII characters and survived
cleaning untouched, because the XML parser in Word/Writer decodes
the entity back into the invisible carrier after the cleaner ran
(#129).
A shared _decode_xml_entities resolves exactly what a conforming
XML parser resolves (numeric references and the five predefined
entities) and leaves everything else literal. The DOCX/XLSX/PPTX
text-run scrubbers decode before clean_text and re-encode on the
way out; the ODT scrubber splits paragraph content into markup and
text segments and only round-trips the text segments, because a
whole-paragraph decode/re-encode would entity-escape the nested
text:span/text:tab markup. Untouched runs keep their original
bytes.
* fix: satisfy ruff lint and format checks
---------
Co-authored-by: yzxcj797 <yzxcj797@users.noreply.github.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
COPY preserves source file and directory modes, so a build host with a
restrictive umask (e.g. 027) landed the scripts as 0640 root:root and
the /app/scripts directory as 0750 -- unreadable and untraversable for
the unprivileged runtime user (uid 10001). The image built fine and
failed only at container start, making it easy to miss (#131).
Normalize after the copy with chmod -R a+rX: world-readable everywhere,
execute bit for directories only, intentionally-executable scripts keep
their bit -- deterministic regardless of the builder's umask.
Co-authored-by: yzxcj797 <yzxcj797@users.noreply.github.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
* fix: report unmeasurable burstiness instead of scoring it as max LLM-like
compute_burstiness returned cv=0.0 for an empty (or single-sentence)
list, which the tiering read as perfectly uniform prose — the strongest
LLM-likeness signal — so any file whose body yielded no parseable
sentences (e.g. entirely wrapped in a code fence) scored falsely high
while word_count stayed high enough to defeat the small-sample dampener
(#132). On a real export this hit 159 of 162 files as pure artifact.
compute_burstiness now returns None for the CV when it cannot be
measured; the scoring site drops the burstiness component and
renormalizes the composite over AI-phrase density and lexical
diversity, with an explicit note. burstiness_cv is null in JSON for
unmeasurable cases, the CLI prints n/a instead of crashing, and the
uniform-cadence finding guards on cv not being None.
* style: satisfy ruff import sorting and formatting
---------
Co-authored-by: yzxcj797 <yzxcj797@users.noreply.github.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
Co-authored-by: Guillaume Meyer <guillaumemeyer@users.noreply.github.com>
* fix: parse single-quoted attributes in relationship/manifest pruning
XML allows attribute values in single or double quotes. The Target
extraction in _prune_dangling_relationships and the full-path
extraction in _prune_odt_manifest_entries matched only double quotes,
so a valid single-quoted Relationship parsed as an empty target,
resolved to the package base directory, and was deleted outright —
corrupting DOCX/XLSX/PPTX and ODT packages produced by tools that
emit single quotes (#130).
Both extractions now use a backreference pattern
((["'])(.*?)\1) so either quote style resolves the real target, and
pruning still only drops relationships whose parts were genuinely
removed. Regression tests cover the single-quoted kept relationship
(the reported corruption) and the double-quoted dropped part.
* fix: satisfy ruff lint and format checks
---------
Co-authored-by: yzxcj797 <yzxcj797@users.noreply.github.com>
Co-authored-by: Guillaume Meyer (The Opinionated Man) <1385518+guillaumemeyer@users.noreply.github.com>
