fix(archives): wrap the bare EOFError a truncated tar.gz raises (#3938)

* fix(archives): wrap the bare EOFError a truncated tar.gz raises

`tarfile` wraps most decompression failures in `TarError`, but a gzip
stream that ends before its end-of-stream marker escapes as a bare
`EOFError` from the gzip layer. `EOFError` derives from neither
`TarError` nor `OSError`, so it bypassed all three of the tar handlers
added with tar archive support (#3874):

- the format probe in `detect_archive_format`, which caught only
  `tarfile.TarError`;
- `tarfile.open` in `safe_extract_tar`;
- member iteration in `safe_extract_tar`.

A truncated `.tar.gz` — an interrupted download, a partially written
file — therefore raised a raw `EOFError` straight through the caller's
`error_type`, so callers catching `ValueError`/`ExtensionError`/
`PresetError` never saw it. In `specify workflow add` the effect is worse
than a traceback: Typer treats a bare `EOFError` as a Ctrl-D abort, so
the command printed only "Aborted." with no diagnostic at all. The ZIP
twin reports "Invalid workflow archive: Invalid ZIP archive: <path>".

Route all three sites through a shared `_TAR_DECOMPRESSION_ERRORS`
tuple so they stay in sync. `zlib.error` is included alongside
`EOFError`: it is likewise neither a `TarError` nor an `OSError` and can
surface from a corrupt deflate block. `OSError` is kept only on the two
`safe_extract_tar` sites, which report genuine I/O failures; adding it
to the probe would silently swallow them instead.

Truncated tar.gz now reports the same clean, domain-typed error as the
ZIP path. Tests cover both the short prefix that fails in
`tarfile.open` and the longer ones that fail during member iteration —
`tarfile` decompresses lazily, so the leak surfaced at different sites
depending on how much of the stream survived.

Assisted-by: Claude Opus 5 (1M context)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(archives): cover the bare zlib.error a corrupt deflate block raises

Review feedback: the `zlib.error` arm of `_TAR_DECOMPRESSION_ERRORS` was
not exercised. Every regression added with the fix truncates a valid
deflate stream, which raises `EOFError`, so `zlib.error` could regress
independently of the EOF handling.

It is genuinely reachable, but only under a narrower condition than the
truncation cases. `tarfile` converts `zlib.error` to `ReadError` while
reading a member *header*, but the forward seek it performs to skip
member *data* (`tarfile.next`) sits outside that conversion, so a corrupt
region past the first header escapes raw. Reaching that seek needs
members larger than the gzip read buffer: with small members the whole
stream is decompressed during the first header read and the error is
wrapped. The new fixture therefore uses two 256 KiB members at
`compresslevel=1` — a ~7 KiB archive — corrupted past the midpoint so
the first header still reads clean.

Adds four tests: the two `safe_extract_tar` sites (plain and with a
caller-supplied `error_type`), the `safe_extract_archive` entry point
with a caller-supplied `error_type`, and a guard asserting the fixture
still reaches the module as a bare `zlib.error` — so if a future Python
wraps it, that fails loudly instead of the coverage silently decaying
into a duplicate of the `EOFError` cases.

Verified test-the-test: the three wrapping tests fail against the
unmodified `_download_security.py` with a raw
`zlib.error: Error -3 while decompressing data: invalid distance code`,
and pass with the fix.

Also corrects the scope claimed for the probe site. Fuzzing 2800 corrupt
archives never produced a bare `zlib.error` from `tarfile.open` alone,
because the only read it performs is the header read that `tarfile`
already converts. The probe's `zlib.error` arm is defensive, not
load-bearing; the tuple comment and a detection test now say so rather
than implying coverage that cannot exist.

Assisted-by: Claude Opus 5 (1M context)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(archives): make the corrupt-deflate fixture zlib-version independent

CI failure on macos-latest/3.13: `test_corrupt_deflate_fixture_raises_bare_zlib_error`
failed with `gzip.BadGzipFile: CRC check failed`. The other five pytest jobs
were fail-fast cancellations, not real failures, and ruff was already green.

The fixture built its corruption by XOR-ing 64 arbitrary bytes mid-stream.
Whether that produces a *structural* deflate error is zlib-version dependent:
on the macOS runner the mangled bytes still decoded, so the stream instead
failed the trailing gzip CRC check and raised `BadGzipFile` -- an `OSError`,
which the pre-fix `(TarError, OSError)` handler already caught. The guard test
exists precisely to catch that degradation, and it did its job.

Replaces the XOR with a deflate block header whose `BTYPE` is the reserved
value `0b11`. Every zlib rejects that identically as "invalid block type", and
it fails during decompression rather than at the CRC check, so no version can
turn it into a `TarError` or `OSError`. The stream is assembled by hand
(`compressobj(-15)` + explicit gzip header/trailer) so the invalid block lands
a controlled 256 KiB into the first member's data -- past the gzip read buffer,
so the first header still reads clean and the failure surfaces from the forward
seek in `tarfile.next`, which is the site the raw `zlib.error` escapes from.

A sweep over clean-prefix sizes confirms a wide margin: with 512 KiB members
every prefix from 160 KiB up yields a bare `zlib.error`, versus the transition
below ~131 KiB where `tarfile` still wraps it as `ReadError`. The hand-built
gzip header also zeroes the mtime field, so the fixture is now byte-identical
across builds instead of embedding a timestamp.

Strengthens the guard to assert what the fix actually depends on -- that the
exception is neither a `TarError` nor an `OSError` -- so the fixture cannot
silently decay into an already-caught type again.

Production code is unchanged from ef49acc; this is test-only.

Verified test-the-test by dropping the `zlib.error` arm from
`_TAR_DECOMPRESSION_ERRORS`: the three wrapping tests fail with the raw
`zlib.error: Error -3 while decompressing data: invalid block type`, and pass
with it restored. `tests/test_download_security.py`: 193 passed.
`ruff check src tests` (the exact CI command): all checks passed.

Assisted-by: Claude Opus 4.8 (1M context)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Noor ul ain
2026-08-05 10:57:51 -05:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 71125fc346
commit e9710ae45e
2 changed files with 209 additions and 3 deletions
+21 -3
View File
@@ -10,6 +10,7 @@ import struct
import tarfile
import unicodedata
import zipfile
import zlib
from collections.abc import Iterator
from contextlib import ExitStack, contextmanager
from ipaddress import IPv4Address, IPv6Address, ip_address
@@ -69,6 +70,19 @@ _ZIP_MAX_COMMENT_BYTES = (1 << 16) - 1
_BOUNDED_ZIP_COMPRESSION_METHODS = frozenset(
(zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED)
)
#: Decompression failures a truncated or corrupt gzip stream raises from
#: ``tarfile``. Most are wrapped in ``TarError``, but two escape raw, and
#: neither derives from ``TarError`` or ``OSError``, so both bypass a
#: ``(TarError, OSError)`` handler:
#:
#: * ``EOFError`` -- from the gzip layer when the stream ends before its
#: end-of-stream marker, i.e. a truncated archive.
#: * ``zlib.error`` -- from a corrupt deflate block. ``tarfile`` converts this
#: to ``ReadError`` while reading a member *header*, but the forward seek it
#: performs to skip member *data* sits outside that conversion, so a corrupt
#: region past the first header escapes raw.
_TAR_DECOMPRESSION_ERRORS = (tarfile.TarError, EOFError, zlib.error)
_ARCHIVE_CONTENT_TYPES: dict[str, ArchiveFormat] = {
"application/gzip": "tar.gz",
"application/x-gzip": "tar.gz",
@@ -166,7 +180,11 @@ def detect_archive_format(
try:
with tarfile.open(fileobj=archive_file, mode="r:gz"):
is_tar_gz = True
except tarfile.TarError:
except _TAR_DECOMPRESSION_ERRORS:
# A truncated gzip stream raises a bare EOFError here rather
# than a TarError, so catching only TarError let it escape
# this probe as a raw exception instead of leaving
# ``is_tar_gz`` False and reporting the format mismatch.
pass
archive_file.seek(0)
except OSError as exc:
@@ -1077,7 +1095,7 @@ def safe_extract_tar(
mode="r:gz",
fileobj=archive_file,
)
except (tarfile.TarError, OSError) as exc:
except (*_TAR_DECOMPRESSION_ERRORS, OSError) as exc:
_raise_from(error_type, f"Invalid tar.gz archive: {archive_path}", exc)
with archive:
@@ -1149,7 +1167,7 @@ def safe_extract_tar(
f"of {max_total_bytes} bytes",
)
validated.append((member, normalized_name, is_dir))
except (tarfile.TarError, OSError) as exc:
except (*_TAR_DECOMPRESSION_ERRORS, OSError) as exc:
_raise_from(
error_type,
f"Invalid tar.gz archive: {archive_path}",