mirror of
https://github.com/github/spec-kit.git
synced 2026-08-15 14:02:04 +02:00
main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
e9710ae45e |
fix(archives): wrap the bare EOFError a truncated tar.gz raises (#3938)
* fix(archives): wrap the bare EOFError a truncated tar.gz raises `tarfile` wraps most decompression failures in `TarError`, but a gzip stream that ends before its end-of-stream marker escapes as a bare `EOFError` from the gzip layer. `EOFError` derives from neither `TarError` nor `OSError`, so it bypassed all three of the tar handlers added with tar archive support (#3874): - the format probe in `detect_archive_format`, which caught only `tarfile.TarError`; - `tarfile.open` in `safe_extract_tar`; - member iteration in `safe_extract_tar`. A truncated `.tar.gz` — an interrupted download, a partially written file — therefore raised a raw `EOFError` straight through the caller's `error_type`, so callers catching `ValueError`/`ExtensionError`/ `PresetError` never saw it. In `specify workflow add` the effect is worse than a traceback: Typer treats a bare `EOFError` as a Ctrl-D abort, so the command printed only "Aborted." with no diagnostic at all. The ZIP twin reports "Invalid workflow archive: Invalid ZIP archive: <path>". Route all three sites through a shared `_TAR_DECOMPRESSION_ERRORS` tuple so they stay in sync. `zlib.error` is included alongside `EOFError`: it is likewise neither a `TarError` nor an `OSError` and can surface from a corrupt deflate block. `OSError` is kept only on the two `safe_extract_tar` sites, which report genuine I/O failures; adding it to the probe would silently swallow them instead. Truncated tar.gz now reports the same clean, domain-typed error as the ZIP path. Tests cover both the short prefix that fails in `tarfile.open` and the longer ones that fail during member iteration — `tarfile` decompresses lazily, so the leak surfaced at different sites depending on how much of the stream survived. Assisted-by: Claude Opus 5 (1M context) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(archives): cover the bare zlib.error a corrupt deflate block raises Review feedback: the `zlib.error` arm of `_TAR_DECOMPRESSION_ERRORS` was not exercised. Every regression added with the fix truncates a valid deflate stream, which raises `EOFError`, so `zlib.error` could regress independently of the EOF handling. It is genuinely reachable, but only under a narrower condition than the truncation cases. `tarfile` converts `zlib.error` to `ReadError` while reading a member *header*, but the forward seek it performs to skip member *data* (`tarfile.next`) sits outside that conversion, so a corrupt region past the first header escapes raw. Reaching that seek needs members larger than the gzip read buffer: with small members the whole stream is decompressed during the first header read and the error is wrapped. The new fixture therefore uses two 256 KiB members at `compresslevel=1` — a ~7 KiB archive — corrupted past the midpoint so the first header still reads clean. Adds four tests: the two `safe_extract_tar` sites (plain and with a caller-supplied `error_type`), the `safe_extract_archive` entry point with a caller-supplied `error_type`, and a guard asserting the fixture still reaches the module as a bare `zlib.error` — so if a future Python wraps it, that fails loudly instead of the coverage silently decaying into a duplicate of the `EOFError` cases. Verified test-the-test: the three wrapping tests fail against the unmodified `_download_security.py` with a raw `zlib.error: Error -3 while decompressing data: invalid distance code`, and pass with the fix. Also corrects the scope claimed for the probe site. Fuzzing 2800 corrupt archives never produced a bare `zlib.error` from `tarfile.open` alone, because the only read it performs is the header read that `tarfile` already converts. The probe's `zlib.error` arm is defensive, not load-bearing; the tuple comment and a detection test now say so rather than implying coverage that cannot exist. Assisted-by: Claude Opus 5 (1M context) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(archives): make the corrupt-deflate fixture zlib-version independent CI failure on macos-latest/3.13: `test_corrupt_deflate_fixture_raises_bare_zlib_error` failed with `gzip.BadGzipFile: CRC check failed`. The other five pytest jobs were fail-fast cancellations, not real failures, and ruff was already green. The fixture built its corruption by XOR-ing 64 arbitrary bytes mid-stream. Whether that produces a *structural* deflate error is zlib-version dependent: on the macOS runner the mangled bytes still decoded, so the stream instead failed the trailing gzip CRC check and raised `BadGzipFile` -- an `OSError`, which the pre-fix `(TarError, OSError)` handler already caught. The guard test exists precisely to catch that degradation, and it did its job. Replaces the XOR with a deflate block header whose `BTYPE` is the reserved value `0b11`. Every zlib rejects that identically as "invalid block type", and it fails during decompression rather than at the CRC check, so no version can turn it into a `TarError` or `OSError`. The stream is assembled by hand (`compressobj(-15)` + explicit gzip header/trailer) so the invalid block lands a controlled 256 KiB into the first member's data -- past the gzip read buffer, so the first header still reads clean and the failure surfaces from the forward seek in `tarfile.next`, which is the site the raw `zlib.error` escapes from. A sweep over clean-prefix sizes confirms a wide margin: with 512 KiB members every prefix from 160 KiB up yields a bare `zlib.error`, versus the transition below ~131 KiB where `tarfile` still wraps it as `ReadError`. The hand-built gzip header also zeroes the mtime field, so the fixture is now byte-identical across builds instead of embedding a timestamp. Strengthens the guard to assert what the fix actually depends on -- that the exception is neither a `TarError` nor an `OSError` -- so the fixture cannot silently decay into an already-caught type again. Production code is unchanged from ef49acc; this is test-only. Verified test-the-test by dropping the `zlib.error` arm from `_TAR_DECOMPRESSION_ERRORS`: the three wrapping tests fail with the raw `zlib.error: Error -3 while decompressing data: invalid block type`, and pass with it restored. `tests/test_download_security.py`: 193 passed. `ruff check src tests` (the exact CI command): all checks passed. Assisted-by: Claude Opus 4.8 (1M context) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
a3e183d069 |
feat: support tar archives for installs (#3874)
* feat: support tar archives for installs Add secure .tar.gz and .tgz parity with ZIP installation for extensions, presets, and workflows, including full workflow package preservation. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd * chore: clean rebased archive imports Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd * fix: preserve hardened archive install behavior Keep malformed ZIP diagnostics, filesystem-independent manifest selection, and reserved workflow overlays consistent after adding generic archive support. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd * fix: extract staged workflow archives by descriptor Avoid reopening a held staging path on Windows while retaining authoritative-inode archive validation. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd * fix: extract catalog archives from verified bytes Use the already bounded and SHA-verified response bytes directly so Windows file-sharing semantics cannot affect archive detection. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd * fix: address archive install review feedback Preserve forced preset reinstalls, sniff suffixless workflow archives without weakening YAML limits, and restore prior workflow packages before failed-install cleanup. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd --------- Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd |
||
|
Pascal THUETandGitHub
|
118062eac4 |
harden: secure extension and preset archive downloads (#3141)
* harden: secure extension and preset archive downloads Adopt the shared download-security primitives from #3140 across extension and preset catalog, package, direct-URL, and ZIP-install flows: - bound catalog, package, and inline manifest reads; - verify catalog SHA-256 values when present; - replace path-only extraction with bounded traversal/symlink-safe extraction; - validate malformed hosts and ports before opening download URLs; - handle normalized trailing-backslash directory entries consistently. Redirect enforcement and checksum verification remain owned by the shared helpers already on main; this commit wires them into extension and preset behavior. Assisted-by: OpenAI Codex (model: GPT-5, autonomous) * harden: close archive and catalog download edge cases Preflight ZIP central directories before ZipFile allocates them, bound both declared and actual payload sizes, and reject ambiguous or non-portable archive paths before extraction. Keep extension update manifest selection consistent with extraction, reject unsafe catalog-derived output filenames and malformed URL types, and escape untrusted values in download errors. Add regression coverage for parser differentials, collisions, platform-specific filenames, bounded call sites, and failure ordering. Assisted-by: OpenAI Codex (model: GPT-5, autonomous) * harden: address download security review feedback Assisted-by: OpenAI Codex (model: GPT-5, autonomous) * harden: close ZIP preflight review gaps Assisted-by: OpenAI Codex (model: GPT-5, autonomous) * fix: harden extension update preflight and rollback Assisted-by: OpenAI Codex (model: GPT-5, autonomous) * fix: harden extension update rollback Assisted-by: OpenAI Codex (model: GPT-5, autonomous) |
||
|
Pascal THUETandGitHub
|
3a7a8758f7 |
fix: harden bounded reads and redirect validation (#3671)
Assisted-by: OpenAI Codex (model: GPT-5, autonomous) |
||
|
Pascal THUETandGitHub
|
5601830ba3 |
harden: bound HTTP reads and enforce strict redirects (#3140)
* harden: bound HTTP reads and enforce strict redirects Add a shared _download_security module (read_response_limited, is_https_or_localhost_http, size constants) and route the GitHub release and Azure DevOps token network reads through bounded reads so an oversized response can't exhaust memory. Add a strict_redirects mode to authentication.open_url: the redirect handler now rejects any redirect whose target isn't HTTPS (or HTTP to localhost), composing with the existing per-hop redirect_validator and auth-stripping. The Azure DevOps token POST is routed through that handler so a 307/308 cannot forward the client_secret body to a non-HTTPS host. Assisted-by: Codex (model: GPT-5, autonomous) * test: align HTTP fakes with bounded reads Assisted-by: Codex (model: GPT-5, autonomous) * fix: tolerate invalid token response encoding Assisted-by: Codex (model: GPT-5, autonomous) * test: align GHES fakes with bounded reads Assisted-by: Codex (model: GPT-5, autonomous) * test: reuse shared upgrade HTTP response helper Assisted-by: Codex (model: GPT-5, autonomous) * fix: include rejected redirect target in error Assisted-by: Codex (model: GPT-5, autonomous) * fix: enforce strict redirects by default Assisted-by: Codex (model: GPT-5, autonomous) * fix: close redirect credential and SSRF gaps Assisted-by: Codex (model: GPT-5, autonomous) |