[rasmusfaber]

This PR contains:

• New features
• Changes to dev-tools e.g. CI config / github tooling
• Docs
• Bug fixes
• Code refactor

What is the current behavior? (You can also link to an open issue here)

On interpreters whose zipfile carries CPython gh-156002 (CVE-2026-15310, "Memory exhaustion via crafted zip file"), every read of a zstd-compressed .eval member fails:

  File ".../zipfile/__init__.py", line 1126, in _read1
    if _decompressor_needs_input(self._decompressor):
  File ".../zipfile/__init__.py", line 818, in _decompressor_needs_input
    return decompressor._needs_input if needs_input is None else needs_input
AttributeError: '_MultiFrameZstdDecompressObj' object has no attribute '_needs_input'

The patched ZipExtFile._read1 now drives non-deflate decompressors like the stdlib bz2/lzma/compression.zstd ones: it probes needs_input before reading more compressed bytes and calls decompress(data, max_length) with two positional arguments. _MultiFrameZstdDecompressObj (src/inspect_ai/_util/zipfile.py), which is registered for ZIP_ZSTANDARD on every Python version and wraps the zstandard backport's one-argument decompressobj, exposes neither.

Where this bites today: Docker Hardened Images' python:3.13-dev republished on 2026-09-03 with the open 3.13 backport python/cpython#156738 applied verbatim (same 3.13.15 version string). The fix is on CPython main and 3.15; backports #156737–#156741 are open for 3.10–3.14, so every maintained CPython will ship this (3.13.16 / 3.14.8 are scheduled for 2026-10-06). User-facing symptom: eval_set writes a success log, cannot read it back, treats the task as incomplete and re-runs it 1 + retry_attempts times.

What is the new behavior?

_MultiFrameZstdDecompressObj.decompress accepts max_length (not enforced, see below) and the object reports needs_input = True. For this wrapper the patched _read1 thereby reduces exactly to the pre-patch one: it always reads, always gets everything, and _eof is still driven by compress_left <= 0. .eval reads work on stock and patched interpreters alike.

Why max_length is not enforced: the zstandard decompressobj has no output bound of its own. Withholding bytes in Python would only add copies on top of an expansion that is already allocated (peak RSS measured identical), and a naive re-slice is quadratic under small read(n) calls (256 MiB of zeros under read(4096): >400 s vs 0.4 s). Rationing compressed input instead is ~50× slower on the whole-member zf.read() path inspect uses and bounds nothing on a real bomb whose compressed stream is smaller than the ration. So the gh-156002 memory bound is not delivered for zstd members — the same exposure as before this change; a truly bounded wrapper would need compression.zstd / backports.zstd as the inner decompressor, which is a larger change.

Tests: a small double of the post-gh-156002 _read1 loop exercises the new call shape on any interpreter (including the stdlib's getattr(..., "needs_input", None) fallback and the new termination formula), plus the unchanged one-argument shape, partial input across a frame boundary, and read/read1/readline round trips of a genuinely multi-frame entry. With the fix reverted, 6 of the 7 new tests fail on a stock interpreter (AttributeError / TypeError).

Verified: tests/util/test_zipfile_multiframe.py + tests/util/test_async_zip.py pass on 3.13.12 stock, 3.13.12 with the DHI/PR-156738 zipfile shimmed onto PYTHONPATH, 3.14.3 stock (native compression.zstd present), 3.14.3 with the PR-156737 zipfile shimmed in, and inside the real dhi.io/python:3.13-dev image. ruff check, ruff format --check and mypy are clean on the changed files.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

No.

Other information:

Note for maintainers: CI here tests 3.10 and 3.11 only, so this will not surface upstream until those backports ship (their PRs are open); a 3.14 leg would catch it first. Every other third-party zipfile._get_decompressor patcher (zipfile-zstd itself, zipfile-deflate64, zipfile-inflate64) breaks identically and none has adapted yet.

Tooling: authored with Claude Code (Claude Fable 5.1).

Agent review

• Reviewer: Claude Fable 5.1, 5 independent passes in fresh contexts, each with a distinct lens (patched _read1 state machine; stdlib decompressor contract; design alternatives with measurements; test quality incl. mutation testing; downstream blast radius). Passes 1–3 ran executed differential/fuzz harnesses (e.g. 18 entry shapes × 17 access patterns incl. random seeks, data-descriptor and encrypted entries, 20,000-frame streams; 1,500 random multi-frame slicings per runtime) on stock and patched zipfiles across 3.13/3.14 and the real DHI image — all byte-identical to stock.
• Findings: 6 — 3 fixed (drain-loop test could hang instead of fail → replaced by a bounded _read1 double; over-absolute "cannot bound its output" comment → reworded; docstring cited a vendor image rather than CPython issue/PR numbers → cites gh-156002 and the backport PRs), 3 dismissed (a fresh re-import of zipfile_zstd after our patch would shadow the wrapper — only reachable by deleting it from sys.modules; max_length=0 returns everything where the stdlib returns b"" — zipfile never passes 0, observed minimum 4096; pre-existing bytes += copies put read() peak at ~2.4× output — predates this change, out of scope). No pass refuted the fix.

[github-actions]

Suppression ledger changed: 788 → 787 (-1)

File	Rule	Change
tests/util/test_zipfile_multiframe.py	noqa:F401	-1 (reason-less 1 → 0)

Reason-less (baselined) suppressions: 777 → 776

[epatey]

Approved at 956dd30.

The fix does what it says. With needs_input always True and eof always False, the post-gh-156002 _read1 loop reads on every iteration and stops when compress_left reaches zero, the same as before the CPython change. I checked that against the merged 3.14 backport (python/cpython#156737), and the _read1 double in the test matches that code. Accepting max_length without enforcing it keeps the existing behavior for zstd members, which is fine here.

One note, take or leave: the 3.14 backport merged a few hours after this PR opened, so 3.14.8 will carry the change for everyone, not only hardened-image users.