fix(parquet/compress): allocate uncompressed output for short destinations - #1004
Conversation
zeroshade
left a comment
There was a problem hiding this comment.
Verified against current main: nocodec.Encode/EncodeLevel/Decode do copy(dst, src); return dst, which silently truncates when dst is non-nil but shorter than src — directly contradicting the documented Codec contract that a nil or too-small dst results in a newly allocated slice. Switching to append(dst[:0], src...) is the right fix: it returns full-length output, reuses dst's capacity when it is large enough (so no extra allocation in the common case), and allocates only when required. dst == nil works correctly since nil[:0] is valid.
The internal reader path is unaffected because buf is always sized to lenUncompressed, making the lengths equal. The test genuinely catches the bug — pre-fix, Encode(make([]byte, 1), "arrow") returns just "a". LGTM.
Note: I originally flagged this as stacked on #1003, but I re-verified against current main and the two are now independent — I applied #1003 to a scratch worktree and this still applied cleanly, so no ordering constraint remains.
The codec contract allows a
nilor undersized destination and expects the codec to allocate when needed. The uncompressed codec was just returning the unchanged destination, which could leave callers with empty or truncated output.This switches
Encode,EncodeLevel, andDecodeto append-based copying and adds coverage forniland undersized destinations.Tests:
go test ./parquet/compress