Skip to content

fix(parquet/encoding): reject invalid delta length payloads#1013

Open
fallintoplace wants to merge 3 commits into
apache:mainfrom
fallintoplace:fix/parquet-delta-length-validation
Open

fix(parquet/encoding): reject invalid delta length payloads#1013
fallintoplace wants to merge 3 commits into
apache:mainfrom
fallintoplace:fix/parquet-delta-length-validation

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

The DELTA_LENGTH_BYTE_ARRAY decoder slices payload data using unchecked lengths and keeps the logical page count after decoding fewer physical values. Invalid lengths can panic, and valid nullable pages can index past the physical length table.

What changes are included in this PR?

  • Propagate errors and short reads from delta prefix and length streams.
  • Reject negative lengths and cumulative lengths beyond the suffix payload.
  • Track the encoded physical length count in the byte-array decoder while retaining logical page-count validation.
  • Require DELTA_BYTE_ARRAY prefix and suffix streams to contain the same number of physical values.

Are these changes tested?

Yes. Focused tests cover negative and oversized lengths plus a nullable page with fewer physical values than logical positions. The full encoding and Parquet file suites pass.

@fallintoplace
fallintoplace requested a review from zeroshade as a code owner July 25, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant