Skip to content

feat(uploader): verify uploaded MIME against magic bytes (closes #639) - #1499

Open
61465 wants to merge 5 commits into
supabase:masterfrom
61465:feat/mime-magic-byte-validation-639
Open

61465 wants to merge 5 commits into
supabase:masterfrom
61465:feat/mime-magic-byte-validation-639

Conversation

@61465

@61465 61465 commented Oct 10, 2026

Copy link
Copy Markdown

What

Adds magic-byte verification to the uploader so that bucket-level
allowed_mime_types restrictions catch files whose declared MIME does not
match the actual signature.

Fixes #639 — the reporter's
case was uploading a renamed .gif with Content-Type: image/jpeg to a
bucket restricted to image/jpeg,image/png; today's validateMimeType
accepts it because only the client-declared value is checked.

How

Pre-existing path (unchanged):
validateMimeType(mimeType, allowedMimeTypes) — compares the declared
Content-Type / multipart field against the bucket's allowlist.

Added path (src/storage/validators/file-signature.ts):

  1. detectMimeFromBuffer(buf) — 23 signatures covering images, documents,
    archives, audio/video, and executables. Refine functions distinguish
    RIFF/WEBP from RIFF/WAVE, and ISO BMFF ftyp sub-brands
    (mp4 / heic / avif / mov / 3gp / m4a / webm). Returns undefined on
    unknown signatures so novel formats continue to work.
  2. mimeFamiliesMatch(detected, declared) — intentionally loose:
    image/jpg == image/jpeg, application/zip accepted for OOXML office
    MIMEs, application/x-cfb accepted for legacy OLE2 formats.
  3. wrapWithSignatureDetection(source) — returns a PassThrough and a
    promise that resolves with the detected MIME once the first 4 KiB has
    flowed. No bytes are swallowed; the backend sees the stream unmodified.

Wiring in uploader.ts (after the existing declared-MIME check):

if (allowedMimeTypes?.length) {
  const { stream, detected } = wrapWithSignatureDetection(body)
  body = stream
  detected.then((actualMime) => {
    if (actualMime && !mimeFamiliesMatch(actualMime, mimeType)) {
      stream.destroy(ERRORS.InvalidMimeType(`${actualMime} (declared as ${mimeType})`))
    }
  })
}

Buckets with no mimetype restriction stay on the fast path — no wrapping,
no extra CPU, zero behavioral change.

Why this design

  • Pure helper, zero new deps. Avoids pulling in file-type (ESM-only in
    current versions) and keeps the supply-chain surface flat.
  • Pure helper, no logger/db/config imports. Follows the pattern that
    worked on JWK not used in signObjectURL #629 — anything circular is caught at compile time.
  • Stream-safe. Peek-and-flush PassThrough; the backend never sees a
    truncated or delayed body. Max 4 KiB extra buffered.
  • Fail-closed, 415-compatible. Mismatch raises ERRORS.InvalidMimeType,
    same error code and HTTP status (415) as a header-level rejection, so
    existing clients already handle it.
  • Loose family matching. image/jpg → image/jpeg, zip → OOXML office,
    x-cfb → legacy office. Prevents false positives on legitimate uploads
    while still catching the attack cases from the report.

Tests

41 vitest cases in src/storage/validators/file-signature.test.ts:

  • Detection — real-signature match for each supported format (23 cases).
  • Edge cases — empty buffer, 1-byte buffer, unknown prefix.
  • False-positive defense — RIFF with an unknown form chunk, ftyp with
    an unknown brand (both must return undefined, not fall back to the
    entry's default MIME).
  • Family matching — alias cases (jpg/jpeg, OOXML/zip, OLE2/x-cfb) and
    the eight security-relevant mismatch scenarios the fix is meant to catch:
    • renamed .gif → image/jpeg (the exact mime-type does not check uploaded files, only the filename #639 repro)
    • polyglot JPG claiming to be text/html
    • PE executable declared as application/pdf
    • ELF executable declared as image/jpeg
    • zip claiming to be image/png
    • gzip claiming to be image/jpeg
    • mp4 claiming to be image/jpeg

Also verified mime-type.test.ts still passes (37/37) — no behavior change
to the existing declared-MIME check.

Run:

npx vitest run --config vitest.unit.config.ts src/storage/validators/file-signature.test.ts
# Test Files  1 passed (1)
#      Tests  41 passed (41)

Impact

  • Buckets with no allowed_mime_types → zero code-path change.
  • Buckets with restrictions → one extra PassThrough + 4 KiB peek per
    upload. Negligible overhead vs. the network / storage write cost.
  • Detection failure (unknown signature) is non-fatal — the declared MIME
    stands. We only refuse when we're certain the actual family is wrong.

Security notes

This closes a real file-upload hardening gap. Common attack cases this now
catches:

  1. Payload smuggling — upload an EXE/ELF declared as image/png to a
    bucket that only allows images, then trigger it from a victim client.
  2. Polyglot files — crafted files that are valid JPG and valid HTML.
  3. Backup exfil via image buckets — upload a zip (contains db dump)
    declared as image/jpeg.

Report: #639 by @flogesell,
open 853 days.


Prior work in this repo for context:

61465 added 5 commits October 8, 2026 00:31
Self-hosted deployments that supply only the JWT_JWKS env config could not
sign storage URLs with an asymmetric key: urlSigningKey was auto-populated
only in the multi-tenant JWKS loader, so single-tenant installs with an EC
signing key silently fell back to the HMAC jwtSecret.

- add isUrlSigningCapableJwk / pickUrlSigningKey helpers in config.ts
  (oct with k or EC with d; reject use="enc" per RFC 7517 §4.2)
- parse JWT_JWKS with auto-populated urlSigningKey when the field is
  absent, preserving an explicit value when provided
- mergeTenantJwksWithLegacyKeys now preserves the tenant urlSigningKey
  with a fallback picked from the merged key list
- refactor jwksManager tenant loader to use the shared capability check
- document the URL signing contract in .env.sample
- 9 unit tests covering selection, exclusion, and backward compatibility

HS256 default deployments (no JWT_JWKS) are unaffected.
…supabase#629)

Adds a unit-level round-trip test that provisions JWT_JWKS with only an
ES256 EC key, lets the config parser auto-populate urlSigningKey, calls
the real signJWT, and verifies the token header reports alg=ES256 with
the expected kid — proving asymmetric signing actually happens rather
than silently falling back to the HS256 jwtSecret.

Also adds a negative control that an RSA-only JWT_JWKS still leaves
urlSigningKey unset (RSA cannot sign storage URLs per the maintainer
whitelist) and that the keys are still parsed for verification.
Operators who provisioned JWT_JWKS without a URL-signing-capable key
previously had storage URL signing silently fall back to the HMAC
jwtSecret, with the only observable symptom being the signed-URL
header bytes. The github issue reporter spent debugging time tracing
this to the config. Now the first call that reads the single-tenant
JWT config emits a one-shot startup warning describing the mismatch.

- describeJwtJwksMisconfiguration helper in config.ts (pure, no logger
  dependency so it stays importable from the module graph that logger
  itself depends on)
- getSingleTenantJwtConfig emits the warning via logSchema.warning
  once per process, keeping the fast path allocation-free afterwards
- warning metadata exposes only key count, kty list, and configured
  urlSigningJwkType — no key material, no HMAC secret
- 5 unit tests pinning the dangerous shape and the four excluded
  silent-safe scenarios, with an explicit red-team guard that the
  serialized description never carries key material

HS256 default deployments and well-configured JWT_JWKS tenants stay
silent.
…pabase#629)

The two capability helpers are consumed by three independent code
paths (JWT_JWKS env parser, tenant JWKS merge, per-tenant jwksManager
loader). Previously they were only reached indirectly through the
config-parsing and misconfiguration-detection tests. Add dedicated
table-driven tests that pin every branch of the capability predicate
(oct/EC with and without private material, RSA, OKP, use="enc" for
both key types) and the ordering/empty-list semantics of the picker,
so a future refactor that touches these helpers surfaces a direct
failure instead of a confusing cross-module regression.
…base#639)

The bucket-level allowed_mime_types check only compared the client-declared
Content-Type / multipart field. A caller could rename a .gif to .jpg, send
Content-Type: image/jpeg, and pass the check even though the actual bytes
were a GIF (issue supabase#639, repro from flogesell, 2024).

Add a magic-byte verifier that peeks the first 4 KiB of the upload stream
and, when a bucket actually restricts mimetypes, rejects uploads whose
detected signature does not belong to the declared MIME family.

- src/storage/validators/file-signature.ts
  * Pure helper, zero new dependencies, no logger/db imports (avoids the
    circular-import pattern that bit us on supabase#629).
  * Covers 23 signatures spanning images, documents, archives, A/V, and
    executables. Refine functions distinguish RIFF/WEBP from RIFF/WAVE and
    the ISO BMFF ftyp sub-brands (mp4 / heic / avif / mov / 3gp / m4a).
  * Returns undefined on unknown signatures so novel formats keep working.
  * mimeFamiliesMatch is intentionally loose: image/jpg == image/jpeg,
    application/zip accepted for OOXML office MIMEs, x-cfb for legacy OLE2.
  * wrapWithSignatureDetection returns a PassThrough + a promise that
    resolves with the detected MIME once enough bytes have flowed. No bytes
    are swallowed; the backend sees the stream unmodified.

- src/storage/uploader.ts
  * After the existing validateMimeType header check, if the bucket has
    allowed_mime_types set, wrap the body with the detector. On mismatch
    the stream is destroyed with ERRORS.InvalidMimeType, so the backend
    upload aborts cleanly and the client gets the same 415 as a header-
    level rejection.
  * Buckets without allowed_mime_types stay on the fast path (no wrapping).

- src/storage/validators/file-signature.test.ts
  * 41 vitest cases: real-signature detection for each format, edge cases
    (empty / 1-byte / unknown prefix), RIFF/ftyp false-positive defenses,
    and the eight security-relevant mismatch scenarios the fix is meant
    to catch (renamed gif, polyglot jpg-html, exe-as-pdf, zip-as-image,
    …).
@61465
61465 requested a review from a team as a code owner October 10, 2026 16:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mime-type does not check uploaded files, only the filename JWK not used in signObjectURL

1 participant