Fix Pydantic annotation metadata handling across 2.10-2.13#3863
Fix Pydantic annotation metadata handling across 2.10-2.13#3863
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
docs | d4bcea4 | Commit Preview URL Branch Preview URL |
May 05 2026, 12:20 PM |
4200fb4 to
4cb4e32
Compare
| if is_annotated(t_): | ||
| a_t, *a_m = get_args(t_) | ||
| return Annotated[_process_annotation(a_t), tuple(a_m)] # type: ignore[return-value] | ||
| return Annotated[(_process_annotation(a_t), *a_m)] # type: ignore[return-value] |
There was a problem hiding this comment.
To me it's unclear why we previously wrapped other annotation metadata into tuples, losing their semantics:
Annotated[str, FieldInfo(...), AfterValidator(...)]
would be turned into:
Annotated[str, (FieldInfo(...), AfterValidator(...)].
Pydantic then ignores this tuple and we effectively lose the validator (or other annotation metadata)
There was a problem hiding this comment.
this is certainly a bug. I probably overlooked that in claude-generated code.
rudolfix
left a comment
There was a problem hiding this comment.
LGTM! it is very hard to verify Pydantic quirks - I added two more tests to have (hopefully) complete coverage - test_apply_contract_preserves_nested_annotated_metadata_entries checks tuple bug directly
| if is_annotated(t_): | ||
| a_t, *a_m = get_args(t_) | ||
| return Annotated[_process_annotation(a_t), tuple(a_m)] # type: ignore[return-value] | ||
| return Annotated[(_process_annotation(a_t), *a_m)] # type: ignore[return-value] |
There was a problem hiding this comment.
this is certainly a bug. I probably overlooked that in claude-generated code.
Fixes #3862 #3861
Summary
RootModelhandling across Pydantic 2.10 → 2.13Annotated[...]metadata when dlt rebuilds Pydantic models for schema contractsProblem 1 — Pydantic 2.13 moved where the discriminator lives
For tagged unions wrapped in a
RootModel, Pydantic 2.13 moved the discriminator off theAnnotatedmetadata of the root field and ontoroot_field.discriminatordirectly:root_field.annotationroot_field.discriminatorAnnotated[Union[...], FieldInfo(discriminator="kind")]NoneUnion[...](bare)"kind"dlt assumed the discriminator was always recoverable from
root_field.annotation, so on 2.13 discriminated root models were treated like a plainRootModelwith a singlerootcolumn, producing e.g.NOT NULL constraint failed: click_events.root.This is partially fixed already, but only for the case where 2.13 strips the annotation down to a bare
Union. It misses the case whereroot_field.annotationis still anAnnotated[...]on 2.13 because additional metadata (e.g. anAfterValidator) is present — the discriminator lives onroot_field.discriminatorbut the annotation is not bare, so the rescue path was skipped.Problem 2 — dlt was corrupting
Annotated[...]metadataWhile rebuilding models for schema contracts, dlt did:
This collapses N metadata entries into one tuple entry, changing Pydantic's semantics. For example:
was effectively rebuilt as:
For regular
BaseModelthis went unnoticed becausecreate_model(__base__=model, ...)re-inherits the original, correctly-spread metadata. ForRootModelthere is no__base__rescue (it's built withtype(name, (PydanticRootModel[ann],), ...)), so validators and other root-level metadata were silently dropped.Fix
Annotated[..., FieldInfo(...)]root_field.discriminator, including whenroot_field.annotationis stillAnnotateddue to other metadataRootModel[...]Annotated[...]metadata by preserving entries individually instead of packing them into a tuple