Tolerate unknown properties in resolved schema / published manifest

[lmolkova]

Plan (incomplete, with follow ups)

Forward-compatibility for resolved schemas and publication manifests

Goal

Older weaver must read resolved schemas and publication manifests produced by newer weaver (minor version ahead). Unknown fields tolerated and ignored. Major version mismatch must fail.

Scope

• Resolved schema v2 (ResolvedTelemetrySchema and all nested types)
• Publication manifest (PublicationRegistryManifest)
• NOT definition schemas (SemConvSpecV2, CommonFields, etc.) — these keep #[serde(deny_unknown_fields)]
• NOT materialized schemas (never read back)
• Only v2 types

Rules

1. Minor version ahead: deserialize successfully, ignore unknown fields
2. Major version mismatch: fail with clear error
3. Current or previous version with unknown fields: fail (typo detection)

Implementation pattern

1. Every struct in resolved schema and publication manifest: remove #[serde(deny_unknown_fields)], add:

   #[serde(flatten, skip_serializing)]
   #[schemars(skip)]
   pub unknown_fields: HashMap<String, serde_yaml::Value>,

2. This applies to ALL nested types: Registry, Refinements, Span, Metric, Event, Entity, AttributeGroup, SpanAttributeRef, MetricAttributeRef, EventAttributeRef, EntityAttributeRef, Attribute (catalog), SpanName, SpanRefinement, MetricRefinement, EventRefinement

3. When a definition type is reused in resolved schema:

   • Via #[serde(flatten)] (e.g., CommonFields): no changes needed — outer struct's unknown_fields HashMap absorbs everything, inner deny_unknown_fields is effectively ignored
   • Via direct nesting (e.g., SpanName): must add unknown_fields to that type, removing its deny_unknown_fields. JSON schema validator still catches typos for definition files.

4. Typo detection (two-pass)

   • Always deserialize tolerantly first
   • Then collect_unknown_fields walks all unknown_fields HashMaps in the deserialized struct tree
   • If file_format version is current or previous AND unknown fields found → fail with error naming the unexpected fields
   • If file_format version is minor-ahead → skip this check

5. collect_unknown_fields must cover every type that has unknown_fields: HashMap

6. Version parsing

   • Use parse_file_format_version("type/MAJOR.MINOR") → (prefix, major, minor).

7. Resolved schema: prefix "resolved", current 2.0

8. Publication manifest: prefix "manifest", current 2.0

9. Tests

   • Minor-ahead integration test: resolved schema YAML with file_format: resolved/2.99 containing a future_* unknown field on EVERY nested type. Must load successfully.
   • Major-mismatch test: file_format: resolved/99.0 → must fail with major version error
   • Unknown field in current version test: file_format: resolved/2.0 with a typo field → must fail naming the field
   • Same three tests for publication manifest
   • Unit tests per type: each resolved schema type deserializes successfully with an unknown field

10. Convention

• All unknown_fields fields use #[serde(flatten, skip_serializing)] and #[schemars(skip)]. Never written back out. Never appear in generated JSON schemas.

[codecov]

Codecov Report

❌ Patch coverage is 85.61644% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.9%. Comparing base (5c9baa5) to head (284b5b6).

Files with missing lines	Patch %	Lines
crates/weaver_semconv/src/manifest.rs	76.4%	12 Missing ⚠️
crates/weaver_resolved_schema/src/v2/version.rs	91.4%	3 Missing ⚠️
crates/weaver_semconv/src/unknown_fields.rs	92.1%	3 Missing ⚠️
crates/weaver_resolver/src/loader.rs	66.6%	2 Missing ⚠️
crates/weaver_resolved_schema/src/v2/mod.rs	93.7%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##            main   #1365    +/-   ##
======================================
  Coverage   81.9%   81.9%            
======================================
  Files        115     117     +2     
  Lines       9850    9978   +128     
======================================
+ Hits        8069    8175   +106     
- Misses      1781    1803    +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.