Shipped in #205: execution-audit.json now retains every JUnit case, exact executed/contracted/unclassified denominators, validated operator-work counts and fallback reasons. Five additional unchanged Calc contracts pass (32 cases total, 8 required native records, 24 explicitly unclassified). Missing/stale evidence and process failures remain blocking. This artifact does not infer native coverage for uncontracted cases or match duplicate parameterized reports without an invocation ID.
Shipped in #206 (merge b94064cc, all 26 checks passed): six unchanged Table API UDF methods now have route contracts across both backend variants. The focused run passes all 12 invocations: six require native Calc work and six require the exact Calc: unsupported function/operator: AS fallback. The full Table API class passes 56 cases with one upstream skip; its other 44 executed cases remain explicitly unclassified. The alias-wrapper admission gap is still open under #110.
Shipped in #277 (merge 6441e27b, all 49 checks passed): per-invocation operator work, submitted graphs, SQL translation links, job outcomes and shared invocation identities now establish native/mixed/fallback, batch host-only, scan-only, preserved-plan and confirmed host-failure routes. Versioned scopes fail closed on missing, stale, ambiguous or inconsistent evidence. Both released-line scopes passed hosted CI: Flink 2.2.1 has 18 passed invocations / 16 required variants; Flink 1.18.1 has 17 / 15. Two early returns per line remain explicitly unclassified, and simulated agent evidence stays outside SQL totals.
Remaining work: expand these proven scopes across the remaining upstream SQL methods and parameter variants, add observation support where no authoritative SQL/job association or outcome can yet be established, and extend verified operator/type contracts. Preserve exact denominators and explicit unclassified records outside supported scopes; the current artifacts do not establish a full-suite native percentage. Do not treat planner installation as execution.
Original audit context:
The upstream suite's green test total does not establish native coverage. At main aea6056352c402e0c7008f6f223c7574324d796a, native-execution.tsv covers ten Flink test methods plus three Delta methods; all other passing tests lack that per-invocation execution contract.
In runtime job https://github.com/datafusion-contrib/StreamFusion/actions/runs/35210877386/job/105167816104?pr=150, the summary reports 8,571 passed tests but only 28 contracted invocations (15 native, 13 expected fallback). Those counts are not an overall native percentage. Agent unit tests also print simulated evidence, so raw log matches must not be mixed with actual SQL invocations.
The agent intentionally retains stock Flink for batch and selected unmodified-plan fixtures. Ordinary planner fallback messages are not present in that console log, and NativeExecution ignores callbacks outside an active contracted method. The newer portable audit improves representative coverage but does not classify the entire upstream suite.
Acceptance:
- Record test identity and parameter/configuration variant, actual native operator work, and precise fallback reasons for upstream SQL invocations.
- Distinguish native/mixed/full fallback, batch host-only, deliberately unmodified plans, scan-only plans, skips and host failures.
- Keep simulated agent-unit-test evidence out of SQL totals.
- Retain complete machine-readable artifacts and fail on missing/unclassified evidence within each declared supported audit scope.
- Expand verified operator/type coverage incrementally and report the exact denominator; do not count installation alone as operator execution.
This extends the representative harness delivered for #106/#107 and the operator execution sampling. PR #31's installation check does not provide this route accounting. Related audit umbrella: #110.
Shipped in #205:
execution-audit.jsonnow retains every JUnit case, exact executed/contracted/unclassified denominators, validated operator-work counts and fallback reasons. Five additional unchanged Calc contracts pass (32 cases total, 8 required native records, 24 explicitly unclassified). Missing/stale evidence and process failures remain blocking. This artifact does not infer native coverage for uncontracted cases or match duplicate parameterized reports without an invocation ID.Shipped in #206 (merge
b94064cc, all 26 checks passed): six unchanged Table API UDF methods now have route contracts across both backend variants. The focused run passes all 12 invocations: six require native Calc work and six require the exactCalc: unsupported function/operator: ASfallback. The full Table API class passes 56 cases with one upstream skip; its other 44 executed cases remain explicitly unclassified. The alias-wrapper admission gap is still open under #110.Shipped in #277 (merge
6441e27b, all 49 checks passed): per-invocation operator work, submitted graphs, SQL translation links, job outcomes and shared invocation identities now establish native/mixed/fallback, batch host-only, scan-only, preserved-plan and confirmed host-failure routes. Versioned scopes fail closed on missing, stale, ambiguous or inconsistent evidence. Both released-line scopes passed hosted CI: Flink 2.2.1 has 18 passed invocations / 16 required variants; Flink 1.18.1 has 17 / 15. Two early returns per line remain explicitly unclassified, and simulated agent evidence stays outside SQL totals.Remaining work: expand these proven scopes across the remaining upstream SQL methods and parameter variants, add observation support where no authoritative SQL/job association or outcome can yet be established, and extend verified operator/type contracts. Preserve exact denominators and explicit unclassified records outside supported scopes; the current artifacts do not establish a full-suite native percentage. Do not treat planner installation as execution.
Original audit context:
The upstream suite's green test total does not establish native coverage. At main
aea6056352c402e0c7008f6f223c7574324d796a,native-execution.tsvcovers ten Flink test methods plus three Delta methods; all other passing tests lack that per-invocation execution contract.In runtime job https://github.com/datafusion-contrib/StreamFusion/actions/runs/35210877386/job/105167816104?pr=150, the summary reports 8,571 passed tests but only 28 contracted invocations (15 native, 13 expected fallback). Those counts are not an overall native percentage. Agent unit tests also print simulated evidence, so raw log matches must not be mixed with actual SQL invocations.
The agent intentionally retains stock Flink for batch and selected unmodified-plan fixtures. Ordinary planner fallback messages are not present in that console log, and NativeExecution ignores callbacks outside an active contracted method. The newer portable audit improves representative coverage but does not classify the entire upstream suite.
Acceptance:
This extends the representative harness delivered for #106/#107 and the operator execution sampling. PR #31's installation check does not provide this route accounting. Related audit umbrella: #110.