Skip to content

Add native Kev 0.8B decisions and validate local Laya inference - #59

Merged
elbruno merged 1 commit into
mainfrom
elbruno-local-decision-inference
Oct 22, 2026
Merged

elbruno merged 1 commit into
mainfrom
elbruno-local-decision-inference

Conversation

@elbruno

@elbruno elbruno commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #58.

  • Validate actual Laya Choice, Score and Assessment through the shared ElBruno.AI.Decisions.IDecisionClient, including contrastive cases, full probability invariants, local HTTP isolation, calibration diagnostics and a documented model-quality counterexample.
  • Add a separate KevOnnxDecisionClient and AddLocalKevDecisions() backend over explicitly acquired CPU FP32 ONNX assets. Laya stays the default; Ollama is only the model-discovery catalog, not a backend or conversion input.
  • Convert the pinned original Apache-2.0 Kev 0.8B adapter/base and trained pointer head with exact original temperature, strict context limits, manifest/provenance and mandatory PyTorch reference parity. Preserve exact free-text tokenization, semantic labels and all option probabilities.
  • Record Tev1's unresolved weight license and the remaining model families' distinct head/export contracts. No unexecuted model is advertised as supported.
  • Prepare coordinated NuGet release 0.24.0 and require ONNX Runtime 1.31.0 for Kev's linear-attention operator.

Verification

  • Re-ran the repository Kev converter end-to-end, then all three shared operations from free text in .NET. Maximum probability difference against original FP32 PyTorch: approximately 5.614e-6, below 1e-4.
  • Exact token/marker parity, Unicode/special-token cases, cancellation recovery, concurrent callers and no additional HTTP attempts during local loading/inference.
  • 99 decision tests passed with both live models enabled; 97 pass and two explicitly skip without opt-in.
  • All five publishing test suites passed locally: 1,947 tests total.
  • Solution Release build passed with the repository CI's existing -p:NoBuild=false override.
  • All five NuGet packages packed and passed package/assembly version validation for 0.24.0.
  • SharedDecisions ran on the converted Kev export, including decision routing.

Scope and limitations

Only Kev 0.8B text/CPU/FP32 is validated. Conversion uses Python once; application runtime needs neither Python nor a server. Weights are not included in Git or NuGet. See docs/kev-guide.md and docs/decision-model-validation.md. Tev1 remains unimplemented and blocked on its weight license; six synthetic cases are not an accuracy or calibration benchmark.

Convert the pinned original Kev checkpoint and trained pointer head to CPU ONNX, preserve reference tokenization and complete probabilities, and prepare coordinated 0.24.0 packages. Document Tev1 licensing and remaining decision-model candidates without an Ollama backend.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b4f53065-9fd2-4ae9-86c1-c5086fc1c8ae
@elbruno
elbruno merged commit b3c6adb into main Oct 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate local IDecisionClient inference with Laya and investigate Tev1 0.8B ONNX backend

1 participant