diff --git a/Directory.Build.props b/Directory.Build.props index 8bf1b7a..0ce91d1 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -23,7 +23,7 @@ Copyright © Bruno Capuano $([System.DateTime]::Now.Year) MIT nuget_logo.png - 0.22.0 + 0.23.0 + diff --git a/README.md b/README.md index ef56ed9..b4f7650 100644 --- a/README.md +++ b/README.md @@ -17,11 +17,11 @@ Run local LLMs in .NET through `IChatClient` — the same interface you'd use fo > The last 5 notable additions to the library. Updated with each NuGet release. -- 🎯 **`ElBruno.LocalLLMs.Decisions`** — new package for local **System One** decision models. Typed `choice`, `score` and yes/no answers with full probability distributions in a single forward pass (~600 ms on CPU), running [Laya](https://github.com/NandhaKishorM/laya) **fully in-process on ONNX Runtime** — no server, no Python. Built for routing, triage, moderation and intent detection, where a chat model is slow and overkill. Call `services.AddLocalDecisions()` and inject `IDecisionClient`. See the [Local Decisions Guide](docs/decisions-guide.md) and the [LocalDecisions sample](src/samples/LocalDecisions/). +- 🔌 **`ElBruno.AI.Decisions` provider** — `ElBruno.LocalLLMs.Decisions` now implements the shared [`ElBruno.AI.Decisions`](https://www.nuget.org/packages/ElBruno.AI.Decisions) `IDecisionClient`, so the in-process ONNX Laya model drops in next to the Foundry, Ollama and Jev providers and into `DecisionRoutingChatClient`. The rich API moved to `ILayaDecisionClient`. **Breaking** for code that injected the v0.22.0 `IDecisionClient`. See the [SharedDecisions sample](src/samples/SharedDecisions/). +- 🎯 **`ElBruno.LocalLLMs.Decisions`** — new package for local **System One** decision models. Typed `choice`, `score` and yes/no answers with full probability distributions in a single forward pass (~600 ms on CPU), running [Laya](https://github.com/NandhaKishorM/laya) **fully in-process on ONNX Runtime** — no server, no Python. Built for routing, triage, moderation and intent detection, where a chat model is slow and overkill. Call `services.AddLocalDecisions()` and inject `ILayaDecisionClient`. See the [Local Decisions Guide](docs/decisions-guide.md) and the [LocalDecisions sample](src/samples/LocalDecisions/). - ⬆️ **.NET 10 only** — every project now single-targets `net10.0`. .NET 8 reaches end of support in November 2026, so multi-targeting was dropped ahead of it. **Breaking:** consumers still on .NET 8 must stay on `v0.21.0` or upgrade to .NET 10. - 🧩 **`ElBruno.LocalLLMs.BlazorComponents`** — new Razor Class Library with 7 ready-to-use Blazor components: `ModelStatusCard` (download progress bar + actions), `ModelGallery` (filterable grid), `ModelSelector` (two-way-bindable dropdown), `ChatBox` (streaming token display), `EnvironmentDashboard` (CPU/CUDA/DirectML badges), `LocalLLMHealthBadge` (nav-bar status dot), and `RagPlayground`. Call `services.AddLocalLLMsBlazorComponents()` to register. See the [Blazor Components Guide](docs/blazor-components.md) and the [BlazorDemo sample](src/samples/BlazorDemo/). - 🧠 **GPT-OSS 20B support** — OpenAI's open-weight MoE model (Apache-2.0) now runs locally via the official `onnxruntime/gpt-oss-20b-onnx` artifacts. Adds the **Harmony** prompt format, channel-aware output filtering (chain-of-thought is stripped, never shown to users), Harmony tool calling, and a `ReasoningEffort` option. Two model IDs: `gpt-oss-20b` (CPU INT4) and `gpt-oss-20b-cuda`. See the [GptOssChat sample](src/samples/GptOssChat/). Also fixes a token-duplication bug that repeated the final token of every generation. -- 🚀 **`v0.20.12`** — Corrects sibling-package assembly versions, hardens vision token probing against model context limits, and verifies Fara smart image resizing for screenshot workflows. ## Features diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index c8a028f..17c6206 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- +## [0.23.0] - 2026-10-09 + +### Changed +- **BREAKING: `ElBruno.LocalLLMs.Decisions` is now a provider for the shared [`ElBruno.AI.Decisions`](https://www.nuget.org/packages/ElBruno.AI.Decisions) abstractions.** `LayaOnnxDecisionClient` implements `ElBruno.AI.Decisions.IDecisionClient` (`ChooseAsync`, `ScoreAsync`, `AssessAsync`), so it can replace or sit next to the Foundry, Ollama and Jev providers and plug into `DecisionRoutingChatClient`. The v0.22.0 `ElBruno.LocalLLMs.Decisions.IDecisionClient` is renamed `ILayaDecisionClient` and now extends the shared interface; it keeps `EvaluateAsync` batching and the `CalibrationClamped` results. `DecisionException` is now the shared type from `ElBruno.AI.Decisions`. Migration: inject `ILayaDecisionClient` instead of the old `IDecisionClient`, or inject `ElBruno.AI.Decisions.IDecisionClient` to stay provider-neutral. +- `AddLocalDecisions()` registers `ILayaDecisionClient` and the concrete client, and registers the shared `IDecisionClient` with `TryAdd` so an existing provider is not displaced. +- Neutral results clamp probabilities into `[0, 1]`, since the shared result types reject values slightly outside that range. + +### Added +- **`SharedDecisions` sample** — provider-neutral decisions plus `DecisionRoutingChatClient` routing. +- Dependency on `ElBruno.AI.Decisions` 0.6.2 (early access). + +--- + ## [0.22.0] - 2026-09-30 ### Changed diff --git a/docs/decisions-guide.md b/docs/decisions-guide.md index fa3744f..4e15f36 100644 --- a/docs/decisions-guide.md +++ b/docs/decisions-guide.md @@ -9,7 +9,7 @@ which is exactly where a chat model is the wrong tool. ## Decision model vs. chat model -| | Chat model (`IChatClient`) | Decision model (`IDecisionClient`) | +| | Chat model (`IChatClient`) | Decision model (`ILayaDecisionClient`) | |---|---|---| | Output | free text, token by token | one typed answer + probabilities | | Passes | autoregressive, one token at a time | a single forward pass | @@ -225,6 +225,35 @@ It is far too small to flip a routing decision, but large enough to matter if yo Batching is still the right default: four questions in one pass cost about what one costs. Just treat the probabilities as approximate, which the calibration caveats above already require. +## Using with ElBruno.AI.Decisions + +`LayaOnnxDecisionClient` is a provider for the shared [`ElBruno.AI.Decisions`](https://www.nuget.org/packages/ElBruno.AI.Decisions) abstractions, alongside its Foundry, Ollama and Jev providers. It is the in-process ONNX one: no server and no network call at inference time. + +```csharp +using ElBruno.AI.Decisions; + +builder.Services.AddLocalDecisions(); + +// Provider-neutral: swap the registration to change providers, nothing else changes. +IDecisionClient decisions = provider.GetRequiredService(); + +ChoiceDecision team = await decisions.ChooseAsync(ticket, "Which team should handle this?", teams); +AssessmentDecision angry = await decisions.AssessAsync(ticket, "The customer is frustrated."); +``` + +The same client also plugs into `DecisionRoutingChatClient`, which picks the `IChatClient` that answers each message. + +Two interfaces are registered, both resolving to the same singleton: + +| Inject | When | +|---|---| +| `ElBruno.AI.Decisions.IDecisionClient` | You want to stay provider-neutral. Registered with `TryAdd`, so the first provider registered keeps this slot. | +| `ILayaDecisionClient` | You want Laya extras: `EvaluateAsync` batching into one forward pass, `CalibrationClamped`, `ProbabilityResult.IsTrue`. It extends `IDecisionClient`. | + +The neutral results clamp every probability into `[0, 1]`, because the shared types reject values a few ulps outside that range. `CalibrationClamped` is only available on the rich results. + +`DecisionException` now comes from `ElBruno.AI.Decisions`. The shared package is early access (0.6.x), so expect its surface to move. + ## Dependency injection ```csharp @@ -235,7 +264,7 @@ builder.Services.AddLocalDecisions(options => }); ``` -Then inject `IDecisionClient`. The client is thread-safe and registered as a singleton, because +Then inject `ILayaDecisionClient` (or the provider-neutral `IDecisionClient`). The client is thread-safe and registered as a singleton, because loading the model is expensive and the loaded session is safe to share. Registration does not touch the network — the model is loaded lazily on the first call. @@ -251,11 +280,11 @@ touch the network — the model is loaded lazily on the first call. ## Testing -`IDecisionClient` is a plain interface, so unit tests can implement it directly with whatever +`IDecisionClient` and `ILayaDecisionClient` are plain interfaces, so unit tests can implement them directly with whatever distributions the test needs — no model download and no ONNX session: ```csharp -internal sealed class StubDecisionClient : IDecisionClient { /* ... */ } +internal sealed class StubDecisionClient : ILayaDecisionClient { /* ... */ } ``` ## Error handling diff --git a/docs/samples.md b/docs/samples.md index 80bf44d..5ad2575 100644 --- a/docs/samples.md +++ b/docs/samples.md @@ -566,6 +566,29 @@ for what to do about both. --- +## SharedDecisions: Provider-Neutral Decisions and Routing + +Uses the local Laya model through the shared `ElBruno.AI.Decisions` `IDecisionClient`, then hands the same client to `DecisionRoutingChatClient` so the decision model picks which `IChatClient` answers a message. + +```bash +dotnet run --project src/samples/SharedDecisions + +# or against a model directory you already have +dotnet run --project src/samples/SharedDecisions -- D:\models\laya +``` + +Expected output (probabilities vary slightly by CPU): + +```text +Team: billing (confidence 98.3%) +Frustrated: 85.3% +[billing agent] I'll look into the duplicate charge. +``` + +See the [Local Decisions Guide](decisions-guide.md#using-with-elbrunoaidecisions). + +--- + ## Next Steps - 📖 [Getting Started](getting-started.md) — full setup guide with GPU configuration diff --git a/src/ElBruno.LocalLLMs.Decisions/DecisionClientExtensions.cs b/src/ElBruno.LocalLLMs.Decisions/DecisionClientExtensions.cs index 67c23f0..6765b93 100644 --- a/src/ElBruno.LocalLLMs.Decisions/DecisionClientExtensions.cs +++ b/src/ElBruno.LocalLLMs.Decisions/DecisionClientExtensions.cs @@ -1,7 +1,7 @@ namespace ElBruno.LocalLLMs.Decisions; /// -/// Convenience overloads for . +/// Convenience overloads for . /// public static class DecisionClientExtensions { @@ -20,7 +20,7 @@ public static class DecisionClientExtensions /// "payment, invoice and refund problems". Prefer the dictionary overload when accuracy matters. /// public static Task ChooseAsync( - this IDecisionClient client, + this ILayaDecisionClient client, string state, IEnumerable options, string? instructions = null, @@ -43,10 +43,10 @@ public static Task ChooseAsync( /// when the probability meets the configured threshold. /// /// Discarding the probability discards the information you need to tune the threshold later. - /// Prefer unless you are certain a boolean is enough. + /// Prefer unless you are certain a boolean is enough. /// public static async Task IsTrueAsync( - this IDecisionClient client, + this ILayaDecisionClient client, string state, string proposition, CancellationToken cancellationToken = default) diff --git a/src/ElBruno.LocalLLMs.Decisions/DecisionException.cs b/src/ElBruno.LocalLLMs.Decisions/DecisionException.cs deleted file mode 100644 index 88b50b1..0000000 --- a/src/ElBruno.LocalLLMs.Decisions/DecisionException.cs +++ /dev/null @@ -1,16 +0,0 @@ -namespace ElBruno.LocalLLMs.Decisions; - -/// -/// Thrown when a local decision model cannot be reached or returns something unusable. -/// -public sealed class DecisionException : Exception -{ - /// Creates an exception with a message. - /// The error description. - public DecisionException(string message) : base(message) { } - - /// Creates an exception with a message and underlying cause. - /// The error description. - /// The underlying failure. - public DecisionException(string message, Exception innerException) : base(message, innerException) { } -} diff --git a/src/ElBruno.LocalLLMs.Decisions/DecisionServiceExtensions.cs b/src/ElBruno.LocalLLMs.Decisions/DecisionServiceExtensions.cs index e8799da..8194f8a 100644 --- a/src/ElBruno.LocalLLMs.Decisions/DecisionServiceExtensions.cs +++ b/src/ElBruno.LocalLLMs.Decisions/DecisionServiceExtensions.cs @@ -1,4 +1,5 @@ using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.DependencyInjection.Extensions; namespace ElBruno.LocalLLMs.Decisions; @@ -8,8 +9,8 @@ namespace ElBruno.LocalLLMs.Decisions; public static class DecisionServiceExtensions { /// - /// Registers an that runs a Laya decision model in-process - /// through ONNX Runtime. + /// Registers a Laya decision model that runs in-process through ONNX Runtime, as the shared + /// provider and as . /// /// The service collection. /// Optional action to configure the model source and threshold. @@ -18,6 +19,8 @@ public static class DecisionServiceExtensions /// The client is registered as a singleton because loading the model is expensive and the /// loaded session is safe to share. The model is downloaded lazily on first use, so /// registration does not block startup. + /// The shared uses TryAdd, like the other ElBruno.AI.Decisions providers, + /// so the first provider registered keeps that slot; is always available. /// /// /// @@ -39,7 +42,9 @@ public static IServiceCollection AddLocalDecisions( options.Validate(); services.AddSingleton(options); - services.AddSingleton(_ => new LayaOnnxDecisionClient(options)); + services.AddSingleton(_ => new LayaOnnxDecisionClient(options)); + services.AddSingleton(provider => provider.GetRequiredService()); + services.TryAddSingleton(provider => provider.GetRequiredService()); return services; } diff --git a/src/ElBruno.LocalLLMs.Decisions/ElBruno.LocalLLMs.Decisions.csproj b/src/ElBruno.LocalLLMs.Decisions/ElBruno.LocalLLMs.Decisions.csproj index aa91778..2f85685 100644 --- a/src/ElBruno.LocalLLMs.Decisions/ElBruno.LocalLLMs.Decisions.csproj +++ b/src/ElBruno.LocalLLMs.Decisions/ElBruno.LocalLLMs.Decisions.csproj @@ -14,6 +14,8 @@ + + diff --git a/src/ElBruno.LocalLLMs.Decisions/IDecisionClient.cs b/src/ElBruno.LocalLLMs.Decisions/ILayaDecisionClient.cs similarity index 87% rename from src/ElBruno.LocalLLMs.Decisions/IDecisionClient.cs rename to src/ElBruno.LocalLLMs.Decisions/ILayaDecisionClient.cs index b7cc5c1..bcf320f 100644 --- a/src/ElBruno.LocalLLMs.Decisions/IDecisionClient.cs +++ b/src/ElBruno.LocalLLMs.Decisions/ILayaDecisionClient.cs @@ -1,7 +1,8 @@ namespace ElBruno.LocalLLMs.Decisions; /// -/// Makes typed decisions about text using a local System One model. +/// The in-process Laya decision client: the provider-neutral from +/// ElBruno.AI.Decisions plus the extras that only a local model can offer. /// /// /// @@ -13,8 +14,12 @@ namespace ElBruno.LocalLLMs.Decisions; /// Use it for routing, triage, moderation gates, intent detection and similar classification work, /// and hand the result to a generative model when you actually need prose. /// +/// +/// Depend on when you want to swap providers (Foundry, Ollama, Jev). +/// Depend on this interface when you want request batching and CalibrationClamped. +/// /// -public interface IDecisionClient +public interface ILayaDecisionClient : IDecisionClient { /// /// Selects one of for the supplied text. diff --git a/src/ElBruno.LocalLLMs.Decisions/LayaOnnxDecisionClient.cs b/src/ElBruno.LocalLLMs.Decisions/LayaOnnxDecisionClient.cs index 0eaf7d3..284d51d 100644 --- a/src/ElBruno.LocalLLMs.Decisions/LayaOnnxDecisionClient.cs +++ b/src/ElBruno.LocalLLMs.Decisions/LayaOnnxDecisionClient.cs @@ -4,7 +4,7 @@ namespace ElBruno.LocalLLMs.Decisions; /// -/// An that runs a Laya decision model in-process through ONNX Runtime. +/// An provider that runs a Laya decision model in-process through ONNX Runtime. /// /// /// @@ -16,7 +16,7 @@ namespace ElBruno.LocalLLMs.Decisions; /// A single instance is safe to share and expensive to create, so register it as a singleton. /// /// -public sealed class LayaOnnxDecisionClient : IDecisionClient, IDisposable +public sealed class LayaOnnxDecisionClient : ILayaDecisionClient, IDisposable { private readonly DecisionOptions _options; private readonly SemaphoreSlim _initializationLock = new(1, 1); @@ -75,6 +75,41 @@ public async Task AskAsync( return result.Probability("ask"); } + /// + async Task IDecisionClient.ChooseAsync( + string input, + string instructions, + IReadOnlyDictionary options, + CancellationToken cancellationToken) + { + DecisionRequestValidation.Choice(input, instructions, options); + var result = await ChooseAsync(input, options, instructions, cancellationToken).ConfigureAwait(false); + return NeutralDecisionMapper.ToDecision(result); + } + + /// + async Task IDecisionClient.ScoreAsync( + string input, + string instructions, + IReadOnlyList rubric, + CancellationToken cancellationToken) + { + DecisionRequestValidation.Score(input, instructions, rubric); + var result = await ScoreAsync(input, rubric, instructions, cancellationToken).ConfigureAwait(false); + return NeutralDecisionMapper.ToDecision(result); + } + + /// + async Task IDecisionClient.AssessAsync( + string input, + string proposition, + CancellationToken cancellationToken) + { + DecisionRequestValidation.Assess(input, proposition); + var result = await AskAsync(input, proposition, cancellationToken).ConfigureAwait(false); + return NeutralDecisionMapper.ToDecision(result); + } + /// public async Task EvaluateAsync( DecisionRequest request, diff --git a/src/ElBruno.LocalLLMs.Decisions/NeutralDecisionMapper.cs b/src/ElBruno.LocalLLMs.Decisions/NeutralDecisionMapper.cs new file mode 100644 index 0000000..e493112 --- /dev/null +++ b/src/ElBruno.LocalLLMs.Decisions/NeutralDecisionMapper.cs @@ -0,0 +1,22 @@ +namespace ElBruno.LocalLLMs.Decisions; + +/// +/// Converts Laya's rich results into the provider-neutral ElBruno.AI.Decisions types. +/// +internal static class NeutralDecisionMapper +{ + internal static ChoiceDecision ToDecision(ChoiceResult result) => + new(result.Choice, ClampDistribution(result.Probabilities), Clamp(result.Confidence)); + + internal static ScoreDecision ToDecision(ScoreResult result) => + new(result.Probabilities.OrderBy(pair => pair.Key).Select(pair => Clamp(pair.Value)).ToArray()); + + internal static AssessmentDecision ToDecision(ProbabilityResult result) => + new(Clamp(result.Probability)); + + // Softmax output can land a few ulps outside [0,1]; the shared types reject that outright. + private static double Clamp(double value) => Math.Clamp(value, 0.0, 1.0); + + private static Dictionary ClampDistribution(IReadOnlyDictionary probabilities) => + probabilities.ToDictionary(pair => pair.Key, pair => Clamp(pair.Value), StringComparer.Ordinal); +} \ No newline at end of file diff --git a/src/samples/DecisionCalibration/Program.cs b/src/samples/DecisionCalibration/Program.cs index 6b773a8..4c1cf6e 100644 --- a/src/samples/DecisionCalibration/Program.cs +++ b/src/samples/DecisionCalibration/Program.cs @@ -28,7 +28,7 @@ // the logits by it sharpens the distribution roughly tenfold, which would report a // genuinely uncertain answer as near-certain. The client clamps it back to 0.5 and sets // CalibrationClamped so you can tell the difference. -static async Task ShowCalibrationClampAsync(IDecisionClient client) +static async Task ShowCalibrationClampAsync(ILayaDecisionClient client) { Console.WriteLine("=== 1. Calibration clamp (choice:11+) ===\n"); @@ -95,7 +95,7 @@ static void Report(string title, ChoiceResult result) // matrix multiplications accumulate in a different order. The difference is around 1e-4: // far too small to flip a routing decision, but large enough to break an exact-equality // assertion or a cache keyed on the probability value. -static async Task ShowBatchVarianceAsync(IDecisionClient client) +static async Task ShowBatchVarianceAsync(ILayaDecisionClient client) { Console.WriteLine("=== 2. fp16 batch variance ===\n"); diff --git a/src/samples/LocalDecisions/Program.cs b/src/samples/LocalDecisions/Program.cs index 9fd3af6..6ce70dc 100644 --- a/src/samples/LocalDecisions/Program.cs +++ b/src/samples/LocalDecisions/Program.cs @@ -1,3 +1,4 @@ +using ElBruno.AI.Decisions; using ElBruno.LocalLLMs.Decisions; // Triages a support ticket with a local System One decision model. diff --git a/src/samples/SharedDecisions/Program.cs b/src/samples/SharedDecisions/Program.cs new file mode 100644 index 0000000..7387b07 --- /dev/null +++ b/src/samples/SharedDecisions/Program.cs @@ -0,0 +1,65 @@ +using ElBruno.AI.Decisions; +using ElBruno.LocalLLMs.Decisions; +using Microsoft.Extensions.AI; +using Microsoft.Extensions.DependencyInjection; + +// Uses the local Laya model through the provider-neutral ElBruno.AI.Decisions API. +// +// The app below only knows IDecisionClient. Swap AddLocalDecisions for the Foundry, +// Ollama or Jev provider and nothing else changes. The same client also drives a +// DecisionRoutingChatClient that picks which IChatClient answers each message. +// +// Pass a local model directory as the first argument to skip the download. + +var services = new ServiceCollection(); +services.AddLocalDecisions(options => options.ModelPath = args.Length > 0 ? args[0] : null); +using var provider = services.BuildServiceProvider(); + +IDecisionClient decisions = provider.GetRequiredService(); + +var teams = new Dictionary +{ + ["billing"] = "Payment, invoice and refund problems", + ["technical"] = "Bugs, outages, API errors and anything with a stack trace", + ["sales"] = "Pricing, plan comparisons, upgrades and new purchases" +}; + +const string ticket = "My invoice charged me twice for the same subscription month and I want my money back."; + +ChoiceDecision team = await decisions.ChooseAsync(ticket, "Which team should handle this ticket?", teams); +Console.WriteLine($"Team: {team.Choice} (confidence {team.Confidence:P1})"); + +AssessmentDecision angry = await decisions.AssessAsync(ticket, "The customer is frustrated."); +Console.WriteLine($"Frustrated: {angry.Probability:P1}"); + +// Routing: the decision model picks the chat client, so only one generative model is invoked. +var routes = new Dictionary +{ + ["billing"] = new CannedChatClient("[billing agent] I'll look into the duplicate charge."), + ["technical"] = new CannedChatClient("[technical agent] Let's look at that stack trace."), + ["sales"] = new CannedChatClient("[sales agent] Happy to compare the plans.") +}; + +using var router = new DecisionRoutingChatClient( + decisions, + routes, + new DecisionRoutingOptions { Instructions = "Which team should handle this message?", Descriptions = teams }); + +ChatResponse reply = await router.GetResponseAsync(ticket); +Console.WriteLine(reply.Text); + +internal sealed class CannedChatClient(string text) : IChatClient +{ + public Task GetResponseAsync(IEnumerable messages, ChatOptions? options = null, CancellationToken cancellationToken = default) => + Task.FromResult(new ChatResponse(new ChatMessage(ChatRole.Assistant, text))); + + public async IAsyncEnumerable GetStreamingResponseAsync(IEnumerable messages, ChatOptions? options = null, [System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken = default) + { + yield return new ChatResponseUpdate(ChatRole.Assistant, text); + await Task.CompletedTask; + } + + public object? GetService(Type serviceType, object? serviceKey = null) => null; + + public void Dispose() { } +} \ No newline at end of file diff --git a/src/samples/SharedDecisions/SharedDecisions.csproj b/src/samples/SharedDecisions/SharedDecisions.csproj new file mode 100644 index 0000000..0f15ab0 --- /dev/null +++ b/src/samples/SharedDecisions/SharedDecisions.csproj @@ -0,0 +1,12 @@ + + + net10.0 + Exe + false + + + + + + + \ No newline at end of file diff --git a/src/tests/ElBruno.LocalLLMs.Decisions.Tests/DecisionOptionsTests.cs b/src/tests/ElBruno.LocalLLMs.Decisions.Tests/DecisionOptionsTests.cs index dbecedb..ce3e53f 100644 --- a/src/tests/ElBruno.LocalLLMs.Decisions.Tests/DecisionOptionsTests.cs +++ b/src/tests/ElBruno.LocalLLMs.Decisions.Tests/DecisionOptionsTests.cs @@ -86,8 +86,8 @@ public void AddLocalDecisions_RegistersASingletonClient() services.AddLocalDecisions(options => options.DecisionThreshold = 0.75); using ServiceProvider provider = services.BuildServiceProvider(); - var first = provider.GetRequiredService(); - var second = provider.GetRequiredService(); + var first = provider.GetRequiredService(); + var second = provider.GetRequiredService(); Assert.Same(first, second); Assert.IsType(first); diff --git a/src/tests/ElBruno.LocalLLMs.Decisions.Tests/ElBruno.LocalLLMs.Decisions.Tests.csproj b/src/tests/ElBruno.LocalLLMs.Decisions.Tests/ElBruno.LocalLLMs.Decisions.Tests.csproj index b3ae8a2..2e06d2b 100644 --- a/src/tests/ElBruno.LocalLLMs.Decisions.Tests/ElBruno.LocalLLMs.Decisions.Tests.csproj +++ b/src/tests/ElBruno.LocalLLMs.Decisions.Tests/ElBruno.LocalLLMs.Decisions.Tests.csproj @@ -5,6 +5,10 @@ true + + + + diff --git a/src/tests/ElBruno.LocalLLMs.Decisions.Tests/SharedInterfaceTests.cs b/src/tests/ElBruno.LocalLLMs.Decisions.Tests/SharedInterfaceTests.cs new file mode 100644 index 0000000..35cc1cb --- /dev/null +++ b/src/tests/ElBruno.LocalLLMs.Decisions.Tests/SharedInterfaceTests.cs @@ -0,0 +1,96 @@ +using Xunit; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.DependencyInjection.Extensions; + +namespace ElBruno.LocalLLMs.Decisions.Tests; + +public class NeutralDecisionMapperTests +{ + [Fact] + public void Choice_ClampsProbabilitiesOutsideUnitInterval() + { + var rich = new ChoiceResult( + "a", + new Dictionary { ["a"] = 1.0000000002, ["b"] = -1e-12 }, + 1.0000000002); + + ChoiceDecision decision = NeutralDecisionMapper.ToDecision(rich); + + Assert.Equal("a", decision.Choice); + Assert.Equal(1.0, decision.Probabilities["a"]); + Assert.Equal(0.0, decision.Probabilities["b"]); + Assert.Equal(1.0, decision.Confidence); + } + + [Fact] + public void Score_OrdersLevelsAndClamps() + { + var rich = new ScoreResult( + 1.5, + new Dictionary { [2] = 0.5, [1] = 0.5000000001 + 0.5, [3] = -1e-13 }, + new[] { "low", "mid", "high" }, + 0.9); + + ScoreDecision decision = NeutralDecisionMapper.ToDecision(rich); + + Assert.Equal(3, decision.LevelProbabilities.Count); + Assert.Equal(1.0, decision.LevelProbabilities[0]); + Assert.Equal(0.5, decision.LevelProbabilities[1]); + Assert.Equal(0.0, decision.LevelProbabilities[2]); + } + + [Fact] + public void Assessment_ClampsProbability() + { + AssessmentDecision decision = NeutralDecisionMapper.ToDecision(new ProbabilityResult(1.0000001, 0.5)); + + Assert.Equal(1.0, decision.Probability); + } +} + +public class SharedInterfaceRegistrationTests +{ + [Fact] + public void AddLocalDecisions_ResolvesSharedAndRichInterfacesToSameInstance() + { + var services = new ServiceCollection(); + services.AddLocalDecisions(); + + using ServiceProvider provider = services.BuildServiceProvider(); + + var shared = provider.GetRequiredService(); + Assert.Same(shared, provider.GetRequiredService()); + Assert.Same(shared, provider.GetRequiredService()); + } + + [Fact] + public void AddLocalDecisions_DoesNotReplaceAnExistingSharedProvider() + { + var existing = new StubClient(); + var services = new ServiceCollection(); + services.AddSingleton(existing); + services.AddLocalDecisions(); + + using ServiceProvider provider = services.BuildServiceProvider(); + + Assert.Same(existing, provider.GetRequiredService()); + Assert.IsType(provider.GetRequiredService()); + } + + [Fact] + public async Task SharedInterface_ValidatesBeforeLoadingTheModel() + { + IDecisionClient client = new LayaOnnxDecisionClient(new DecisionOptions()); + var options = new Dictionary { ["only"] = null }; + + await Assert.ThrowsAnyAsync(() => + client.ChooseAsync("text", "pick", options, CancellationToken.None)); + } + + private sealed class StubClient : IDecisionClient + { + public Task ChooseAsync(string input, string instructions, IReadOnlyDictionary options, CancellationToken cancellationToken) => throw new NotSupportedException(); + public Task ScoreAsync(string input, string instructions, IReadOnlyList rubric, CancellationToken cancellationToken) => throw new NotSupportedException(); + public Task AssessAsync(string input, string proposition, CancellationToken cancellationToken) => throw new NotSupportedException(); + } +} \ No newline at end of file