ControlPlane.ai

Accenture Innovation Challenge 2026 — Team KQV

A sidecar reliability proxy for LLM APIs. Point your OpenAI-compatible client at ControlPlane instead of your provider directly, and it runs a three-tier, cheapest-first detection funnel on every request and response — blocking what's knowable before a response finishes reaching the client, and flagging in real time everything that's only knowable after the fact.

LLM provider: OpenRouter only, free-tier models. No paid API key is required anywhere in this stack. Every supporting tool — presidio, spaCy, pysbd, sentence-transformers, Redis — is free and open source, or free to self-host.

---

Table of contents

• Honest framing
• Architecture
• Setup
• Running
• Testing
• Demo script
• Known limitations
• Project structure
• Tech stack

---

Honest framing

ControlPlane does not claim to stop hallucinations or cost blowups mid-response — nothing can, because the tokens are already delivered by the time those signals resolve. What it genuinely stops:

• PII / unsafe content, mid-stream, before it reaches the client — chunk-buffered and overlap-safe, so a pattern straddling a chunk boundary is never split and missed.
• Tool-call loops, but only in SDK mode (a decorator wrapping your own tool functions). Proxy mode — just swapping your base URL — can only alert on loops after the fact, since it structurally sits beside the agent's loop, never inside it.

Everything else — confidence scoring, cost/latency anomalies, Tier 2 judge escalation — is real-time flagging, surfaced live on the dashboard, never a block.

---

Architecture

Client app --(own OpenRouter key, per-request header, never stored)-->
  ControlPlane FastAPI proxy
    --> forwards to OpenRouter (via litellm)
    --> sentence-boundary chunker (pysbd) + overlap-safe cap fallback
    --> Tier 1: sync PII/unsafe scan (presidio + spaCy + regex)
          -> pass / redact / block   (block ends the stream cleanly)
    --> proxy-mode tool-call loop alerting (passive, no halt)
    --> Tier 3: policy engine (tenant / provider / model fallback chain)
    --> single-writer queue --> SQLite (WAL mode)
    --> (async, post-response) Tier 2:
          confidence: logprob entropy (free) OR tail-checkpoint
            self-consistency (free, local embeddings) for non-logprob models
          cost / latency vs. warmed-up per-(tenant, provider, model) baseline
          judge escalation on ambiguous confidence scores
            (fail-closed = flag + tighten, NEVER block)
  HTMX dashboard: live feed, Tier-1-only % stat, drill-down, chaos mode

SDK mode (separate, in-process, NOT part of the proxy):
  HaltingToolGuard decorator + Redis-backed rolling-window loop detector
  -- wraps your own LangGraph (or any Python) tool functions directly,
  so it can genuinely halt execution mid-loop. Fails open if Redis is down.

---

Setup

python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

pip install -r requirements.txt
python -m spacy download en_core_web_sm   # free, local, one-time

cp .env.example .env
# fill in OPENROUTER_API_KEY  -- free key: https://openrouter.ai/keys
# confirm DEFAULT_MODEL is still free: https://openrouter.ai/models?max_price=0

python scripts/init_db.py

Optional — required only for SDK-mode loop detection and its chaos-mode scenario:

docker run -d -p 6379:6379 --name controlplane-redis redis:7-alpine

---

Running

uvicorn app.main:app --reload

Endpoint	Purpose
POST /v1/chat/completions	OpenAI-compatible proxy endpoint. Supports stream: true/false.
GET /dashboard	Live HTMX dashboard: feed, stats, drill-down, chaos mode.
GET /health	Health check.

Example streaming call:

curl -N http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Tenant-Id: your-tenant" \
  -d '{"messages": [{"role": "user", "content": "Say hello."}], "stream": true}'

---

Testing

pytest

The full suite runs offline against mocked LLM calls — no OpenRouter key or network access required to verify correctness.

---

Demo script

1. Open the dashboard, trigger Chaos Mode → PII Leak — watch it get blocked or redacted live, with a different fake value on every click.
2. Trigger Unsafe Content — always blocks.
3. Trigger Proxy-Mode Loop — alerts, never halts.
4. Trigger SDK-Mode Loop with Redis running — genuinely halts. Stop Redis and trigger again — it fails open instead of crashing.
5. Send a real ambiguous prompt through the proxy and watch a Tier 2 confidence or judge verdict appear in that request's drill-down view.
6. Point at the live % resolved at Tier 1 only stat as the closing line — the cost argument is a number on screen, not a claim in the pitch.

---

Known limitations

Stated up front, not left for a judge to find.

• Review workflow — alerts.reviewed is a flag, not a workflow. No assignment, resolution states, or audit trail.
• False-positive feedback loop — no mechanism to mark a verdict wrong and retrain or adjust thresholds from that signal.
• Self-consistency coverage — tail-checkpoint resampling catches drift at two sampled points (~40% / 75%), not exhaustive coverage of a full response. A directional signal, not a guarantee.
• SDK-mode halt requires framework-level integration — wrapping your own tool functions. It is not part of the base "swap your base URL" proxy pitch, and runs as a separate, standalone module (app/tiers/sdk_halt.py), outside the FastAPI proxy process.
• Cold-start baselines — no latency/cost verdict for a (tenant, provider, model) combination until 20+ samples exist. Baselines reset on server restart (in-memory, not persisted sample-by-sample).
• Single-writer queue is lock-based, not a literal asyncio.Queue plus worker task — functionally equivalent (strict serialization, FIFO among waiters) but simpler to reason about across test runs and server restarts.
• Ambiguity-tightening state is in-memory, per-process — doesn't survive a restart and doesn't share across multiple server workers.
• Latency measurement includes ControlPlane's own processing overhead, not just upstream LLM time — not split apart in this build.
• Free OpenRouter models often don't return logprobs — in practice, most responses use the self-consistency path, not the entropy path. This is expected, not a bug.

---

Project structure

app/
main.py                     FastAPI app, global exception handler
api/
chat.py                    proxy endpoint (streaming + non-streaming)
dashboard.py               dashboard, feed, drill-down, chaos trigger
core/
config.py                  settings (env-driven)
db.py                      SQLite schema + connections
repository.py              all DB reads/writes
writer_queue.py            single-writer serialization
tiers/
chunker.py                 sentence-boundary chunking, overlap-safe
scanner.py                 Tier 1 PII/unsafe detection
policy.py                  Tier 3 policy engine
loop_detector.py           proxy-mode loop alerting
sdk_loop_detector.py       SDK-mode loop detection (Redis-backed)
sdk_halt.py                SDK-mode halting decorator
confidence.py              logprob entropy + self-consistency scoring
judge.py                   Tier 2 judge escalation
ambiguity_state.py         tightening on judge timeout
baseline.py                latency baseline tracking
chaos.py                   chaos-mode scenarios
dashboard/templates/            HTMX/Jinja2 templates
tests/                           pytest suite, one file per module above
scripts/init_db.py               standalone DB init

---

Tech stack

Everything free — no paid API key anywhere in this stack.

Layer	Choice
Backend	FastAPI + httpx
LLM provider	OpenRouter (free-tier models), via litellm
Tier 2 judge model	Same free OpenRouter model, narrow single-question prompt
Self-consistency embeddings	sentence-transformers (all-MiniLM-L6-v2), local
Sentence boundaries	pysbd
PII detection	presidio + spaCy (en_core_web_sm)
SDK-mode loop state	Redis — free, local via Docker; fails open if unavailable
Database	SQLite, WAL mode
Frontend	HTMX + Jinja2 + Picnic CSS (CDN)