Skip to content

fix(mpc): free models rank on ability, AA slugs match, Jev stops duplicating - #13

Merged
JeffreyJYZ merged 2 commits into
mainfrom
fix/free-val
Nov 9, 2026
Merged

JeffreyJYZ merged 2 commits into
mainfrom
fix/free-val

Conversation

@jyzbot

@jyzbot jyzbot commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Four defects from the pasted table, all measured with mpcdev --check --bench aa plus a direct dump of AA's free list (727 rows).

1. VAL 100 for a weak free model

assignValueIndex short-circuited every unlimited model to valueIndex = 100 and kept it out of the composite, discarding ability, tps, cache and output:

if (!Number.isFinite(m.requestsPerMonth)) { m.valueIndex = 100; continue; }

Free rows now enter the composite with the volume term pinned at 1 — infinite requests is unbeatable on volume and nothing else — and are ranked with everything else. Beam 1.0 (ability 17.7) went 100 → 85, behind DeepSeek V4.1 Flash, Fast and V4 Flash.

COST is untouched: that column is volume only, and a free model really is unbounded, so it still indexes 0.

2. AA slugs that never matched

AA keeps what the catalogs drop, and normalizeKey kept it too:

reflection-beam   → reflectionbeam  ≠ beam10  (AA index 17.7 unread)
tencent-hy4       → tencenthy4      ≠ hy4     (AA index 34.6 unread)
mimo-v2-5-0424    → mimov250424     ≠ mimov25 (AA index null, speed only)

Beam 1.0's 17.7 in the table came from the CommandCode fill, not AA. Four aliases added; Hy4 now shows 34.6 and Beam resolves from AA.

3. Duplicate Jev row

Jev and Jev 1.14.0 are the same CommandCode build listed twice with identical allowance and $/1K. jev1140 → jev collapses them: 79 rows → 78.

4. Unscored rows looked like a parser failure

ability missing fell from 19 to 17. The remainder are genuinely unscored by AA under every name — crouton, glyph, iceman, the LongCat free tier, ornith, Qwen 3.8 Flash (AA only publishes qwen3-8-flash-next, which is aliased and still null). The legend now says a free row wins volume only, so COST 0 beside a mid VAL reads as intended rather than broken.

Gates

bun check         No type errors in 89 files
bun test          114 pass, 0 fail  (+4 alias assertions)
bun fmt --check   All 95 files are formatted
biome lint        clean

Version 0.1.9.

@vercel

vercel Bot commented Nov 9, 2026

Copy link
Copy Markdown

Deployment failed for project cmdcode-tools-site with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/jeffreys-projects-b51c8adf?upgradeToPro=build-rate-limit

assignValueIndex short-circuited every unlimited model to valueIndex 100 and kept
it out of the composite, so ability, tps, cache and output rates were discarded:
Beam 1.0 (ability 17.7) outranked every paid model for being free.

Free rows now enter the composite with the volume term at 1 (infinite requests
cannot be beaten on volume) and are ranked with everything else. Beam reads 82,
behind DeepSeek V4.1 Flash 100, Fast 91 and V4 Flash 84.

COST is untouched: a free model still indexes 100 there, because that column is
volume only and volume really is unbounded.

Version 0.1.9.
AA writes slugs the catalogs do not: reflection-beam, tencent-hy4,
mimo-v2-5-0424. normalizeKey kept the prefix, so those rows never matched and a
scored model showed no ability (Hy4 34.6, Beam 1.0 17.7 were both sitting unread).

Jev 1.14.0 is the same CommandCode build as Jev listed twice, so it aliases back
and the row collapses instead of pairing two identical halves.

Ability-missing rows 19 to 17; the rest are genuinely unscored by AA (crouton,
glyph, iceman, longcat free tiers, ornith, qwen3.8 flash-next under all names).
@JeffreyJYZ
JeffreyJYZ merged commit d49d670 into main Nov 9, 2026
8 of 9 checks passed
@JeffreyJYZ
JeffreyJYZ deleted the fix/free-val branch November 9, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants