Repository navigation
fix(mpc): free models rank on ability, AA slugs match, Jev stops duplicating - #13
Merged
Merged
Conversation
|
Deployment failed for project cmdcode-tools-site with the following error: Learn More: https://vercel.com/jeffreys-projects-b51c8adf?upgradeToPro=build-rate-limit |
JeffreyJYZ
approved these changes
Nov 9, 2026
assignValueIndex short-circuited every unlimited model to valueIndex 100 and kept it out of the composite, so ability, tps, cache and output rates were discarded: Beam 1.0 (ability 17.7) outranked every paid model for being free. Free rows now enter the composite with the volume term at 1 (infinite requests cannot be beaten on volume) and are ranked with everything else. Beam reads 82, behind DeepSeek V4.1 Flash 100, Fast 91 and V4 Flash 84. COST is untouched: a free model still indexes 100 there, because that column is volume only and volume really is unbounded. Version 0.1.9.
AA writes slugs the catalogs do not: reflection-beam, tencent-hy4, mimo-v2-5-0424. normalizeKey kept the prefix, so those rows never matched and a scored model showed no ability (Hy4 34.6, Beam 1.0 17.7 were both sitting unread). Jev 1.14.0 is the same CommandCode build as Jev listed twice, so it aliases back and the row collapses instead of pairing two identical halves. Ability-missing rows 19 to 17; the rest are genuinely unscored by AA (crouton, glyph, iceman, longcat free tiers, ornith, qwen3.8 flash-next under all names).
JeffreyJYZ
approved these changes
Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four defects from the pasted table, all measured with
mpcdev --check --bench aaplus a direct dump of AA's free list (727 rows).1. VAL 100 for a weak free model
assignValueIndexshort-circuited every unlimited model tovalueIndex = 100and kept it out of the composite, discarding ability, tps, cache and output:Free rows now enter the composite with the volume term pinned at
1— infinite requests is unbeatable on volume and nothing else — and are ranked with everything else. Beam 1.0 (ability 17.7) went100 → 85, behind DeepSeek V4.1 Flash, Fast and V4 Flash.COST is untouched: that column is volume only, and a free model really is unbounded, so it still indexes 0.
2. AA slugs that never matched
AA keeps what the catalogs drop, and
normalizeKeykept it too:Beam 1.0's 17.7 in the table came from the CommandCode fill, not AA. Four aliases added; Hy4 now shows 34.6 and Beam resolves from AA.
3. Duplicate Jev row
JevandJev 1.14.0are the same CommandCode build listed twice with identical allowance and$/1K.jev1140 → jevcollapses them: 79 rows → 78.4. Unscored rows looked like a parser failure
ability missingfell from 19 to 17. The remainder are genuinely unscored by AA under every name — crouton, glyph, iceman, the LongCat free tier, ornith, Qwen 3.8 Flash (AA only publishesqwen3-8-flash-next, which is aliased and still null). The legend now says a free row wins volume only, soCOST 0beside a midVALreads as intended rather than broken.Gates
Version
0.1.9.