Repository navigation
fix(mpc): match AA's dated slugs, keep a last-good answer for quota days - #26
Merged
Merged
Conversation
AA scores DeepSeek under dated variant slugs -- deepseek-v4-1-flash-1017 (32.6), -1017-high (28.6), -1017-non-reasoning (18.5) -- while the catalogs name the model bare, so normalizeKey never matched and DeepSeek read as unscored. aaKeyVariants strips the bake date and the effort suffix, and only when a 3-4 digit date is present: without that anchor the effort rule would eat a real name token (Qwen3.8-Max -> qwen38). The free tier is 100 requests/day, and a spent quota used to drop the table to the partial page scrape, blanking a third of ability and speed mid-day. lastGood.ts now stores the last complete answer and replays it only when the live call throws, labelled with its age; an empty answer is never written, which is what killed the old TTL cache. Version 0.1.11.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
JeffreyJYZ
approved these changes
Nov 17, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You were right — AA does score DeepSeek, and mpc was throwing the scores away.
The mismatch
AA writes the bake date and the effort variant into its slugs; the catalogs name the model bare:
normalizeKey("deepseek-v4-1-flash-1017-high")→deepseekv41flash1017high, which never equalsdeepseekv41flash, so 45 rows read as unscored while AA had them.aaKeyVariants()now strips the bake date and then the effort suffix — only when a 3–4 digit date is present, because without that anchor the effort rule would eat a real name token (Qwen3.8-Max→qwen38). The exact key stays first.Quota days
The free tier is 100 requests/24 h. A spent quota used to drop the whole table onto the partial page scrape, so ability and speed vanished mid-day:
lastGood.tsstores the last complete answer and replays it only when the live call throws, labelledlast good, Nh old — API: …. An empty answer is never written — that is exactly what killed the old 7-day TTL cache.Gates
Version
0.1.11. Live proof of the ranking shift needs the quota to reset (429 right now), at which point the first successful run also populates the last-good file.