fix(checkworker): stop the rate limiter starving long-period checks

[fclairamb]

Problem

The prod watchdog reported two hourly SSL checks of one org showing "No data" for 27 hours while their region was healthy.

The gravelines worker logged Check execution rate-limited; deferring to next period (limit 10) at every hourly attempt. The org schedules about 10.5 checks/min against the SaaS cap of 10/min.

The fair rotation of spec 2026-08-26-02 keeps a deferred job's effective_scheduled_at, so it wins the next contended claim. That works for 1-minute checks (they all ran 1139 to 1426 times in 24h). An hourly check gets one attempt per hour, always at the same second. Those two land one second after a burst of 1-minute siblings has drained the bucket, so they lose every time.

Fix

• A job the gate already deferred (CheckJob.RateLimitDeferred: scheduled_at about a period ahead of effective_scheduled_at) may borrow one token.
• The bucket goes into debt, down to one burst below zero. The next on-time arrivals are deferred instead, so the deficit keeps rotating.
• Refill repays the debt, so the org's long-run rate stays at the cap.
• Both gates (in-process worker and agent claim) call the new entitlements.ReserveJobExecution.

Tests

• TestDeferredJobBorrowsSoLongPeriodChecksRun simulates 12 hours of the prod org second by second, on the real bucket, with the phases read from check_jobs.
  • Control without borrowing: the hourly check runs 0 times, and a 5-minute check runs 2 times out of 144.
  • With the fix: every check runs at least every other period (hourly 6/12, that 5-minute check 73/144). The total stays at or below the cap: 7206 runs against a ceiling of 7220.
• TestReserveDeferredCheckExecutionDebtIsBounded covers the debt floor and its repayment.
• TestCheckJobRateLimitDeferred covers the predicate: on time, cost offset, tier credit, deferred.

Under extreme overload (demand far above the cap) the debt floor can still be reached, and then the same arrival-time lottery applies. That is out of scope here.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.36%. Comparing base (2a8b86d) to head (b071a91).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #609      +/-   ##
==========================================
- Coverage   83.36%   83.36%   -0.01%     
==========================================
  Files        1238     1238              
  Lines      127020   127026       +6     
==========================================
+ Hits       105894   105896       +2     
- Misses      21117    21121       +4     
  Partials        9        9              

Files with missing lines	Coverage Δ
server/internal/checkworker/worker.go	90.46% <100.00%> (ø)
server/internal/db/models/check_job.go	100.00% <100.00%> (ø)
server/internal/entitlements/service.go	88.69% <100.00%> (+0.46%)	⬆️
server/internal/handlers/agentws/handler.go	83.39% <100.00%> (ø)

... and 8 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.