feat: add adaptive rate-limit tracking and backoff for metadata agent

[Copilot]

Summary

Adds comprehensive GitHub API quota tracking and adaptive retry behaviour for Reporting Agent v2 Phase 2.4, including independent core/GraphQL/search limit monitoring, exponential backoff with jitter, recovery estimation, and quota health reporting.

Changes Made

Added

• agents/metadata-agent/lib/rate-limit/rate-limit-types.js — Canonical rate-limit type definitions and defaults
• agents/metadata-agent/lib/rate-limit/rate-limit-tracker.js — Independent tracking for core, GraphQL, search quotas with header updates and recovery estimates
• agents/metadata-agent/lib/rate-limit/backoff-calculator.js — Adaptive exponential backoff with jitter and Retry-After header support
• agents/metadata-agent/lib/rate-limit/quota-monitor.js — Quota health states and global bottleneck reporting
• Comprehensive test suite under agents/metadata-agent/__tests__/rate-limit/

Changed

• agents/metadata-agent/lib/api/rate-limiter.js — Now extends RateLimitTracker, updates limits from GitHub responses while preserving threshold behaviour
• agents/metadata-agent/lib/api/retry-strategy.js — Delegates delay calculation to BackoffCalculator, adapts retries by rate-limit resource context
• agents/metadata-agent/__tests__/api/retry-strategy.test.js — Added adaptive rate-limit delay test coverage
• agents/metadata-agent/README.md — Updated with Phase 2.4 module documentation

Key Features

// Adaptive backoff based on rate-limit type and pressure
const delay = backoffCalculator.calculateDelay({
  attemptNumber: 1,
  error, // 429 or rate-limit message
  rateLimitType: "search",
  quotaRemainingPercent: 5,
});

// Quota health monitoring
const status = quotaMonitor.getStatus("search");
// Returns: { state: 'warning'|'critical'|'exhausted', percentage, recoveryInMs, ... }

// Global bottleneck detection
const global = quotaMonitor.getGlobalStatus();
// { bottleneck: 'search', core: {...}, graphql: {...}, search: {...} }

Fixed

• Numeric parsing edge case in quota normalisation where valid zero values could be overwritten
• Global status aggregation now avoids mutable in-place sort side effects
• Proper handling of nullable/undefined rate-limit headers

Testing

• 24+ unit tests for rate-limit tracking, backoff calculation, and quota monitoring
• Tests cover edge cases: zero values, missing headers, recovery estimates
• Adaptive delay testing with different quota pressure scenarios

Linked Issue

Resolves #2034

---

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

Rate limit normalisation currently coerces missing/non-numeric remaining values to 0 (and may clear reset), which can incorrectly mark quotas as exhausted and trigger unnecessary backoff.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces Phase 2.4 rate-limit infrastructure for the metadata agent, adding independent quota tracking (core/GraphQL/search) and an adaptive backoff strategy (including Retry-After support), then wiring that into the existing API rate limiter and retry strategy.

Changes:

• Added new rate-limit modules (RateLimitTracker, BackoffCalculator, QuotaMonitor, and canonical types/defaults).
• Updated retry delay logic to delegate to BackoffCalculator, with rate-limit resource awareness.
• Updated documentation and expanded test coverage for the new rate-limit behaviour.

File summaries

File	Description
agents/metadata-agent/README.md	Documents Phase 2.4 rate limit modules in the agent README.
agents/metadata-agent/lib/rate-limit/rate-limit-types.js	Defines canonical rate limit type constants plus defaults/thresholds.
agents/metadata-agent/lib/rate-limit/rate-limit-tracker.js	Implements per-resource quota tracking, header updates, bottleneck detection, and recovery estimates.
agents/metadata-agent/lib/rate-limit/quota-monitor.js	Provides quota health states and a global bottleneck summary derived from the tracker.
agents/metadata-agent/lib/rate-limit/backoff-calculator.js	Implements adaptive exponential backoff with jitter and Retry-After support.
agents/metadata-agent/lib/api/retry-strategy.js	Switches retry delays to use BackoffCalculator and detects rate-limit resource context.
agents/metadata-agent/lib/api/rate-limiter.js	Refactors the rate limiter to extend and reuse RateLimitTracker.
agents/metadata-agent/tests/rate-limit/rate-limit-tracker.test.js	Adds unit tests for tracker updates, quota recovery, and bottleneck detection.
agents/metadata-agent/tests/rate-limit/quota-monitor.test.js	Adds unit tests for quota health state classification and global bottleneck reporting.
agents/metadata-agent/tests/rate-limit/backoff-calculator.test.js	Adds unit tests for jitter, Retry-After precedence, and pressure-based delay adaptation.
agents/metadata-agent/tests/api/retry-strategy.test.js	Expands retry strategy tests to cover adaptive delay under search rate limit pressure.

Review details

• Files reviewed: 11/11 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

📄 README Validation

❌ One or more README checks failed.

Check	Result
❌ Frontmatter	Failed
✅ Structure	Passed

[github-actions]

🔍 Reviewer Summary for PR #2487

CI Status: ❌ pending
Files changed: 11

Recommendations

• CI checks not green

[github-actions]

🔍 Reviewer Summary for PR #2487

CI Status: ❌ pending
Files changed: 11

Recommendations

• CI checks not green

[github-actions]

⚠️ A merged PR reached develop without passing the template content guardrail.

Missing required section(s): Linked issues, Changelog, Global DoD checklist

This is a post-merge backstop for admin bypasses. Please review branch protection for develop.

[github-actions]

🔍 Reviewer Summary for PR #2487

CI Status: ❌ pending
Files changed: 11

Recommendations

• CI checks not green

[github-actions]

⚠️ Branch Naming Issue

This PR's branch name copilot/reporting-agent-v2-rate-limiting doesn't follow the standard pattern.

Standard pattern: {type}/{scope}-{title} (e.g., feat/something, fix/bug-name)

Based on linked issue type: task

Correct template should be: pr_feature.md

No action required — this is informational. Future PRs should use the correct branch name.

See Branching Strategy for more info.

[github-actions]

🔍 Reviewer Summary for PR #2487

CI Status: ❌ pending
Files changed: 11

Recommendations

• CI checks not green

[github-actions]

🔍 Reviewer Summary for PR #2487

CI Status: ❌ pending
Files changed: 11

Recommendations

• CI checks not green

[ashleyshaw]

Approved