-
-
Notifications
You must be signed in to change notification settings - Fork 920
feat/rate limit implementation #742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sujal12344
wants to merge
7
commits into
VoltAgent:main
Choose a base branch
from
sujal12344:feat/rate-limit-implementation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
46ea0ce
feat: implement rate limiting to control external API requests
sujal12344 8de4af4
chore: add example usage for rate limiting
sujal12344 59575ae
feat: simplify rate limit checks and remove provider-specific configu…
sujal12344 486f11b
feat: implement multiple rate limiting strategies with hook support
sujal12344 a2682f9
refactor: streamline delay handling in FixedWindowCounterLimiter
sujal12344 295afe3
Merge branch 'main' into feat/rate-limit-implementation
sujal12344 1edd8d3
feat: add rate-limiting implementation in Agent class with hook support
sujal12344 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| GOOGLE_GENERATIVE_AI_API_KEY=your_api_key_here |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| node_modules | ||
| dist | ||
| .DS_Store | ||
| .voltagent | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # Rate Limiting Example | ||
|
|
||
| This example demonstrates VoltAgent's rate limiting feature to control the frequency of LLM calls and tool executions. | ||
|
|
||
| ## Features Demonstrated | ||
|
|
||
| 1. **Basic LLM Rate Limiting** - Limit requests per minute with error throwing | ||
| 2. **Delay Strategy** - Automatic waiting when limits are exceeded | ||
| 3. **Provider-Specific Limits** - Different limits for different LLM providers | ||
| 4. **Tool Rate Limiting** - Control tool execution frequency | ||
| 5. **Combined Limits** - Multiple rate limits working together | ||
| 6. **Monitoring Stats** - Track rate limit usage in real-time | ||
|
sujal12344 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| Set your OpenAI API key: | ||
|
sujal12344 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```bash | ||
| export GOOGLE_GENERATIVE_AI_API_KEY=your_api_key_here | ||
| ``` | ||
|
sujal12344 marked this conversation as resolved.
Outdated
sujal12344 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Running Examples | ||
|
|
||
| Edit `src/index.ts` and uncomment the examples you want to run in the `main()` function: | ||
|
|
||
| ```typescript | ||
| async function main() { | ||
| await example1_basicLLMRateLimit(); | ||
| await example2_delayStrategy(); | ||
| // ... etc | ||
| } | ||
| ``` | ||
|
|
||
| Then run: | ||
|
|
||
| ```bash | ||
| pnpm start | ||
| ``` | ||
|
|
||
| ## Rate Limit Configuration | ||
|
|
||
| ### LLM Rate Limiting | ||
|
|
||
| ```typescript | ||
| rateLimits: { | ||
|
sujal12344 marked this conversation as resolved.
Outdated
|
||
| llm: { | ||
| maxRequestsPerMinute: 10, | ||
| strategy: "fixed_window", | ||
| onExceeded: "throw" // or "delay" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Provider-Specific Limits | ||
|
|
||
| ```typescript | ||
| rateLimits: { | ||
| providers: { | ||
| openai: { | ||
| maxRequestsPerMinute: 5, | ||
| onExceeded: "throw" | ||
| }, | ||
| anthropic: { | ||
| maxRequestsPerMinute: 3, | ||
| onExceeded: "delay" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
sujal12344 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### Tool Rate Limiting | ||
|
|
||
| ```typescript | ||
| rateLimits: { | ||
| tools: { | ||
| search_tool: { | ||
| maxRequestsPerMinute: 3, | ||
| onExceeded: "delay" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Strategies | ||
|
|
||
| ### `onExceeded: "throw"` | ||
|
|
||
| - Immediately throws `RateLimitExceededError` when limit is reached | ||
| - Good for strict enforcement and error handling | ||
|
|
||
| ### `onExceeded: "delay"` | ||
|
|
||
| - Automatically waits until the rate limit resets | ||
| - Good for background jobs and retry scenarios | ||
|
|
||
| ## Use Cases | ||
|
|
||
| - **Cost Control**: Limit expensive LLM API calls | ||
| - **API Quota Management**: Stay within provider rate limits | ||
| - **Resource Protection**: Prevent tool overuse | ||
| - **Fair Usage**: Distribute resources across multiple agents | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| { | ||
| "name": "with-rate-limiting", | ||
| "version": "1.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "scripts": { | ||
| "start": "tsx src/index.ts", | ||
| "dev": "tsx watch src/index.ts", | ||
| "build": "tsc", | ||
| "run": "node dist/index.js" | ||
| }, | ||
| "dependencies": { | ||
| "@ai-sdk/google": "^2.0.23", | ||
| "@voltagent/core": "workspace:*", | ||
| "dotenv": "^17.2.3", | ||
| "zod": "^3.25.0" | ||
| }, | ||
| "devDependencies": { | ||
| "tsx": "^4.19.2", | ||
| "typescript": "^5.8.2" | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.