[Doc][Router] Clarify RequestStatsMonitor TTFT timing - #1053
Conversation
Signed-off-by: 姚远 <asmile.yy@gmail.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the docstrings and comments in request_stats.py to clarify that TTFT (Time To First Token) is calculated when the first response token is received. Feedback suggests adding a guard check in on_request_response to prevent duplicate calls for the same request from corrupting the statistics.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3fad59c6e1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: 姚远 <asmile.yy@gmail.com>
1 participant
Summary
RequestStatsMonitorrecords TTFT when the first response token is receivedget_request_stats()description of when TTFT is unavailableTesting
git diff --checkblack --check src/vllm_router/stats/request_stats.pyisort --check-only src/vllm_router/stats/request_stats.pycodespell --ignore-words .codespell-ignore src/vllm_router/stats/request_stats.py