-
Notifications
You must be signed in to change notification settings - Fork 511
[Bugfix][Router] Log request headers as a lazy argument so they can be redacted #1097
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
chrikrah
wants to merge
2
commits into
vllm-project:main
Choose a base branch
from
chrikrah:fix/redact-headers-lazy-arg
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| """The session-extraction debug line must not defeat TokenRedactionFilter. | ||
|
|
||
| TokenRedactionFilter only inspects ``record.args`` for a Starlette ``Headers`` | ||
| object, so a call site that formats the headers into the message string leaves | ||
| nothing for it to redact. | ||
| """ | ||
|
|
||
| import logging | ||
|
|
||
| from starlette.datastructures import Headers | ||
|
|
||
| from vllm_router import log | ||
|
|
||
|
|
||
| def _record(msg, args): | ||
| return logging.LogRecord( | ||
| name="test", | ||
| level=logging.DEBUG, | ||
| pathname="test.py", | ||
| lineno=1, | ||
| msg=msg, | ||
| args=args, | ||
| exc_info=None, | ||
| ) | ||
|
|
||
|
|
||
| def test_formatted_headers_are_not_redacted_but_a_lazy_arg_is(): | ||
| redaction = log.TokenRedactionFilter() | ||
| headers = Headers( | ||
| {"authorization": "Bearer super-secret-token", "host": "example.invalid"} | ||
| ) | ||
|
|
||
| formatted = _record(f"Request headers: {dict(headers)}", None) | ||
| redaction.filter(formatted) | ||
| assert "super-secret-token" in formatted.getMessage(), ( | ||
| "a pre-formatted dict carries nothing in record.args, so the filter has " | ||
| "nothing to act on; this is what the call site must avoid" | ||
| ) | ||
|
|
||
| lazy = _record("Request headers: %s", (headers,)) | ||
| redaction.filter(lazy) | ||
| assert "super-secret-token" not in lazy.getMessage() | ||
| assert "Bearer ****" in lazy.getMessage() | ||
|
|
||
|
|
||
| def test_the_filter_leaves_headers_alone_when_nothing_is_sensitive(): | ||
| """Documented behaviour, pinned by test_filter_preserves_non_sensitive_headers. | ||
|
|
||
| The consequence for this call site is that the line reads as a dict when a | ||
| secret is present and as Headers({...}) when it is not. That is a format | ||
| difference rather than a leak, and changing it would change the filter's | ||
| own contract. | ||
| """ | ||
| redaction = log.TokenRedactionFilter() | ||
|
|
||
| plain = _record("Request headers: %s", (Headers({"host": "x"}),)) | ||
| redaction.filter(plain) | ||
| assert "Headers({'host': 'x'})" in plain.getMessage() | ||
|
|
||
| secret = _record( | ||
| "Request headers: %s", (Headers({"authorization": "Bearer s3cret"}),) | ||
| ) | ||
| redaction.filter(secret) | ||
| assert secret.getMessage() == "Request headers: {'authorization': 'Bearer ****'}" | ||
|
|
||
|
|
||
| def test_the_call_site_passes_headers_as_a_lazy_argument(): | ||
| """Read the call site as a syntax tree, not as text. | ||
|
|
||
| A string match on the source breaks on quote style or on a formatter | ||
| rewrapping the line. The AST answers the only question that matters: does | ||
| the ``logger.debug`` call for this message carry a second argument, or is | ||
| everything baked into the first one? | ||
| """ | ||
| import ast | ||
| import inspect | ||
|
|
||
| from vllm_router.services.request_service import request as request_module | ||
|
|
||
| tree = ast.parse(inspect.getsource(request_module.route_general_request)) | ||
|
|
||
| calls = [ | ||
| node | ||
| for node in ast.walk(tree) | ||
| if isinstance(node, ast.Call) | ||
| and isinstance(node.func, ast.Attribute) | ||
| and node.func.attr == "debug" | ||
| and node.args | ||
| and isinstance(node.args[0], ast.Constant) | ||
| and isinstance(node.args[0].value, str) | ||
| and "Request headers" in node.args[0].value | ||
| ] | ||
|
|
||
| assert len(calls) == 1, "expected exactly one headers debug call" | ||
| call = calls[0] | ||
|
|
||
| assert len(call.args) >= 2, ( | ||
| "the headers must be a separate argument; TokenRedactionFilter only " | ||
| "inspects record.args, so anything formatted into the message survives" | ||
| ) | ||
| assert "%s" in call.args[0].value | ||
|
|
||
|
|
||
| def test_the_call_site_logs_headers_as_a_lazy_argument(): | ||
| """Capture on the module's own logger, which does not propagate to root.""" | ||
| from vllm_router.services.request_service import request as request_module | ||
|
|
||
| captured: list[logging.LogRecord] = [] | ||
|
|
||
| class _Capture(logging.Handler): | ||
| def emit(self, record: logging.LogRecord) -> None: | ||
| captured.append(record) | ||
|
|
||
| handler = _Capture(level=logging.DEBUG) | ||
| logger = request_module.logger | ||
| previous_level = logger.level | ||
| logger.addHandler(handler) | ||
| logger.setLevel(logging.DEBUG) | ||
| try: | ||
| logger.debug( | ||
| "Debug session extraction - Request headers: %s", | ||
| Headers({"authorization": "Bearer super-secret-token"}), | ||
| ) | ||
| finally: | ||
| logger.removeHandler(handler) | ||
| logger.setLevel(previous_level) | ||
|
|
||
| record = next( | ||
| r for r in captured if "Debug session extraction - Request headers" in r.msg | ||
| ) | ||
| assert record.args, "the headers must arrive as a lazy argument" | ||
|
|
||
| log.TokenRedactionFilter().filter(record) | ||
| assert "super-secret-token" not in record.getMessage() | ||
| assert "Bearer ****" in record.getMessage() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Passing
request.headersdirectly as a lazy argument introduces an inconsistency in the log format depending on whether sensitive headers are present.\n\nThis happens becauseTokenRedactionFilter(insrc/vllm_router/log.py) only converts theHeadersobject to adictwhen a sensitive header is found and modified:\n- With sensitive headers: The filter redacts them and replaces the argument with a standard Pythondict. The log output formats as a dictionary:{'authorization': 'Bearer ****', 'host': '...'}.\n- Without sensitive headers: The filter does not modify the argument, leaving it as a StarletteHeadersobject. The log output formats using theHeaders__repr__:Headers(headers=[('host', '...')]).\n\nTo ensure consistent log formatting,TokenRedactionFiltershould be updated to always convertHeadersandMutableHeadersto adict(or always reconstruct them asHeaders/MutableHeadersif type preservation is desired) regardless of whether any sensitive headers were modified.