Skip to content

[Router][Bugfix] Keep the KV-aware /tokenize fallback off the event loop - #1101

Open
David-Wu1119 wants to merge 3 commits into
vllm-project:mainfrom
David-Wu1119:fix/kvaware-tokenize-nonblocking
Open

David-Wu1119 wants to merge 3 commits into
vllm-project:mainfrom
David-Wu1119:fix/kvaware-tokenize-nonblocking