Skip to content

fix: send max_completion_tokens to GPT-5 and o-series models - #155

Open
xh123637 wants to merge 1 commit into
Optima-CityU:mainfrom
xh123637:fix/153-reasoning-model-sampling-fields
Open

xh123637 wants to merge 1 commit into
Optima-CityU:mainfrom
xh123637:fix/153-reasoning-model-sampling-fields

Conversation

@xh123637

@xh123637 xh123637 commented Nov 2, 2026

Copy link
Copy Markdown

Summary

Fixes #153. Sampler calls go out through OpenAICompatibleProvider.chat() / chat_stream(), which copied the caller's kwargs straight into the request. With an OpenAI reasoning model (gpt-5, gpt-5-mini, gpt-5.4, o3, …) every call was rejected with a 400 — Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead — so no algorithm was generated, and the provider connectivity test in the web UI failed the same way with a valid key.

Changes

In src/llm4ad/infra/provider/openai_compatible.py:

  • New _normalize_request_kwargs(), applied in both chat() and chat_stream() before the request is built.
  • For GPT-5-family and o-series models (the gpt-5-chat variants excluded — they are plain chat models), a caller's max_tokens is renamed to max_completion_tokens; an explicitly passed max_completion_tokens wins.
  • For the o-series and the original GPT-5 generation (gpt-5, gpt-5-mini, gpt-5-nano, including dated snapshots), which reject any temperature other than 1, a non-default temperature is dropped so the request does not fail on the next parameter. Later GPT-5.x models keep their temperature.
  • Every other model keeps the caller's request shape exactly as before.

Tests

Eleven new request-shape tests in tests/unit/provider/test_openai_compatible.py, using that file's mocked-client style:

  • gpt-5.4-mini chat: budget arrives as max_completion_tokens=4096, no max_tokens, temperature kept;
  • o3 / o4-mini / gpt-5 / gpt-5-mini / gpt-5-nano: budget renamed and non-default temperature dropped;
  • gpt-5 with temperature=1: kept (it is the models' own default);
  • guards: gpt-4o, gpt-5-chat-latest, and deepseek-chat pass max_tokens and temperature through unchanged;
  • chat_stream with gpt-5.4-mini: same normalization on the streaming path.

Verification: the provider test file goes from 24 to 35 passing; with the provider change reverted, 8 of the new tests fail (the 3 guards stay green by design). A full pytest run shows a failure set identical to the unmodified baseline (9 pre-existing failures in unrelated AST/evaluator/settings tests), with exactly the 11 new tests added to the pass count. ruff check and mypy are clean on the changed files.

1. Rename max_tokens to max_completion_tokens in OpenAICompatibleProvider for GPT-5 family and o-series models, which reject max_tokens on Chat Completions
2. Drop a non-default temperature for the o-series and the original gpt-5/gpt-5-mini/gpt-5-nano, which only accept the default temperature
3. Apply the normalization in both chat() and chat_stream() and add request-shape regression tests

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gpt-5 and o-series models rejected with 400: samplers always send max_tokens

1 participant