Skip to content

fix: restore ready=false for unresolved model in TRITONSERVER_ServerModelIsReady - #508

Merged
mc-nv merged 1 commit into
mainfrom
mchornyi/TRI-1529/model-is-ready-not-found-regression
Jul 8, 2026
Merged

fix: restore ready=false for unresolved model in TRITONSERVER_ServerModelIsReady#508
mc-nv merged 1 commit into
mainfrom
mchornyi/TRI-1529/model-is-ready-not-found-regression

Conversation

@mc-nv

@mc-nv mc-nv commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Description

TRITONSERVER_ServerModelIsReady regressed in #502: a model that was never
loaded, has been unloaded, or is unregistered now returns a NOT_FOUND error
instead of ready=false.

#502 refactored InferenceServer::ModelIsReady to take an already-resolved
Model& and moved the GetModel() lookup up into the C-API function behind
RETURN_IF_STATUS_ERROR. Before #502 the lookup failure was swallowed and the
call returned ready=false with a success status. The refactor turned that
"unknown model -> not ready" case into a hard error.

Impact -- every model_is_ready caller:

  • Python in-process API (test_binding.py::test_server_explicit regressed:
    assert not server.model_is_ready(<unloaded model>) raised NotFoundError
    instead of evaluating).
  • KServe /v2/models/{name}/ready HTTP/gRPC readiness endpoints.
  • C++ and other language bindings.

Changes

src/tritonserver.cc -- in TRITONSERVER_ServerModelIsReady, treat an
unresolvable model as not-ready (*ready = false, return success) instead of
surfacing the lookup error, while keeping #502's resolved-Model path for the
found case. This restores the pre-#502 public contract.

Test plan

  • Existing python/test/test_binding.py::TestBindings::test_server_explicit
    (unchanged) exercises this exact path via its
    assert not model_is_ready(<unloaded>) assertions -- it fails on main and
    is expected to pass with this change. Confirming on the L0_python_api job.

Follow-up

  • Consider adding C++/HTTP readiness coverage (not only the Python binding
    test) so this contract is guarded at the C-API level.

Note for reviewers

This branch temporarily includes the TRI-1527 wheel-packaging fix
(fix(TRI-1527): don't vendor libtritonserver stub into python wheel) so that
the L0_python_api suite can run at all -- without it the wheel crashes at
TRITONSERVER_ServerOptions() before reaching the readiness assertion. That
commit ships separately in #507; once #507 merges this branch will be rebased
on main and the build_wheel.py change will drop out, leaving only the
src/tritonserver.cc readiness fix.

Related Issues:

  • Resolves: TRI-1529

@mc-nv mc-nv self-assigned this Jul 2, 2026
@whoisj

whoisj commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Will approve if the changes are split into 2 PRs. Thanks!

…odelIsReady

#502 refactored ModelIsReady to take an already-resolved Model, moving the
GetModel() lookup into TRITONSERVER_ServerModelIsReady with
RETURN_IF_STATUS_ERROR. This changed the public contract: a model that was
never loaded, was unloaded, or is unregistered now returns a NOT_FOUND error
instead of ready=false. Before #502 the lookup failure was swallowed and the
call returned ready=false with a success status.

The regression surfaced as a failure in test_binding.py::test_server_explicit
(and affects every model_is_ready caller, incl. KServe /ready endpoints and the
Python in-process API). Treat an unresolvable model as not-ready again while
keeping #502's resolved-Model path for the found case.
@mc-nv
mc-nv force-pushed the mchornyi/TRI-1529/model-is-ready-not-found-regression branch from a6933ac to 25105e3 Compare July 6, 2026 19:00
@mc-nv
mc-nv requested review from whoisj and yinggeh July 7, 2026 15:36
@mc-nv
mc-nv marked this pull request as ready for review July 8, 2026 17:17
@mc-nv
mc-nv merged commit 2ff9f5e into main Jul 8, 2026
1 check passed
mc-nv added a commit that referenced this pull request Jul 8, 2026
…odelIsReady (#508) (#510)

#502 refactored ModelIsReady to take an already-resolved Model, moving the
GetModel() lookup into TRITONSERVER_ServerModelIsReady with
RETURN_IF_STATUS_ERROR. This changed the public contract: a model that was
never loaded, was unloaded, or is unregistered now returns a NOT_FOUND error
instead of ready=false. Before #502 the lookup failure was swallowed and the
call returned ready=false with a success status.

The regression surfaced as a failure in test_binding.py::test_server_explicit
(and affects every model_is_ready caller, incl. KServe /ready endpoints and the
Python in-process API). Treat an unresolvable model as not-ready again while
keeping #502's resolved-Model path for the found case.
@mc-nv
mc-nv deleted the mchornyi/TRI-1529/model-is-ready-not-found-regression branch July 8, 2026 18:13
@yinggeh

yinggeh commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

The change will discard genuine error from lserver->GetModel(model_name, model_version, &model). I will create a separate PR to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants