Skip to content

fix(client): preserve slow-starting managed services - #59246

Open
ggrabarz wants to merge 1 commit into
anomalyco:devfrom
ggrabarz:service-startup
Open

ggrabarz wants to merge 1 commit into
anomalyco:devfrom
ggrabarz:service-startup

Conversation

@ggrabarz

@ggrabarz ggrabarz commented Oct 26, 2026 •

Copy link
Copy Markdown

Issue

Related to #54810, #54883, and #59043.

What does this PR do?

  • Keeps a service contender spawned by ensure() alive through the existing startup deadline instead of evicting it after three short health-probe timeouts.
  • Prevents other clients from replacing that contender while it is still inside its shared startup window.
  • Records optional startedAt metadata in the managed-service registration so concurrent clients can make the same startup decision.
  • Keeps the new registration field optional for older service registrations.

This is scoped to the slow-starting managed-service retry loop. The separate live-but-busy incumbent case is handled by #59043.

How did you verify your code works?

  • bun test in packages/client: 235 passed, 4 skipped.
  • bun test test/service.test.ts test/service-lifecycle.test.ts in packages/cli: 26 passed, 10 skipped.
  • Client and CLI typechecks passed.
  • Root bun run check passed with Bun 1.4.4; lint reported existing warnings but no errors, and all 35 typecheck tasks passed.
  • The local compiled CLI was exercised through restart, authenticated health, and stop in an isolated state/data directory.

Visual

<svg width="240" height="72" viewBox="0 0 240 72" fill="none" xmlns="http://www.w3.org/2000/svg">
  <style>
    .pulse { animation: pulse 1.8s ease-in-out infinite; transform-origin: center; }
    @keyframes pulse { 0%,100% { opacity:.25; transform:scale(.85); } 50% { opacity:1; transform:scale(1); } }
    @media (prefers-reduced-motion: reduce) { .pulse { animation:none; opacity:.9; } }
  </style>
  <path d="M24 36h192" stroke="currentColor" stroke-opacity=".18" stroke-width="4" stroke-linecap="round"/>
  <circle cx="48" cy="36" r="8" fill="currentColor" opacity=".28"/>
  <circle class="pulse" cx="96" cy="36" r="8" fill="currentColor" style="animation-delay:-1.2s"/>
  <circle class="pulse" cx="144" cy="36" r="8" fill="currentColor" style="animation-delay:-.6s"/>
  <circle cx="192" cy="36" r="8" fill="currentColor" opacity=".28"/>
</svg>

Added by the opencode bot (AI-generated).

Related: #54810

Public API: adds optional startedAt to the client Info type/schema and the service registration file (backwards compatible)

@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 72 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The owned-contender part works, but the shared startedAt window blocks recovery from a service that crashes soon after it registers.

I checked this with the client and CLI service test suites (all pass) and tsc in both packages. I also ran a scratch test where the registration has a fresh startedAt but its process is dead. On the base, ensure() spawned a replacement in ~240 ms. With this PR, it never spawns and fails with "Timed out waiting for the background service to start" at the deadline. With the real timing, every client would wait 120 s and then error. register() runs as soon as the server starts listening, before the slow startup work described in #54810, so a crash in that phase is exactly the case this window catches. See the inline comment.

The updated "prior failure" tests also show a behaviour change: a hung contender this client spawned is no longer evicted. ensure() now waits out the whole deadline, reports the other contender's failure, and leaves the hung process registered. If that's intended, please call it out in the description.

The owned-contender guard is the same as the one in the open #54883 (same starting(pid) helper and the same two guards), so maintainers will likely want to choose between the two.

The reformatting of the rename/retry chain in service-registration.ts is unrelated and can be dropped.

Public API: adds an optional startedAt (epoch ms) to the client Info type, the Effect Info schema and the service.json registration file. This is backwards compatible: older registrations without it still decode, and the field is optional for readers.

Related: #54810

}

/** Keep a registered service's shared startup window open for every concurrent client. */
export function withinStartupWindow(info: Info, timing: EnsureTiming) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This window only looks at the clock, so it also covers registrations whose process is already dead. When the probe gets connection refused (not a timeout), starting is still true, so !starting && pool.shouldRecruit(...) never spawns, and every client fails with a timeout at the deadline. On the base, a replacement is spawned after one spawn delay. I reproduced this with a registration that has a fresh startedAt and an exited pid: the base recovered in ~240 ms, while this PR timed out. Since register() runs as soon as the server is listening, a crash during the slow startup phase hits this. Could the shared window apply only while the registered pid is alive (e.g. process.kill(pid, 0)), and only to the eviction-after-timeouts path rather than to recruitment?

if (Date.now() >= deadline)
throw pool.failure() ?? new Error("Timed out waiting for the background service to start")
const registration = await registered(options.file, timing.requestTimeout)
const starting =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

starting now also stops recruitment for any registration inside its window, including one owned by another client. Before, an unanswered registration got one spawnDelay before a lock-probe contender competed with it. So the PR now changes more than eviction: a registration that never answers (dead, or bound to a port it no longer serves) blocks every client for up to promiseTimeout. The same logic is in effect/service.ts.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs:compliance This means the issue will auto-close after 2 hours.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant