flame-edge caches a null variant into the 30-day cookie and never re-resolves

[madmikeross]

flame-edge signs a "no variant" answer into the 30-day cookie and never re-asks.

What happens

assignVariant copies body.variant_id straight into the stored assignment:

// packages/flame-edge/src/api.ts:73-79
const body = (await res.json()) as AssignResponse;
return {
  experimentId: body.experiment_id,
  variantId: body.variant_id,   // typed `string`, can be null on the wire
  ...
};

AssignResponse declares variant_id: string (api.ts:18), but the API has
several 200 responses that name no variant — the caller is in a project's
holdout, or the workspace is at an event cap, or (new) the workspace has been
archived. All of them return 200 with every variant-shaped field null,
deliberately: the SDK must not fault, it must render the original experience.

resolveAssignments only discards entries whose fetch threw:

// packages/flame-edge/src/resolve.ts:118-129
} catch {
  // One bad experiment shouldn't kill the whole resolution.
  return null;
}
...
const assignments = Object.fromEntries(
  assignmentEntries.filter((x): x is NonNullable<typeof x> => x !== null)
);

A 200 with a null variant is not a throw, so { variantId: null } lands in
assignments, gets signed into the cookie (resolve.ts:138-143), and is
served with DEFAULT_COOKIE_TTL_SECONDS — 30 days.

Then step 1 of the next request short-circuits on it:

// packages/flame-edge/src/resolve.ts:84-90
const existing = cookies[cookieName];
if (existing) {
  const state = await verifyAndDecode(existing, opts.secret, { now: opts.now });
  if (state) {
    return { prehydrated: state };   // no check that the assignments name variants
  }
}

Nothing re-resolves until the cookie expires.

Why it matters

flame-react's provider degrades correctly — provider.tsx:51 looks the
variant up by id, finds nothing for null, and skips the experiment, so the
visitor sees the original experience. That is the intended graceful
degradation. The bug is that they are stuck there:

• Cap reached — the cap clears at midnight. The cookie lasts 30 days.
• Workspace archived — the API just added a third no-variant arm for an
  archived workspace, and an archive can stand for months before the owner
  revives it. Every edge-prehydrated visitor who cold-resolves during that
  window stays unenrolled for up to 30 days after the revive.
• Holdout — arguably the one case where caching is right, since a holdout
  carve-out is meant to be sticky.

So an experiment that is live and serving everyone else keeps rendering the
original experience for a slice of traffic, silently, with nothing in the
report to explain the gap.

Not a problem in the browser SDK

Worth contrasting, because it shows the intended shape. packages/flame
re-checks a stored assignment against the experiment's variants before
trusting it:

// packages/flame/src/assignment.ts:55-59
const variantExists = experiment.variants?.some((v) => v.id === existing.variantId);
if (variantExists) {
  return existing;
}
// Variant no longer exists, need new assignment

null never matches, so the browser SDK re-requests and self-heals. It costs a
redundant /assign per resolution, but no visitor gets stuck.

Repro

1. Point resolveAssignments at a project whose workspace is archived (or at
   one sitting on its event cap) with at least one running experiment.
2. Cold-resolve with no cookie. The response is 200; the cookie is set.
3. Revive the workspace (or wait for the daily cap to reset).
4. Send the same request with the cookie from step 2 — prehydrated still
   carries variantId: null and no /assign call is made. Holds until the
   cookie's Max-Age elapses.

Possible directions

Not prescribing one, but the options seem to be:

• Don't cache a non-answer. Drop entries with no variantId before
  building assignments, the way thrown fetches are already dropped. Simplest,
  and it makes the next request re-ask.
• Correct the type. variant_id: string | null on AssignResponse — the
  root cause is that the type says this cannot happen, so no code guards it.
  This makes the compiler point at every site that needs a decision.
• Shorten the TTL for a partial resolution, so a cookie that resolved
  nothing expires soon rather than in 30 days.
• Optionally read the discriminator flags the API sends alongside
  (held_out, cap_reached, workspace_archived) to tell a deliberate
  sticky carve-out apart from a temporary one. Note this is additive on the
  wire and none of them are read today.

Filed from the API side, where the third no-variant arm was just added.