Skip to content

antigravity interactions harness - improve TTFT (per-turn -> per chunk streaming) #314

Description

@joycel-github

Problem statement

The Antigravity Interactions harness's output granularity is per turn, coarser than the SDK harness (which is per-Content / block).

Code pointers

  • Buffer text deltas into one string:
    case "text":
    if textChunk, ok := delta["text"].(string); ok {
    // Text deltas are mostly incremental, but the server
    // periodically re-sends a full cumulative restatement of the
    // text so far (e.g. when finalizing a turn before a tool
    // call). Detect that case and replace rather than append, so
    // the restated text is not duplicated.
    if strings.HasPrefix(textChunk, turn.modelText) && turn.modelText != "" {
    turn.modelText = textChunk
    } else {
    turn.modelText += textChunk
    }
    }
  • Emit once per turn (emitText at the top of the Run loop):
    for turn := 0; turn < e.harness.cfg.MaxTurns; turn++ {
    e.harness.debugTurn(e.conversationID, turn+1, len(res.toolCalls))
    if err := emitText(ctx, handler, e.id, res.modelText); err != nil {
    return err
    }
  • emitText builds a single TextContent per turn:
    // emitText forwards non-empty model text to the handler as a Message.
    func emitText(ctx context.Context, handler harness.Handler, execID, text string) error {
    if strings.TrimSpace(text) == "" {
    return nil
    }
    return handler.OnMessage(ctx, execID, &proto.Message{
    Role: "assistant",
    Content: &proto.Content{
    Type: &proto.Content_Text{Text: &proto.TextContent{Text: text}},
    },
    })
    }

Desired behavior

  • Emit the first visible output as soon as the SSE stream produces it, rather than buffering the whole turn.
  • Stream per chunk / per Content instead of per turn to improve TTFT.
  • Keep final/durable content semantics correct.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions