Skip to content

[PoC] Support linking spans connected with Go chan#1727

Draft
MrAlias wants to merge 6 commits intoopen-telemetry:mainfrom
MrAlias:go-chan
Draft

[PoC] Support linking spans connected with Go chan#1727
MrAlias wants to merge 6 commits intoopen-telemetry:mainfrom
MrAlias:go-chan

Conversation

@MrAlias
Copy link
Copy Markdown
Contributor

@MrAlias MrAlias commented Apr 2, 2026

This PoC adds Go span linking for work handed between goroutines through a chan.

When OBI observes a matched channel send/receive, it captures the active OBI trace context on the sending and receiving goroutines in eBPF space and emits a dedicated channel-link event. User-space holds that link until the corresponding OBI spans are decoded, then exports the resolved spans with OTLP span links.

What Changed

  • Added Go runtime uprobes for channel handoff detection:
    • runtime.chansend1
    • runtime.chanrecv1
    • runtime.chanrecv2
  • Added eBPF-side handoff correlation for direct and buffered channels.
  • Added a dedicated EVENT_GO_CHANNEL_LINK ringbuf event for cross-goroutine handoffs.
  • Added user-space pending-link reconstruction keyed by trace_id + span_id.
  • Attached reconstructed links to resolved OBI spans before OTLP export.
  • Added a runnable example in examples/go-channel-links using docker compose, local OBI build, and Grafana LGTM.
  • Updated docs to describe the actual design and the example workflow.

Behavior

Given two separate goroutines with active OBI spans:

  • sender goroutine sends work through a Go chan
  • receiver goroutine receives that work
  • OBI exports the resulting spans as separate traces with reciprocal span links

In the example, this shows up as links between the OBI HTTP server spans for:

  • GET /dispatch
  • GET /receive

Known Limitations

  • This PoC covers direct runtime channel wrappers only:
    • runtime.chansend1
    • runtime.chanrecv1
    • runtime.chanrecv2
  • select-based channel paths are not covered yet; those go through runtime.selectgo
  • The current example intentionally uses a plain blocking receive (<-workCh) for that reason
  • This is focused on linking active OBI spans, not app-created manual Go SDK spans

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 2, 2026

Codecov Report

❌ Patch coverage is 40.79602% with 119 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.09%. Comparing base (6fc1474) to head (16c1f19).

Files with missing lines Patch % Lines
pkg/ebpf/common/go_channel_link_transform.go 55.29% 28 Missing and 10 partials ⚠️
examples/go-channel-links/app/main.go 0.00% 34 Missing ⚠️
pkg/internal/ebpf/gotracer/gotracer.go 36.58% 25 Missing and 1 partial ⚠️
pkg/ebpf/common/common.go 36.66% 19 Missing ⚠️
pkg/export/otel/tracesgen/tracesgen.go 80.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1727      +/-   ##
==========================================
+ Coverage   45.08%   45.09%   +0.01%     
==========================================
  Files         339      341       +2     
  Lines       37298    37484     +186     
==========================================
+ Hits        16814    16902      +88     
- Misses      19423    19509      +86     
- Partials     1061     1073      +12     
Flag Coverage Δ
integration-test 20.86% <29.16%> (+0.04%) ⬆️
integration-test-arm 0.00% <0.00%> (ø)
integration-test-vm-x86_64-5.15.152 0.00% <0.00%> (ø)
integration-test-vm-x86_64-6.10.6 0.00% <0.00%> (ø)
k8s-integration-test 2.10% <0.00%> (-0.02%) ⬇️
oats-test 0.00% <0.00%> (ø)
unittests 48.55% <41.01%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant