Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 29 additions & 31 deletions content/en/docs/zero-code/obi/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ cascade:
OTEL_RESOURCE_ATTRIBUTES_APPLICATION: obi
OTEL_RESOURCE_ATTRIBUTES_NAMESPACE: obi
OTEL_RESOURCE_ATTRIBUTES_POD: obi
cSpell:ignore: asyncio CAP_PERFMON uvloop
cSpell:ignore: CAP_PERFMON
---

OpenTelemetry libraries provide telemetry collection for popular programming
Expand Down Expand Up @@ -40,11 +40,13 @@ OBI offers the following features:
- **Visibility into encrypted communications**: Capture transactions over
TLS/SSL without decryption
- **Context propagation**: Propagate trace context across services automatically
- **Protocol support**: HTTP/S, gRPC, gRPC-Web, MQTT, Memcached, and more
- **Protocol support**: HTTP/S, gRPC, gRPC-Web, JSON-RPC, MQTT, Memcached, and
more
- **Database instrumentation**: PostgreSQL (including pgx driver), MySQL,
MongoDB, Redis, Couchbase (N1QL/SQL++ and KV protocol)
- **GenAI instrumentation**: Trace and metrics for OpenAI and Anthropic Claude
API calls with automatic payload extraction
- **GenAI instrumentation**: Trace and metrics for OpenAI, Anthropic Claude,
Google AI Studio (Gemini), and AWS Bedrock API calls with automatic payload
extraction
- **Low cardinality metrics**: Prometheus-compatible metrics with low
cardinality for cost reduction
- **Network observability**: Capture network flows between services with
Expand All @@ -54,36 +56,32 @@ OBI offers the following features:
- **Collector integration**: Run OBI as an OpenTelemetry Collector receiver
component

## Recent highlights (v0.7.0)

OBI v0.7.0 introduces several significant improvements:

- **StatsO11y**: New statistical metrics pipeline for host-level network
statistics, starting with TCP RTT metrics
- **Expanded protocol coverage**: Added Memcached protocol tracing support
- **Enhanced GenAI instrumentation**: Added support for Anthropic Claude with
automatic payload extraction
- **Python asyncio improvements**: Added context propagation support for Python
asyncio workloads using `uvloop`
- **New example scenario**: Added an NGINX example covering direct routing,
reverse proxying, and route-based telemetry across Docker, Kubernetes, and
standalone deployments
- **Prometheus exemplars**: Support for exemplars in metrics export for better
correlation with traces
- **New span types**: SQL instrumentation now emits server spans for database
calls
- **Operational controls**: Added configurable `log_format` and Kubernetes API
reconnect interval settings
- **Network diagnostics**: Added `obi_bpf_network_ignored_packets_total` for
troubleshooting dropped network packets
- **Release artifacts**: CycloneDX SBOMs now included in release packages for
supply chain security
## Recent highlights (v0.8.0)

OBI v0.8.0 expands protocol coverage, payload extraction, and deployment
documentation:

- **Generic Go tracing improvements**: Added generic Go protocol support,
including context propagation for generic requests
- **Expanded protocol coverage**: Added JSON-RPC support across all languages
- **Deeper HTTP payload extraction**: Added full HTTP body extraction, with
bounded decompression for response bodies
- **Broader GenAI coverage**: Added Google AI Studio (Gemini) and AWS Bedrock
payload extraction, and fixed Vertex AI Gemini support
- **Named CIDR labels**: Network metrics can now label configured CIDR ranges
with human-readable names
- **New example scenario**: Added an Apache HTTP Server example alongside the
existing NGINX walkthroughs
- **Support documentation**: Added a project support matrix for release
artifacts and supported environments

For a complete list of changes and upgrade notes, see the
[release notes](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.7.0).
[release notes](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.8.0).

If you want to explore the new NGINX example, see the
[example walkthrough](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/v0.7.0/examples/nginx).
If you want to explore the upstream examples, see the
[NGINX walkthrough](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/v0.8.0/examples/nginx)
and the
[Apache walkthrough](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/v0.8.0/examples/apache).

## Requirements

Expand Down
6 changes: 3 additions & 3 deletions content/en/docs/zero-code/obi/configure/export-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ The list of instrumentation areas OBI can collect data from:
- `mqtt`: MQTT publish/subscribe message metrics (MQTT 3.1.1 and 5.0)
- `couchbase`: Couchbase N1QL/SQL++ query metrics and KV (Key-Value) protocol
metrics based on memcached protocol
- `genai`: GenAI client metrics (OpenAI and Anthropic)
- `genai`: GenAI client metrics (OpenAI, Anthropic, Gemini, and AWS Bedrock)
- `gpu`: GPU performance metrics
- `mongo`: MongoDB client call metrics
- `dns`: DNS query metrics
Expand Down Expand Up @@ -218,7 +218,7 @@ The list of instrumentation areas OBI can collect data from:
- `mqtt`: MQTT publish/subscribe message traces (MQTT 3.1.1 and 5.0)
- `couchbase`: Couchbase N1QL/SQL++ query traces and KV (Key-Value) protocol
traces, with query text and operation details
- `genai`: GenAI client traces (OpenAI and Anthropic)
- `genai`: GenAI client traces (OpenAI, Anthropic, Gemini, and AWS Bedrock)
- `gpu`: GPU performance traces
- `mongo`: MongoDB client call traces
- `dns`: DNS query traces
Expand Down Expand Up @@ -472,7 +472,7 @@ The list of instrumentation areas OBI can collection data from:
- `kafka`: Kafka client/server message queue metrics
- `mqtt`: MQTT publish/subscribe message metrics
- `couchbase`: Couchbase N1QL/SQL++ query metrics and KV protocol metrics
- `genai`: GenAI client metrics (OpenAI and Anthropic)
- `genai`: GenAI client metrics (OpenAI, Anthropic, Gemini, and AWS Bedrock)

For example, setting the `instrumentations` option to: `http,grpc` enables the
collection of `HTTP/HTTPS/HTTP2` and `gRPC` application metrics, and disables
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,10 @@ YAML section: `ebpf`
You can configure the component under the `ebpf` section of your YAML
configuration or via environment variables.

| YAML<br>environment variable | Description | Type | Default |
| ---------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -------- |
| `context_propagation`<br>`OTEL_EBPF_BPF_CONTEXT_PROPAGATION` | Controls trace context propagation method. Accepted: `all`, `headers`, `ip`, `disabled`. For more information, refer to the [context propagation section](#context-propagation). | string | disabled |
| `track_request_headers`<br>`OTEL_EBPF_BPF_TRACK_REQUEST_HEADERS` | Track incoming `Traceparent` headers for trace spans. For more information, refer to the [track request headers section](#track-request-headers). | boolean | false |
| YAML<br>environment variable | Description | Type | Default |
| ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | -------- |
| `context_propagation`<br>`OTEL_EBPF_BPF_CONTEXT_PROPAGATION` | Controls trace context propagation method. Accepted: `all`, `headers`, `tcp`, `headers,tcp`, `disabled`. For more information, refer to the [context propagation section](#context-propagation). | string | disabled |
| `track_request_headers`<br>`OTEL_EBPF_BPF_TRACK_REQUEST_HEADERS` | Track incoming `Traceparent` headers for trace spans. For more information, refer to the [track request headers section](#track-request-headers). | boolean | false |

### Context propagation

Expand All @@ -102,17 +102,21 @@ that also use TC must chain correctly with OBI. For more information about
chaining programs, see the
[Cilium compatibility documentation](../../cilium-compatibility/).

You can disable the TCP/IP level encoding and TC programs by setting
`context_propagation="headers"`. This context propagation is fully compatible
with any OpenTelemetry distributed tracing library.
You can disable the TCP-level propagation and Linux Traffic Control programs by
setting `context_propagation="headers"`. This mode is fully compatible with any
OpenTelemetry distributed tracing library.

Context propagation values:

- `all`: Enable both HTTP and IP options context propagation
- `all`: Enable both HTTP header and TCP context propagation
- `headers`: Enable context propagation via the HTTP headers only
- `ip`: Enable context propagation via the IP options field only
- `tcp`: Enable context propagation via the TCP packet path only
- `headers,tcp`: Enable both methods explicitly
- `disabled`: Disable trace context propagation

`http` is accepted as an alias for `headers`, but `headers` is the preferred
name in examples and configuration.

To use this option in containerized environments (Kubernetes and Docker), you
must:

Expand All @@ -121,7 +125,7 @@ must:
path
- Grant the `CAP_NET_ADMIN` capability to the OBI container

gRPC and HTTP/2 are not supported.
gRPC and HTTP/2 are not supported for this network-level mode.

For an example of how to configure distributed traces in Kubernetes, see our
[Distributed traces with OBI](../../distributed-traces/) guide.
Expand Down Expand Up @@ -154,22 +158,24 @@ because it can create false positives, for example, if an application sends SQL
text for logging through a TCP connection. Currently, OBI natively supports the
PostgreSQL and MySQL binary protocols.

### HTTP header enrichment for spans
<a id="http-header-enrichment-for-spans"></a>
Comment thread
MrAlias marked this conversation as resolved.
Outdated

### HTTP header and body enrichment for spans

OBI can attach selected HTTP headers to spans through the
`ebpf.payload_extraction.http.enrichment` configuration section. This is useful
when you want to carry business or routing headers into traces without manually
instrumenting the application.
OBI can attach selected HTTP headers and selected HTTP body fields to spans
through the `ebpf.payload_extraction.http.enrichment` configuration section.
This is useful when you want to carry business or routing headers into traces
without manually instrumenting the application.

The enrichment engine is rule-based:

- Set `enabled: true` to activate HTTP header enrichment.
- Use `policy.default_action` to define whether unmatched headers are included
or excluded. The default is `exclude`.
- Use `policy.match_order` to control rule evaluation. The default is
`first_match_wins`.
- Use `obfuscate` rules to redact sensitive values while still exposing the
header key on the span.
- Set `enabled: true` to activate HTTP header and body enrichment.
- Use `policy.default_action.headers` and `policy.default_action.body` in YAML
to define whether unmatched headers or body content are included or excluded.
The default for both is `exclude`.
- Use `obfuscate` rules to redact sensitive header values or JSON body fields
before they are attached to spans.
- Rules are evaluated in order.

For example:

Expand All @@ -182,8 +188,9 @@ ebpf:
enrichment:
enabled: true
policy:
default_action: exclude
match_order: first_match_wins
default_action:
headers: exclude
body: exclude
obfuscation_string: '***'
rules:
- action: obfuscate
Expand All @@ -202,17 +209,40 @@ ebpf:
- X-Custom-*
- X-Dice-Roll
case_sensitive: false
- action: include
type: body
scope: request
match:
methods: [POST]
url_path_patterns:
- /v1/chat/completions
- action: obfuscate
type: body
scope: request
match:
methods: [POST]
url_path_patterns:
- /v1/chat/completions
obfuscation_json_paths:
- $.messages[*].content
```

The following environment variables control the policy defaults:
The following environment variables control the global enrichment behavior:

- `OTEL_EBPF_HTTP_ENRICHMENT_ENABLED`
- `OTEL_EBPF_HTTP_ENRICHMENT_DEFAULT_ACTION`
- `OTEL_EBPF_HTTP_ENRICHMENT_MATCH_ORDER`
- `OTEL_EBPF_HTTP_ENRICHMENT_OBFUSCATION_STRING`

Comment thread
MrAlias marked this conversation as resolved.
Rules themselves are configured in YAML. If you expect large headers, increase
`ebpf.buffer_sizes.http` so OBI can capture the relevant values.
The `policy.default_action.headers` and `policy.default_action.body` settings
are configured in YAML only; there are no environment variables for these
defaults.

Rules themselves are configured in YAML. Header rules use `match.patterns` and
optional `case_sensitive`. Body rules use `match.url_path_patterns`,
`match.methods`, and `match.obfuscation_json_paths`.

Body extraction requires HTTP payload capture. Increase `ebpf.buffer_sizes.http`
so OBI can capture the request or response bytes you want to enrich. The limit
applies independently to requests and responses.

## Instance ID decoration

Expand Down
3 changes: 3 additions & 0 deletions content/en/docs/zero-code/obi/distributed-traces.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ from OpenTelemetry SDK instrumented services still works.

gRPC and HTTP/2 are not supported at the moment.

If you need finer control, `context_propagation` also accepts `headers`, `tcp`,
and `headers,tcp`. `http` is accepted as an alias for `headers`.

This type of context propagation works for any programming language and doesn't
require that OBI runs in `privileged` mode or has `CAP_SYS_ADMIN` granted. For
more details, see the
Expand Down
19 changes: 14 additions & 5 deletions content/en/docs/zero-code/obi/network/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,9 @@ socket filter to capture the network events. This mode doesn't conflict with
Cilium CNI or other eBPF programs, which use the Linux Traffic Control egress
and ingress filters.

| YAML | Environment variable | Type | Default |
| ------- | ------------------------- | -------- | ------- |
| `cidrs` | `OTEL_EBPF_NETWORK_CIDRS` | []string | (empty) |
| YAML | Environment variable | Type | Default |
| ------- | ------------------------- | -------------------- | ------- |
| `cidrs` | `OTEL_EBPF_NETWORK_CIDRS` | list of CIDR strings | (empty) |

CIDRs list, to be set as the `src.cidr` and `dst.cidr` attribute with the entry
that matches the `src.address` and `dst.address` respectively.
Expand All @@ -93,8 +93,17 @@ address matches multiple CIDR definitions, the flow is decorated with the
narrowest CIDR. As a result, you can safely add a `0.0.0.0/0` entry to group all
the traffic that does not match any of the other CIDRs.

If you set this property via environment variable each entry must be separated
by a comma, for example:
In YAML, each entry can be either a plain CIDR string or an object with `cidr`
and `name` fields. The `OTEL_EBPF_NETWORK_CIDRS` environment variable accepts a
comma-separated list of CIDR strings only. For example:

```yaml
network:
cidrs:
- cidr: 10.0.0.0/8
name: cluster-internal
- 192.168.0.0/16
```

```sh
OTEL_EBPF_NETWORK_CIDRS=10.0.0.0/8,192.168.0.0/16
Expand Down
4 changes: 2 additions & 2 deletions content/en/docs/zero-code/obi/setup/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ You can verify the signature of the container image using the following
commands:

```sh
export VERSION=v0.7.0
export VERSION=v0.8.0

# Verify a release image from Docker Hub
cosign verify --certificate-identity-regexp 'https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/' --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' otel/ebpf-instrument:${VERSION}
Expand Down Expand Up @@ -80,7 +80,7 @@ don't have one, you can use this
[simple blog engine service written in Go](https://macias.info):

```sh
export VERSION=v0.7.0
export VERSION=v0.8.0
docker run -p 18443:8443 --name goblog mariomac/goblog:dev
```

Expand Down
14 changes: 13 additions & 1 deletion content/en/docs/zero-code/obi/trace-log-correlation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ weight: 35
description:
Learn how OBI correlates application logs with distributed traces for faster
debugging and troubleshooting.
cSpell:ignore: BPFFS ringbuffer
cSpell:ignore: BPFFS PYTHONUNBUFFERED ringbuffer
---

OpenTelemetry eBPF Instrumentation (OBI) correlates application logs with
Expand Down Expand Up @@ -119,6 +119,18 @@ and `span_id` fields into JSON log objects:
Plain text logs are passed through unchanged and are **not enriched** with trace
context.

#### Runtime buffering limitations

The log enricher only sees trace context when the log write happens on the
request-handling thread. Runtimes that buffer stdout asynchronously can break
this assumption.

- Python in Docker commonly needs `PYTHONUNBUFFERED=1`
- .NET `Console.Out` is buffered by default when stdout is a pipe; use a
`StreamWriter` with `AutoFlush = true`
- ASP.NET Core's default `Microsoft.Extensions.Logging.AddConsole()` pipeline is
not compatible because it writes from a background thread

### 2. Trace export and log enrichment enabled

Traces must be exported and log enrichment enabled:
Expand Down
12 changes: 12 additions & 0 deletions static/refcache.json
Original file line number Diff line number Diff line change
Expand Up @@ -11407,6 +11407,10 @@
"StatusCode": 206,
"LastSeen": "2026-04-03T18:40:19.553351857Z"
},
"https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.8.0": {
"StatusCode": 206,
"LastSeen": "2026-04-17T18:11:47.603377278Z"
},
"https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/2feaeb84ef7aeeb1e57e642b46c96b965ae4b804/internal/test/integration/k8s/manifests?from_branch=main": {
"StatusCode": 206,
"LastSeen": "2026-03-17T09:54:42.999934807Z"
Expand All @@ -11423,6 +11427,14 @@
"StatusCode": 206,
"LastSeen": "2026-04-03T19:48:49.65041184Z"
},
"https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/v0.8.0/examples/apache": {
"StatusCode": 206,
"LastSeen": "2026-04-17T18:11:56.141549709Z"
},
"https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/v0.8.0/examples/nginx": {
"StatusCode": 206,
"LastSeen": "2026-04-17T18:11:52.877605927Z"
},
"https://github.com/open-telemetry/opentelemetry-ebpf-profiler": {
"StatusCode": 206,
"LastSeen": "2026-04-10T09:57:36.748982346Z"
Expand Down
Loading