Skip to content

feat: Implement process context publishing (OTEP-4719)#3460

Open
scottgerring wants to merge 6 commits intoopen-telemetry:mainfrom
scottgerring:feat/otep-4719
Open

feat: Implement process context publishing (OTEP-4719)#3460
scottgerring wants to merge 6 commits intoopen-telemetry:mainfrom
scottgerring:feat/otep-4719

Conversation

@scottgerring
Copy link
Copy Markdown
Member

@scottgerring scottgerring commented Apr 14, 2026

Summary

Implements the publisher side of OTEP-4719 (Process Context Sharing) behind the process-context feature flag on opentelemetry-proto. This is helpful for signal correlation - and together with the thread context sharing (OTEP-4947), will allow the Rust community to have request-correlated profiling support. This is a linux-only mechanism by design and the API to support it provides a no-op impl for other targets.

On Linux, this publishes SDK resource attributes to a named memory mapping (OTEL_CTX) so external readers (such as the OpenTelemetry eBPF Profiler) can discover process metadata without direct integration or process activity.

This is a precursor for OTEP-4947 which will enable per-thread resource attribution building on top of the process-level mechanism implemented here.

The implementation is adapted from the implementation used in libdatadog, a shared library we use underneath many of our own datadog libraries.

Usage

use opentelemetry_sdk::Resource;

let resource = Resource::builder()
    .with_service_name("my-service")
    .build();

// Publish resource as process context (no-op on non-Linux)
opentelemetry_proto::process_context::publish(&resource);

// Later, during shutdown:
opentelemetry_proto::process_context::unpublish();

Decisions

Why opentelemetry-proto?

I reckon this probably belongs in opentelemetry-sdk, but because opentelemetry-proto depends on that for its transformation layer we can't reference our proto types there (see #3045).

We have options:

  1. Publish it from the proto crate
  2. Create a new crate just for this (in the future perhaps it would also contain the thread-local context publishing mechanism)
  3. Fix Simplify opentelemetry-proto: SDK decoupling, gRPC separation #3045 first, put it in sdk

I'd like to do (3) (and am happy to impl it myself), but as I poked this issue recently and heard nothing back I am hesitant to put it in the dependency path for this.

Publishing mechanism
I've made this explicit - the user must explicitly use this API to publish - look at the updated example code. I'd prefer if this happened automatically, but because each signal stands alone and is provided its own resource, it is difficult to envisage how an automatic publishing mechanism should work. If anyone has any better ideas lmk.

Merge requirement checklist

  • CONTRIBUTING guidelines followed
  • Unit tests added/updated (if applicable)
  • Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
  • Changes in public API reviewed (if applicable)

@scottgerring scottgerring added enhancement New feature or request A-common Area:common issues that not related to specific pillar labels Apr 14, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 14, 2026

Codecov Report

❌ Patch coverage is 83.81295% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.2%. Comparing base (75a0b7e) to head (27cc658).

Files with missing lines Patch % Lines
opentelemetry-proto/src/process_context/linux.rs 82.0% 41 Missing ⚠️
opentelemetry-proto/src/process_context/mod.rs 91.8% 4 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main   #3460     +/-   ##
=======================================
- Coverage   83.2%   83.2%   -0.1%     
=======================================
  Files        128     130      +2     
  Lines      25086   25364    +278     
=======================================
+ Hits       20896   21125    +229     
- Misses      4190    4239     +49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@@ -0,0 +1,139 @@
//! Process context sharing via memory-mapped regions.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pulled this up to represent the public API, subbing in either a linux-64 or a no-op impl based on the target

tonic-prost-build = { workspace = true }
tempfile = { workspace = true }
serde_json = { workspace = true }
serial_test = { workspace = true }
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes it easier for testing

@scottgerring
Copy link
Copy Markdown
Member Author

@cijothomas @lalitb it would be great to get alignment on where this should live (see my idea of the options in the PR text) first!

@scottgerring scottgerring marked this pull request as ready for review April 14, 2026 11:06
@scottgerring scottgerring requested a review from a team as a code owner April 14, 2026 11:06
Comment thread opentelemetry-proto/src/process_context/linux.rs Outdated
/// is Undefined Behavior, for example. We assume that a forking runtime (such as Python or
/// Ruby) that doesn't follow with an immediate `exec` is already "taking that risk", so to
/// speak (typically, if no thread is ever spawned before the fork, things are mostly fine).
pub(crate) fn publish_raw_payload(payload: Vec<u8>) -> Result<(), Error> {
Copy link
Copy Markdown

@yannham yannham Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the whole fork business should really be here? I suppose support for languages with a forking runtime isn't really Datadog-specific, but one could also leave that out for a first implementation. I don't really have an opinion myself, just asking.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch; this is certainly not a common pattern in Rust! I will remove it.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also meant the whole pid business in the code. That is, I think we can simplify the code a bit if we don't care about forks (basically removing the pid field and the case Some(handle) => below when pid != getpid()). We would also get away from the whole "memory allocation is UB after a fork" trap: we don't pretend to handle arbitrary forks anymore.

Comment thread opentelemetry-proto/src/process_context/linux.rs
Comment thread opentelemetry-proto/src/process_context/linux.rs Outdated
Co-authored-by: Yann Hamdaoui <yann.hamdaoui@gmail.com>
Co-authored-by: Scott Gerring <scottgerring@users.noreply.github.com>
@linux-foundation-easycla
Copy link
Copy Markdown

CLA Missing ID CLA Not Signed

One or more co-authors of this pull request were not found. You must specify co-authors in commit message trailer via:

Co-authored-by: name <email>

Supported Co-authored-by: formats include:

  1. Anything <id+login@users.noreply.github.com> - it will locate your GitHub user by id part.
  2. Anything <login@users.noreply.github.com> - it will locate your GitHub user by login part.
  3. Anything <public-email> - it will locate your GitHub user by public-email part. Note that this email must be made public on Github.
  4. Anything <other-email> - it will locate your GitHub user by other-email part but only if that email was used before for any other CLA as a main commit author.
  5. login <any-valid-email> - it will locate your GitHub user by login part, note that login part must be at least 3 characters long.

Alternatively, if the co-author should not be included, remove the Co-authored-by: line from the commit message.

Please update your commit message(s) by doing git commit --amend and then git push [--force] and then request re-running CLA check via commenting on this pull request:

/easycla

@scottgerring
Copy link
Copy Markdown
Member Author

CLA Missing ID CLA Not Signed

One or more co-authors of this pull request were not found. You must specify co-authors in commit message trailer via:

Co-authored-by: name <email>

Supported Co-authored-by: formats include:

  1. Anything <id+login@users.noreply.github.com> - it will locate your GitHub user by id part.
  2. Anything <login@users.noreply.github.com> - it will locate your GitHub user by login part.
  3. Anything <public-email> - it will locate your GitHub user by public-email part. Note that this email must be made public on Github.
  4. Anything <other-email> - it will locate your GitHub user by other-email part but only if that email was used before for any other CLA as a main commit author.
  5. login <any-valid-email> - it will locate your GitHub user by login part, note that login part must be at least 3 characters long.

Alternatively, if the co-author should not be included, remove the Co-authored-by: line from the commit message.

Please update your commit message(s) by doing git commit --amend and then git push [--force] and then request re-running CLA check via commenting on this pull request:

/easycla

Hey @yannham , do you mind signing the CTA now that i've taken your suggestion ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-common Area:common issues that not related to specific pillar enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simplify opentelemetry-proto: SDK decoupling, gRPC separation

2 participants