Skip to content

Fix Netty 4.1 HTTP/1.1 pipelined response tracing#11937

Open
ygree wants to merge 3 commits into
masterfrom
ygree/fix-netty41-http11-pipelining-context-queue
Open

Fix Netty 4.1 HTTP/1.1 pipelined response tracing#11937
ygree wants to merge 3 commits into
masterfrom
ygree/fix-netty41-http11-pipelining-context-queue

Conversation

@ygree

@ygree ygree commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Fixes Netty 4.1 HTTP/1.1 pipelining by storing server request state in a per-channel queue instead of a single overwritten channel attribute.

Each queued entry now carries the tracing context, request headers, and AppSec response blocking state, so outbound responses are matched with the correct inbound request.

Motivation

HTTP/1.1 pipelining can place multiple in-flight requests on the same Netty channel. The previous instrumentation kept only one server tracing context per connection, so later requests overwrote earlier request state before their responses were written. This caused responses to finish or annotate the wrong span, losing trace context for pipelined requests.

Additional Notes

  • HTTP/1.1 pipelined responses are matched to request contexts in FIFO order.
  • The existing CONTEXT_ATTRIBUTE_KEY is still mirrored for generic Netty fire* span activation.
  • If a channel exceeds the pending server request context limit, pending contexts are closed and server tracing is disabled for that channel to avoid unbounded retention or response misattribution.
  • The per-channel state is removed on channelInactive.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@ygree ygree self-assigned this Jul 13, 2026
@ygree ygree added type: bug fix Bug fix inst: netty Netty instrumentation labels Jul 13, 2026
@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Jul 13, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 69.73% (+12.68%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 57f9d65 | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.90 s [-0.1%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent 12.94 s 12.94 s [-0.6%; +0.6%] (no difference)
startup:petclinic:appsec:Agent 16.98 s 16.59 s [+1.3%; +3.4%] (significantly worse)
startup:petclinic:iast:Agent 16.85 s 16.96 s [-1.6%; +0.2%] (no difference)
startup:petclinic:profiling:Agent 16.84 s 16.34 s [-1.5%; +7.6%] (no difference)
startup:petclinic:sca:Agent 16.89 s 16.35 s [-1.2%; +7.8%] (no difference)
startup:petclinic:tracing:Agent 16.05 s 16.20 s [-1.6%; -0.2%] (maybe better)

Commit: 57f9d657 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@ygree ygree force-pushed the ygree/fix-netty41-http11-pipelining-context-queue branch from 6f4df04 to 7f7fdf4 Compare July 13, 2026 23:23
@ygree ygree marked this pull request as ready for review July 14, 2026 00:10
@ygree ygree requested a review from a team as a code owner July 14, 2026 00:10
@ygree ygree requested review from jordan-wong and removed request for a team July 14, 2026 00:10

@datadog-datadog-us1-prod datadog-datadog-us1-prod Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The PR correctly fixes HTTP/1.1 pipelining by switching from single-attribute storage to a per-channel queue, allowing multiple in-flight requests to maintain independent tracing contexts. Code review of request queuing, response FIFO matching, error handling, and channel cleanup pathways reveals no behavioral regressions—the fix is architecturally sound.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit 7f7fdf4 · What is Autotest? · Any feedback? Reach out in #autotest

@ygree ygree added the tag: ai generated Largely based on code generated by an AI or LLM label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: netty Netty instrumentation tag: ai generated Largely based on code generated by an AI or LLM type: bug fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant