Skip to content

docs(rfc): propose HttpConfigLayer for per-operator HTTP transport config - #8526

Open
trim21 wants to merge 2 commits into
apache:mainfrom
trim21:rfc/http-config
Open

trim21 wants to merge 2 commits into
apache:mainfrom
trim21:rfc/http-config

Conversation

@trim21

@trim21 trim21 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Related: #6871 (per-operator proxy support for bindings). This RFC proposes the design; implementation lands in follow-up PRs.

Rationale for this change

Bindings cannot configure the HTTP transport of an operator. Every binding builds operators from a flat HashMap<String, String>, so a user who needs a proxy for one operator must set HTTPS_PROXY process-wide, changing the network path of every other operator and thread.

Rust users can already replace the transport via Operator::with_context(op.context().with_http_transport(...)), but a reqwest::Client cannot cross an FFI boundary, so bindings have no equivalent. Layers are already the mechanism OpenDAL uses to expose operator behavior to bindings (RetryLayer, TimeoutLayer, ConcurrentLimitLayer, CapabilityOverrideLayer), and Layer::apply_context already owns the transport slot that needs to change.

What changes are included in this PR?

A new RFC, core/core/src/docs/rfcs/0000_http_config.md, proposing:

  • A portable HttpConfig value type in opendal-core (proxy, no_proxy, connect_timeout, read_timeout, timeout, user_agent).
  • A provided HttpTransport::configure hook that builds a configured transport; unsupported transports fail with Unsupported instead of ignoring options.
  • A new HttpConfigLayer (crate core/layers/http-config/) that replaces the operator's transport at construction time, so invalid options fail when the layer is created.
  • Thin wrappers in every binding that already exposes layers.

Service configuration is untouched: no new config keys, no Configurator or OperatorUri changes.

The RFC documents the replace semantics precisely, including layer-ordering rules (apply before transport-wrapping layers such as ConcurrentLimitLayer), that the configured transport overrides later with_context replacements, and why wrapping was rejected (proxy/connect/TLS options must be owned by the client that establishes connections).

Are there any user-facing changes?

No code changes yet; documentation only.

Breaking changes

None.

AI Usage Statement

  • Harness: pi coding agent
  • Model: Claude (Anthropic)
  • Role: fact-checked the RFC against the current codebase (HttpTransport/HttpTransporter in opendal-core, layer replay in Operator, binding layer surfaces, pyo3/jiff conversion) and revised the reference-level sections accordingly: corrected the stated precedence between HttpConfigLayer and Operator::with_context, specified rejection of no_proxy without proxy, added credential masking for proxy values in error context, and fixed the TimeoutLayer precedent claim.

@trim21
trim21 requested a review from Xuanwo as a code owner November 18, 2026 13:06
@github-actions github-actions Bot added core releases-note/docs The PR modifies docs related content or has a title that begins with "docs" size:L This PR changes 100-499 lines, ignoring generated files. labels Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core releases-note/docs The PR modifies docs related content or has a title that begins with "docs" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant