client: don't synthesize the default port into the Host header#1318
Open
krynju wants to merge 1 commit into
Open
client: don't synthesize the default port into the Host header#1318krynju wants to merge 1 commit into
krynju wants to merge 1 commit into
Conversation
For a URL without an explicit port, the client built the request `Host` header from the connection address, which always carries the scheme's default port (`example.com:443`). The Host header should mirror the URL authority as written, so a bare-host URL must send a bare `Host`. Sending `Host: host:443` is legal per RFC 9110 but breaks any server that treats the Host verbatim. Concretely it breaks AWS SigV4 presigned URLs: the signature is computed over the canonical host (`s3.amazonaws.com`), so a request whose Host carries `:443` is rejected with SignatureDoesNotMatch (403). Go's net/http, curl and HTTP.jl 1.x all keep the Host as written. Add a `host_header` view on the parsed URL that preserves an explicit port but never synthesizes the default one (keeping IPv6 brackets), and build `request.host` from it. The connection address is unchanged, so dialing still targets the right port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1318 +/- ##
==========================================
+ Coverage 87.67% 87.72% +0.04%
==========================================
Files 30 30
Lines 11739 11746 +7
==========================================
+ Hits 10292 10304 +12
+ Misses 1447 1442 -5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found this issue when porting to HTTP.jl 2 and AWS S3 presigned links were not working as is.
The problem
For a URL without an explicit port, the client puts the scheme's default
port into the
Hostheader:The
Hostheader should mirror the URL's authority as written. A bare-hostURL should send a bare
Host: example.com. HTTP.jl 1.x did this; 2.0 regressedbecause
request.hostis now built from the connection address(
example.com:443), which always carries a port for dialing.Why it matters
Host: example.com:443is technically legal (RFC 9110 §7.2 allows a port), butmany servers compare the
Hostvalue verbatim and a synthesized default portbreaks them. The case that bit us is AWS S3 presigned URLs (SigV4):
s3.amazonaws.com.Host: s3.amazonaws.com:443.Host, gets a different value,and rejects the request:
403 SignatureDoesNotMatch.The same request succeeds with
curland with HTTP.jl 1.x, which send a bareHost.What other clients do
Go's
net/httpkeeps theHostheader equal to the URL authority and adds thedefault port only to the dial address, never to
Host:curland HTTP.jl 1.x behave the same way. HTTP.jl 2.0 is the outlier.The fix
Add a
host_headerview on the parsed URL that mirrors the authority aswritten — an explicit port is preserved, the default port is never synthesized
(and IPv6 brackets are kept) — and build
request.hostfrom it. The connectionaddress is unchanged, so dialing still targets the correct port.
Hostheader (before → after)https://example.com/example.com:443example.com:443→example.comhttps://example.com:443/example.com:443example.com:443→example.com:443http://minio:9000/minio:9000minio:9000→minio:9000https://[2001:db8::1]/[2001:db8::1]:443[2001:db8::1]:443→[2001:db8::1]Unit tests added in
test/http_client_tests.jlcovering bare host, explicitdefault port, http default port, a custom port, and IPv6 (bracketed) forms.
Verified end-to-end: a real S3 presigned PUT that returns
403on currentmasterreturns200with this change.