Repository navigation
Conversation
WalkthroughThis PR reworks URLDecomposition's origin, host, hostname, and port handling in Bun's JSC bindings to follow WHATWG URL state semantics, and fixes URLPatternComponent's capture group emission to only expose named groups. Test suites for URL setters and URLPattern nested captures are added, and URLPattern test fixtures are reformatted and extended. ChangesURL origin/host/port state semantics
URLPattern nested capture group fix
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Updated 3:17 PM PT - Aug 17th, 2026
✅ @robobun, your commit 947a725a7452a72c0bc48fb3fa7e3b3bf6491f5e passed in 🧪 To try this PR locally: bunx bun-pr 32826That installs a local version of the PR into your bun-32826 --bun |
There was a problem hiding this comment.
I didn't find any bugs, but this rewrites the host/hostname/port setter state-machine logic and changes parsePort semantics (with an acknowledged knock-on effect on URLPatternCanonical::canonicalizePort), so it's worth a human pass — in particular the hand-written nested-group test asserting { a: "12", b: "1" } where :b(3) matched 3 looks counterintuitive enough to double-check against Chrome.
Extended reasoning...
Overview
This PR audits Bun's URL and URLPattern against the WHATWG specs and fixes several conformance gaps. It touches three C++ binding files: URLDecomposition.cpp (a full rewrite of setHost/setHostname, a new findHostPortSeparator helper, a semantic change to parsePort so an empty digit buffer now fails, an explicit empty-string branch in setPort, and tightened origin() rules for ftps/blob:), URLDecomposition.h (comment only), and URLPatternComponent.cpp (clamping the match-result loop to m_groupNameList.size() so nested user-regexp captures don't leak as empty-named groups). Tests are extensive: 26 new setter cases cross-checked against Node 26.3/WPT, updated blob: origin assertions, a refreshed WPT urlpatterntestdata.json (349→369 entries), and a hand-written nested-group test.
Security risks
None apparent. The changes tighten parsing toward spec behavior (more rejections, e.g. hostname = "[::1]:80" now no-ops; origin returns "null" more often). No new allocation patterns, no untrusted size arithmetic; the ovector indexing in componentMatch is now bounded by min(numSubpatterns, groupNameList.size()) which is strictly safer than before.
Level of scrutiny
Moderate-to-high. URL setters are a core, widely-exercised web API and the new setHost is a from-scratch reimplementation of the host/port state-override transitions rather than a small patch. The PR description explicitly calls out an intentional cross-cutting behavior change: parsePort's other caller, URLPatternCanonical::canonicalizePort, now throws for { port: "\t\n" } where it previously returned "". That's defensible per the port-state spec text but is the kind of ripple a human should sign off on.
Other factors
The work is exceptionally well-documented (spec citations, Node/WPT cross-verification, before/after test counts) and the bug-hunting system found nothing. One thing I'd flag for the reviewer: the new urlpattern.test.ts case new URLPattern({ pathname: "/:a((?<x>1)(?<y>2)):b(3)" }).exec({ pathname: "/123" }) asserts groups is { a: "12", b: "1" }. With the simple min(numSubpatterns, groupNameList.size()) clamp, capture index 2 (the nested (?<x>1)) is what gets paired with name b, not capture index 4 (the (3) part). That may well be what upstream WebKit and the current spec text produce, but it reads surprising enough that it's worth a quick sanity check in Chrome before landing. The two WPT-sourced nested-group entries in urlpatterntestdata.json each have only one top-level named group, so they don't exercise this index-shift case.
1dc6efa to
c189810
Compare
|
Good flag on The index shift is pre-existing and unchanged by this PR. Before the fix Bun returned Three sources agree on
I could not run Chrome in this environment. If liburlpattern disagrees it would be splitting from Node, Safari, and the spec text together, which reads like a spec issue to file upstream rather than something for Bun to diverge on. The two WPT On the |
There was a problem hiding this comment.
I didn't find any bugs, but this rewrites the URL host/hostname/port setter state machine and changes observable origin/URLPattern behavior (including the flagged URLPattern({ port: "\t\n" }) Node-parity question), so it's worth a human look before merging.
Extended reasoning...
Overview
This PR touches three C++ binding files and three test files to bring Bun's URL and URLPattern into WHATWG spec conformance:
URLDecomposition.cpp: substantial rewrite ofsetHost,setHostname,setPort,parsePort, andorigin(). Adds a newfindHostPortSeparatorscanner that mimics the spec's host-state':'handling (bracket-aware, terminator-aware). Reorders the opaque-path / file-scheme / empty-host guards and changes how a failed port parse interacts with an already-committed host.URLDecomposition.h: comment-only.URLPatternComponent.cpp: clamps thecomponentMatchloop tom_groupNameList.size()so nested user-regexp captures don't leak as""-keyed groups — a small, well-contained fix matching upstream WebKit.- Tests: 26 new setter cases in
url.test.ts(cross-checked against Node 26.3), updatedblob:/ftps:origin expectations, one hand-written URLPattern nested-group test, and a WPT refresh ofurlpatterntestdata.json(349→369 entries, plus reformatting).
Security risks
None identified. URL setters parse attacker-controllable strings, but the new code only narrows what the setter accepts (more cases now no-op instead of partially mutating). findHostPortSeparator is a bounded linear scan over a StringView with no allocation. The origin() change makes more schemes serialize as "null" rather than fewer, which is the safer direction for any origin-based check downstream.
Level of scrutiny
Medium-high. URL is a foundational web API; these setters are reachable from any user script and the edge-case matrix (file vs special vs non-special scheme, opaque path, IPv6 brackets, terminators, default ports, tab/newline stripping) is large. The setHost rewrite is not mechanical — it replaces a reverseFind(':')/setHostAndPort approach with a forward scan + two-phase commit, and the guard ordering changed. The author has done unusually thorough verification (full WPT urltestdata.json and setters_tests.json pass, Node cross-check, spec citations inline), which raises confidence considerably, but the logic still merits a maintainer's eye.
Other factors
- The PR explicitly leaves one decision open for a maintainer:
parsePort's new empty-buffer failure meansnew URLPattern({ port: "\t\n" })now throws where Node returns"". The author argues this is spec-correct and Node is internally inconsistent, but offers to special-case it back if Node parity is preferred. That's a judgment call this review can't make. - The
origin()change flips previously-passing test assertions (e.g.blob:ws://…→ now"null"). The PR description says the old assertions were written alongside the buggy implementation and that WPT/Node/browsers agree on the new values, but it's a user-visible behavior change worth a maintainer ack. - The bug-hunting pass found nothing, and the URLPattern clamp is straightforward. The bulk of the review weight is on the ~60 lines of new
setHost/setHostnamelogic inURLDecomposition.cpp.
|
Rebased onto Verification after the rebase:
Note on the red Ready for review. |
There was a problem hiding this comment.
I didn't find any bugs, but this rewrites the host/hostname/port setter state machine and changes origin semantics for ftps:/blob: URLs (plus an intentional Node divergence on URLPattern({ port: "\t\n" })), so it's worth a human pass on the spec reasoning before merging.
Extended reasoning...
Overview
This PR rewrites URLDecomposition::setHost/setHostname/setPort/parsePort/origin in src/jsc/bindings/URLDecomposition.{cpp,h} to follow the WHATWG URL spec's state-override parser, and clamps the capture-group loop in URLPatternComponent::componentMatch. Tests in test/js/web/url/url.test.ts add 26 setter cases (cross-checked against Node/WPT), urlpattern.test.ts adds a nested-group case, and urlpatterntestdata.json is refreshed from WPT (349→369 entries, plus reformatting).
Security risks
URL.prototype.origin is security-relevant — it feeds same-origin checks. The change here is strictly more restrictive (ftps:// and blob: with non-http(s) inner URLs now serialize as "null" instead of a tuple origin), which aligns with the spec/Node/browsers and is the safe direction. The host/hostname setter rewrite changes which host gets committed under malformed input (e.g., out-of-range ports now keep the old port but commit the new host), which is spec-correct but a behavior shift. I don't see an exploitable issue, but origin/host parsing is exactly the kind of code where a second pair of eyes on the state-machine edge cases is warranted.
Level of scrutiny
Medium-high. This is not a mechanical change: setHost went from a reverseFind(':') heuristic to a hand-rolled scan (findHostPortSeparator) that mirrors the spec's host-state terminator rules, with new commit-ordering semantics. The parsePort change also has a deliberate ripple into URLPatternCanonical::canonicalizePort, making new URLPattern({ port: "\t\n" }) throw where Node returns "" — the PR argues this is spec-correct and Node is inconsistent, which seems right, but it's an intentional divergence a maintainer should sign off on.
Other factors
Test coverage is excellent (full WPT urltestdata.json + setters_tests.json reportedly pass, CI is green modulo an infra hiccup), the PR description is thorough with spec citations for every case, and the bug-hunting system found nothing. The existing blob urls test had its expectations changed (previously asserting non-spec values), which is justified but means the old behavior was load-bearing in at least one test. No CODEOWNERS apply to these paths. Given the scope of the logic rewrite and the origin/host security adjacency, I'm deferring rather than approving.
ef36b48 to
8708805
Compare
There was a problem hiding this comment.
I didn't find any bugs, but this rewrites the URL host/hostname/port setter semantics and changes observable origin/URLPattern behavior, so it's worth a human sanity-check on the spec interpretation and the intentional parsePort ripple into URLPatternCanonical::canonicalizePort.
Extended reasoning...
Overview
This PR touches three C++ files in src/jsc/bindings/ (URL setter logic in URLDecomposition.cpp/.h and the URLPattern match-result builder in URLPatternComponent.cpp) plus three test files. The core change is a rewrite of setHost/setHostname/setPort to follow the WHATWG URL spec's state-override algorithm rather than the previous ad-hoc reverseFind(':') + setHostAndPort approach, a tightening of origin() to only return tuple origins for the five spec-listed schemes (and only inner http(s) for blob:), and a clamp on the URLPattern group-name loop to stop nested user-regexp captures leaking into exec() results. The WPT urlpatterntestdata.json is refreshed from 349 to 369 entries with a large formatting-only diff mixed in.
Security risks
None identified. URL parsing is security-adjacent (origin computation feeds same-origin decisions in browsers), but Bun's URL.prototype.origin is a getter on a userland object, not a security boundary here. The changes tighten behavior toward the spec (more "null" origins, more setter no-ops on invalid input) rather than loosening it.
Level of scrutiny
This warrants human review. It is a non-mechanical rewrite of user-visible Web API semantics in C++: the old setHost logic is fully replaced, parsePort's contract changes (empty digit buffer now fails instead of returning the default-port sentinel), and existing test expectations are edited to match new behavior (the blob:ws:///blob:file:// origin assertions flip from tuple origins to "null"). The PR description is thorough and cites WPT/Node/spec agreement for each case, and the bug-hunting pass found nothing, but the combination of "rewrites a state machine by hand", "changes existing assertions", and "has a documented intentional side effect on a second API (new URLPattern({ port: "\\t\\n" }) now throws)" is exactly the kind of change a maintainer should sign off on.
Other factors
Test coverage is strong (26-entry setter table cross-checked against Node, refreshed WPT data, hand-written nested-group test). CI ran green on the URL/URLPattern lanes per the author's rebase comment. There are no prior claude[bot] reviews on this PR, and no outstanding human reviewer comments. The one open judgment call the author explicitly flags — spec-over-Node for URLPattern({ port: "\\t\\n" }) — is a reasonable place for a maintainer to weigh in.
|
The port setter case in this PR (case 4, One thing to note: the branch no longer merges cleanly. #38013 added |
|
Three more host setter cases from the #39458 list were handed over separately. Checked against current main (
|
8708805 to
2285206
Compare
2285206 to
42203c3
Compare
URLDecomposition::setHost split the value at the last ':' and handed the
pieces to URL::setHostAndPort, which diverges from the URL Standard's
"basic URL parse with a state override" setter steps in four ways:
- a port equal to the scheme's default was not cleared:
new URL("http://x:8080").host = "example.com:80" kept :8080
- an out-of-range port dropped the existing port instead of only
failing the port state (the host must still be committed):
new URL("http://x:8080").host = "newhost:99999" lost :8080
- file URLs were split at ':' even though the file host state never
enters the port state and ':' is a forbidden host code point:
new URL("file:///a/b").host = "h2:x" applied "h2" instead of no-op
- url.port = "\t\n" cleared the port. Only the empty string does that;
the parser strips ASCII tab/newline, leaving the port state with an
empty buffer, which fails without touching the port.
setHostname also accepted "[::1]:80" even though the hostname state
fails on any ':' outside an IPv6 literal.
Rewrite setHost to scan for the first ':' outside brackets before a
terminator (the host state's transition to the port state), commit the
host via URL::setHost, and then run the existing parsePort for the port
part. parsePort now reports an empty digit buffer as a failure, which
is what the port state does when a state override is given.
URL.prototype.origin also treated "ftps" as a special scheme and gave
blob: URLs with an inner ftp/ws/wss/file URL a tuple origin. Per
https://url.spec.whatwg.org/#concept-url-origin only ftp/http/https/
ws/wss produce a tuple origin directly, and for blob: only an inner
http(s) URL does; everything else serializes as "null". The blob:
expectations in test/js/web/url/url.test.ts were asserting the previous
non-spec values and are updated to match WPT urltestdata.json, Node,
and browsers.
URLPatternComponent::componentMatch iterated every capture group of the
compiled RegExp and fell back to an empty-string group name once it ran
past the component's group name list. A part's user-supplied regexp can
contain its own named groups, so
new URLPattern({ pathname: "/:foo((?<x>a))" }).exec({ pathname: "/a" })
returned pathname groups { foo: "a", "": "a" } instead of { foo: "a" }.
Clamp the loop to the group name list, which is what upstream WebKit and
https://urlpattern.spec.whatwg.org/#create-a-component-match-result do.
test/js/web/urlpattern/urlpatterntestdata.json is refreshed from WPT
(349 -> 369 entries); the two new nested-group entries cover this.
42203c3 to
947a725
Compare
There was a problem hiding this comment.
I re-reviewed after 3e52404 addressed the tab/newline empty-host nit and didn't find further issues. Given that this is a full rewrite of the setHost/setHostname state machine plus a change to origin() semantics, a human look is still worthwhile.
What was reviewed:
findHostPortSeparatorbracket/terminator scan and the fivesetHostCheckedcommit sites — the re-appliedhasAcceptableHostcheck covers every path that writes a host, including the newfile:and host-with-port branches.parsePort's second callerURLPatternCanonical::canonicalizePort— the!foundDigit→ nullopt change makesnew URLPattern({port: "\t\n"})throw, which is the intentional Node divergence called out in the description.origin()narrowing — only removes tuple origins (ftps, blob:ws/wss/ftp/file), never adds one, so it's the safe direction.componentMatchclamp —m_groupNameList[i - 1]is bounded by thestd::min, matching upstream WebKit'screateComponentMatchResult.
Extended reasoning...
Overview
Three C++ source changes and three test files. URLDecomposition.cpp rewrites setHost() and setHostname() around a new findHostPortSeparator() helper that models the WHATWG host-state → port-state transition, changes setPort() so only a literally-empty string clears the port, splits parsePort()'s no-digit case from the default-port case, and narrows origin() to the spec's tuple-origin scheme list. URLPatternComponent.cpp clamps the group-emission loop to min(numSubpatterns, groupNameList.size()). Tests add a 27-row setter table verified against Node/whatwg-url, refresh the WPT URLPattern fixtures to 369 entries, and update the pre-existing blob: origin assertions to spec values.
Security risks
origin() is same-origin-policy relevant. The change strictly narrows what receives a tuple origin (ftps:// and blob: with inner ws/wss/ftp/file now serialize as "null"), which is the conservative direction — nothing that was previously opaque becomes a tuple. The host setters route every commit through setHostChecked() → hasAcceptableHost(), so the UTS #46 punycode gate from #34660/#38013 is preserved on all five new commit paths (previously two). No new untrusted-input parsing beyond what WTF::URL::setHost already does.
Level of scrutiny
High. This is a hand-implemented state machine replacing ~30 lines of ad-hoc reverseFind(':') logic with ~60 lines that must correctly interleave IPv6 bracket tracking, terminator handling (/ ? # \\), tab/newline stripping, file-scheme special-casing, special-vs-opaque host emptiness rules, and the punycode check — in a Web API that every URL-bearing feature depends on. The parsePort change ripples to URLPatternCanonical::canonicalizePort, producing an intentional divergence from Node that the description flags for maintainer sign-off. The rebase manually re-applied a security-relevant check across a rewrite. None of this is mechanical.
Other factors
Test coverage is thorough: gate evidence shows 12 fail → 0 fail across both test files under debug+ASAN and release, WPT parity with Node on the full urltestdata/setters_tests suites, and a 1574-pass directory sweep. The one inline nit I raised on the previous push ("\t:" bypassing the empty-host guard) was fixed in 3e52404 via containsOnly<isASCIITabOrNewline>() with a matching test row. The comment-cop bot flags are all resolved. The two remaining WTF::URLParser bugs the description records are correctly deferred to the WebKit fork. Deferring because the scope (URL setter rewrite + origin semantics + intentional Node divergence) is beyond what I'd approve without a human maintainer's eye, not because of any specific concern with the implementation.
Fixes #28661
WPT-sourced audit of Bun's
URLandURLPatternagainst the WHATWG URL and URLPattern standards. With this change, Bun's results on the full WPTurl/resources/urltestdata.json(891 cases) andsetters_tests.json(278 cases) suites are identical to Node's: the only remaining divergences from the WPT fixtures are the 14xn--punycode assertions that Node (Ada) also fails and that main now intentionally fails too since #34660 (those come from the constructor's and setters' UTS #46 host validation, which this PR preserves; see the rebase note below). Against main's own source the same runs fail 15 and 11 assertions, and the delta is exactly the cases fixed here. Bun also passes the WPTurlpatterntestdata.jsonsuite (refreshed here to 369 entries).URLhost / hostname / port setters (URLDecomposition.cpp)Per the spec, the setters run the basic URL parser with a state override (https://url.spec.whatwg.org/#api).
URLDecomposition::setHostinstead split the value at the last:and handed the pieces toURL::setHostAndPort, which diverges in four ways:1 and 4 are verbatim failures in WPT's
setters_tests.json; the rest are gaps in that suite. Every expected value above, plus the 27-entry table added totest/js/web/url/url.test.ts, was verified against Node 26.3 (Ada), which agrees on all of them.setHostnow scans for the first:outside brackets and before a terminator (the host state's transition to the port state), commits the host viaURL::setHost, and then runs the existingparsePorton the port part.parsePortreports an empty digit buffer as a failure, which is what the port state does once a state override is given.One intentional side effect:
parsePort's other caller isURLPatternCanonical::canonicalizePort, sonew URLPattern({ port: "\t\n" })now throws a TypeError like{ port: "abc" }already did (both hit the same port-state rule). Node returns""for the former and throws for the latter, which is an Ada inconsistency; WPT doesn't cover it.URL.prototype.origin(URLDecomposition.cpp)7 verbatim failures in WPT's
urltestdata.json.origin()treatedftpsas a special scheme and gaveblob:URLs with an innerftp/ws/wss/fileURL a tuple origin. Per https://url.spec.whatwg.org/#concept-url-origin onlyftp/http/https/ws/wsshave a tuple origin directly, and forblob:only an inner http(s) URL does. Everything else serializes as"null".The
blob:expectations in the existingit("blob urls")test were asserting the previous non-spec values (they were written alongside the implementation in #5825) and are updated to the values WPT, Node, and browsers agree on.URLPattern.prototype.execgroup leakage (URLPatternComponent.cpp)URLPatternComponent::componentMatchiterated every capture group of the compiledRegExpand fell back to an empty-string group name once it ran past the component's group name list. A part's user-supplied regexp can contain its own named groups:Clamp the loop to the group name list, matching upstream WebKit's
createComponentMatchResultand https://urlpattern.spec.whatwg.org/#create-a-component-match-result.test/js/web/urlpattern/urlpatterntestdata.jsonis refreshed from WPT master (349 to 369 entries); the two new nested-group entries cover exactly this, and the other 18 new entries already pass.Verification
test/js/node/test/parallel/test-whatwg-url-{toascii,custom-setters,override-hostname,custom-href-side-effect,custom-properties,canparse,custom-parsing}.js,test/js/web/url/,test/js/web/urlpattern/,test/js/node/url/, andtest/js/web/html/URLSearchParams.test.tsall pass with the fix (1574 pass in the directory sweep). The one remaining failure intest/js/node/url/underbun bd(URL.canParse > repeatedly called produces same result, a debug+ASAN timing test) reproduces identically with main's source.Rebase note: punycode host validation
While this PR was open, #34660 and #38013 added a UTS #46 check to the old
setHost/setHostname(hasAcceptableHost, so thaturl.host = "xn--<invalid>"is a silent no-op on special schemes, matching the constructor and Node). The rewrite here deletes the code those commits patched, so the check is re-applied by hand: every path that commits a new host insetHost/setHostnamenow goes through a smallsetHostChecked()helper that runsURL::setHostfollowed byhasAcceptableHost. That is five commit sites (the old code had two), so the check also covers thefile:and host-with-port paths. Verified bytest-whatwg-url-toascii.js, which assigns every WPT toascii input through both setters, plus a direct probe of all five paths with an invalid and a validxn--label.Not fixed here: two
WTF::URLParserbugsThe same audit turned up two parser (not setter) bugs, but they live in the prebuilt WebKit, so they need an
oven-sh/WebKitchange plus aWEBKIT_VERSIONbump rather than anything in this repo. Recording them with exact locations so they are not lost:new URL("sc://u@h?x").pathnameis"/"instead of"".URLParser.cpp'sState::Hostbranch (around line 1502 atcd821fec) appends'/'on?/#with nom_urlIsSpecialguard; the siblingState::AuthorityOrHostbranch, reached when there is no@, has the guard. Sonew URL("sc://h?x").pathnameis already correct.new URL("sc:/a/..\\x").hrefis"sc:/x"instead of"sc:/a/..\\x".isSingleDotPathSegment/isDoubleDotPathSegmentcallisSlashQuestionOrHash, whosecharacterClassTableentry for'\\'is unconditional, so..\is misdetected as a double-dot segment for non-special schemes. The mainState::Pathslash check is correctly guarded, which is why the plainnon-special://host/a\bcases in WPT'surltestdata.jsonpass and the suite never catches this.Not a bug, recorded so it is not re-filed:
new URL("?a#b", "data:,x")throws in Bun, and that is correct. The URL Standard's no-scheme state only allows#against an opaque-path base (https://url.spec.whatwg.org/#no-scheme-state step 1). Node accepts"?a#b"but rejects"?a"against the same base, which is an Ada inconsistency, not something to copy.Supersedes #28662
#28662 fixes only case 1 by delegating the whole value to
URL::setHostAndPort, but that function has cases 2 and 3 baked in (it rebuilds the string frompathStart(), dropping the existing port when the new port is invalid, and splits file hosts at:), so it cannot cover the class. Closing it in favor of this.[review] gate passed · iteration 11 · 6 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 3 passed · 0 rejected · iteration 11
evidence per changed file