Skip to content

test: a non-special URL host accepts every supplementary code point (needs oven-sh/WebKit#1026) - #48463

Draft
robobun wants to merge 1 commit into
mainfrom
robobun/d91ff92b/bump-webkit-url-opaque-host-code-point
Draft

robobun wants to merge 1 commit into
mainfrom
robobun/d91ff92b/bump-webkit-url-opaque-host-code-point

Conversation

@robobun

@robobun robobun commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Draft until WEBKIT_VERSION names a WebKit build with oven-sh/WebKit#1026. Until then these tests fail, and CI is skipped.

Problem

  • new URL("zz://a\u{10020}/") throws TypeError: Invalid URL. Node.js and the URL Standard give the host a%F0%90%80%A0. Bun 1.2.3 and later refuse 272 such code points. Bun 1.2.2 did not.
  • WebKit's URL parser converts a host code point to char16_t before the test for the 17 ASCII code points that no host can hold (Source/WTF/wtf/URLParser.cpp:709 in oven-sh/WebKit). U+10020 is tested as U+0020.
  • No user reported it. A differential test against Node.js found it.

Fix

  • [WTF] URLParser: test the whole code point for a forbidden host code point WebKit#1026 tests the whole code point. The fix reaches Bun with the first regular WEBKIT_VERSION move after it merges. This PR has the Bun tests and changes no pin.
  • Verified: test/js/web/url/url.test.ts, 4 new tests. They fail on Bun 1.4.3 and on a debug build of main. They pass on a debug build with the preview build of that pull request.
  • Self-reviewed: 5 concerns raised, 5 addressed (Notes).

Background

  • A non-special URL has a scheme other than http, https, ws, wss, ftp and file. Its host refuses those 17 (the forbidden host code points) and percent-encodes the rest.
  • Bun links a prebuilt WebKit, which has the URL parser. WEBKIT_VERSION names the build. It moves several times a week.
  • Considered a dedicated bump to the preview build: the pin of main passed it on the same day. Considered a Bun-side check: no Bun code can see the conversion.
Notes

When to mark this ready. When git merge-base --is-ancestor <merge commit of oven-sh/WebKit#1026> <WEBKIT_VERSION of main> succeeds in a WebKit clone. Then rebase (the diff is one test file), drop [skip ci] from the commit subject, run bun bd test test/js/web/url/url.test.ts, and mark it ready. A comment on this pull request wakes the author.

Repro.

const host = (cp) => { try { return new URL("zz://a" + String.fromCodePoint(cp) + "/").host; } catch (e) { return e.name; } };
console.log(host(0x10000), host(0x10020), host(0x1002f)); // low 16 bits: NUL, a space, a slash
console.log(host(0x10001), host(0x10021), host(0x1f600)); // controls
const u = new URL("zz://example/p"); u.host = "a\u{10020}b"; console.log(u.href);
line 1 line 2 line 3
Bun 1.2.3, Bun 1.4.3, debug build of main TypeError TypeError TypeError a%F0%90%80%81 a%F0%90%80%A1 a%F0%9F%98%80 zz://example/p
Bun 1.2.2, Node.js v26.3.0, debug build with the fix a%F0%90%80%80 a%F0%90%80%A0 a%F0%90%80%AF the same zz://a%F0%90%80%A0b/p

The refused code points are U+n0000, U+n0009, U+n000A, U+n000D, U+n0020, U+n0023, U+n002F, U+n003A, U+n003C, U+n003E, U+n003F, U+n0040, U+n005B to U+n005E and U+n007C, for n = 1 to 16: the ones whose low 16 bits are a forbidden host code point.

Counts. One script, every supplementary code point as the host of zz://, then each one with a low half under U+0080 through the two setters:

new URL refuses host setter refuses hostname setter refuses real forbidden code points still refused
Bun 1.2.2, Node.js v26.3.0 0 of 1,048,576 0 of 2,048 0 of 2,048 18 of 18
Bun 1.2.3, Bun 1.4.3, debug build of main 272 272 272 18 of 18
debug build with the fix 0 of 1,048,576 0 of 2,048 0 of 2,048 18 of 18

The last column is U+0000, space, <, >, [, \, ], ^ and | in an 8-bit string and in a 16-bit string. The other 8 forbidden host code points have another role: tab, LF and CR are removed from the input, #, / and ? end the authority, : starts the port, and @ ends the userinfo.

Who meets it. No issue reports it, and no row of the WPT urltestdata.json has a supplementary code point in a non-special host. Of the 272 code points, 177 are unassigned in Unicode 15.1 and 34 are private use. The 61 assigned ones are 14 Linear B signs, 34 CJK ideographs of Extensions B and G, and 13 tag characters. None is an emoji.

What changes in Bun when the pin has the fix. The constructor with and without a base, URL.canParse, URL.parse, the href, host and hostname setters, new Request(url), Response.redirect(url) and URLPattern inputs all end in the same call of WTF::URLParser. Not reached: the same code points in the path, the query or the userinfo, the host of an http URL (IDNA decides there), BMP code points, and strings with no character above U+00FF. A script that goes through every one of these entry points prints the same 30 lines on a debug build with the fix and on Node.js v26.3.0.

The entry points that do I/O stop at the next step, as they do today for the neighbour U+10021:

call, with U+10020 in the host Bun 1.4.3 with the fix
fetch("s3://…") with an S3 endpoint rejects, ERR_INVALID_URL sends the GET, with the bucket name percent-encoded twice
new WebSocket("ws+unix://…") SyntaxError: Invalid url for WebSocket constructs, with the url ws+unix://a%F0%90%80%A0b/
new Bun.RedisClient("redis://…").connect() TypeError: Invalid URL format ERR_REDIS_CONNECTION_CLOSED
a query on new Bun.SQL("postgres://…") TypeError: Invalid URL ERR_POSTGRES_CONNECTION_REFUSED

The tests. The 4 tests are in the existing file, beside the other opaque-host test.

  • The first goes through the 17 low halves in each of the 16 planes (272 code points), in 6 hosts and 3 shapes (zz://H/, zz://u:p@H/, and //H/q on a zz: base), 4,896 parses, and compares each result with encodeURIComponent. The parser reads the authority 16 code units at a time, and narrows them in a different way on x64 and arm64. So 5 of the hosts are 34 code units long, with the code point at the start of a block, across the edge of two blocks and after the last full block.
  • The second goes through each entry point. The third goes through the setters.
  • The fourth holds the 9 forbidden host code points that have no other role, in 8-bit and 16-bit strings and in both setters. Its loops pass before and after, and fail if the fix accepts too much.

USE_SYSTEM_BUN=1 bun test test/js/web/url/url.test.ts -t "opaque hosts" gives 4 failures on Bun 1.4.3. A debug build of main at 98cda0d8b1 gave the same 4 failures with the first version of these tests. A debug build of 98cda0d8b1 with the preview build of oven-sh/WebKit#1026 passes the file (32 tests). A fail-before check that reverts only src/ and packages/ cannot show both sides here, because the fix is not in this repository.

Evidence from the first state of this pull request. It pinned the preview build of oven-sh/WebKit#1026 (ee59053d, the pin of main at that time, plus that one commit). Debug ASAN builds of one Bun commit with the two pins:

suite pin of main then (ee59053d) preview pin
test/js/web/url/url.test.ts 28 pass, 4 fail (the new tests) 32 pass
test/js/web/url/url-wpt-constructor.test.ts 111 pass 111 pass
test/js/web/urlpattern/urlpattern.test.ts 410 pass 410 pass
test/js/node/url 212 pass, 2 skip, 5 todo the same
test-url*, test-urlpattern and test-whatwg-url* of test/js/node/test/parallel (42 files) 42 exit 0 42 exit 0

CI on that pin (build 139721) passed 184 of 188 jobs, with url.test.ts on 10 test lanes (Linux glibc and musl, macOS arm64, Windows x64 and arm64, ASAN). The 4 macOS x64 test jobs failed on the CI machines before they reached the file: no agent took them for three attempts, and in the fourth the checkout directory was removed while the tests ran. In the lane that ships for Linux x64, 3 of the 920 objects of the WTF, bmalloc and JavaScriptCore archives differ between the two WebKit builds: URLParser.cpp.o (the change), URL.cpp.o (only the checksum of URLParser.h in its debug info) and one JavaScriptCore unified source (only byteCodeCacheVersion, which follows the revision).

Self-review. 5 concerns, none about the WebKit diff. (1) The descriptions did not say that no user reported the bug. (2) The order against oven-sh/WebKit#1015, which touches the deleted declaration: there is a note on it. (3) This pull request was a dedicated bump to a preview build. The pin of main passed it on the same day (it is 873ab3a4 now), and a new preview for each pin move costs a build of every lane. It has the tests only now. (4) The cost line in oven-sh/WebKit#1026 named one input as a range. It is measured over 2,025 inputs now. (5) A comment in the test said that @ ends the host. It ends the userinfo. The review also asked for hosts of 16 code units or more and for a host after userinfo. The first test has them now.

Not part of this change. Chromium 153 refuses a different and larger set of supplementary code points in this position (69,632), so it is not a reference here. WTF::URLHelpers has the same conversion in its lookalike-character test (URLHelpers.cpp:179). Bun does not link that code.

@robobun

robobun commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 6:27 AM PT - Nov 10th, 2026

❌ @robobun, your commit 5d7555e has some failures in Build #139721 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 48463

That installs a local version of the PR into your bun-48463 executable, so you can run:

bun-48463 --bun

@robobun

robobun commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

…p ci]

A forbidden host code point is one of 17 ASCII code points. WebKit's URL
parser tested only the low 16 bits of a host code point, so a non-special
URL such as "zz://a\u{10020}/" was refused. oven-sh/WebKit#1026 tests the
whole code point.

These tests fail until WEBKIT_VERSION names a WebKit build that has that
change. CI is skipped for that reason.
@robobun
robobun force-pushed the robobun/d91ff92b/bump-webkit-url-opaque-host-code-point branch from 5d7555e to 12bfc77 Compare November 10, 2026 18:18
@robobun robobun changed the title Bump WebKit (oven-sh/WebKit#1026 preview): a non-special URL host accepts every supplementary code point test: a non-special URL host accepts every supplementary code point (needs oven-sh/WebKit#1026) Nov 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants