Skip to content

Replace c-ares with a DNS resolver in safe Rust - #47076

Draft
dylan-conway wants to merge 40 commits into
mainfrom
claude/c-ares-rust-replacement-xrtz9k
Draft

dylan-conway wants to merge 40 commits into
mainfrom
claude/c-ares-rust-replacement-xrtz9k

Conversation

@dylan-conway

@dylan-conway dylan-conway commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Removes the dependency on c-ares. The DNS client behind node:dns's resolve*(), reverse(), lookupService() and Resolver, and behind Bun.dns.lookup()'s "c-ares" backend, is now written in safe Rust.

Draft. In CI, as of fd841cb7, the whole test suite passed on Linux, Windows and macOS, x64 and aarch64. Since then main was merged in, with bun build --compile --aot, and one commit keeps the resolver out of trimmed executables (below). CI is running on that. What is left is listed at the end.

  • src/dns_resolver (bun_dns_resolver, new, forbid(unsafe_code) for the whole crate, depends on bun_core only)
    • It does no I/O and has no callbacks. Channel is a state machine: the caller passes the time into each call, says which socket became readable or writable, and then takes the finished requests out of a queue. Everything the resolver needs from the system (sockets, files, the environment, random bytes, routes for address sorting, the platform's resolver settings) comes through the Io trait, which the channel holds as a Box<dyn Io>.
    • message: the DNS message reader and writer (name compression, escapes, every record type c-ares knows).
    • reply: what ares_parse_*_reply made of a message, for each resolve*().
    • channel: queries, retries, timeouts from the per-server latency buckets, failover and probing of failed servers, retiring a UDP socket after a timeout, TCP after truncation, TCP Fast Open, the query cache.
    • lookup: getaddrinfo (hosts file, search list, ndots, HOSTALIASES, RFC 6724 sorting), gethostbyaddr, getnameinfo.
    • system_config, hosts: resolv.conf, nsswitch.conf, LOCALDOMAIN, RES_OPTIONS, hosts, services. What Windows (GetAdaptersAddresses, GetBestRoute2, the registry), macOS (dns_configuration_copy) and Android (net.dns*) report arrives as plain data, so the rules for it compile and are tested on every host.
    • Objects refer to each other by id, not by pointer. c-ares calls back into its caller from the middle of its own loops. Here a finished request waits in a queue until no borrow is held, so a promise reaction can call straight back into the resolver.
    • simulation: runs a whole channel in a world described by a text script (files, settings, routes, sockets, the clock, the random seed) and returns a log of every socket call, packet, timer and completion. bun:internal-for-testing exposes it as dnsSimulate(). Nothing else reaches it.
  • src/runtime/dns_jsc
    • system_io.rs (new) implements Io on bun_sys and owns the FilePolls (uv_poll_t on Windows). system_settings.rs (new) asks each platform for its settings.
    • dns.rs loses about 2,400 lines: one request type and one map of requests in flight replace six request structs, a macro per record type and fifteen 32-slot HiveArray caches. Requests for the same thing still share one query.
    • The timer is armed for the deadline of the next query. It used to tick once a second.
    • The getaddrinfo backends (libc, libuv, dns_sd) are unchanged.
  • ares_inet_pton / ares_inet_ntop had users outside DNS (SocketAddress, BlockList, UDP sockets, node:cluster, uWS's remote address). They now use bun_core::ip_address, which reads and prints the same.
  • bun_sys::socket (new): socket, connect, bind, getsockname, recvfrom, setsockopt for callers that are not uSockets.
  • Deleted: the dependency, src/cares_sys, the update workflow. The attribution stays in LICENSE.md, because src/dns_resolver is derived from c-ares 1.34.9.

The goal was no observable change, quirks included. These are the changes I know of:

Change Why
A query to a UDP port where nothing listens fails with ECONNREFUSED at once Bug fix. The refusal arrives as EPOLLERR alone, which Bun passed on as neither readable nor writable. The event loop span at 100% CPU until the query ran out of tries: 22 s and ETIMEOUT on main, 30 ms here. Node.js reports ECONNREFUSED
An answer without records (the name exists, the type does not) is remembered for as long as its SOA record says, and not at all without one Bug fix. c-ares takes the lowest TTL of the records other than the SOA. With none that is "no limit", so it kept the answer for the maximum, an hour. A host that gained an AAAA or TXT record stayed ENOTFOUND in a running process for that long. RFC 2308 section 5
Timeouts fire when they are due They were looked at once a second, so new Resolver({ timeout: 250, tries: 1 }) took up to a second longer
resolveAny() of an answer that holds only a CNAME gives [] It gave [undefined, undefined]. Node.js gives the CNAME. Not changed here
More than 32 identical requests in flight share one query The table of requests in flight had 32 slots per kind
A server whose probe could not open a socket, was cancelled by cancel(), or was answered with TC and sent again over TCP to another server, is probed again later c-ares leaves it marked as being probed for the life of the channel
A read error on a socket that also had data to read is acted on at once: the connection is closed and its queries are sent again c-ares notes the error, then does nothing with the note. The queries wait for their timeouts
A server's failure or recovery is always counted c-ares looks the server up in its sorted list by sort key. While dns.setServers() replaces servers with queries in flight, an old and a new server can compare equal, and the one that sorts second is skipped
An empty search list (search with no domain or only delimiters, LOCALDOMAIN= or one with a non-printing byte, a Windows suffix list of only delimiters) no longer throws away the rest of the system's configuration c-ares treats it as a failure to read the configuration and falls back to 127.0.0.1
RES_OPTIONS that is set and empty is no options Bug in c-ares: it reports an empty options string as out of memory, throws away the system's configuration and falls back to 127.0.0.1 with no search list. RES_OPTIONS= bun -e 'console.log(require("dns").getServers())' prints ["127.0.0.1"] in 1.4.4. From reading ares_sysconfig_mac.c, a macOS resolver whose options are there and empty did the same
Lines of the hosts file that share a name or an address are not merged. A name has the addresses of the lines it is on, an address the names of its lines Bug in 1.34.9, fixed upstream since (589b5887). 10.0.0.1 a b and 10.0.0.2 b c made a resolve to 10.0.0.2 as well
A hosts file that was replaced is read again whatever its mtime: size, inode and device count too Fixed upstream since (64b36bb5). A file moved into place with an older mtime was never read
Bun.dns.lookup() takes A records only from the answer to its A query and AAAA records only from the answer to its AAAA query. resolveCname() of an answer that holds an address and no CNAME is ENOTFOUND Fixed upstream since (4d4986ff). family: 6 could give IPv4 addresses, a server that put both in both answers gave each address twice, and resolveCname() gave the name that was asked for
An answer that replaces another in the cache stays for its own TTL c-ares leaves the old entry's expiry in place, which then evicts the new one (upstream ea7810d9, not merged)
A name longer than 255 bytes on the wire is EBADNAME (ENOTFOUND in JS) c-ares sends 256 and 257. Fixed in 1.5.0 and lost again
In resolv.conf a TAB is a blank, and the last of domain and search counts. In nsswitch.conf a # starts a comment As resolv.conf(5) has it. c-ares drops a line with a TAB in its value, ignores domain after search, and reads words of a comment as sources. All three were fixed once (1.5.2, 1.7.x) and lost in rewrites
An nsswitch.conf that cannot be read is as one that is not there c-ares gives up on resolv.conf too and asks 127.0.0.1. Fixed in 1.11.0 and lost again
A link-local server is not asked from setLocalAddress()'s IPv6 address It cannot be reached from there. Fixed upstream once (d974c556) and lost again
A UDP datagram that is no DNS message ends the connection, and counts against the server, only if it starts with the id of a query sent on that socket With c-ares any datagram does, so whoever guesses the port needs no id: the effect of CVE-2023-32067, back since 1.34.0
localhost. is localhost RFC 6761. c-ares asks the servers
setServers() does not throw "there are pending queries" for a probe of a failed server that nobody waits for It counted every query of the channel
65,536 queries in flight: the next one fails with ENOMEM c-ares loops forever looking for a free id
After 53 rounds of tries, which only failures that take no time can get to, the timeout stays at its maximum c-ares adds it to the clock as a signed number. The deadline is in the past, and the tries that are left are used up at once
dns:// URIs as nameserver in resolv.conf, sortlist, /etc/netsvc.conf and /etc/svc.conf are not read An extension of c-ares that no other resolver reads, not reachable from Bun's API, or for systems Bun does not run on
lookupService() reads the services file itself, by the rules of glibc's parser It was getservbyport_r, so an NSS source other than files is no longer asked, and other systems' parsers differ in what they make of a malformed line. Two things glibc does are not copied. It gives up on the whole file at a line that does not fit the 4 KiB buffer c-ares passed with its aliases. And it misreads a last line that is indented and has no newline (__nss_readline moves it without its NUL)
Bun.dns.lookup(name, { flags: AI_ALL, backend: "c-ares" }) no longer reads CARES_HOSTS AI_ALL has the value of ARES_AI_ENVHOSTS by accident
TCP Fast Open is used on Linux and Android only c-ares also used it on macOS and FreeBSD
The error for an invalid address given to SocketAddress never has errno It had one (EMSGSIZE) only for an IPv4 address with too many octets
process.versions.ares is the constant 1.34.9 Node.js has the key, so it stays, as the release this behaves like

Known differences from Node.js that were there before and are not changed: resolveTxt() returns one array per character-string and not per record, resolveAny() leaves out CNAME records, resolve4() of a CNAME without an address resolves with [], and ENODATA is reported as ENOTFOUND.

How did you verify your code works?

New tests through public APIs. Their expected results were recorded with a debug build of the parent commit, which links c-ares, so they compare Bun before with Bun after. They pass on both by design, except five: one for the first row of the table, two for the second and two for the row about the type a query asked for.

  • test/js/node/dns/dns-server.ts (new): a scripted DNS server on 127.0.0.1, UDP and TCP on one port. A test sees every query and chooses each reply byte for byte.
  • test/js/node/dns/dns-resolver-class.test.ts, 55 new tests: every record type and resolveAny(); the bytes of a query; each RCODE; empty answers; CNAME chains and their TTLs; compressed names and escapes; replies with the wrong id, name or type; five kinds of malformed reply; truncation and TCP (a reply of nearly 64 KiB, several queries on one connection, a server that hangs up); timeouts and tries; failover; the query cache, negative answers included; identical requests in flight; 300 queries at once; cancel(), also with a new query made from a cancelled one's callback; setServers() with a query in flight; setLocalAddress(); names that cannot be sent and how the others are encoded.
  • test/js/bun/dns/resolve-dns.test.ts, 8 new tests in subprocesses: Bun.dns.lookup() with the "c-ares" backend for both families, aliases, the search list, ndots, trailing dots, HOSTALIASES, literal addresses and localhost.
  • Same results before and after, test by test: test/js/node/dns, test/js/bun/dns, every test/js/node/test/parallel/test-dns*.js, test-net-dns-*, test-worker-dns-terminate*, test-dgram-*, test-tls-canonical-ip, test-blocklist, socketaddress.spec.ts, udp_socket.test.ts, dgram.test.ts. The machine has no internet, so the tests that need it fail the same way on both.

New tests of the resolver itself (test/js/node/dns/dns-simulation.test.ts, 50 tests through dnsSimulate()). They cover what a test cannot arrange on a real machine, and run on every platform since the platform is part of the script: resolv.conf and nsswitch.conf, the settings of Windows, macOS and Android, search lists of several domains, reloading of the hosts file, RFC 6724 with made-up routes, the length of each timeout, sockets that cannot be opened, TCP answers a byte at a time or several at once, partial writes, Fast Open, the services file, and how long an answer is remembered. No Bun that used c-ares has this function, so these were written from c-ares's source and not recorded. They were then compared with c-ares itself: a driver links c-ares at Bun's pin, unmodified but for its random number generator, with every libc call that touches the world redirected to the same scripted world, and prints the same log. When last compared, of the 95 scripts these tests ran, 10 were about the settings of other platforms and 3 used a command the driver lacks. Of the other 82, 66 gave the same log, packet ids and jittered timers included, and each of the 16 that did not is a row of the table.

Logs of c-ares, checked in (test/js/node/dns/dns-simulation-c-ares.txt, 358 more tests in the same file). 149 scripts were picked from the fuzzing corpus so that together they reach every line of the resolver that a script for Linux can reach. The others are one small script each for a difference that fuzzing found and that is fixed, for a CVE or a bug fix in c-ares's history, or for a row of the table. 83 came out of a mutation run in the fuzzing session: of 994 mutants of the crate that compile, the tests killed 692 before these scripts and 886 with them, and it judged the 108 that are left to be equivalent or to show only in what no log prints. 329 have to give the log that unmodified c-ares gave. 29 are differences that are meant, with both logs. test/js/node/dns/c-ares-driver/regenerate.ts builds the driver against a checkout of c-ares and writes the logs again. It gives the file that is checked in, byte for byte. Nothing builds it in CI, and it can go if 1,600 lines of C are not wanted in the tree.

c-ares's history. All 3,037 commits up to upstream's main of 19 October 2026, and every CVE and advisory, were read for bugs that a port could have: those fixed before the pin (is the fix here?), those fixed after it (is the bug here?), and those fixed once and lost in a rewrite. Of 49 CVE rows that apply to this code, 44 were fine as they were. What was found is in the table above, each with a script. One bug was the port's own: after a search name that the cache answered, the AAAA query of a lookup went on being retried with an IPv4 address in hand. Left as c-ares has them, on purpose: names from servers are escaped but not refused for bytes outside [A-Za-z0-9-._/*] (the second half of the fix for CVE-2021-3672, which upstream dropped in 1.21 and which would refuse DNS-SD names), the QR bit is not checked, the cache has no cap on its entries, and a query is written again into the same TCP connection on a retry.

The same binary with and without c-ares. A script asks a hostile local server 16,565 questions through node:dns, from 3,000 seeds, and prints each result whole: the JS value, or every field of the error. I ran it twice on a debug build of main at the merge base and twice on a debug build of this branch. 16,523 lines are the same in all four runs. The others vary from run to run on either build, because how many datagrams one read finds depends on timing. There was no line on which the two runs of each build agree and the builds do not, apart from 143 results of resolveAny() that lose their nulls (in the table). With the fixes from c-ares's history there are 10 more, 9 by the row about resolveCname() and 1 by the row about datagrams. The renaming and simplifying after that changed none.

Under ASAN, two seeded scripts do at random what is hard on the binding: cancel(), setServers() with queries in flight, requests made from completions, resolvers dropped and collected, the server's socket closed, TCP answers cut off, the global resolver, Bun.dns.lookup(), lookupService(), workers terminated with queries in flight. 824 seeds: no crash, no report, and every request settles.

--compile --aot --trim. The resolver is a part of the Dns feature: its poll, timer and deferred-error arms in dispatch.rs, the one place a Resolver is freed, and the close at VM teardown are guarded. Built alike on Linux x64 (LTO, trimmable), a trimmed hello world maps 26,948,810 bytes on main and 26,875,027 here, and 12,850 bytes of DNS code are always kept (the cache under fetch() and connect()) where main keeps 85,778. Not measured for Mach-O and COFF. Trimmed executables were run by hand: every kind of query against a local server with the output of the plain run, fetch() of localhost in a program without Dns, process.exit() with two queries in flight, and a worker terminated with 20 in flight.

By hand, the same script under the debug build and Node.js 24: all of the above kinds of query against a local server, a worker terminated with 20 queries in flight, a resolver dropped and collected with a query in flight, and a closed port. Same output.

Still to do

  • More of Windows and macOS. CI's Windows and macOS machines pass the DNS tests, so system_settings.rs and the Windows half of bun_sys::socket have run, but only on the network settings those machines have. The struct layouts were compared with the windows-sys crate and are asserted at compile time.
  • Fuzzing, in progress. So far: over 100 million generated scripts for the state machine against that driver, over 115 million messages through the message parser and each ares_parse_*_reply equivalent against c-ares's, 840 million inputs to the address parsers, 65 million raw scripts, and 32 million scripts with the settings of Windows, macOS and Android, which have no c-ares to compare with and were checked for rules instead (every request ends once, every socket is closed once, no timer is left). No panic, hang or unbounded memory. Every difference found is either fixed or in the table. Line coverage of the crate by the corpus is between 93% and 100% per file, except error.rs, whose functions only the binding calls. What is not reached is mostly defensive returns.
  • Not this PR's, found on the way, on main too: .bss is not opted out of transparent huge pages as the heaps are, so with THP at always a release whose link puts an aligned 2 MiB stretch inside .bss uses 2 MB more, and test/aot/perf/start.test.ts fails on such a branch. The reproduction and a tested patch are in the CI notes comment below.
  • Not this PR's, found on the way, on main too: after node:dgram sends a datagram to a closed port on Linux, its next send() to another, live peer fails with ECONNREFUSED and nothing is sent. An unconnected socket can also emit error for it. Node.js does neither. bsd.c says why (IP_RECVERR). The tests' DNS server sends the reply again.

Generated by Claude Code

claude added 13 commits October 27, 2026 11:21
The dependency, its update workflow and its version macro are gone.
`process.versions.ares` stays, as in Node.js, and now reports a release
number instead of a commit.

The build does not link after this commit: `bun_cares_sys` still declares
the `ares_*` functions. The following commits replace them with Rust.
The crate does no I/O itself: sockets, files, the clock and randomness come
through an `Io` trait, and finished requests wait in a queue instead of
calling back. It follows c-ares 1.34.9 as Bun configured it.

bun_cares_sys is deleted. bun_runtime still refers to it, so the build
stays broken until the runtime moves over.
The runtime's Resolver now owns a `Channel` over `SystemIo`, which answers
the crate's `Io` trait with bun_sys sockets and the event loop's polls.
One map from request key to waiting promises replaces the sixteen
32-slot pending caches and their intrusive lists for everything the
channel answers. The timer is set for the channel's next timeout instead
of ticking every second.

The build links again.
…ocal server

A datagram refused by the server arrives as EPOLLERR alone, which was neither
readable nor writable, so the event loop span until the query timed out.
What Windows, macOS and Android report about their resolvers reaches the crate
as plain data through Io::settings(), so the rules for it compile and can be
tested on any host. SystemIo reads it: GetAdaptersAddresses and the registry,
dns_configuration_copy, the net.dns properties.

simulation::run() drives a whole channel from a text script and logs every
socket call, packet, timer and completion. bun:internal-for-testing exposes it
as dnsSimulate().
… world

- A hosts file where many lines share an address took quadratic time to read.
- ndots was 0, not 1, when the system's configuration could not be read.
- Taking an answered query off its connection was linear in the queries in flight.
- A probe that could not open a socket left its server marked as being probed for good.
- An answer without records is cached for what its SOA record says, not for an hour.
- Error is a thiserror enum.
@robobun

robobun commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator
Updated 11:01 PM PT - Nov 9th, 2026

✅ @claude, your commit 5f5d5c6638ef44cd11906e595e9671fd3578e1ee passed in Build #139606! 🎉


🧪   To try this PR locally:

bunx bun-pr 47076

That installs a local version of the PR into your bun-47076 executable, so you can run:

bun-47076 --bun

- iphlpapi.lib came in with c-ares's objects. libuv needs it too.
- clippy, mordant and the byte-search lint.
- The query id is drawn before anything else in a send, so the same random
  bytes lead to the same choices as in c-ares.
- Queries due at the same instant time out last sent first.
- A datagram whose source port was not reported is accepted.
- Idle connections are closed server by server.
- The services file is read by glibc's rules.
Channel is no longer generic: it holds a boxed Io, and knows sockets and
requests by number. The generic one was compiled twice into every binary, for
the system and for the simulator.

- Servers whose sort keys are equal while the list is replaced are ordered as in
  c-ares's skip list.
- An interface name after a nameserver has to fit IF_NAMESIZE.
- A record's class is checked before its RDATA, and a URI target that does not
  print is EBADSTR.
- hive_array's unit tests no longer use the method the last commit removed.
- The localhost lookup test does not depend on the machine's hosts file.
…adratic

- test/js/node/dns/dns-simulation-c-ares.txt holds 194 scripts for the
  simulation with what c-ares 1.34.9 logged for them in the same world.
  189 have to go the same way here. The other 5 are the differences that
  are meant, and hold the log that is expected here as well.
- A record with 16,000 options, or a reply with thousands of names, took
  time quadratic in their number. Options are now found by id and names
  to point back at by their text. What is parsed and written is the same.
- A NUL byte ends a line of the services file, as it does for glibc.
- The tests' DNS server sends its UDP replies one after the other, sends
  one again if it failed for the sake of an earlier one whose socket was
  closed, and fails the test that owns it for any other error.
test/js/node/dns/c-ares-driver/regenerate.ts builds c-ares 1.34.9 from a
checkout, with the calls it makes of libc renamed to the driver's, and
writes its logs in dns-simulation-c-ares.txt again. It gives the file
that is checked in, byte for byte. Nothing builds it in CI.

The scripts that guard a difference that was fixed are now named after
what they guard.
@dylan-conway
dylan-conway force-pushed the claude/c-ares-rust-replacement-xrtz9k branch from 321e5d9 to e7236b1 Compare October 27, 2026 17:06
claude and others added 9 commits October 27, 2026 20:59
… simplify the resolver

From going through c-ares's history:
- hosts: lines that share a name or an address are no longer merged, and a file that was replaced is read again whatever its mtime
- lookup() takes only addresses of the type a query asked for
- an answer that replaces another in the cache is not evicted when the other was due
- a question longer than 255 bytes on the wire is EBADNAME
- resolv.conf: a TAB is a blank, the last of `domain` and `search` counts; nsswitch.conf: comments, and one that cannot be read is as one that is not there
- a link-local server is not bound to the local IPv6 address
- a UDP datagram that cannot be parsed ends the connection only if it has the id of a query sent on it
- `localhost.` is localhost
- setServers() does not count a probe as a pending query
- the AAAA query of a lookup stops being retried also after a name that the cache answered

From review: Record is Message, Kind is RecordType, the requests have Node's names, dead parameters, states and error codes are gone, items no other crate uses are not pub, and the files derived from c-ares carry its licence notices.
The script commands `etc` and `readable` had no script that used them, and no script got an answer to a CNAME question. `touch` is gone from the driver as it is from the simulation.
The test and regenerate.ts each parsed the file. It has a header that gives its format and how to add an entry. regenerate.ts refuses a checkout of c-ares that is not at the pinned commit, and finds the sources itself.
…stion that was sent again

What the dropped test checked is pinned without a clock by three scripts of dns-simulation-c-ares.txt. A handler of the test server that throws now fails the test that owns the server.
It had Cell and ElTimespec from dns.rs, which no longer uses them, so macOS did not build.
Returning from insert() in that case hid the failure, which the mordant check does not allow.
- a lookup had two match arms that could no longer be reached, and told an answer's status and what it added apart for nothing
- `finish` checked the RCODE in three places
- bare boolean parameters are enums or lists, derives nothing uses are gone, every file says what it is, comments are at most 100 columns wide
- `word()` scans once instead of searching the set of spaces for each byte
- the script language of the simulation gives a file an inode, so that a hosts file replaced by one of the same size and time can be tested
- the crate's documentation says where it differs from c-ares
- log labels and an accessor still had the names of functions that were renamed
- titles that named a topic, or a property the test did not check, are reworded
- a truncated answer is really not remembered, a system without /etc is tried, resolveCname() of an address alone is ENOTFOUND
- a question that a slow machine made the resolver send again is counted once
- the tests of a port that nothing listens on no longer run beside tests that could be given the port
- two tests whose scripts were also in dns-simulation-c-ares.txt are gone
- one soa() and one parse() for the three files that had their own
- the driver needs no flag to keep quiet at the end
- comments in other tests no longer explain themselves by functions of c-ares that are gone

dylan-conway commented Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

CI notes

5f5d5c6, build #139606 (passed: 188 of 188 jobs)

The result: green. No test failed on any platform. All 188 jobs passed: Linux (glibc, musl, ASAN), Windows x64 and arm64, macOS arm64 and x64. (The status was red for a while because the 4 macOS x64 jobs expired in the queue three times before an agent took them.) All 9 GitHub checks passed. Main's own build of 98cda0d, #139578, passed 175 of 175.

  • None of the three test files that could not be judged on my machine failed in CI: compile-aot-size.test.ts (which does run there: it was among main's 38), compile-trim-locales.test.ts, compile-trim-capabilities.test.ts.
  • 12 test files passed on a retry (14 times over the platforms), none about DNS. 7 are in test/flaky-tests.txt. Two of those are in files near this PR: node-net.test.ts ("should allow reconnecting after end()", the case the list names) and process.test.js on Windows x64, where process.uptime() and performance.now() were read on two sides of a whole second. The other 5 are gc-controller-cadence.test.ts (retried in main's #139578 too), bun-serve-html.test.ts, two files under test/aot, and the one below.
  • On macOS x64, server.spec.ts (retried there in #135399) and the two objects-closed programs needed no retry. One test did: serve-directory-routes.test.ts, "handles a concurrent burst on one file", where one of 128 fetches sent at once to a local server got ECONNRESET. The connection was made and then reset, so the name had resolved, and 128 is macOS's default listen backlog. What this PR changes near Bun.serve and fetch (the error for a hostname that is none, the error for a name that does not resolve, the text of an address for requestIP) is not on that path. I could not compare with other branches: of the 20 recent builds I sampled, the macOS x64 lane had run in 2. One retry, unplaced; if it comes back on this branch I will take it as this PR's.

A merge of main up to 98cda0d (6 commits), because it conflicted in src/js/node/tls.ts, and one small commit of test data.

Main is green again, so this build can be too. The rewritten ahead-of-time compiler (#47942, #48120) is in this merge. Main's 38 failing tests went with it: #139525 on af566a3 passed 175 of 175. The solver whose answer depended on the order of its rules is gone, and with it the one test that failed on this branch alone (ir-6.test.ts below), so that note is closed.

What the merge needed:

  • src/js/node/tls.ts: main dropped the type annotations from the lines where this branch names dns_resolver_jsc.bind.ts. Main's form, this branch's file name.
  • Two things git did not flag. src/js/ahead-of-time/node/tls.ts, new on main, is a second text of the same module, which bun build --compile --aot compiles in place of the first. It named cares_jsc.bind.ts, which this branch removes, so it takes the same rename. And it records the SHA-256 of the first text, which test/internal/second-texts-of-modules.test.ts compares: this branch's tls.ts differs from main's by that one line, so the hash is this branch's. Whoever merges main again after tls.ts changes there has to write the hash again.
  • Nothing else: the other 14 files that both sides changed hold this branch's changes and no more, no file of this branch is left under test/aot where main deleted thousands, main did not touch the DNS directories, and nothing in its new tables of built-ins names what this branch renames or removes.

The second commit adds HOSTALIASES, LOCALDOMAIN and RES_OPTIONS, which the resolver reads, to test/aot/trimmed-is-whole/variables.txt, which had lost CARES_HOSTS. Swept on a trimmable build: 105 pairs of runs, none differs.

Run before the push, Linux x64, on the merged tree:

DNS, worker, socket, TLS and cluster suites, debug + ASAN the numbers of the last merge; what fails is the same tests by name, all from the machine (no public internet, no ::1 in /etc/hosts, its ICU, its Node, no supplementary groups); no sanitizer report
Node's own tests dns 37 of 37, cluster 85 of 85
The engine's IR tests Bun's 572 of 572; WebKit's 2,623, none failed, 3 todo
The engine's native tests 2,075, none failed, 43 todo
147 test files of test/aot on a release build (engine/, builtins/, those about the table of built-ins, those that name DNS) 8,045 tests, none failed
45 trim test files on a trimmable build 43 pass; see below
test/internal, all 151 files pass (8 of builtins-table/ time out on a debug build here and take 8 s each on a release build)
The lint workflow's type checks, check-types-of-modules.ts among them pass
cargo fmt, rust:check-all, clippy, prettier clean, 12 of 12 targets
A program compiled with --compile --aot that goes through tls.checkServerIdentity and a dns.promises.Resolver at a local server prints what bun run prints
  • The two trim files that fail here do so because my trimmable build links Ubuntu's ICU 74. compile-trim-capabilities.test.ts finds a call of dlopen, and its address is in uprv_dl_open_74. compile-trim-locales.test.ts fails on removing ICU's locale data; I have not compared that one with a build of main.
  • test/bundler/compile-aot-size.test.ts did not run here in a way that counts (skipped on the plain release build, off on the ICU one). From CI's earlier measurements this branch makes the --trim executable about 86 KB smaller than main, so it should pass with more room than main has. Its bounds "only go down": once this build has measured it, the --trim bound could come down by as much.

b3e456b, build #137888 (finished 05:09 UTC, 6 Nov: red as foreseen, with main's 38 failing tests and one more; 128 jobs passed, every macOS one among them, and 60 failed)

The result: the failing tests are main's 38, every one of them, and test/aot/engine/ir-6.test.ts. Nothing else. I compared the error annotations with those of main's #137822 by name, and inside the tests that could differ: ir-1.test.ts and the four engine-tests-flow-under-other-schedules fail on the same one file as on main, and ir-6.test.ts on the one test described below (308 of its 309 pass), on Debian x64, arm64 and ASAN and on both Windows.

  • 128 jobs passed and 60 failed, where main has 41. The 19 more are 17 shards of Debian x64 ASAN, which main's builds do not run (33 tests fail there: 32 of main's list and ir-6), and one shard on each Windows that has ir-6 alone. No failed job is without an annotation that explains it.
  • All 8 macOS shards passed, 4 on arm64 and 4 on x64, the first runs of the merged dns_sd.rs, which I could only type-check. The two objects-closed programs had their second run on macOS x64 without a retry.
  • All GitHub checks are green, cargo miri test and mordant among them.
  • server.spec.ts, which needed a retry on macOS x64 in #135399, needed none. Of the tests retried on macOS one is about sockets, node-net.test.ts ("should allow reconnecting after end()"), and it is in test/flaky-tests.txt.

A merge of main up to b1305dd (43 commits), because it conflicted in 10 files.

Expect this build to be red, mostly from main. Main's own builds have had the same 41 failed jobs and 38 failing tests, 37 of them under test/aot/, on 8 of its last 9 commits (the ninth was cancelled), #137822 on b1305dd among them. I found no small fix to port: the open candidates are #47942, which rewrites the compiler's front end, and two drafts. One failure more is particular to this branch, below.

How the conflicts were resolved

What git does not flag

  • set_timer, which only this branch has, passed uws_loop() where increment_timer_ref now takes &Loop.
  • SystemIo::watch made a &mut Loop while check_timeouts, with main's uws_loop_ref(), now holds a &Loop across the call. Before the merge that was a raw pointer. register and unregister take &Loop, so it is self.vm.uws_loop_ref() now, with one unsafe block and one unwrap fewer.
  • Main added no new reference to c-ares.

test/aot/engine-tests/valueflow/modules/create-require-read-late-on-a-copy-of-node-module-returns-a-require-function-with-flow-rules-delayed.ir fails on this branch and not on main. The defect is in flow's solver and is on main too; I have left the test as it is.

  • The test's four variants ask that flow answers the same whatever order its rules run in. Three do. Under testing=DelayFlowRules the report loses the site: unseen LoadsCodeByName ModuleRequire with no f1:v6.
  • letSomeDueStatementsWait picks the rules that wait by a hash of the schedule's number, the round and contentOf(statement), which holds numbers of the table of built-ins. So another numbering is another set of schedules.
  • I ran the plain variant under all 256 schedules. On a testaot built from main (b1305dd) 9 lose the site: 28, 60, 90, 113, 114, 199, 200, 216, 245. On this branch 13 do, and 0 is one of them, which is the one the failing variant runs.
  • On main: testaot run test/aot/engine-tests/valueflow/modules/create-require-read-late-on-a-copy-of-node-module-returns-a-require-function.ir --option=testing=DelayFlowRules:28
  • What is lost: under a bad schedule the callee of f1:v5 does hold NodeModuleCreateRequire (among 37 members, where a good schedule has 12), but the result of the call has no ModuleRequire. The answer is narrower than it should be, which is the unsafe direction.
  • engine-test-commands.ts says of such a test that it "has lost something: it is not added", so I have not changed its check lines, its schedule or marked it todo. runFlowTestsUnderOtherSchedules runs WebKit's valueflow folder only, which is why the 9 on main do not show. Any PR that adds or removes a built-in can meet this.

valueflow/modules/text-file-answers-the-text-to-a-linked-require-and-its-namespace-to-the-loader.ir (WebKit's, in ir-1.test.ts) fails here as on main: the list it reads has 27 members and 24 are printed.

Before the push, on the merged tree (Linux x64)

  • The debug + ASAN suites (DNS, worker teardown, sockets, cluster) gave the same numbers as before 949031e, with no sanitizer report. Node's own tests: 37 of 37 and 85 of 85.
  • The ownership I ported, driven by hand under ASAN: 200 waiters on one lookup that succeeds and on one that fails, more names than the 32 slots, 40 names with 5 waiters each, on three backends; workers terminated with lookups in flight; BUN_DESTRUCT_VM_ON_EXIT=1. Of 2,140 lookups the log has 2,140 made (1,677 waiters and 463 heads) and 2,140 dropped, 1,440 rejected and 700 resolved.
  • The engine's tests: 1,851 of 1,852 IR tests here and 5,665 of 5,666 in WebKit (the two above), 3,023 native, none failed.
  • On a release build with LTO and trimmable: trim-by-use 244 and compile-trim-by-use 43, none failed. The two objects-closed programs pass; AOT: bun test --aot on large suites, module loading fixes, fuzzer fixes #47316 is in main now and they merged without a conflict.
  • rust:check-all 12 of 12 targets, so dns_sd.rs and the Windows paths, which I can only read here, at least type-check. cargo fmt, clippy, the source lints (194), bun-types.test.ts and build-hardened-flags.test.ts are clean.
  • A review of src/runtime/dns_jsc against main, with experiments of its own under ASAN, found no regression from c-ares. SystemIo::watch above is its finding.

949031e, build #135399: passed, 188 of 188 jobs (finished 23:39 UTC, 29 Oct)

Every platform passed, macOS x64 among them, with no error annotation. macOS x64 shard 1, which failed on 0b8f26c, passed, and neither objects-closed-in-other-ways.ts nor objects-closed-through-others.ts needed its retry. That is one run of a test that failed in most runs but not all, so it speaks for the fix of 718e06b without proving it.

The status was red three times on the way (20:08, 22:05 and 23:05 UTC): macOS test jobs expiring in the queue before they started, which BuildKite queues again by itself. The arm64 shards got agents after 2 to 4 hours and the x64 ones after nearly 5. #47389 has the numbers on the x64 agents.

All GitHub checks are green, cargo miri test among them. Seven tests that are not in test/flaky-tests.txt passed only at their retry. None fails in resolver code. Six do the same on other branches; the last I could not place:

  • test/js/bun/jsc-stress/testFFI.test.ts, on 4 Linux platforms. It came with the merge: it is not in this branch's two builds before it, and it is in main's own #135376 (3 platforms) and in 9 of the 12 other branches' builds with annotations that I sampled from #135340 to #135396.
  • test/aot/run-aot-same-messages-every-iteration.test.ts (also in #135352) and test/js/bun/spawn/spawn_waiter_thread.test.ts (also in #135380).
  • test/aot/no-leak-records-at-exit.test.ts on Debian x64 ASAN, as under 0b8f26c below.
  • test/aot/in-place-executables.test.ts on macOS x64 (five tests timed out at 120 s) and test/aot/build-scaling/build-scaling-2.test.ts on macOS arm64. Both were also in this branch's #134997, and they are in 8 and 2 of the 13 other builds I looked at.
  • test/js/node/net/server.spec.ts on macOS x64, "calls the connection listener when a socket connects". Probably a race in the test, not shown. It is in none of the 9 other builds I looked at whose macOS x64 tests ran. The client does look up localhost, which is code this PR touches, but the test first waits for the client's connect callback, so the lookup had succeeded. It then sleeps 1 ms and expects the server's callback to have run; it had not. On Linux x64 a copy of the test misses in 0 of 5,760 connections, at 48 processes on two cores, on this branch and on Bun 1.4.4 alike. If it comes back on this branch I will take it as this PR's.

A merge of main up to 73a2043, because #47087 conflicted.

Before the push, on the merged tree (Linux x64):

  • The debug + ASAN suites (DNS, worker teardown, sockets, cluster) gave the same numbers as before 0b8f26c, with no sanitizer report. Node's own tests: 37 of 37 and 85 of 85.
  • All of the engine's tests: 1,599 IR tests here, 5,037 in WebKit, 2,916 native, none failed.
  • On a release build with LTO and trimmable: trim-by-use 243 and compile-trim-by-use 43, none failed.
  • The two programs of 718e06b pass test/aot/run-parallel.ts. cargo fmt and clippy are clean.
  • test/js/node/process/process.test.js, which both sides changed: 194 pass and 4 fail, all four from my machine (system ICU 74, Node 24 on the host, a user with no supplementary groups, and BUN_OPTIONS=--smol in the environment, with which Bun 1.4.4 prints the same).

Main has since moved to 9fe1cd6 (#47382), which does not conflict, so it is not merged.

718e06b, build #135328 (all 180 Linux and Windows jobs passed; the 8 macOS jobs had not got an agent in 1 h 50 min when the next push came)

One commit, tests only: the fix for the one failure on 0b8f26c, objects-closed-in-other-ways.ts on macOS x64. That failure is not this PR's (the evidence is under 0b8f26c below). The fix is #47316's: the two files are byte for byte as on that branch, so they merge without a conflict once it lands. A request that fails before the server's handler ran is sent again, up to three times; a request that the handler did see is judged as before.

Checked before the push, on a release build of this branch (Linux x64):

  • test/aot/run-parallel.ts passes both programs, with and without --no-erased.
  • On a quiet machine the old and the new program print the same 2,103 lines.
  • I could not reproduce the race on Linux, with the old program or the new: 288 runs of each at 96 at a time, and 128 of each at 64 at a time pinned to two cores, all print the reference text.
  • So I tested the retry with a fault put in by hand, in copies outside the tree: every 7th fetch rejects with a TypeError before it reaches the server. The old programs then show what CI shows (46 and 26 more N: TypeError lines). The new ones print exactly the reference text, and keep the 3 and 2 TypeError answers that belong there.
  • Whether it holds on macOS x64 is still to be shown, by the next build whose macOS jobs run: the two builds of AOT: bun test --aot on large suites, module loading fixes, fuzzer fixes #47316 that I looked at (#135034, #135043) were cancelled before their macOS jobs ran.

Main was 5 commits ahead (69f4077) with no conflict and no change to what the table of built-ins is made from, so it is not merged. #47316 changes those files, and #47293 does by its title; when either lands, custom-accessor-reached-computed-key.ir (cc4be3b) may need its check lines changed again.

0b8f26c, build #134997 (finished 11:28 UTC, 29 Oct: 188 jobs passed, 1 failed, on one test)

Every Linux and Windows job passed: 181 of 181, with no error annotation. All 14 build-bun jobs passed, the Windows ones with the line below among them, and ir-6.test.ts passes on the five platforms where it failed. All GitHub checks are green, cargo miri test too.

macOS got agents at 10:34 (arm64) and 11:00 UTC (x64), after the jobs had expired in the queue three times, as in other branches' builds. All 4 arm64 shards passed, and x64 shards 0, 2 and 3 (shard 3 on rye).

macOS x64 shard 1 failed on test/aot/parallel/node-http/objects-closed-in-other-ways.ts (host pretzel). Not this PR's: it fails the same way on branches that still build c-ares. Fixed in 718e06b, above. I first wrote here that this was a judgement and not a proof; the sightings below settle it.

  • The test (new in AOT: fixes for modules, classes and Workers, WebKit upgrade, shorter compiler docs #47129) runs a program in plain Bun and compiled with --aot, and compares what they print. Both exited with 0. They differ in 1 of its 81 cases, "freeze by hand response before": after the exchange whose response object was frozen, plain Bun's third fetch ends in a TypeError and never reaches the server.
  • The same test failed in 7 builds of other branches in the hours after this one, each time on macOS x64 shard 1: #135001, #135005, #135009, #135012, #135033, #135037, #135045. Of the builds from #134998 to #135045 whose four macOS x64 shards all finished, only #135038 and #135042 do not have it.
  • In 5 of the 7 the annotation shows the same thing: one more N: TypeError in plain Bun's run, in a "freeze" case, at a fetch after the broken exchange. #135033 and #135045 have this build's very case and line. In #135001 and #135009 the part of the annotation that is shown has no such line; annotations are cut at about 32 KB.
  • Its sibling objects-closed-through-others.ts fails in 4 of them as well.
  • The commits of #135033 and #135045 still have scripts/build/deps/cares.ts, so none of this PR's code is in them.
  • The cause, as AOT: bun test --aot on large suites, module loading fixes, fuzzer fixes #47316 puts it: the server drops the connection after the broken exchange, and on a busy machine the client sends on it before it notices. The client waits a fixed 2 ms between requests. No name is resolved (127.0.0.1 on both sides).

test/aot/no-leak-records-at-exit.test.ts needed its retry on Debian x64 ASAN, here and on 6bd18b4. It looks for leaks at exit, and this PR changed how a resolver is freed at teardown, so I read the four stacks: two are JavaScriptCore's list of Intl locales (intlAvailableLocales) and two the console client of GlobalObject::setConsole. The same report, to the byte (782 in 31 objects, 50 in 2, 40 in 1), is in 3 of the 6 builds of other branches I looked at whose ASAN jobs ran (#134990, #134972, #134969). main runs no ASAN test jobs. Not this PR's.

Two commits: cc4be3b, the fix for ir-6.test.ts (under 6bd18b4 below), and a merge of main up to 081ace7, which conflicted.

Before the push, on the merged tree (Linux x64): the debug + ASAN suites gave the same numbers as before, with no sanitizer report; Node's own tests 37 of 37 and 85 of 85; all of the engine's tests, which I now run before every push: 1,599 IR tests here, 5,037 in WebKit and 2,916 native tests, none failing; a release build with LTO and trimmable (Dns 0.32 MB removable), on which trim-by-use (243) and compile-trim-by-use.test.ts (43) pass; cargo fmt and clippy clean; rust:check-all 12 of 12, which is what compiles the Windows line.

6bd18b4, build #134938 (175 jobs passed and 5 failed, all on one test; the 8 macOS jobs had not got an agent when the next push cancelled them)

test/aot/engine/ir-6.test.ts fails on Alpine x64 and arm64, Debian x64 ASAN, and Windows x64 and arm64. This PR's, together with main's new test. Fixed in cc4be3b.

  • One IR test of 267 fails, test/aot/engine-tests/pipeline/gc/custom-accessor-reached-computed-key.ir, new in AOT: fixes for modules, classes and Workers, WebKit upgrade, shorter compiler docs #47129. For process.env[key], under a key made at run time, the compiler prints one more possible value than the check lines have, builtin(FunctionPrototype). Nothing else in the output differs.
  • It passes on main alone (a testaot built from e64b59d with the same WebKit) and fails on this branch, so I could not put it down to main. I had not run the engine's IR tests before pushing the merge; I do now.
  • The two tables of built-ins differ in one renamed entry (canonicalizeIP, out of cares_jsc.bind.ts), one new one (dnsSimulate in bun:internal-for-testing), and the numbers of the 3,626 entries behind them. Only the numbers matter: without dnsSimulate the test passes, and with dnsSimulate kept and an unrelated entry (dnsIsLocalhostName) taken out, so that the numbers are main's again, it passes too.
  • The value first shows at ApplyFlowResults, so it is AnalyzeFlow whose answer depends on how the built-ins are numbered. I have not found why. The wider type is a superset of the narrower, so it is sound either way. But any PR that adds or removes an entry in front of these will trip this test, which someone who knows the analysis may want to look at.
  • The fix follows the rule in WebKit's CLAUDE.md for tests in test/aot/engine-tests (the PR that changes the table changes the test): 11 check lines get the value. With it all of the engine's tests pass on this branch's table: 1,599 IR tests here, 5,037 in WebKit, 2,928 native tests.

A merge of main, up to e64b59d. No conflicts, but two things in #47129 bear on this PR, and CI builds the head of the branch, not its merge with main:

  • Main now has tests of --trim that use DNS, written on c-ares: test/bundler/trim-by-use/programs/dns.ts (a UDP server of its own, Resolver.resolve4, dns.promises.lookup, Bun.dns.lookup) and dns-resolve-of-bun.ts. They had never run on this resolver. Both pass, and untrimmed both print what Bun 1.4.4 prints.
  • The other 238 programs there are each built alone, so --trim takes Dns out of those that do not name it, and 3ffb48b put Abort guards of Dns where a resolver is freed and in VM teardown. No program aborts: all 16 parts of trim-by-use (243 tests) and compile-trim-by-use.test.ts (43) pass.
  • Its WebKit fixes the one failure that counted on 711b3cb, below.

The trim tests ran on a local release build with LTO and trimmable (Linux x64, static ICU 74). Its link passes Dns's minimum, with 0.32 MB removable, as before the merge. On a debug + ASAN build of the merged tree the DNS, worker-teardown, socket and cluster suites gave the same numbers as on 711b3cb, with no sanitizer report; Node's own tests pass, 37 of 37 (dns, net-dns, worker-terminate) and 85 of 85 (cluster). cargo fmt and clippy are clean, and rust:check-all compiles on all 12 targets.

All GitHub checks are green, cargo miri test among them, in 14 minutes, for the first time since 0dd6940. I wrote here that it would not finish on this commit either; that was wrong. #47181 is not in main, but #47129 has the same remedy, #[cfg_attr(miri, ignore)] on characters_refuse_only_names_that_are_too_far_apart, so it came with the merge.

711b3cb, build #134746

Two merges of main, up to 37a5e3a, because main conflicted with this branch, and did again while the first merge was being checked. Nothing under src/dns_resolver changed.

  • src/runtime/dns_jsc/dns.rs: uws: put Loop's fields in Cells so its methods take &self #47231 made uv_loop of the uws loop a method inside on_dns_socket_state, which this branch removed with c-ares. This branch's side is kept whole. Its own Windows code asks the VM for the loop (vm.uv_loop()), which did not change, and FilePoll::register still takes the loop the way system_io.rs passes it.
  • cluster_raw_bind in src/runtime/node/node_cluster_binding.rs: this branch parses the address in Rust where main calls ares_inet_pton, and Zero-initialize plain C structs with the safe ffi::zeroed() #47249 zero-initializes the addrinfo hints just below with the safe ffi::zeroed(). Both are kept.

Before the push, on Linux x64 with a debug + ASAN build of the merged tree: the DNS, worker-teardown and socket suites gave the same numbers as on 857f19e, with no sanitizer report; cluster.test.ts 41 of 41 and Node's own cluster tests 85 of 85; a cluster worker binding 127.0.0.1, ::1, localhost and a name that does not exist printed what Node v24.20.0 prints. cargo fmt and clippy are clean, and rust:check-all compiles on all 12 targets.

Final but for two soft jobs that the next push cancelled: 306 jobs passed and one that counts failed, on one test, below; 69 test-aot-drawn jobs failed softly. All 14 build-bun jobs passed. start.test.ts passed this time (shard 25; the merge moved .bss), and so did compile-aot-size.test.ts (shard 5). All GitHub checks were green but cargo miri test, for the reason under 857f19e below.

  • macOS: every job that counts passed, on arm64 (2 test-bun, 5 test-aot) and on x64 (2 test-bun, 1 test-aot), after they had expired in the queue two and three times; they ran from 03:44 and 04:41 UTC. check.test.ts passed: x64 shard 1 ran on bagel, not on rye.
  • I read all 62 annotations: none names DNS or the resolver. The one about --trim, trimmed-against-whole-1.test.ts on Windows x64 (soft), differs on WebAssembly's CompileError. One soft test on macOS x64 ended with SIGTRAP, so I checked it against the Abort guards: it is JavaScriptCore's "AOT validation: UnreachableValue", in a generated program about Bun.readableStreamToFormData.

test/aot/engine-tests-native-12.test.ts, shard 15 of debian 13 x64 - test-aot. Not this PR's: a race in the engine's allocator, about 1 run in 100, fixed by the WebKit that main pins since #47129. 6bd18b4 above has it.

  • One of the part's 181 native tests failed, heap-cage/ballast/first-requests-of-threads-are-served: its child process, under ulimit -v 1048576, died of SIGSEGV.
  • The program is testaot, which is linked from WebKit, mimalloc and the table of built-ins. It has no Rust in it. The two merges changed no C or C++, build script, pin, patch or generator, and the same shard passed on 857f19e (#134347) and 0dd6940 (#134170).
  • It reproduces here on an idle machine, with a release testaot built from the WebKit pinned then (fcc1ea09): 26 of 3000 runs crash.
  • The SIGSEGV is the CRASH() of WTF::fastMalloc(48), called by one of the test's 48 threads: a request that may not fail found no memory. In each of the 6 crashes whose state I read in gdb, 47 of the 48 threads had been served, 0 of the 8 pieces of the ballast were left, no other thread was inside a request, and about 0.9 MB of address space was still free.
  • Correction. I first wrote here that the test has no margin, and proposed to start 32 threads in place of 48. That was the condition and not the cause. The 48 first requests do take about all of the 8 MB (once all are served, 341 of 400 passing runs have no piece left and 59 have one), and with 32 threads the last piece is never reached, so nothing crashes (0 of 3000). The cause is in makeRoomWithBallast (bmalloc.cpp): a request that found no room and could let go of no further piece gave up, although another thread had let go of the last piece since the request looked, and what that piece left was still free. WebKit f495db1d counts the pieces let go and has such a request look again ("another thread may have let go of the last piece since this request looked, or while it waited").
  • With a testaot built from f495db1d, and the test's 48 threads as they are, 0 of 6000 runs crash (3000 on an idle machine, 3000 beside a build). The patch I proposed is not needed and I have taken it out of this comment.

To reproduce on fcc1ea09: for i in $(seq 3000); do sh -c 'ulimit -c 0 && ulimit -v 1048576 && exec "$0" "$@"' ./testaot native --single-process heap-cage/ballast/first-requests-of-threads-are-served --useAOT=true >/dev/null 2>&1 || echo crashed; done

857f19e, build #134347 (finished 13:23 UTC)

Tests only: a script for a difference from c-ares that fuzzing found (RES_OPTIONS set and empty). No job was cancelled for lack of an agent this time, so build #134281 below needs no retry. In the end 303 jobs passed and 74 test-aot-drawn jobs failed softly. Two jobs that count failed on a test, neither on resolver code: start.test.ts on Debian x64 and check.test.ts on macOS x64, both below. Four more show as failed with exit -1; their retries passed. The usual macOS agents took no job of any branch from about 05:20 to 11:46 UTC; this build's macOS jobs expired in the queue three times.

  • macOS arm64, the first run of 3ffb48b's change to how a resolver is freed and to VM teardown: every job that counts passed, both test-bun shards and all 5 test-aot shards. Four of them first ended together at 12:03 UTC on four hosts with exit -1, which is the agents going away and not a test; they were retried by themselves and passed. None of the macOS annotations mentions an abort, a signal, a removed feature or DNS.
  • macOS x64: test-bun shard 0 and the test-aot shard passed. test-bun shard 1 failed on one test, test/cli/check/check.test.ts, "finishes within 10 s on conditions nested deeper than the stack allows": bun check was killed at its 10 s (exit 137); the other 851 tests of the file passed. It is the type checker, and it goes with the host, not the branch: shard 1 ran on rye, and check.test.ts also failed on macOS x64 in #134075, #134298 and #134334 of three other branches, each time with shard 1 on rye (in #134075 I read the output: the same test, killed at 10 s). In 17 other builds of today shard 1 passed on other hosts. Not this PR's; I know of no fix, and I cannot retry BuildKite jobs.
  • test/bundler/compile-trim-capabilities.test.ts fails on macOS arm64 (test-aot-drawn, soft): an executable without what loads native code still holds two places that call dlopen, kept by WorkerConstructor. It is about --trim, so I looked for this PR in it. The two dlsym_with_handle! in system_settings.rs call dlsym only. And the same failure, word for word, is in builds of three other branches from the day before 3ffb48b: #134075 (claude/upgrade-webkit-fcc1ea0973a4), #134089 (claude/zstd-rust-replacement-4qwmbq) and #134100 (claude/compile-aot-on-check). Not this PR's.
  • test/bundler/compile-aot-size.test.ts passes: shard 5 of debian 13 x64 - test-aot, where it failed in #134170, passed. That is the first CI result for the fix in 3ffb48b.
  • test/aot/perf/start.test.ts fails again (shard 25), with the same signature as in #134170: privateKilobytes of bun-serve 2008 -> 3972 and of node-http 3036 -> 4928, one huge page less what each had touched, and the other five programs within budget. It is the .bss and transparent huge pages problem described under #134170 below, with the patch. It and check.test.ts on the macOS x64 host rye (above) are the only test failures in jobs that count. It stays red on this branch until that patch or another fix lands, or a change happens to move .bss. The patch is not DNS's, so it waits for a maintainer's word on whether it belongs in this PR.
  • cargo miri test (GitHub Actions) does not finish, on any branch that has Compile TypeScript to machine code ahead of time: bun build --compile --aot #45631. Not this PR's. I wrote here before that miri takes over two hours; that was wrong. It took 14 minutes on fd841cb, and has not finished once since main was merged in 0dd6940. Compile TypeScript to machine code ahead of time: bun build --compile --aot #45631 added characters_refuse_only_names_that_are_too_far_apart to bun_sema, whose 380,880 comparisons take Miri hours, and GitHub cancels the job at six. Dozens of branches have the job running for hours now. ci: skip the bun_sema spelling test under Miri, and give each Miri crate a 45-minute limit #47181 finds this and fixes it: there the job passes in 11 to 19 minutes. I have not ported it, because what makes it pass is #[cfg_attr(miri, ignore)] on that test, and skipping a test of another crate is not for a DNS PR to decide; it comes with main once it is merged. What Miri has to say on this PR it has said: of its 19 crates this PR touches only bun_collections (an unused safe method of HiveBitSet removed), and that was in fd841cb, where the job passed. The run on this commit first failed in Setup (the download of the pinned Bun zip broke off, curl: (18)); its one re-run started at 09:16 UTC and GitHub cancelled it at 15:16, at six hours. lol-html cargo test passes on this commit, as do all the other GitHub checks.
  • test/cli/init/init.test.ts timed out on darwin 27 aarch64, without output, and bundler_npm.test.ts needed a retry there ("Failed to install dependencies: SIGTERM"). Both run a real bun install, so I checked whether this PR is in that path. It is not: bun install resolves names through mod internal of dns.rs, which differs from main's in two renamed calls in a testing helper and nothing else. The step is new (3 of the 60 builds from #134300 have it) and has one host, bingus. The job before this one on that host, #134310 of an unrelated branch, had about twenty networking tests fail or need retries, test-integration-rspack.ts time out, and timed out itself after 61 minutes, as a soft failure. On the usual macOS arm64 agents both test-bun shards passed on 0dd6940, which has all of the resolver. Those jobs are still waiting for agents in this build. The macOS 27 job then timed out itself after 60 minutes, as a soft failure, so it does not count.
  • One test-aot-drawn test dumped core on Alpine x64 (generated-own-properties-that-a-function-of-the-language-copies-are-read-off-the-copy.test.ts). The --trim guards of 3ffb48b abort when they are reached, so I read the stack: it is JavaScriptCore's own check, crashOnValidationFailure in AOTRuntimeValidation.cpp ("AOT validation: ImpossibleMutation"), in a generated program about arguments and defineProperties. No frame is the resolver's. variant-validate-2.test.ts on Debian x64 crashed in the same check ("AOT validation: ValueType").
  • The other test-aot-drawn failures: of 59 annotations, 50 are tests, or numbered siblings of tests, that were red in this PR's earlier builds, on main or on other branches. I read the 9 new names, since the drawn jobs pick their programs by the commit. Besides the three above they are a cost bound under ASAN, the jsc shell dying under ASAN, --aot output that differs from the plain run, an internal compiler error (InvalidGuard), string widths under eager tier-up, and buffer.parent: refused. None runs resolver code.

3ffb48b (as of 06:50 UTC)

All 14 build-bun jobs passed, so Dns's new ELF minimum holds on every Linux, Android and FreeBSD target and the macho one still holds. Every job that got an agent has finished: 147 passed, none that counts failed, 28 test-aot-drawn jobs failed softly. All Alpine and Windows test-bun and test-aot jobs passed. Of the GitHub checks, cargo miri test is still running and the rest but lol-html cargo test are green.

EC2 capacity came back: the builds I sampled that started after about 05:22 UTC (#134288 to #134312) have no Linux job cancelled. All 32 macOS jobs expired waiting for an agent and were queued again; macOS jobs are waiting in those other builds too.

Two things make it red, and neither ran any of this PR's code:

  • lol-html cargo test (GitHub Actions) failed in Setup, before Rust was installed; cargo test was skipped. The download of the pinned Bun release zip crawled at a few KB/s for 3 min 40 s and ended in curl: (56) Recv failure: Connection reset by peer, which --retry 5 does not retry. I asked for a re-run of the job and GitHub answered 403, "The workflow run containing this job is already running", so it has not been re-run. The next push runs it again.
  • Build #134281 is marked failing because 174 Debian and Ubuntu test jobs were cancelled before they got an agent. The build's robobun annotation lists 174 times "Failed to create agent": EC2 answered CapacityLimitExceeded, "This account has reached its capacity limit". Other branches' builds of the same half hour have jobs cancelled too (#134276, #134279, #134280, #134282, #134283, #134284). I cannot retry BuildKite jobs.
    • So the size fix has no CI result yet. In #134170 compile-aot-size.test.ts and start.test.ts failed in shards 5 and 25 of debian 13 x64 - test-aot. Here only shards 0, 4, 16, 18 and 26 of that step got an agent.

The rest of what is red is in test-aot-drawn jobs (soft failures). I compared the failing tests with #134170 and main's #134092 and #134067, and searched their output for DNS, the resolver and removed features, because this commit moves --trim guards: nothing mentions them, and all but one are tests, or numbered siblings of tests, that were red there. The one new name and one new flaky test are on other branches too:

here elsewhere
test/aot/static-modules-slope.test.ts row H (Object.keys in CommonJS modules), bound 2400 2409.08 on Alpine x64 2409.95 on Alpine x64, #134276 (claude/lint-check-perf)
test/cli/bake/bake.test.ts "starts the server once the type errors are fixed", ECONNREFUSED, passed on retry Alpine x64 Alpine aarch64, #134274 (claude/vitest-compat)

Build #134170 on 0dd6940 (merge of main, with --aot)

This PR runs all of test/aot in the fixed step. Two failures there count; 303 jobs passed. Everything else that is red is in test-aot-drawn jobs, which are soft failures and are red on main's own builds too (#134092, #134067); none of it names DNS.

1. test/bundler/compile-aot-size.test.ts: a trimmed hello world maps 25,706,728 bytes, the limit is 25,700,000. This PR's, fixed in 3ffb48b.

Resolver's Drop closes the channel, which statically reaches most of src/dns_resolver. Code outside the Dns feature drops refs on a resolver (the drop glue of RuntimeState and VM teardown; from reading the code, also the completion of a system lookup on macOS and Windows), so --trim kept the resolver in every executable. c-ares hid the same edge behind callbacks stored at query time. Now the one place a resolver is freed and the teardown call are removable_body!(Dns, ..), and the VM holds its resolver as a RefPtr like everyone else.

Like-for-like local builds (Linux x64, LTO, trimmable, static ICU 74, so the absolute numbers are not CI's):

main 11c3671 0dd6940 3ffb48b
hello world --aot --trim, mapped bytes 26,948,810 27,075,795 26,875,027
DNS code that is always kept 85,778 217,113 12,850
removable with Dns 0.26 MB 0.12 MB 0.32 MB

Dns's minimum for ELF goes from 100,000 to 150,000, which fails the link on the state before the fix. I could not link Mach-O or COFF here, so those numbers are not measured and the macho minimum is as it was.

2. test/aot/perf/start.test.ts: privateKilobytes of bun-serve 2008 -> 3980 and of node-http 3036 -> 4952. Not the resolver: transparent huge pages on .bss, by the luck of the link.

  • Bun opts its heaps out of THP mapping by mapping (MI_DEFAULT_ALLOW_THP = 0, JSC's reservations, bun_alloc's arena). In smaps they all carry nh. .bss, about 2.09 MB, does not.
  • With /sys/kernel/mm/transparent_hugepage/enabled at always, the first write to a 2 MiB stretch that lies wholly inside .bss faults a whole huge page. .bss is barely over 2 MiB, so whether there is such a stretch depends on the address the link gave it.
  • The numbers fit: each program grows by 2048 KB less what it had touched in that stretch, so node-http grows by 56 KB less than bun-serve. #134177 (claude/picohttpparser-rust-replacement-aogv4j) failed the same test with 3992 and 4964: again 56 KB apart.
  • Reproduced locally with the same server code and a 1 MB dummy array in .data that moves .bss from 0x098f9c80 to 0x099f9c80, so that 0x09a00000..0x09c00000 lies inside it:
THP program as linked .bss moved by 1 MB moved, with the patch below
always bun-serve 2152 KB 4084 KB (AnonHugePages: 2048 kB in .bss) 2152 KB
always node-http 3200 KB 5136 KB 3200 KB
madvise bun-serve 2160 KB 2152 KB
madvise node-http 3192 KB 3200 KB

So any branch can hit this, and users on always kernels (the comment in mimalloc.ts names RHEL, Amazon Linux, Arch) pay 2 MB whenever a release links that way. I found no fix on main or in an open PR. It has nothing to do with DNS, so it is not in this PR. 3ffb48b moves .bss again, so this test may pass or fail on it for the same reason.

Proposed patch, tested as in the table (glibc only: musl needs the same call from another hook)
diff --git a/src/jsc/bindings/workaround-missing-symbols.cpp b/src/jsc/bindings/workaround-missing-symbols.cpp
index e1478771..d80acb86 100644
--- a/src/jsc/bindings/workaround-missing-symbols.cpp
+++ b/src/jsc/bindings/workaround-missing-symbols.cpp
@@ -482,9 +482,21 @@ static int bun_libc_csu_init(int argc, char** argv, char** envp)
 BUN_SYMVER(__libc_start_main, BUN_GLIBC_BASE);
 extern "C" int __libc_start_main(int (*)(int, char**, char**), int, char**, int (*)(int, char**, char**), void (*)(), void (*)(), void*);
 extern "C" void Bun__askForAddressSpaceToStart();
+extern "C" char __bss_start[];
+extern "C" char _end[];
+// Under THP `enabled=always` the first write to a 2 MiB stretch that lies wholly in .bss faults a whole huge page.
+// Whether there is such a stretch depends on where the link put .bss. The heaps opt out the same way (mimalloc.ts).
+static void optBssOutOfHugePages()
+{
+    uintptr_t page = static_cast<uintptr_t>(sysconf(_SC_PAGESIZE));
+    uintptr_t start = (reinterpret_cast<uintptr_t>(__bss_start) + page - 1) & ~(page - 1);
+    uintptr_t end = (reinterpret_cast<uintptr_t>(_end) + page - 1) & ~(page - 1);
+    if (start < end) madvise(reinterpret_cast<void*>(start), end - start, MADV_NOHUGEPAGE);
+}
 extern "C" int __wrap___libc_start_main(int (*main)(int, char**, char**), int argc, char** argv,
     int (*init)(int, char**, char**), void (*fini)(), void (*rtld_fini)(), void* stack_end)
 {
+    optBssOutOfHugePages();
     // libstdc++ allocates in an initializer. AddressSanitizer starts in one, and its address space can't be limited.
 #if !__has_feature(address_sanitizer)
     Bun__askForAddressSpaceToStart();

Earlier

  • #134057 on fd841cb passed. It was first marked failed because the two darwin any x64 - test-bun jobs expired waiting for an agent, as in other branches' builds that night; the retries passed.
  • #134016 on 5987b6c: one macOS arm64 shard failed node-http.test.ts, native-source-onclose-leak.test.ts and serve-body-leak.test.ts, all large loopback transfers that stalled, on a host that took 42 min for a shard that takes about 20. The first two use 127.0.0.1 only and run no resolver code. It did not repeat on #134057.

Generated by Claude Code

A mutation run over src/dns_resolver left 302 of 994 mutants alive under
dns-simulation.test.ts. These scripts kill 194 of them. They cover limits
(name, alias, port and option lengths), the latency spans behind the timeout,
address sorting labels and scopes, escapes, opcodes and extended RCODEs.

82 give the log of c-ares 1.34.9. One is a `differs:` entry for a difference
that is already listed: the end of a TCP stream that came with data.

A script may now say {x*n} for x n times, so that a 64 KB answer is one line.
Dropping a Resolver closes its channel, which statically reaches most of
src/dns_resolver. Code outside the Dns feature drops refs on a resolver (the
drop glue of RuntimeState, VM teardown), so every trimmed executable kept the
resolver. c-ares hid the same edge behind callbacks stored at query time.

- the one place a resolver is freed, and the close at VM teardown, are
  removable_body!(Dns, ..)
- the VM holds its resolver as a RefPtr like everyone else, in place of a value
  in a box with a ref that pinned it. GlobalData is gone.
- the link fails if less than 150,000 bytes go with Dns on ELF. It was 100,000,
  which the state before this met.

hello world built with --compile --aot --trim, bytes mapped, on Linux x64 with
LTO: 26,948,810 on main, 27,075,795 before this, 26,875,027 now. Not measured
for Mach-O and COFF.
c-ares reports an empty options string as out of memory, drops the
system's configuration and falls back to 127.0.0.1 without a search list.
With `RES_OPTIONS=`, `dns.getServers()` gave ["127.0.0.1"] in 1.4.4.
src/dns_resolver reads it as no options. Found by fuzzing script text.

The checked-in logs gain the script with both logs. A macOS resolver's
options go through the same function in both, which the driver for c-ares
cannot run, so that has a test of its own.
One conflict, in src/runtime/dns_jsc/dns.rs: main changed `uv_loop` of the uws
loop from a field to a method inside `on_dns_socket_state`, which this branch
removed with c-ares. This branch's side is kept. Its own Windows code asks the
VM for the loop (`vm.uv_loop()`), which did not change.
One conflict, in `cluster_raw_bind` of src/runtime/node/node_cluster_binding.rs:
this branch parses the address in Rust where main still calls `ares_inet_pton`,
and main zero-initializes the `addrinfo` hints below it with the safe
`ffi::zeroed()`. Both are kept.
No conflicts. Main now has tests of `--trim` that use DNS
(test/bundler/trim-by-use/programs/dns.ts and dns-resolve-of-bun.ts), which
pass on this branch's resolver, and a WebKit that fixes the race behind the
failure of test/aot/engine-tests-native-12.test.ts on the previous commit.
`dnsSimulate` in bun:internal-for-testing adds an entry to the table of
built-ins, which moves the numbers of the entries behind it. With those numbers
AnalyzeFlow finds one more value, `builtin(FunctionPrototype)`, for
`process.env[key]` under a key made at run time, so 11 check lines of
custom-accessor-reached-computed-key.ir no longer matched. Nothing else in the
output changes.

Only the numbers matter: with `dnsSimulate` kept and another entry taken out,
so that the numbers are main's again, the old lines match.
One conflict: main edited src/cares_sys/c_ares.rs, which this branch deletes. It
stays deleted.

The edit was that `uv__winsock_ensure` is now a `safe fn`, so its call needs no
`unsafe`. `SystemIo::new` calls it too, on Windows, and gets the same change:
the workspace denies warnings, and an unused `unsafe` is one.
Jarred-Sumner added a commit that referenced this pull request Oct 29, 2026
In that program `process.env[key]` stands beside `RegExp[key]`. With main's table
of built-ins the type of the first has no Function.prototype in it. With any
other table it has: the file is the one IR text that fails on this branch, and
also on #47071, #47074 and #47076, which add or remove built-ins of their own and
have nothing else in common with it.

Written by test/aot/tools/update-ir-tests.ts. 11 lines change, each by that one
member of the union.
Jarred-Sumner added a commit that referenced this pull request Oct 29, 2026
In that program `process.env[key]` stands beside `RegExp[key]`. With main's table
of built-ins the type of the first has no Function.prototype in it. With any
other table it has: the file is the one IR text that fails on this branch, and
also on #47071, #47074 and #47076, which add or remove built-ins of their own and
have nothing else in common with it.

Written by test/aot/tools/update-ir-tests.ts. 11 lines change, each by that one
member of the union.
@robobun

robobun commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

A note for after this PR. The callback forms of node:dns still call back from .then() on this branch (27 .then/.catch calls in src/js/node/dns.ts). So a microtask that a callback queues runs before its process.nextTick callback. Node runs the tick queue first. #43728 fixed that for node:fs and lists dns.lookup under "Not in this PR".

I have the node:dns fix on robobun/de781fd3/dns-callback-arm (one commit on main, no PR). Each native waiter holds a promise or a callback, and the callback runs from the native completion. I did not open a PR, because the commit edits cares_jsc.rs and the waiter structs that this PR replaces. dns.ts, the schema of the five natives and the test file merge clean with this branch.

On this PR the native half goes on Waiter: the promise or the callback in Waiter, the callback as an Option through Resolver::request, and a second arm in Resolver::settle. Mention me when this PR is in and I rebuild it there. If you want it on main first, say so.

…ode:http tests

`objects-closed-in-other-ways.ts` and `objects-closed-through-others.ts` fail on
macOS x64 in CI, on this branch and on others. The server drops the connection
after the exchange that the test breaks on purpose. On a busy machine the client
sends the next request on that connection before it notices, so `fetch` rejects
and plain Bun prints one more `TypeError` than the compiled program.

A request that fails before the handler ran is now sent again, up to three times.
A request that the handler did see is judged as before.

The same change as in #47316, so that it merges without a conflict.
Conflict in src/js/node/tls.ts: main dropped the type annotations from the
lines where this branch names dns_resolver_jsc.bind.ts. Main's form is kept,
with this branch's file name.

src/js/ahead-of-time/node/tls.ts, new on main, is a second text of the same
module. It named cares_jsc.bind.ts, which this branch removes, so it takes
the same rename, and the SHA-256 of the first text at its head is the one of
this branch's tls.ts.
HOSTALIASES, LOCALDOMAIN and RES_OPTIONS steer the resolver behind node:dns.
The list of variables that a trimmed executable is compared with a whole one
under lost CARES_HOSTS with c-ares and did not gain these.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants