Skip to content

SDK: make injected glue and sysroot paths reproducible - #943

Closed
brandonpayton wants to merge 1 commit into
mainfrom
fix/sdk-stable-glue-debug-paths
Closed

SDK: make injected glue and sysroot paths reproducible#943
brandonpayton wants to merge 1 commit into
mainfrom
fix/sdk-stable-glue-debug-paths

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 14, 2026

Copy link
Copy Markdown
Member

Why

The software development kit (SDK) must produce the same portable artifact regardless of where a developer checked out the repository. DWARF is compiler-generated debugging information stored inside a WebAssembly file; it must not expose private machine paths.

Make Wasm artifacts independent of the checkout paths that contain Kandelo's SDK glue and sysroot.

The Homebrew PCRE2 source gate found 20 occurrences of the local Kandelo checkout in the final pcre2test executable. All 20 were in DWARF custom sections (.debug_str and .debug_line). The SDK compiler wrapper caused them by injecting absolute paths for channel_syscall.c, compiler_rt.c, cxxrt.c, and --sysroot into every executable link.

This is an SDK reproducibility contract, not a PCRE2 formula concern. Fixing it in an individual formula would leave every other SDK consumer vulnerable to the same host-path leak.

What changed

  • Add file, debug, and macro prefix maps for SDK-owned glue and sysroot paths.
  • Map wasm32 and wasm64 sysroots to distinct stable identities under /usr/src/kandelo-sdk.
  • Append the SDK-specific maps after caller flags. Clang resolves overlapping prefix maps in option order, so this prevents a broader caller-owned mapping from retaining a checkout-relative suffix for the SDK glue.
  • Leave package source trees and dependency prefixes under the build system's ownership.
  • Document the division of responsibility in the SDK guide.

Merge order

Merge #913 first. Both PRs change the SDK compiler wrapper and therefore the
registry package cache generation. After #913 lands, this PR must be rebased
onto the exact merged main head and its staging workflow rerun there. Building
the pre-#913 generation would produce artifacts that cannot be reused after
that rebase, so the current staging run was intentionally cancelled.

Validation

  • Rebuilt musl from this exact main base in a clean validation worktree.
  • Ran the complete SDK Vitest suite against that fresh sysroot: 7 files and 65 tests passed.
  • Added linked and compile-only artifact regressions that assert raw SDK paths are absent and stable identities are present.
  • Ran ./scripts/dev-shell.sh bash scripts/check-abi-version.sh: the ABI snapshot and generated C/TypeScript bindings are in sync.
  • Ran git diff --check.

The SDK package does not declare a TypeScript compiler, so a separate tsc --noEmit command was not available; Vitest exercised all SDK TypeScript units and integrations. Runtime, conformance, and browser suites were not run because this changes compiler arguments and build metadata, not guest or host runtime behavior.

ABI and artifacts

This does not change the Kandelo ABI and does not require an ABI_VERSION bump.

It can change package artifact bytes when SDK-injected paths appear in debug or macro metadata. Those packages must be rebuilt and receive their normal package/formula revision bump rather than reusing earlier bytes. The PCRE2 and Bash Homebrew source gate will be rerun against this exact PR head.

Map SDK-owned glue and sysroot paths to fixed identities in file, debug, and macro metadata. Package build systems remain responsible for mapping their own source and dependency paths.

This removes checkout-specific Kandelo paths from linked Wasm debug sections without changing the ABI.
@brandonpayton
brandonpayton force-pushed the fix/sdk-stable-glue-debug-paths branch from aa808bd to a0ab3fd Compare July 14, 2026 06:11
@brandonpayton

Copy link
Copy Markdown
Member Author

Independent review: accepted at exact head a0ab3fd0e835e443e982104792c600660f58a034.

The review found one documentation mismatch: the guide initially named only the wasm32 stable sysroot identity. That is corrected on this head; it now names /usr/src/kandelo-sdk/sysroot for wasm32 and /usr/src/kandelo-sdk/sysroot64 for wasm64.

No functional blocker remains. A direct LLVM 21.1.7 overlap experiment confirmed the required ordering behavior: a broad caller map followed by the narrow SDK map emits only the stable SDK glue identity across file, debug, and macro material. The worktree-local musl build, complete SDK suite (65/65), ABI snapshot/version gate, and diff check pass. No ABI surface changes.

This changes SDK-generated metadata and possible __FILE__ bytes. Downstream package revisions will be decided from exact before/after hashes; Bash and PCRE2 are the first rebuild gate.

The current staging run was intentionally cancelled because #913 must merge first. After #913 lands, rebase this PR and build the combined current package generation once.

@brandonpayton

Copy link
Copy Markdown
Member Author

Absorbed by rebase-merged #967. Main is now 6efb411 with the sealed tree b133bfa018f7e2bc84c4845edf2f945b2f844954, and ABI 41 activation completed successfully in run 29550069085. The absorption audit confirmed this PR\x27s behavior and tests are present in that merged tree, so this source PR is closed and its branch is deleted.

@brandonpayton
brandonpayton deleted the fix/sdk-stable-glue-debug-paths branch July 17, 2026 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant