Skip to content

Support pnpm via pnpm import from yarn.lock - #2110

Open
robhogan wants to merge 1 commit into
robhogan/jsx-a11y-drop-core-js-purefrom
pr2109
Open

robhogan wants to merge 1 commit into
robhogan/jsx-a11y-drop-core-js-purefrom
pr2109

Conversation

@robhogan

@robhogan robhogan commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Metro is developed with Yarn v1 and yarn.lock remains the source of truth, but this lets pnpm users work in the repo by generating a lockfile with pnpm import, useful for disk-efficient local development, and CI (next PR).

It adds a small pnpm-workspace.yaml (pnpm doesn't read workspaces from package.json), git-ignores the generated pnpm-lock.yaml and documents the two commands in the README.

linkWorkspacePackages is required to prevent pnpm resolving our exact-version internal dependencies like "metro-core": "0.88.0" from npm.

This depends on Flow 0.341 and the stack up to meta/flow#9535 - previously Flow didn't support pnpm layouts.

Changelog: Internal

Test plan:
On a fresh clone of this PR, with each of pnpm 11.28.12 and 12.21.0, pnpm import then pnpm install --frozen-lockfile succeed, packages/metro/node_modules/metro-core is a symlink to ../../metro-core, and pnpm exec jest --no-watchman packages passes (167 suites, 3262 tests), as do build, lint, format-check, verify-api-snapshots, typecheck-ts and test-smoke.

flow check passes under pnpm 12.15.0 and under Yarn.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 26, 2026
@robhogan
robhogan changed the base branch from main to pr2116 October 26, 2026 16:13
@robhogan
robhogan added this pull request to stack #2119 October 26, 2026 16:14
Base automatically changed from pr2116 to main November 2, 2026 14:40
@robhogan
robhogan force-pushed the pr2109 branch 2 times, most recently from db15151 to c8f5b82 Compare November 16, 2026 15:37
meta-codesync Bot pushed a commit to meta/flow that referenced this pull request Nov 18, 2026
Summary:
pnpm links each package from its store into the `node_modules` of every package that depends on it, and the crawl reaches the package's files through each of those links. Files reached through a symlink are reported by their real path, so every link to a package repeats all of its files. On react/metro under pnpm, `flow ls` lists 125,443 files, of which 22,572 are unique.

This keeps the set of real paths reported so far and skips repeats. The set only holds files the crawl canonicalises (under `node_modules` or below a symlink), and only for the duration of the crawl.

This doesn't fix the hang in #9529 - that needs the crawl to stop following the links themselves.

Pull Request resolved: #9533

Test Plan:
New `node_modules_symlink_duplicates` test, which creates its links in `test.sh` (native symlinks on Windows, checked with Git Bash). On `main` it lists the store package's files four times each.

On react/metro under pnpm (react/metro#2110, macOS), `flow ls` goes from 125,443 lines to 22,574. The two remaining duplicates are file symlinks beside their targets, which the crawl also reports by their logical path. The full `flow dev-tools runtests` suite passes.

Reviewed By: SamChou19815

Differential Revision: D132631246

fbshipit-source-id: d2047e815e0b938525a758c80c1c15e29e2397b5
meta-codesync Bot pushed a commit to meta/flow that referenced this pull request Nov 19, 2026
Summary:
Refs #9529

Flow's file crawl follows every directory symlink. jwalk's loop check compares the raw link target with the paths of the directories above it, so a relative target like `../../b` never matches, jwalk descends with a longer path and walks again:

https://github.com/Byron/jwalk/blob/a5b1ea634fcacee06f7ef6e9411e0cfecfccb128/src/core/dir_entry.rs#L219-L229

Packages linked into each other's `node_modules` (eg `packages/a/node_modules/b -> ../../b` and back, as pnpm links workspace packages) are walked until the OS refuses to resolve the path, once for every path through the links, which in practice never ends.

Files reached through a symlink under the root are reported by their real path, so following one to a directory under the root that the crawl reads anyway reports nothing new. This stops following those links, deciding with `path_is_in_crawl_scope` (which can now answer for a whole directory, counting `[include]` globs that cover it) and the crawl's prune check. Each skipped link is recorded in `SymlinkMap`, which now also maps paths under a directory link, so module resolution still works through it without IO. Other links are still followed - a follow-up stops at cycles among them.

One difference: the initial check now resolves through a link whose own path is `[ignore]`d but whose target isn't, as rechecks and saved-state init already do. Before, it only resolved the files it had visited through the link.

Pull Request resolved: #9534

Test Plan:
`node_modules_symlink_duplicates` now also links three workspace packages to each other, which `main` never finishes crawling. `node_modules_with_symlinks` now also resolves through two links, through a link inside an ignored directory, and through a link whose own path is ignored. New `symlinks_out_of_root` test for links the crawl must still follow. Each fails without the part of this change it covers, and the full `flow dev-tools runtests` suite passes (macOS).

On react/metro installed with pnpm (react/metro#2110) with its `[ignore]` workaround removed, `main` hangs on `flow ls` and `flow full-check`. With this, `flow full-check` finishes in 2-3s with the same result as with the workaround.

Reviewed By: SamChou19815

Differential Revision: D132920137

fbshipit-source-id: c601b95997f3a927aace7858e18c25e962bcdae4
meta-codesync Bot pushed a commit to meta/flow that referenced this pull request Nov 19, 2026
Summary:
Fixes #9529

Last part of the stack #9532, #9533, #9534 (split from #9530).

#9534 stops the file crawl following a symlink to a directory it reads anyway, but it still follows any other symlink, eg out of the root or out of the directory passed to `flow ls`, and goes round a cycle of them until the OS refuses to resolve the path. With a few packages linked to each other, eg `flow ls packages/a` in a pnpm workspace, that never ends in practice.

This stops following a symlink to a directory the crawl passed through to reach it. Every walk now ends, though a dense web of links out of the crawl is still walked once per path through it, which grows quickly with the number of links.

Pull Request resolved: #9535

Test Plan:
`symlinks_out_of_root` now also has a cycle out of the root, whose files `main` lists once per lap. The full `flow dev-tools runtests` suite passes (macOS).

On react/metro installed with pnpm (react/metro#2110), `flow ls packages/metro-config` hangs on `main`. With this, `flow ls` of each workspace package finishes in under a second, and `flow full-check` gives the same result in the same time.

Reviewed By: SamChou19815

Differential Revision: D133227488

fbshipit-source-id: 889fee75094aed805ff360dbd21939cbe8cf465b
@robhogan
robhogan force-pushed the pr2109 branch 2 times, most recently from fc346b3 to 0f533a2 Compare November 21, 2026 09:03
@robhogan
robhogan removed this pull request from stack #2119 November 21, 2026 09:05
@robhogan
robhogan changed the base branch from main to robhogan/no-hoisted-type-imports November 21, 2026 09:05
@robhogan
robhogan added this pull request to stack #2268 November 21, 2026 09:05
@robhogan
robhogan removed this pull request from stack #2268 November 21, 2026 09:07
@robhogan
robhogan changed the base branch from robhogan/no-hoisted-type-imports to robhogan/remove-type-only-edges November 21, 2026 09:08
@robhogan
robhogan added this pull request to stack #2269 November 21, 2026 09:08
@robhogan
robhogan removed this pull request from stack #2269 November 21, 2026 09:23
@robhogan
robhogan changed the base branch from robhogan/remove-type-only-edges to robhogan/jsx-a11y-drop-core-js-pure November 21, 2026 09:23
@robhogan
robhogan added this pull request to stack #2271 November 21, 2026 09:23
@robhogan
robhogan force-pushed the pr2109 branch 2 times, most recently from 8e6a43f to a0a3d2f Compare November 21, 2026 09:30
@robhogan
robhogan marked this pull request as ready for review November 21, 2026 09:49
@robhogan
robhogan requested review from huntie and vzaidman November 21, 2026 09:58
Metro is developed with Yarn v1 and `yarn.lock` stays the source of truth, but this lets pnpm users work in the repo by generating a lockfile with `pnpm import`. It adds a `pnpm-workspace.yaml` (pnpm doesn't read `workspaces` from `package.json`), git-ignores the generated `pnpm-lock.yaml` and documents the two commands in the README.

The workspace file also sets `linkWorkspacePackages` - without it, pnpm (since v9) resolves our exact-version internal dependencies like `"metro-core": "0.88.0"` from npm, which fails after a version bump that isn't published yet. pnpm 11+ also fails the install over a dependency's unapproved build script, and with #2270 none has one.

Linking workspace packages gives each one its own `node_modules` pointing at its siblings, where Yarn hoists them. Those links form cycles (`packages/metro-config/node_modules/metro` -> `packages/metro`, whose `node_modules/metro-config` links back), which Flow crawls without hanging since 0.341.0, and with #2257 no type imports rely on hoisting, so Flow needs no config changes.

Changelog: Internal

Test plan:
On a fresh clone of this PR, with each of pnpm 11.28.12 and 12.21.0, `pnpm import` then `pnpm install --frozen-lockfile` succeed, `packages/metro/node_modules/metro-core` is a symlink to `../../metro-core`, and `pnpm exec jest --no-watchman packages` passes (167 suites, 3262 tests), as do `build`, `lint`, `format-check`, `verify-api-snapshots`, `typecheck-ts` and `test-smoke`.

`flow check` passes under pnpm 12.15.0 and under Yarn.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant