Skip to content

Flow hangs crawling pnpm workspaces (symlink cycles in node_modules) #9529

Closed

Description

@robhogan

When workspace packages are linked into each other's node_modules, as pnpm does with linkWorkspacePackages: true, flow ls and flow check never finish.

We ran into this in the wild while adding pnpm tests to Metro. Under pnpm, flow check got past ~1M parsed files before I killed it - under Yarn's hoisted layout the same project has ~23k files and checks in a few seconds. The draft PR adding pnpm support has to work around it in .flowconfig:

react/metro#2110

Minimal repro (0.340.0, macOS):

mkdir -p repro/packages/{a,b,c,d,e,f} && cd repro
printf '[options]\nall=true\n' > .flowconfig
for p in a b c d e f; do
  echo '// @flow' > packages/$p/index.js
  mkdir packages/$p/node_modules
  for q in a b c d e f; do [ $p != $q ] && ln -s ../../$q packages/$p/node_modules/$q; done
done
flow ls # doesn't return

With only two packages linked to each other, flow ls does return, but lists each file 33 times.

The crawl follows symlinks with jwalk, whose loop check compares the raw link target (../../b) with the paths of the directories above it, so it never fires for a relative link. The walk only stops when the OS refuses to resolve the path (ELOOP, after 32 links on macOS), visiting every path through the link graph up to that depth.

The only workaround I've found, and what the Metro PR does, is to [ignore] the linked packages under every node_modules, eg **/node_modules/metro*/**. That also hides any unrelated npm package that happens to match (in Metro's case, metro-memory-fs).

I have a fix - PR to follow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions