Skip to content

Commit 91c14fe

Browse files
jpowersdeveffect-bottim-smart
committed
Add Glob module and portable FileSystem.glob (#9438)
Co-authored-by: Effect Bot <tech-ops@effectful.co> Co-authored-by: Tim Smart <hello@timsmart.co>
1 parent 89fd619 commit 91c14fe

11 files changed

Lines changed: 1432 additions & 59 deletions

File tree

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
"effect": minor
3+
"@effect/platform-node-shared": minor
4+
"@effect/platform-deno": minor
5+
---
6+
7+
Add experimental `Glob` path matching and a shared `FileSystem.glob` implementation for Node, Bun, and Deno.
8+
9+
- Breaking: custom `FileSystem` implementations, including inputs to `FileSystem.make`, must provide `readDirectoryEntries`; mapping `readDirectory` names to `{ name, type: "Unknown" }` is supported. Wrappers must apply directory-listing changes to both methods.
10+
- `FileSystem.make` now derives `glob` instead of accepting it. Apply custom overrides afterward with `FileSystem.of({ ...fs, glob: customGlob })`.
11+
- Glob patterns use `/` separators and backslash escapes; Node extglobs no longer expand. Results use `/` separators and are relative to `root` unless the pattern is absolute. Deno wildcards now skip dotfiles unless explicitly named or `dot: true`; unreadable wildcard-reached directories are skipped, while read failures at the root or a leading literal directory propagate.

‎packages/effect/src/FileSystem.ts‎

Lines changed: 65 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import * as Cause from "./Cause.ts"
1717
import * as Context from "./Context.ts"
1818
import * as Effect from "./Effect.ts"
1919
import { pipe } from "./Function.ts"
20+
import * as Glob from "./internal/glob.ts"
2021
import * as Layer from "./Layer.ts"
2122
import * as Option from "./Option.ts"
2223
import { badArgument, type PlatformError, systemError } from "./PlatformError.ts"
@@ -130,13 +131,36 @@ export interface FileSystem {
130131
gid: number
131132
) => Effect.Effect<void, PlatformError>
132133
/**
133-
* Glob a directory.
134+
* Finds paths under `root` that match a glob pattern.
135+
*
136+
* **Details**
137+
*
138+
* Patterns follow `Glob.make`: wildcards skip names that start with `.`
139+
* unless the pattern segment starts with `.` or `dot` is `true`, and
140+
* `dir/**` includes `dir` itself. Results are `/`-separated paths relative
141+
* to `root` (or absolute for an absolute pattern), visited depth-first with
142+
* each directory's entries sorted by name. `**` does not traverse symbolic
143+
* links to directories, but a link named by another segment is followed.
144+
* Paths that match an `exclude` pattern are dropped, and excluded
145+
* directories are not read. Entries whose type cannot be determined, such
146+
* as broken links, are treated as files.
147+
*
148+
* Read failures at the root or a leading literal directory propagate. Missing
149+
* paths return an empty result; unreadable wildcard-reached directories are
150+
* skipped.
151+
*
152+
* **Gotchas**
153+
*
154+
* Wildcard matching is case-sensitive. Leading literal segments use `stat`,
155+
* so their case sensitivity follows the file system and results retain the
156+
* pattern's spelling.
134157
*/
135158
readonly glob: (
136159
pattern: string,
137160
options?: {
138161
readonly root?: string | undefined
139162
readonly exclude?: ReadonlyArray<string> | undefined
163+
readonly dot?: boolean | undefined
140164
}
141165
) => Effect.Effect<Array<string>, PlatformError>
142166
/**
@@ -247,6 +271,22 @@ export interface FileSystem {
247271
readonly recursive?: boolean | undefined
248272
}
249273
) => Effect.Effect<Array<string>, PlatformError>
274+
/**
275+
* Lists a directory with the type of each entry.
276+
*
277+
* **Details**
278+
*
279+
* Returns the same bare names as `readDirectory(path)`, excluding `.` and
280+
* `..`, in no particular order. Types describe entries, not symlink targets.
281+
* `"Unknown"` is allowed when the listing cannot determine the type, and
282+
* `glob` then falls back to `stat`.
283+
*
284+
* **Gotchas**
285+
*
286+
* Wrappers that change `readDirectory`, such as by prefixing paths, must
287+
* apply the same change here for `glob` to list the correct directory.
288+
*/
289+
readonly readDirectoryEntries: (path: string) => Effect.Effect<ReadonlyArray<DirectoryEntry>, PlatformError>
250290
/**
251291
* Read the contents of a file.
252292
*/
@@ -478,6 +518,21 @@ export type OpenFlag =
478518
*/
479519
export const FileSystem: Context.Service<FileSystem, FileSystem> = Context.Service("effect/FileSystem")
480520

521+
/**
522+
* A directory entry with its type, independent of any symlink target.
523+
* `"Unknown"` means the listing could not determine the type.
524+
*
525+
* @see {@link FileSystem} `readDirectoryEntries`, which returns these entries
526+
*
527+
* @stability unstable
528+
* @category models
529+
* @since 4.0.0
530+
*/
531+
export interface DirectoryEntry {
532+
readonly name: string
533+
readonly type: File.Type
534+
}
535+
481536
/**
482537
* Creates a FileSystem implementation from a partial implementation.
483538
*
@@ -490,8 +545,10 @@ export const FileSystem: Context.Service<FileSystem, FileSystem> = Context.Servi
490545
* **Details**
491546
*
492547
* This function takes a partial FileSystem implementation and automatically provides
493-
* default implementations for `exists`, `readFileString`, `stream`, `sink`, and
494-
* `writeFileString` methods based on the provided core methods.
548+
* default implementations for `exists`, `glob`, `readFileString`, `stream`, `sink`,
549+
* and `writeFileString` methods based on the provided core methods. `glob` walks
550+
* the tree with `readDirectoryEntries`, using `stat` and `readLink` for entries
551+
* whose type is `"Unknown"`.
495552
*
496553
* @see {@link makeNoop} for a testing stub that accepts method overrides without requiring a complete implementation
497554
* @see {@link layerNoop} for providing a no-op `FileSystem` as a `Layer` in tests
@@ -501,11 +558,12 @@ export const FileSystem: Context.Service<FileSystem, FileSystem> = Context.Servi
501558
* @since 4.0.0
502559
*/
503560
export const make = (
504-
impl: Omit<FileSystem, typeof TypeId | "exists" | "readFileString" | "stream" | "sink" | "writeFileString">
561+
impl: Omit<FileSystem, typeof TypeId | "exists" | "glob" | "readFileString" | "stream" | "sink" | "writeFileString">
505562
): FileSystem =>
506563
FileSystem.of({
507564
...impl,
508565
[TypeId]: TypeId,
566+
glob: Glob.make(impl),
509567
exists: (path) =>
510568
pipe(
511569
impl.access(path),
@@ -692,6 +750,9 @@ export const makeNoop = (fileSystem: Partial<FileSystem>): FileSystem =>
692750
readDirectory(path) {
693751
return Effect.fail(notFound("readDirectory", path))
694752
},
753+
readDirectoryEntries(path) {
754+
return Effect.fail(notFound("readDirectoryEntries", path))
755+
},
695756
readFile(path) {
696757
return Effect.fail(notFound("readFile", path))
697758
},

‎packages/effect/src/Glob.ts‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
/**
2+
* Matches `/`-separated paths without accessing the file system, using the
3+
* same pattern syntax as `FileSystem.glob`.
4+
*
5+
* @stability experimental
6+
* @since 4.0.0
7+
*/
8+
import { format } from "./Formatter.ts"
9+
import { dual } from "./Function.ts"
10+
import { type Inspectable, NodeInspectSymbol } from "./Inspectable.ts"
11+
import * as internal from "./internal/glob.ts"
12+
import { type Pipeable, pipeArguments } from "./Pipeable.ts"
13+
14+
const TypeId = "~effect/Glob"
15+
16+
/**
17+
* A compiled glob pattern.
18+
*
19+
* **Details**
20+
*
21+
* `pattern` is the source text and `dot` records whether wildcards match names
22+
* that start with `.`. Match paths against it with {@link match}.
23+
*
24+
* @see {@link make} for compiling a pattern
25+
*
26+
* @stability experimental
27+
* @category models
28+
* @since 4.0.0
29+
*/
30+
export interface Glob extends Pipeable, Inspectable {
31+
readonly [TypeId]: typeof TypeId
32+
readonly pattern: string
33+
readonly dot: boolean
34+
}
35+
36+
interface GlobImpl extends Glob {
37+
readonly compiled: internal.Compiled
38+
}
39+
40+
const GlobProto = {
41+
[TypeId]: TypeId,
42+
toJSON(this: Glob) {
43+
return { _id: "Glob", pattern: this.pattern, dot: this.dot }
44+
},
45+
toString(this: Glob) {
46+
return `Glob(${format(this.toJSON())})`
47+
},
48+
[NodeInspectSymbol](this: Glob) {
49+
return this.toJSON()
50+
},
51+
pipe() {
52+
return pipeArguments(this, arguments)
53+
}
54+
}
55+
56+
/**
57+
* Compiles a glob pattern into a reusable `Glob`.
58+
*
59+
* **Details**
60+
*
61+
* Patterns use `/` as the separator. `*` matches any run of characters within
62+
* one path segment, `?` matches one character, and `[...]` matches one
63+
* character from a set, with `!` or `^` negating it and POSIX classes such as
64+
* `[[:digit:]]` allowed. `{a,b}` expands to each alternative and may nest. A
65+
* segment that is exactly `**` matches zero or more whole segments. A trailing
66+
* `/` matches only directories, and `\` makes the next character literal.
67+
*
68+
* **Gotchas**
69+
*
70+
* Wildcards and `**` skip names that start with `.` unless the pattern segment
71+
* itself starts with `.` or `dot` is `true`; they never match `.` or `..`.
72+
* `x/.` is the same as `x/`, and a `.` or `..` segment after `**` makes the
73+
* pattern match nothing, as does the empty pattern. UNC patterns and paths
74+
* such as `//server/share` are not supported and never match. Braces and
75+
* commas inside `[...]` are class members. Extended glob syntax and leading
76+
* `!` negation are not interpreted, but ordinary wildcards still apply within
77+
* them (for example, `?(a)` matches `x(a)`). Malformed syntax, such as an
78+
* unclosed `[`, matches literally. Braces also stay literal if expansion would
79+
* exceed 10,000 alternatives.
80+
*
81+
* **Example** (Matching source files)
82+
*
83+
* ```ts import.meta.vitest
84+
* import { Glob } from "effect"
85+
*
86+
* const sources = Glob.make("src/**\/*.{ts,tsx}")
87+
*
88+
* Glob.match(sources, "src/index.ts") // => true
89+
* Glob.match(sources, "src/ui/Button.tsx") // => true
90+
* Glob.match(sources, "src/.cache/index.ts") // => false
91+
* Glob.match(sources, "test/index.ts") // => false
92+
* ```
93+
*
94+
* @stability experimental
95+
* @category constructors
96+
* @since 4.0.0
97+
*/
98+
export const make = (pattern: string, options?: {
99+
/** Whether wildcards and `**` match names that start with `.`. */
100+
readonly dot?: boolean | undefined
101+
}): Glob => {
102+
const dot = options?.dot ?? false
103+
const self = Object.create(GlobProto)
104+
self.pattern = pattern
105+
self.dot = dot
106+
self.compiled = internal.compile([pattern], dot)
107+
return self
108+
}
109+
110+
/**
111+
* Tests whether a `/`-separated path matches a `Glob`.
112+
*
113+
* **Details**
114+
*
115+
* Empty segments and `./` segments in the path are ignored. A trailing `/`
116+
* marks the path as a directory, which a pattern ending in `/` requires.
117+
*
118+
* **Example** (Filtering paths)
119+
*
120+
* ```ts import.meta.vitest
121+
* import { Glob, pipe } from "effect"
122+
*
123+
* const tests = Glob.make("**\/*.test.ts")
124+
* const paths = ["src/a.ts", "src/a.test.ts", "test/b.test.ts"]
125+
*
126+
* paths.filter((path) => Glob.match(tests, path)) // => ["src/a.test.ts", "test/b.test.ts"]
127+
* pipe(tests, Glob.match("./src/a.test.ts")) // => true
128+
* ```
129+
*
130+
* @stability experimental
131+
* @category predicates
132+
* @since 4.0.0
133+
*/
134+
export const match: {
135+
(path: string): (self: Glob) => boolean
136+
(self: Glob, path: string): boolean
137+
} = dual(2, (self: Glob, path: string): boolean => internal.match((self as GlobImpl).compiled, path))
138+
139+
/**
140+
* Escapes glob syntax so a string matches only itself.
141+
*
142+
* **Example** (Matching a literal file name)
143+
*
144+
* ```ts import.meta.vitest
145+
* import { Glob } from "effect"
146+
*
147+
* const name = "report[1].txt"
148+
*
149+
* Glob.match(Glob.make(name), name) // => false
150+
* Glob.match(Glob.make(Glob.escape(name)), name) // => true
151+
* ```
152+
*
153+
* @stability experimental
154+
* @category utils
155+
* @since 4.0.0
156+
*/
157+
export const escape = (input: string): string => input.replace(/[\\*?[\]{}]/g, "\\$&")

‎packages/effect/src/index.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,12 @@ export * as Formatter from "./Formatter.ts"
290290
*/
291291
export * as Function from "./Function.ts"
292292

293+
/**
294+
* @stability experimental
295+
* @since 4.0.0
296+
*/
297+
export * as Glob from "./Glob.ts"
298+
293299
/**
294300
* @stability unstable
295301
* @since 4.0.0

0 commit comments

Comments
 (0)