Skip to content

Fix JSDoc @param rest parsing (#320) - #321

Open
kungfooman wants to merge 3 commits into
mainfrom
fix-jsdoc-param-parsing-320
Open

kungfooman wants to merge 3 commits into
mainfrom
fix-jsdoc-param-parsing-320

Conversation

@kungfooman

Copy link
Copy Markdown
Owner

Fixes #320

Primer

JSDoc rest annotations come in two spellings. The documented spelling puts the dots inside the braces (@param {...number[]} values): the ... marks the parameter variadic and the number[] inside gives the element type. The bare spelling puts only the dots there (@param {...number} values): in the TypeScript grammar a rest parameter must still be an array type, so ...number gathers the call arguments into number[] and the parameter behaves as one fixed element of that array type, while ...number[] gathers into number[][] so each argument is checked against number[]. Both spellings reach the transpiler as the same string, and the type expander erases the marker: expandTypeDepFree('...number') returns {array, number} the same way it returns it for a plain number[], so after expansion no trace of ... remains.

The overload layer converts each @overload block into a positional tuple in src-transpiler/overloadToTuple.js, where a dotDot flag on a tupleMember makes the validator treat it as variadic (src-runtime/expandTupleElements.js, src-runtime/inspectOverloads.js). Because the expander erased the marker, the old code recovered it with a second regex over the raw section string (/@param\s+\{\s*\.\.\.[^}]*\}\s+(\w+)/g), then applied an array-shape gate: rest applied only when the expanded type was an array whose element was itself an array. Two parses of one string, two regex dialects, and neither knew about the other. Anything the regexes disagreed on silently became a different type.

What broke, gap by gap, each verified against tsc --noEmit --allowJs --checkJs --strict on the same file:

  1. {T[]} ...name (dots before the name) fragmented into dotted-path parts (['', '', '', 'values']), tripped the console.assert in parseJSDoc.js, emitted Skipping warnings, and built a phantom empty-named member plus a non-rest member. tsc rejects the spelling at parse (TS1003 Identifier expected, TS8032 qualified name). RTI accepted it with phantom members.
  2. Bare {...T} lost variadicity in overloads: {array, number} fails the nested-array gate, so @param {...number} values became a fixed number[] member. tsc treats the same tag as variadic-over-number-vs-array-shape (accepts g(1), rejects the gathered g([1,2]) with [number] vs number), and RTI's own plain path agreed with tsc; only the overload tuple diverged, and no probe pinned it because the generateTypeChecks.spec.js rest tests and the overloads-input.mjs sum fixture exercise only ...T[].
  3. The gate could not distinguish nesting origins: for {...number[][]} the unwrap-then-check-array logic yields variadic-over-number[] by counting one level from dots and one from brackets, i.e. string counting, not structure.
  4. Trailing prose survived both regexes by luck (both stop at the name token); pinned by nothing.
  5. Optional rest ([... ]) silently dropped dotDot and stayed fixed-arity. tsc rejects it (TS2370 a rest parameter must be of an array type, equivalently no call matches). RTI accepted it as fixed-arity optional, and no fixture covered it.

What changed

  1. src-transpiler/parseJSDocTags.js (new): a tag-level scanner that walks each @param line, extracts the brace content and the head name token with the existing extractCurlyContent/extractNameAndOptionality helpers, and records whether the rest marker sits before the name ({T[]} ...name). Dots inside braces, optional brackets, dotted paths, same-line tags, and trailing prose each have a spec pinning the scan.
  2. src-transpiler/tagRestElementType.js (new): resolves one tag's rest shape. A brace marker with an optional name is the TS2370 error and resolves to never, so every call warns. A ...T[] marker (written array) turns variadic over the written element type. A ...T marker turns variadic over the gathered element itself, matching the tsc accept/reject matrix call by call. Anything else passes through untouched.
  3. src-transpiler/overloadToTuple.js: consumes the tags instead of the regex. Members whose marker sits before the name are dropped (matching tsc's parse rejection); every other member resolves through tagRestElementType. Branch bodies use braces per review convention, and the poison loop is a plain for with continue instead of a map/filter chain.
  4. src-transpiler/expandTypeDepFree.js: a leading ... element union now splits the gathered element at the first top-level |, ,, ;, or ? before wrapping, so ...number|string gathers only number ((number[])|string, as tsc parses ...number: number) instead of swallowing the whole tail. Depth tracking skips bracketed groups (Array<>, parens, tuples, objects).
  5. Permanent coverage: test/typechecking/overload-rest-input.mjs plus overload-rest-output.mjs (transpiler parity) and overload-rest-errors.json (runtime: exactly documented('x',1,'bad'), bare('x',[1,2]), optionalRest([1,2]), misplaced(1), misplaced([1]) throw, in order); overloadToTuple.spec.js gains bare-rest, optional-rest, and misplaced-rest tests; new parseJSDocTags.spec.js and tagRestElementType.spec.js wired into test_runtime.js. The old expectations fail on the pre-fix tree (bare rest fixed-arity, optional rest non-never, poison phantom kept), and the full gate passes on the new tree.

Worked demonstration (node run-jsdoc.js --emit on the minimal overload input, emission verbatim): docs in are the two-block sum doc above; emission out is a single inspectOverloads([...arguments], [{tuple: {type: tuple, elements: [{tupleMember, label: string}, {tupleMember, values: {array, number}, dotDot: true}]}, templates: {}}], 'sum', 'arguments') guard; at runtime sum('x', 1, 2) passes, sum('x', 1, 'bad') warns Argument of type ["x", 1, "bad"] is not assignable to parameter of type [label: string, ...values: number[]], and no other call shape changes.

Latent holes surfaced

  1. {number[]} ...values in the plain (non-overload) path still writes phantom {'': ...} entries through parseJSDoc with Skipping warnings: the tuple path now drops the member, but parseJSDoc itself still fragments leading dots into dotted-path parts and the Asserter default path warns Missing param. Tolerated deliberately: tsc rejects the spelling before checking, the errors file pins the overload verdict, and restructuring parseJSDoc's name grammar is out of scope.
  2. parseJSDoc's name regex admits [/]/= prose but not {/}: a @param line without braces never scans, and a description containing a literal @param at a line start with a non-tag gap is skipped by the gap rule. The tag scanner mirrors the old line rule (gap must be empty, *, or /**, and must not contain @), so both layers agree; a prose line that defeats one defeats both, which the same-line spec pins.
  3. tagRestElementType decides array-ness from the written text (endsWith('[]'), startsWith('Array<')) rather than the expanded node, because the expander erases which array level the dots contributed. A parenthesized written array (...(number[])) or readonly prefix would misclassify the written shape; tsc probes for readonly show it still gathers variadically, so the text check errs toward the documented path and the nested-union probes (...number|string, ...(number|string), ...Array<number>) pin the covered spellings.

* Scan raw @PARAM tags so rest markers survive type expansion

* Resolve bare, optional, and misplaced rest spellings per tsc

* Split gathered rest unions at top-level separators like tsc

* Pin overload rest parity with fixture, specs, and errors file
@kungfooman kungfooman self-assigned this Nov 4, 2026
@kungfooman kungfooman added type assertions Everything related to type assertions. typings Everything related to type declarations runtime transpiler labels Nov 4, 2026
Drop the written-text array sniff in tagRestElementType: the expander
already gathers ...T into one array level, so variadicity is just
marker plus isArrayNode. Covers paren and generic spellings for free.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

runtime transpiler type assertions Everything related to type assertions. typings Everything related to type declarations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSDoc param parsing: where the string hacking lives, and what it hides

1 participant