Add renderToString to repl-sdk + ember-repl - #2165
Add renderToString to repl-sdk + ember-repl#2165NullVoxPopuli-ai-agent wants to merge 10 commits into
Conversation
|
|
Footnotes
|
|
CI is green except one test: The failure is Verified the non-renderToString gmd path was not functionally changed — only TS narrowings. Happy to rerun the failed jobs if you've got admin rights, or skip-the-flake / merge as-is. |
The build-time variant of `Compiler#compile`: when called with
`renderToString: true` (or via the new `compileToSource(...)` API),
each compiler emits a JavaScript module string instead of evaluating
the result and rendering into the DOM.
- gjs already returned its babel output as a string; the new code
path just propagates that without the blob/import dance.
- hbs now emits a self-contained `template(<source>, …)` call from
`@ember/template-compiler`.
- gmd recursively asks every live code block for its
renderToString form, then inlines each one into one self-contained
module: imports hoisted + deduped, each demo wrapped in
`const Demo<N> = (() => { …; return _component; })();`, and a
trailing `template(prose, { scope: () => ({ …names… }) })` call
whose prose has the `<div id="placeholderId">` HTML holes
rewritten to `<Demo<N> />` Glimmer invocations.
The output is a `.gjs`-shaped JS module that the host app's
content-tag + babel pipeline can precompile to wire format — letting
SSG pipelines skip the runtime kolay → ember-repl → repl-sdk →
parseMarkdown chain entirely.
`ember-repl` exposes the new path via `CompilerService.compileToSource(ext, text, options)`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ember-repl's rollup build runs `ember-tsc` against the published
`repl-sdk` type definitions (src/index.d.ts), not the source. The
first commit only added `compileToSource` to the internal
`#nestedPublicAPI` object — not to the `Compiler` class itself or
to the hand-maintained `.d.ts`. So ember-repl's
`this.compiler.compileToSource(...)` failed to type-check.
- Add a real `compileToSource` method on the `Compiler` class
forwarding to `compile(format, text, { ...options, renderToString: true })`.
- Update the public API forwarder to call the new method.
- Add `compileToSource` and the widened `compile` return type to
`src/index.d.ts`.
- Narrow the runtime-only `await this.#compile(...)` site in
ember-repl's CompilerService to the rendered-element variant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the renderToString work was a parallel code path bolted on
next to gmd's existing runtime compile — `compileToSource()` rebuilt
the demos + prose alongside an unchanged `compile()` that built a
component via runtime `template()` and a render() that DOM-appended
sub-demos into placeholder divs.
Both paths now go through a single `buildGmdModule` call. The only
forks are:
- Which `@ember/template-compiler` to import. Runtime form uses
`/runtime`; renderToString uses the build-time form so the host
app's babel pipeline precompiles the `template()` call.
- How the live runtime scope crosses the source boundary. Runtime
stashes scope on `globalThis[Symbol.for('repl-sdk:gmd-scope:N')]`
and emits a module that destructures its keys into the prose's
`scope: () => ({...})`. renderToString gets an empty scope.
gmd.compile() always returns `{ compiled: source, ... }` (or
`{ source }` when `renderToString: true`). The Compiler class's
existing blob-eval pipeline handles the runtime case — no more
special-casing for "compiler returned a component directly". gmd's
render() is now a thin renderComponent wrapper that also clears the
stashed scope on destroy. The placeholder div + appendChild loop is
gone; demo composition is via Glimmer scope.
`replacePlaceholder` now preserves the placeholder div's wrapping
(including its `class` attribute) so existing CSS targeting e.g.
`repl-sdk__demo` keeps working.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original PR widened `Compiler#compile`'s return type to a union of
`{ element, destroy } | { source }` so the same entry point could route
to either the runtime or build-time path. That widening forced every
caller (including ember-repl's `CompilerService` and the markdown
compiler's render path) to narrow the union with `as` / JSDoc casts.
Cleanup:
- Stop auto-routing inside `compile()`. `compile()` is now narrowly
typed to return `{ element, destroy }`; `compileToSource()` is its
own entry point and returns `{ source }`. A shared `#runCompile`
private method holds the announce/error wrapper so both methods
behave identically around lifecycle logging.
- In `#compileToSource`, replace the source-shape cast with `in`-
based narrowing and return a fresh `{ source }` object instead of
asserting the compiler's broader return value matches.
- Fix `compilers/markdown/parse.d.ts` to accurately type the
`codeBlocks` array (`{ format, flavor, code, placeholderId, meta }`)
instead of the stale `{ lang, format, code, name }` declaration.
With the right types in place, gmd's per-block destructure no
longer needs `@type {string}` casts on every field.
- In gmd's runtime path, store the live scope on `globalThis` under a
string key (`__replSdk__gmdScope__<n>`) read/written via
`Reflect.set` / `Reflect.deleteProperty`. `Reflect` accepts any
`PropertyKey` without requiring an index signature on
`typeof globalThis`, so the casts that wrapped the old
`Symbol.for(...)` indexing go away too.
- Drop the cast in ember-repl's `CompilerService#compile` and in
`markdown.js`'s sub-render: with the narrowed `compile()` return
type, both call sites read cleanly.
The diff against `main` for this PR no longer introduces a single `as`
or runtime-narrowing JSDoc cast; the only remaining `@type` annotations
introduced by this PR are variable-declaration types (equivalent to TS
`const foo: T = ...`), not cast expressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Compilers shouldn't be hand-rolling globalThis namespaces; repl-sdk
already has the `manual:` URL resolver + `cache.resolves` plumbing for
exactly this — registering a JS value behind a virtual import
specifier so blob-eval'd modules can reach it via the regular ES
module graph.
Surface that as `api.provide(specifier, value)` on `PublicMethods`
(returns an unregister callback), and rewire gmd's runtime path to
use it:
- gmd's `compile()` now calls `api.provide('repl-sdk:gmd-scope:N', scope)`
and emits `import * as __scope__ from 'repl-sdk:gmd-scope:N'`. The
Compiler's existing resolver chain handles the rest.
- `buildGmdModule`'s `scope` shape is now `{ specifier, keys }`
instead of `{ expression, keys }`; it emits a top-level
`import * as __scope__ from '<specifier>'` rather than a const
binding to a globalThis expression.
- gmd no longer has any direct `globalThis` access — no
`Reflect.set` / `Reflect.deleteProperty`, no string-keyed
`globalThis[...]`. The unregister callback returned from
`api.provide` is attached to the compile result so gmd's
`render()` destroy can call it.
End-to-end verified against the kolay docs-app: prose stays mounted
through rehydration on every page (no FOUC); typedoc declarations
remain stable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ycle
Previously `api.provide(specifier, value)` returned an unregister
callback that gmd had to thread through `extra` and remember to call
inside its render's destroy. That's the kind of bookkeeping individual
compilers should not have to get right — if one ever forgets (or
throws before the destroy chain is wired up), the cache.resolves entry
leaks indefinitely.
Refactor: the Compiler now hands a per-compile `CompileAPI` to every
`compile()` / `render()` invocation. It extends `PublicMethods` with
`provideScope(value) => { specifier }` — a Compiler-tracked
registration. The Compiler:
- Generates a fresh `Set<string>` of provided specifiers at the
start of every `#compile` / `#compileToSource` call.
- Wraps the rendered-element destroy chain so the registered
specifiers are released after the compiler's own teardown runs.
- Also releases them on the unhappy paths — compile/blob-eval/render
throwing before destroy is wired up, and `#compileToSource`
returning a build-time source string with no render lifecycle.
gmd's `compile()` now takes the per-compile API as its third argument
and calls `compileApi.provideScope(scope)`; its `render()`'s destroy
shrinks back to `result.destroy()`. No more `__replSdkUnregisterScope`
on extras, no `scopeSpecifier`/`scopeNonce` helpers, no top-level
state to leak.
The existing `Compiler#compile` signature gains an optional third
`api?: CompileAPI` parameter; existing compilers (gjs, hbs, markdown,
mermaid, react, svelte, vue, js) accept two arguments and continue
to type-check against the optional-third overload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3782c37 to
c6f5574
Compare
|
Related to emberjs/rfcs#1213 ( What each one produces:
So they are consecutive stages of one SSG pipeline rather than competing designs. This PR builds the module; 1213 would turn that module into HTML. Where 1213 would actually help: right now the output of Where it would not help: none of One naming note while both are in flight. |
The renderToString rework regressed 8 rendering tests. Five separate defects, all in the runtime path that now routes through a generated module instead of rendering each demo as its own island. - Scope specifiers came from a per-Compiler counter, but the es-module-shim registry is global and caches a specifier's exports on first fetch. A second Compiler reused `repl-sdk:scope:1` and got the first compile's export list, so later scope keys read as undefined. Counter is now module-scoped. - Scope keys were destructured into module scope, where they collided with demo imports of the same name (`on`, `fn`, `get`, …) and killed the module with a duplicate declaration. Keys are now read off the `__scope__` namespace. - Live hbs codefences compiled to source with an empty scope and a build-time `template` import, so they lost top-level scope and added a second conflicting `template` binding. gmd now threads its template module and scope namespace into the sub-compile. - splitModule and rewriteDefaultExport matched line-anchored `import` and `export default` inside template literals, silently corrupting demos that quote Ember code as a string. Both now share a scanner that tracks string, template, comment, and brace context. - Inlining a demo dropped the `data-repl-output` wrapper that a separately rendered island produced, changing DOM callers depend on. replacePlaceholder emits it again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
what in the world is this file, are we manually building source now? does babel and oxc not do it for us?
There was a problem hiding this comment.
You were right, and babel was already there. Fixed in 45c8c20.
gjs.js runs babel.transformAsync on every demo, and this file then re-parsed that output with String.split('\n') and regexes. @babel/standalone exposes packages.{parser,generator,traverse,types}, so the AST was available the whole time.
That regex parsing was not a theoretical problem. The "build your own REPL" sample assigns a whole component to a template literal, and the lines inside that string start with import and export default at column 0. They were hoisted out of the string and rewritten. Two demos both importing on from @ember/modifier also collided into a duplicate declaration.
The merge now runs as a babel plugin: parse each demo, register its imports in a shared table, rename its top-level bindings per demo. Same binding imported twice collapses to one declaration; the same name from different modules stays distinct. splitModule, mergeImports, wrapAsConst, rewriteDefaultExport and their scanner are gone.
Second change in the same commit, and the more important one: the runtime path no longer generates source at all. 84444936 unified runtime and renderToString onto one generated module, which is what forced a live scope object through a serialized boundary and needed the provideScope virtual-module bridge to get it back. Rendering each live codefence as its own island needs none of that — the scope object goes to template() by reference, as it did before. renderToString keeps the inlining, because one self-contained module is the point of it.
Net effect across both commits: -432 lines, and provideScope / CompileAPI are deleted.
On oxc: not usable here as far as I can tell. @oxc-project/* is build tooling for vite/rolldown; nothing in repl-sdk resolves it in-browser. Babel is the one already loaded at runtime.
Green locally: lint 63/63, node 78, chrome and firefox 0 failures across ember-repl, tutorial, and limber.
render-to-string.js re-parsed babel's own output with line splitting and regexes, so anything that merely looked like module syntax was treated as module syntax. A demo that quotes Ember code in a template literal had its imports hoisted out of the string and its export default rewritten. Babel is already resolved and running in this path — the gjs compiler transforms every demo through it — so the merge runs through babel too. Demo modules are parsed, their imports registered in a shared table, and their top-level bindings renamed per demo. Two demos can now declare the same name, and the same binding imported twice collapses to one declaration while the same name from different modules stays distinct. Also stop generating source on the runtime path. Rendering each live codefence as its own island needs no source assembly at all: the caller's scope object is passed to template() by reference. Unifying the two paths is what forced a live scope through a generated module, and every defect in the previous commit came from that. renderToString keeps the inlining because a single self-contained module is the whole point of it; the runtime path no longer pays for it. Removes splitModule, mergeImports, wrapAsConst, rewriteDefaultExport, the top-level scanner, and the provideScope/CompileAPI bridge they needed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
repl-sdk does not depend on @babel/standalone; the consumer supplies it through the resolve config, as ember-repl does. Resolving it up front made every renderToString call require it, including documents with no live demos, which need no AST work at all. Resolve lazily, and assert with a message naming what to provide rather than failing somewhere inside the merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2 participants
Summary
Adds a build-time variant of
Compiler#compileto repl-sdk, surfaced asCompiler#compileToSource(format, text, options)(and asoptions.renderToString: trueoncompile). Each compiler emits a JavaScript module string instead of evaluating and rendering — useful for SSG / pre-rendering pipelines that want to hand the output to their host app's bundler.template(<source>, …)call sourced from@ember/template-compiler(build-time variant).const Demo<N> = (() => { …; return _component; })();template(prose, { scope: () => ({ …names… }) })whose prose has the<div id="placeholderId">HTML holes rewritten to<Demo<N> />Glimmer invocations.The output is a
.gjs-shaped JS module that the host app's content-tag + babel pipeline precompiles to wire format. Pre-rendering pipelines can skip the runtime kolay → ember-repl → repl-sdk → parseMarkdown chain entirely.ember-replexposes the new path viaCompilerService.compileToSource(ext, text, options).Motivation
kolay's docs pre-render viavite-ember-ssr(SSG). Without this, even though the markdown HTML is server-rendered, each.gjs.mdpage still has to do a full runtime compile in the browser before its demos hydrate — leading to a visible flash of unstyled content during rehydration.With renderToString,
kolay'sgjs-mdbuild plugin can collapse the runtime chain into a single bundler-precompiled module per page.Test plan
splitModule,mergeImports,wrapAsConst,replacePlaceholder,buildGmdModule) covering single-line / multi-line / side-effect / attributed imports, IIFE wrapping, import dedup, placeholder rewriting, scope-list generation, and zero-demo case.gjs-mdplugin to use this and confirm the docs-app FOUC disappears (follow-up PR).🤖 Generated with Claude Code