feat([issue-2036]): re-render generated images from annotations (phase 2)#2086
Merged
Conversation
…e 2) Adds a user-triggered "Re-render with annotations" flow to the annotate page: flatten the source image + drawn strokes and feed the flattened PNG back through the existing local-FLUX img2img regen as the init image, so the marks reshape the render. Provider/model is named in a confirm dialog before any AI call runs (no cold-bootstrap), with optional prompt + strength. - server: /image-gen/:filename/regenerate gains `annotated` — seeds initImagePath from the saved sketch PNG sidecar (getSketchPngPath), higher default denoise, GPU-only (rejects the light method), 400 when no annotation saved. - client: rerenderWithAnnotations API + Re-render button/modal on MediaAnnotate, gated on getRegenAvailability; saves the annotation then queues the re-render. - tests: buildRegenParams annotated override, getSketchPngPath, the annotated route (no-annotation/ light / enqueue) paths, and the MediaAnnotate flow. Refs #2036
… root The local runner re-validates initImagePath against its approved image roots (gallery/image-refs/visual-templates) and silently drops anything else. The sketch sidecar lives in data/media-sketches/, which isn't approved — so the annotated re-render would have discarded the markup and produced a plain text-to-image render. Stage a snapshot of the flattened annotation into the image-refs root the runner accepts (also freezing it at enqueue time), and add a regression test asserting the enqueued initImagePath resolves via resolveImageInputPath.
The staged img2img init snapshot went to data/image-refs as annot-<uuid>.png, which imageRefsGc.js (issue #1214) doesn't match (it sweeps init-/ref- prefixes only) — so a canceled/failed annotated re-render would leak the full-size file indefinitely. Name it init-<uuid>.png: it IS this job's init image and the existing GC now sweeps it once unreferenced. Test pins the init- naming.
…sure
- P3: stage the flattened annotation into image-refs only AFTER the light-method,
no-annotation, and backend-availability gates pass, so a rejected re-render
never leaves an orphaned init-<uuid>.png behind.
- P2: thread the source image's model through the regen-availability preflight
(GET /regen/availability?filename=, getRegenAvailability({sourceModelId})) so
the annotate confirm dialog names the EXACT model the POST will enqueue on a
multi-model install — honoring the "disclose the model before the render"
requirement. Client passes the source filename.
…regenInfo - P2: the /regen/availability sidecar read used the raw ?filename query value; resolveGalleryImage only validates the basename, so a ../-prefixed value could traverse out of PATHS.images. Read the sidecar by basename(resolvedGalleryPath). - P2: clear regenInfo at the start of the availability effect so a slow probe on a new :mediaKey can't leave the previous image's model/bounds visible in the re-render dialog.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sketch & Annotation Canvas — Phase 2 (Refs #2036)
Ships phase 2: a user-triggered "Re-render with annotations" action that feeds your markup back into img2img. Draw over a generated image, then re-render it guided by the marks.
What shipped (Phase 2)
/media/annotate/:mediaKey) — a toolbar button opens a confirmation dialog that names the exact local model the render will use (source-aware:GET /regen/availability?filename=resolves the model a regen of that image would run, so multi-model installs disclose the real model — no cold-bootstrap AI). Optional prompt + strength slider (bounds from availability), then Save-then-enqueue.POST /image-gen/:filename/regenerategainsannotated: true: it flattens the saved annotation (source + strokes) and stages it as the img2img init image through the existing local-FLUX regen pipeline. Higher default denoise (0.5) so the marks actually reshape the render; GPU-only (rejects the light method); 400 when no annotation is saved.regenOf).Design notes (from review)
initImagePathagainst its approved image roots, anddata/media-sketches/isn't one — so the flattened annotation is staged into theimage-refsroot asinit-<uuid>.png(whichimageRefsGc.jssweeps once unreferenced), only after every gate passes so a rejected request leaves nothing behind. This also freezes the markup at enqueue time.Out of scope (remains open on #2036)
Umbrella issue stays open (
Refs, notCloses) per the issue's phased convention; I'll comment there on what shipped/remains.Tests
buildRegenParamsannotated override;getSketchPngPath; the annotated regenerate route (no-annotation 400, light+annotated 400, staged init path resolves under an approved root — the regression guard for the silently-dropped-init-image bug,init-GC-covered naming); source-aware/regen/availability?filename=(threads source model, traversal-safe basename read).MediaAnnotatere-render flow (saves annotation → enqueues, model disclosed, unavailable-gating).Test plan
https://claude.ai/code/session_013aGBsru3A2x8MHTxeaYXBp