Skip to content

fix: handle missing Flux VAE config values and None prompt inputs#807

Open
octaviomejiadiaz wants to merge 1 commit into
ostris:mainfrom
octaviomejiadiaz:fix/flux-vae-config-and-prompt-normalization
Open

fix: handle missing Flux VAE config values and None prompt inputs#807
octaviomejiadiaz wants to merge 1 commit into
ostris:mainfrom
octaviomejiadiaz:fix/flux-vae-config-and-prompt-normalization

Conversation

@octaviomejiadiaz

Copy link
Copy Markdown

Problem

When Flux is loaded from a ComfyUI single-file checkpoint (.safetensors), the bundled VAE often omits scaling_factor and shift_factor from its config. This causes a crash during sample generation:

`
TypeError: unsupported operand type(s) for +: 'Tensor' and 'NoneType'

diffusers/pipelines/flux/pipeline_flux.py line 1010

latents = (latents / self.vae.config.scaling_factor) + self.vae.config.shift_factor
`

A second related issue: the CLIP/T5 tokenisers raise a TypeError: TextEncodeInput must be Union[TextInputSequence, ...] when they receive None or False as the unconditional (negative) prompt, which can happen through the dropout path or when callers pass bare strings/None to encode_prompt.

Changes

oolkit/stable_diffusion_model.py

  • After loading the VAE, inject the correct Flux defaults (scaling_factor=0.3611, shift_factor=0.1159) only when the values are absent - existing configs are untouched.
  • Wrap the prompt argument in a list inside encode_prompt for the Flux branch, so a bare string or None never reaches encode_prompts_flux as a non-list.

oolkit/train_tools.py

  • Normalise every element of prompts in encode_prompts_flux to a str, replacing None / False with '', before the tokeniser call.

Testing

Verified end-to-end with lux1-dev.safetensors loaded from a local ComfyUI checkpoint directory (24 GB VRAM, 8-bit quantisation, �damw8bit optimiser). Training now proceeds past the baseline sample generation step.

Notes

The correct Flux VAE constants (�.3611 / �.1159) match the values published in the official �lack-forest-labs/FLUX.1-dev model card.

When Flux is loaded from a ComfyUI single-file checkpoint, the VAE may
not carry `scaling_factor` or `shift_factor` in its config, causing a
TypeError during sample generation. Inject the correct Flux defaults
(0.3611 / 0.1159) when those fields are absent.

Also normalise prompts in encode_prompts_flux so the CLIP/T5 tokenisers
never receive None or False values, and wrap the prompt in a list in
encode_prompt when a bare string or None is passed for Flux.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant