Skip to content

Draft: Fix handling of alpha when using luma file - #1276

Open
j-b-m wants to merge 1 commit into
mltframework:masterfrom
j-b-m:work/luma2
Open

Draft: Fix handling of alpha when using luma file#1276
j-b-m wants to merge 1 commit into
mltframework:masterfrom
j-b-m:work/luma2

Conversation

@j-b-m

@j-b-m j-b-m commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Currently, the luma transition behaves differently when using a luma file or not on images with an alpha channel. When a luma file is used, it always behaves as if alpha_over was enabled. Here are some screenshot comparing a transition between 2 images of triangles with a transparent background. Each image below contains 4 different cases:

  • top left, normal dissolve with alpha_over disabled
  • top right, normal dissolve with alpha_over enabled
  • bottom left, luma file "Vertical bar" with alpha_over disabled
  • bottom right, luma file "Vertical bar" with alpha_over enabled

Frame 0: only first image is visible (red triangle in top right corner):

master-frame-000

Last frame of the transition, you can see the behavior difference when a luma is used or not (bottom left should be similar to top left):

master-frame-124

With the changes in this merge request, the result is now as expected:

fixed-frame-124

The only question is do we want to maintain backwards compatibility, and how ?
Feedback is welcome.

@j-b-m

j-b-m commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Hmm don't know how but it's only now that I realize that there is an issue with my change, bottom left part of the blue triangle is missing..
Will work on it!

@j-b-m j-b-m changed the title Fix handling of alpha when using luma file Draft: Fix handling of alpha when using luma file Jul 30, 2026
@ddennedy
ddennedy requested a review from Copilot July 30, 2026 18:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the core luma transition implementation to make alpha handling consistent when a luma file is used, by ensuring the luma-based compositing path respects the alpha_over setting instead of implicitly behaving as if alpha_over were enabled.

Changes:

  • Add an alpha_over parameter to the luma compositing functions (YUV422/RGBA32/RGBA64) and thread it through transition_get_image.
  • Gate the “alpha-over” normalization/alpha-channel updates in the luma compositing loops behind alpha_over.
Comments suppressed due to low confidence (1)

src/modules/core/transition_luma.c:438

  • In the translucent+YUV422 luma path, the alpha pointers are advanced linearly across pixels (immediately after this block) but are not re-based per row/field or adjusted for differing src/dst row strides. If frames are interlaced (field_count=2) or src/dst dimensions differ, alpha reads/writes can become misaligned and produce incorrect alpha output.
                    }
                    *q = sample_mix(*q, *p++, mix_b);
                    q++;

Comment on lines +424 to +435
mix_a = calculate_mix(1.0f - value, alpha_dest ? *alpha_dest : 255);
if (invert && alpha_src) {
float mix2 = mix_b + mix_a;
*alpha_src = 255 * mix2;
if (mix2 != 0.f)
mix_b /= mix2;
} else if (!invert && alpha_dest) {
float mix2 = mix_b + mix_a;
*alpha_dest = 255 * mix2;
if (mix2 != 0.f)
mix_b /= mix2;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants