diff mbox series

[FFmpeg-devel] avfilter/vf_chromakey: The chromakey filter preserves non-opaque alpha transparency.

Message ID 20200611202731.8672-1-gavin.smith@playerbites.com
State New
Headers show
Series [FFmpeg-devel] avfilter/vf_chromakey: The chromakey filter preserves non-opaque alpha transparency.
Related show

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Gavin Smith June 11, 2020, 8:27 p.m. UTC
This is to address trac ticket #8724.  The filter did not preserve alpha transparency. Items that were transparent in the input would appear black on the output or pixels that were semi-tranparent would appear opaque.
---
 libavfilter/vf_chromakey.c | 43 ++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 18 deletions(-)

Comments

Timo Rothenpieler June 11, 2020, 9:03 p.m. UTC | #1
On 11.06.2020 22:27, Gavin Smith wrote:
> This is to address trac ticket #8724.  The filter did not preserve alpha transparency. Items that were transparent in the input would appear black on the output or pixels that were semi-tranparent would appear opaque.
> 

What is the performance impact for inputs without an alpha channel?

Generally looks fine to me, but might need hidden behind an option, as 
to not break existing setups that rely on this filter 
discarding/ignoring the input alpha channel.
Gavin Smith June 11, 2020, 10:31 p.m. UTC | #2
On 11/06/2020 22:03, Timo Rothenpieler wrote:
> On 11.06.2020 22:27, Gavin Smith wrote:
>> This is to address trac ticket #8724.  The filter did not preserve 
>> alpha transparency. Items that were transparent in the input would 
>> appear black on the output or pixels that were semi-tranparent would 
>> appear opaque.
>>
>
> What is the performance impact for inputs without an alpha channel?

Firstly, I'm new to this world of filters.  Now my patch only applies to 
chromakey and not chromahold. On that note, is it not the case that the 
chromakey mandates that all its inputs surfaces contain alpha (as per 
query_formats function)? Would any AVPixelFormat that does not match 
query_formats get converted to a format containing alpha transparency?

>
> Generally looks fine to me, but might need hidden behind an option, as 
> to not break existing setups that rely on this filter 
> discarding/ignoring the input alpha channel.

Yes. No problem. "preserve_transparency" and default to the equivalent 
of 'false'?

> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Timo Rothenpieler June 12, 2020, 11:59 a.m. UTC | #3
On 12.06.2020 00:31, Gavin Smith wrote:
> 
> On 11/06/2020 22:03, Timo Rothenpieler wrote:
>> On 11.06.2020 22:27, Gavin Smith wrote:
>>> This is to address trac ticket #8724.  The filter did not preserve 
>>> alpha transparency. Items that were transparent in the input would 
>>> appear black on the output or pixels that were semi-tranparent would 
>>> appear opaque.
>>>
>>
>> What is the performance impact for inputs without an alpha channel?
> 
> Firstly, I'm new to this world of filters.  Now my patch only applies to 
> chromakey and not chromahold. On that note, is it not the case that the 
> chromakey mandates that all its inputs surfaces contain alpha (as per 
> query_formats function)? Would any AVPixelFormat that does not match 
> query_formats get converted to a format containing alpha transparency?

But it still adds new code, and given it's in the very hot path of the 
filter, it can easily add a performance penalty.

>>
>> Generally looks fine to me, but might need hidden behind an option, as 
>> to not break existing setups that rely on this filter 
>> discarding/ignoring the input alpha channel.
> 
> Yes. No problem. "preserve_transparency" and default to the equivalent 
> of 'false'?

That seems a bit clunky. Maybe something like "use_alpha"?
Gavin Smith June 12, 2020, 3:58 p.m. UTC | #4
On 12/06/2020 12:59, Timo Rothenpieler wrote:
> On 12.06.2020 00:31, Gavin Smith wrote:
>>
>> On 11/06/2020 22:03, Timo Rothenpieler wrote:
>>> On 11.06.2020 22:27, Gavin Smith wrote:
>>>> This is to address trac ticket #8724. The filter did not preserve 
>>>> alpha transparency. Items that were transparent in the input would 
>>>> appear black on the output or pixels that were semi-tranparent 
>>>> would appear opaque.
>>>>
>>>
>>> What is the performance impact for inputs without an alpha channel?
>>
>> Firstly, I'm new to this world of filters.  Now my patch only applies 
>> to chromakey and not chromahold. On that note, is it not the case 
>> that the chromakey mandates that all its inputs surfaces contain 
>> alpha (as per query_formats function)? Would any AVPixelFormat that 
>> does not match query_formats get converted to a format containing 
>> alpha transparency?
>
> But it still adds new code, and given it's in the very hot path of the 
> filter, it can easily add a performance penalty.
Sure. I understand. I'll see what numbers I can get.  Having a quick 
cursory look over the code, I wonder if my code actually might be 
slightly more performant because it checks the case if a0 != 0. However, 
this jmp/cmp may negatively impact performance.
>
>>>
>>> Generally looks fine to me, but might need hidden behind an option, 
>>> as to not break existing setups that rely on this filter 
>>> discarding/ignoring the input alpha channel.
>>
>> Yes. No problem. "preserve_transparency" and default to the 
>> equivalent of 'false'?
>
> That seems a bit clunky. Maybe something like "use_alpha"?
Sure.
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Gavin Smith June 12, 2020, 9:39 p.m. UTC | #5
On 12/06/2020 16:58, Gavin Smith wrote:
> On 12/06/2020 12:59, Timo Rothenpieler wrote:
>> On 12.06.2020 00:31, Gavin Smith wrote:
>>>
>>> On 11/06/2020 22:03, Timo Rothenpieler wrote:
>>>> On 11.06.2020 22:27, Gavin Smith wrote:
>>>>> This is to address trac ticket #8724. The filter did not preserve 
>>>>> alpha transparency. Items that were transparent in the input would 
>>>>> appear black on the output or pixels that were semi-tranparent 
>>>>> would appear opaque.
>>>>>
>>>>
>>>> What is the performance impact for inputs without an alpha channel?
>>>
>>> Firstly, I'm new to this world of filters.  Now my patch only 
>>> applies to chromakey and not chromahold. On that note, is it not the 
>>> case that the chromakey mandates that all its inputs surfaces 
>>> contain alpha (as per query_formats function)? Would any 
>>> AVPixelFormat that does not match query_formats get converted to a 
>>> format containing alpha transparency?
>>
>> But it still adds new code, and given it's in the very hot path of 
>> the filter, it can easily add a performance penalty.
> Sure. I understand. I'll see what numbers I can get.  Having a quick 
> cursory look over the code, I wonder if my code actually might be 
> slightly more performant because it checks the case if a0 != 0. 
> However, this jmp/cmp may negatively impact performance.

vf_chromakey wll now have 2 implementations of chromakey depending on 
whether "use_alpha" is selected. What are your feelings on the best way 
to approach this?

1. Add an additional function pointer to ChromakeyContext: In 
do_chromakey_slice(..), call the said func ptr in the loop each iteration?

2. Add a conditional to do_chromakey_slice(...): In the loop iteration, 
check "use_alpha" variable and execute and jmp to pertinent code?

3. Any other suggestions?

Thanks.

>>
>>>>
>>>> Generally looks fine to me, but might need hidden behind an option, 
>>>> as to not break existing setups that rely on this filter 
>>>> discarding/ignoring the input alpha channel.
>>>
>>> Yes. No problem. "preserve_transparency" and default to the 
>>> equivalent of 'false'?
>>
>> That seems a bit clunky. Maybe something like "use_alpha"?
> Sure.
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/libavfilter/vf_chromakey.c b/libavfilter/vf_chromakey.c
index 4b1669d084..7206f44441 100644
--- a/libavfilter/vf_chromakey.c
+++ b/libavfilter/vf_chromakey.c
@@ -47,7 +47,7 @@  typedef struct ChromakeyContext {
                     int jobnr, int nb_jobs);
 } ChromakeyContext;
 
-static uint8_t do_chromakey_pixel(ChromakeyContext *ctx, uint8_t u[9], uint8_t v[9])
+static uint8_t do_chromakey_pixel(ChromakeyContext *ctx, uint8_t a, uint8_t u[9], uint8_t v[9])
 {
     double diff = 0.0;
     int du, dv, i;
@@ -62,13 +62,13 @@  static uint8_t do_chromakey_pixel(ChromakeyContext *ctx, uint8_t u[9], uint8_t v
     diff /= 9.0;
 
     if (ctx->blend > 0.0001) {
-        return av_clipd((diff - ctx->similarity) / ctx->blend, 0.0, 1.0) * 255.0;
+        return av_clipd((diff - ctx->similarity) / ctx->blend, 0.0, 1.0) * (float)a;
     } else {
-        return (diff > ctx->similarity) ? 255 : 0;
+        return (diff > ctx->similarity) ? a : 0;
     }
 }
 
-static uint16_t do_chromakey_pixel16(ChromakeyContext *ctx, uint16_t u[9], uint16_t v[9])
+static uint16_t do_chromakey_pixel16(ChromakeyContext *ctx, uint16_t a, uint16_t u[9], uint16_t v[9])
 {
     double max = ctx->max;
     double diff = 0.0;
@@ -84,9 +84,9 @@  static uint16_t do_chromakey_pixel16(ChromakeyContext *ctx, uint16_t u[9], uint1
     diff /= 9.0;
 
     if (ctx->blend > 0.0001) {
-        return av_clipd((diff - ctx->similarity) / ctx->blend, 0.0, 1.0) * max;
+        return av_clipd((diff - ctx->similarity) / ctx->blend, 0.0, 1.0) * (float)a;
     } else {
-        return (diff > ctx->similarity) ? max : 0;
+        return (diff > ctx->similarity) ? a : 0;
     }
 }
 
@@ -131,13 +131,17 @@  static int do_chromakey_slice(AVFilterContext *avctx, void *arg, int jobnr, int
 
     for (y = slice_start; y < slice_end; ++y) {
         for (x = 0; x < frame->width; ++x) {
-            for (yo = 0; yo < 3; ++yo) {
-                for (xo = 0; xo < 3; ++xo) {
-                    get_pixel_uv(frame, ctx->hsub_log2, ctx->vsub_log2, x + xo - 1, y + yo - 1, &u[yo * 3 + xo], &v[yo * 3 + xo]);
+            uint8_t *a = frame->data[3] + frame->linesize[3] * y;
+            const uint8_t ao = a[x];
+
+            if (ao != 0) {
+                for (yo = 0; yo < 3; ++yo) {
+                    for (xo = 0; xo < 3; ++xo) {
+                        get_pixel_uv(frame, ctx->hsub_log2, ctx->vsub_log2, x + xo - 1, y + yo - 1, &u[yo * 3 + xo], &v[yo * 3 + xo]);
+                    }
                 }
+                a[x] = do_chromakey_pixel(ctx, ao, u, v);
             }
-
-            frame->data[3][frame->linesize[3] * y + x] = do_chromakey_pixel(ctx, u, v);
         }
     }
 
@@ -163,15 +167,18 @@  static int do_chromakey16_slice(AVFilterContext *avctx, void *arg, int jobnr, in
 
     for (y = slice_start; y < slice_end; ++y) {
         for (x = 0; x < frame->width; ++x) {
-            uint16_t *dst = (uint16_t *)(frame->data[3] + frame->linesize[3] * y);
-
-            for (yo = 0; yo < 3; ++yo) {
-                for (xo = 0; xo < 3; ++xo) {
-                    get_pixel16_uv(frame, ctx->hsub_log2, ctx->vsub_log2, x + xo - 1, y + yo - 1, &u[yo * 3 + xo], &v[yo * 3 + xo]);
+            uint16_t *a = (uint16_t *)(frame->data[3] + frame->linesize[3] * y);
+            const uint16_t ao = a[x];
+
+            if (ao != 0) {
+                for (yo = 0; yo < 3; ++yo) {
+                    for (xo = 0; xo < 3; ++xo) {
+                        get_pixel16_uv(frame, ctx->hsub_log2, ctx->vsub_log2, x + xo - 1, y + yo - 1, &u[yo * 3 + xo], &v[yo * 3 + xo]);
+                    }
                 }
-            }
 
-            dst[x] = do_chromakey_pixel16(ctx, u, v);
+                a[x] = do_chromakey_pixel16(ctx, ao, u, v);
+            }
         }
     }