From patchwork Fri Dec 9 00:28:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 39660 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp792076pzb; Thu, 8 Dec 2022 16:28:22 -0800 (PST) X-Google-Smtp-Source: AA0mqf7SZ8RQ2c4LGpvIoiyzKpGeqkLG6Hr3nRrSpAsJI6uVuYxrKkQjRWvXMUHgObt++5ZmgXy4 X-Received: by 2002:aa7:dbd1:0:b0:461:1a4b:60df with SMTP id v17-20020aa7dbd1000000b004611a4b60dfmr3446080edt.24.1670545702441; Thu, 08 Dec 2022 16:28:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670545702; cv=none; d=google.com; s=arc-20160816; b=hs9ghWCrTHnRs12OgGeJw1fqKmdVfyr17ZFbjjaTrCqQwXKwbqm2Ex4umXRr6R19rN iGDc3IhD1+YMRAK8/hX+mCKyFmH4wadByavdZIigL4x0ozABSfOSAfgpHD1Y49Tc65Ys 4f7U9dCtdjRmtiwR940bQ2yIp6OV//Zv1AI0xYtYVVizA+oMpKhMzQYVdvbnIpndFJUE nMO8doTBM/NtfbflfIZXRaXvAZ+5iYR/0CkKB+rzAl3W4X8rNU8Zj1VvsCksxZcwbkkT BzE0MU5DC6hcjBqipVkaTvAlOSE1bsXCHdanYzy7VdjMxaycgXpTwAwgXnurv5O2i3u4 fTbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=b9n7g2vAcrbTYDlDTo+OksuTkO4+XZNng52F0VsfXyc=; b=rhmpEA4bYbV5VeW3HK/rf5SLJz+gHI+5n+7mIkdP68H6UN7ST+VNUJCyGKcc2xM2mO at+Ylr8nV7P1f0/iApyJABOz3rq4UpFD+zW/xXnyK757kFc0NTteROESw03AqztV6y2H mPl5WKSe/owh7EXpkIiEP4tDGjoLfRgLm7an/oE/I7irlJFsBgAGtt9jZaRPC6UX4L+4 3BCLzPuhpqggWLmVDiSi7tDKW74jcd50tATwqsXR9Q5b/C7DhnGcmQYE/RTknD/9KrbA LjVn3r+2SqWl5gRiQRFNw7KhB0fxXNL4BdlJHDcNzgdRqmpEg4qLvLqGHb1l0AgOueHV rfkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=fMSUrrqg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l8-20020a056402254800b00469dd6e394asi31578edb.579.2022.12.08.16.28.21; Thu, 08 Dec 2022 16:28:22 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=fMSUrrqg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E60A868BC2F; Fri, 9 Dec 2022 02:28:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 76F6068BC2E for ; Fri, 9 Dec 2022 02:28:10 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 6D11C4B5FC; Fri, 9 Dec 2022 01:28:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1670545689; bh=C1t7qkc8CZUTtX80g2OKdIxAytPDgB1Raxd6agecieQ=; h=From:To:Cc:Subject:Date:From; b=fMSUrrqgSD91GMtVfReAujpeheCoF2rjgq6fJj7OsdX7MC3fgPPEZqivrP+xheEJh CM2e7q+fMozEXZkZ79bryPvykbdpIeNZPIhIvQ+S7bmu142kfPY8wkXP9igJNysiN8 ab9BRQ8sKk7ghqsqbB94Pj7Bx8BdN0J01mVyhy2U= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 9 Dec 2022 01:28:05 +0100 Message-Id: <20221209002806.22424-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.38.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avfilter/vf_blackdetect: support full-range YUV X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Fbk011gNGAXN From: Niklas Haas This filter currently makes the distinction between limited and full range by testing for the deprecated YUVJ pixel formats at link setup time. This is deprecated and should be improved to perform the detection based on the per-frame metadata. Rewrite it to calculate the black pixel threshold at the time of filtering a frame, when metadata about the frame's color range is known. Doing it this way has the small side benefit of being able to handle streams with variable metadata, and is not a meaningful performance cost. Signed-off-by: Niklas Haas --- libavfilter/vf_blackdetect.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/libavfilter/vf_blackdetect.c b/libavfilter/vf_blackdetect.c index 99ff1ac606..c937248169 100644 --- a/libavfilter/vf_blackdetect.c +++ b/libavfilter/vf_blackdetect.c @@ -102,8 +102,6 @@ static int config_input(AVFilterLink *inlink) BlackDetectContext *s = ctx->priv; const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); const int depth = desc->comp[0].depth; - const int max = (1 << depth) - 1; - const int factor = (1 << (depth - 8)); s->depth = depth; s->nb_threads = ff_filter_get_nb_threads(ctx); @@ -113,16 +111,10 @@ static int config_input(AVFilterLink *inlink) if (!s->counter) return AVERROR(ENOMEM); - s->pixel_black_th_i = ff_fmt_is_in(inlink->format, yuvj_formats) ? - // luminance_minimum_value + pixel_black_th * luminance_range_size - s->pixel_black_th * max : - 16 * factor + s->pixel_black_th * (235 - 16) * factor; - av_log(s, AV_LOG_VERBOSE, - "black_min_duration:%s pixel_black_th:%f pixel_black_th_i:%d picture_black_ratio_th:%f\n", + "black_min_duration:%s pixel_black_th:%f picture_black_ratio_th:%f\n", av_ts2timestr(s->black_min_duration, &s->time_base), - s->pixel_black_th, s->pixel_black_th_i, - s->picture_black_ratio_th); + s->pixel_black_th, s->picture_black_ratio_th); return 0; } @@ -182,6 +174,14 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) AVFilterContext *ctx = inlink->dst; BlackDetectContext *s = ctx->priv; double picture_black_ratio = 0; + const int max = (1 << s->depth) - 1; + const int factor = (1 << (s->depth - 8)); + const int full = picref->color_range == AVCOL_RANGE_JPEG || + ff_fmt_is_in(picref->format, yuvj_formats); + + s->pixel_black_th_i = full ? s->pixel_black_th * max : + // luminance_minimum_value + pixel_black_th * luminance_range_size + 16 * factor + s->pixel_black_th * (235 - 16) * factor; ff_filter_execute(ctx, black_counter, picref, NULL, FFMIN(inlink->h, s->nb_threads)); From patchwork Fri Dec 9 00:28:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 39661 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp792181pzb; Thu, 8 Dec 2022 16:28:31 -0800 (PST) X-Google-Smtp-Source: AA0mqf7ZT9Jl7iPngGYq76m5NrlGJPORZZngzrywqagTEkrMnzoub9w7oJZwcPE3/NmBwgdSndwN X-Received: by 2002:a17:906:4e1a:b0:7c1:3583:9d6e with SMTP id z26-20020a1709064e1a00b007c135839d6emr1580232eju.65.1670545711642; Thu, 08 Dec 2022 16:28:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670545711; cv=none; d=google.com; s=arc-20160816; b=voFFeafXx7AvCXKJZBer56nsMqSgG6lvvtVzidXKBUnZCUWHCXjdXM04lTDlg3dAEg i3w4XH+kUQ9G9nCNvQlndLWtJnhZmOTuoEhGH+8S1vQceYk4Xi+NKTXjhdqKn8oAKEbZ kkBjzNbV4D4sWchtcUV+G6rYjPVIaadYgANy0iJHlXJnU2eb93boODZngasx4+PcImJ9 bEifnxwdW0Yu9I232ZxLSASFmCKbpCoGOda2blzUaFnkxuzklb1H0Ap744DIu2PCQxcq jzITlqWyVVVGNOCiQBHRR9U+fiX0pCGieuvAIEiN0awnYhq1QxQPTZowT2vIwEZ3k7Ed 1u/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=IcWh8ydPrQQLkumATHTfPCN0OHMul8luQu4QAjbHT5Y=; b=ZyACV2OHdfGFk3yMaZ+FsIJKtrRcw+U5Ji6ANSA3+P+EIYdKve+MwiuDEN8tAj+rNK 2JssLoK3Ds55/VNn514ViMW4TXiqj++tmoBgb8PJNzB6+viveObxDu2uYINRQRzDO1cW 04i9+WECevv7nFvI4XOYFJYinsxntMcbih+BdZJxUFlb3kH+0VLB/xuzqMadu6yzktiI Uvn+T8ooUHM5BZNAjM/a5k5Ef+zDFEYAX9/oMqRS6GKrfU8SDtVdjn9NlvLkrTHjIeIj ReiaHB15uH0BYGaWxRkiW8XfkQrQoPf18XTMin6I+hFDzRR7wlKfjPUmfsYnLUAbAV3B vT0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=fPCAL22T; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fe17-20020a1709072a5100b007c10f6a46c7si5221928ejc.355.2022.12.08.16.28.31; Thu, 08 Dec 2022 16:28:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=fPCAL22T; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EE77C68B9E8; Fri, 9 Dec 2022 02:28:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 426C368B9E8 for ; Fri, 9 Dec 2022 02:28:11 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id E8C374B5FE; Fri, 9 Dec 2022 01:28:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1670545691; bh=zV0urcdmR/3aEshtfKb38kekQO7tXTqMDaUEZv+tdS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fPCAL22TcRDuIPLWHSj745kAZNKUVIMrJ9SglCLMqfiGLDG69hGItmvicVyh2dtjX Mv7tB33hDcuQbJwhDwATfP+6cb23gohSrlp/ZlKhveTCFIQUOVN7CPDQJd/R7CIZWl hqKw2eBcBklTwHKfNqXIQkIJnlaGoRR5AlOy5EyY= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 9 Dec 2022 01:28:06 +0100 Message-Id: <20221209002806.22424-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221209002806.22424-1-ffmpeg@haasn.xyz> References: <20221209002806.22424-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avfilter/vf_tinterlace: support full-range YUV X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qkrckLa2D0s4 From: Niklas Haas This filter, when used in the "pad" mode, currently makes the distinction between limited and full range solely by testing for YUVJ pixel formats at link setup time. This is deprecated and should be improved to perform the detection based on the per-frame metadata. In order to make this distinction based on color range metadata, which is only known at the time of filtering frames, for simplicity, we simply allocate two copies of the "black" frame - one for limited range and the other for full range metadata. This could be done more dynamically (e.g. as-needed or simply by blitting the appropriate pixel value directly), but this change is relatively simple and preserves the structure of the existing code. This commit actually fixes a bug in FATE - the new output is correct for the first time. The previous md5 ref was of a frame that incorrectly combined full-range pixel data with limited-range black fields. The corresponding result has been updated. Signed-off-by: Niklas Haas --- libavfilter/tinterlace.h | 2 +- libavfilter/vf_tinterlace.c | 26 ++++++++++++++------ tests/ref/fate/filter-pixfmts-tinterlace_pad | 2 +- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/libavfilter/tinterlace.h b/libavfilter/tinterlace.h index 4059ebf81a..37b6c10c08 100644 --- a/libavfilter/tinterlace.h +++ b/libavfilter/tinterlace.h @@ -70,7 +70,7 @@ typedef struct TInterlaceContext { int vsub; ///< chroma vertical subsampling AVFrame *cur; AVFrame *next; - uint8_t *black_data[4]; ///< buffer used to fill padded lines + uint8_t *black_data[2][4]; ///< buffer used to fill padded lines (limited/full) int black_linesize[4]; FFDrawContext draw; FFDrawColor color; diff --git a/libavfilter/vf_tinterlace.c b/libavfilter/vf_tinterlace.c index 7c54861de4..032629279a 100644 --- a/libavfilter/vf_tinterlace.c +++ b/libavfilter/vf_tinterlace.c @@ -201,7 +201,8 @@ static av_cold void uninit(AVFilterContext *ctx) av_frame_free(&tinterlace->cur ); av_frame_free(&tinterlace->next); - av_freep(&tinterlace->black_data[0]); + av_freep(&tinterlace->black_data[0][0]); + av_freep(&tinterlace->black_data[1][0]); } static int config_out_props(AVFilterLink *outlink) @@ -225,14 +226,22 @@ static int config_out_props(AVFilterLink *outlink) int ret; ff_draw_init(&tinterlace->draw, outlink->format, 0); ff_draw_color(&tinterlace->draw, &tinterlace->color, black); - if (ff_fmt_is_in(outlink->format, full_scale_yuvj_pix_fmts)) - tinterlace->color.comp[0].u8[0] = 0; - ret = av_image_alloc(tinterlace->black_data, tinterlace->black_linesize, + /* limited range */ + if (!ff_fmt_is_in(outlink->format, full_scale_yuvj_pix_fmts)) { + ret = av_image_alloc(tinterlace->black_data[0], tinterlace->black_linesize, + outlink->w, outlink->h, outlink->format, 16); + if (ret < 0) + return ret; + ff_fill_rectangle(&tinterlace->draw, &tinterlace->color, tinterlace->black_data[0], + tinterlace->black_linesize, 0, 0, outlink->w, outlink->h); + } + /* full range */ + tinterlace->color.comp[0].u8[0] = 0; + ret = av_image_alloc(tinterlace->black_data[1], tinterlace->black_linesize, outlink->w, outlink->h, outlink->format, 16); if (ret < 0) return ret; - - ff_fill_rectangle(&tinterlace->draw, &tinterlace->color, tinterlace->black_data, + ff_fill_rectangle(&tinterlace->draw, &tinterlace->color, tinterlace->black_data[1], tinterlace->black_linesize, 0, 0, outlink->w, outlink->h); } if (tinterlace->flags & (TINTERLACE_FLAG_VLPF | TINTERLACE_FLAG_CVLPF) @@ -360,7 +369,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) AVFilterLink *outlink = ctx->outputs[0]; TInterlaceContext *tinterlace = ctx->priv; AVFrame *cur, *next, *out; - int field, tff, ret; + int field, tff, full, ret; av_frame_free(&tinterlace->cur); tinterlace->cur = tinterlace->next; @@ -418,6 +427,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) out->sample_aspect_ratio = av_mul_q(cur->sample_aspect_ratio, av_make_q(2, 1)); field = (1 + outlink->frame_count_in) & 1 ? FIELD_UPPER : FIELD_LOWER; + full = out->color_range == AVCOL_RANGE_JPEG || ff_fmt_is_in(out->format, full_scale_yuvj_pix_fmts); /* copy upper and lower fields */ copy_picture_field(tinterlace, out->data, out->linesize, (const uint8_t **)cur->data, cur->linesize, @@ -425,7 +435,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *picref) FIELD_UPPER_AND_LOWER, 1, field, tinterlace->flags); /* pad with black the other field */ copy_picture_field(tinterlace, out->data, out->linesize, - (const uint8_t **)tinterlace->black_data, tinterlace->black_linesize, + (const uint8_t **)tinterlace->black_data[full], tinterlace->black_linesize, inlink->format, inlink->w, inlink->h, FIELD_UPPER_AND_LOWER, 1, !field, tinterlace->flags); break; diff --git a/tests/ref/fate/filter-pixfmts-tinterlace_pad b/tests/ref/fate/filter-pixfmts-tinterlace_pad index 81a6961215..29321e542b 100644 --- a/tests/ref/fate/filter-pixfmts-tinterlace_pad +++ b/tests/ref/fate/filter-pixfmts-tinterlace_pad @@ -1,4 +1,4 @@ -gray 7ef396fecd8d1c9fe32173e4415ba671 +gray 227a6fe36a31fbef80210823454131ea yuv410p 35bc11d0d32efc9e9a969be7d720f4e6 yuv411p 17ef3cd22a74f7368b5e02f68779f294 yuv420p 93d5b6a4c44d67e4d4447e8dd0bf3d33