From patchwork Fri Jun 18 12:50:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 28575 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp1631275iol; Fri, 18 Jun 2021 05:50:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxXKIY2jkkecG/Aco4PsJWMWDNPUsHKQoPC9QibaWJGNvmXbr4NMvEWPNxd0Hjbj6vZwmkD X-Received: by 2002:a17:906:b6c5:: with SMTP id ec5mr10824208ejb.290.1624020642896; Fri, 18 Jun 2021 05:50:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1624020642; cv=none; d=google.com; s=arc-20160816; b=Nlgh5lUTAS/oKse80SLHg+aDVgfqpNyjmXuJJ5H39NrpUCpr+voHnkbO2nn7UqOxV3 DXTTHWfvh9oFe6/gHazbsrb8fkHEHEezElm9qMesRp9Epe47mGKusa2ZX3bbfTxSoH5b Hnsx6OETwm4bWRr4Oc7i48tG3bZIp6YuuIu7HOa6cxlCpcBpdNyJXbhYrRgScmgawGTa 5a++CQ1vNTyyc2+QkoiklkGBx23tfzKZyEwMx1mZI4+20unvcDMZWoVFgp4iN6ro/fo8 WhZmKGXGfEOCAMaBfXY/vLOT5FevdPK9Iqb3qjzeMctGEW1sIxTiBG7PVu/LoeX8beub MEqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=E41wnZmaA5fZnWGaVAcdSLW3fzjQmqe0gYxiMvTMcOY=; b=zlIugpRrGmmPm449wnHhDcJ1AH8ZTkS/iOdxjp0KVyiBdGf38/I4idyKvhjVWoRdbY RJipfQTaTPKAaqDjkg18EYtYnjQrxLjO5f2zbSOcGSH2DXQZt9vpeqPHP7FYlzXB8tC5 NwFsddH2rG1J9IfQfiKnbOOMF7sUAqcdIU5TrbWWfLhpRKl2phUv2y/C2JbIQjmCFP+4 wGTitSzX/e/XVMuM95WUWP5lU3PTj6uCjlPX3kWgHGYegl5JCWWhQ/t3kFzIx4x7YwSZ PIhYrfSTid6DiBukQW5YIt1Zccb3CiROfa2A9peB8G6P6GWafhVilMmTpDZ0G862ru3R Bfnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=YdLSmANW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ia21si2303580ejc.285.2021.06.18.05.50.42; Fri, 18 Jun 2021 05:50:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=YdLSmANW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2E2C06802E4; Fri, 18 Jun 2021 15:50:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CD28768094F for ; Fri, 18 Jun 2021 15:50:31 +0300 (EEST) Received: from Kryux.localdomain (unknown [83.135.86.229]) by btbn.de (Postfix) with ESMTPSA id 6038212F21C; Fri, 18 Jun 2021 14:50:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1624020631; bh=om2ICjnFVw3CdMTvKoj/MtnqCe8MwQBNKwKH4qFuMf0=; h=From:To:Cc:Subject:Date; b=YdLSmANWN6uHQFzu11SxdOCdbrA67KOM6Y2tVSHi0MHfZ+NXJVCd4sjvMEjcIgrP7 2B71xDyYMse/jfG+4yIbavx4YsGMWeT7EU0p51Tyt3cj/619j68Qy6bYg+BJsbD/tJ a0gI5zLy51Gafj2SX30edFLd9sEHIoKRVTa8Z8qdQ/LPHPOSjfV+QYJWOnRWMXgM8P 3WYVWpxg2JsMLywM7k1qF2tZ0x5DioURuhHe/B6L7Qskk7cuO8Df5IPRWKL8MP2oFd B8miIAKMogKBYStjeu+Hjy7ol78/f/a2WkN0vzKT9rpzL23XAq7Usv892sujKioQ2C X4yRXl3xXnZTQ== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Fri, 18 Jun 2021 14:50:22 +0200 Message-Id: <20210618125022.5304-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avfilter/select: add metadata detection function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Om+L7A1sR2/U --- doc/filters.texi | 18 ++++++++++ libavfilter/f_select.c | 79 ++++++++++++++++++++++++++++++++++++++++-- libavfilter/version.h | 2 +- 3 files changed, 96 insertions(+), 3 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index da8f7d7726..05fec04b55 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -5404,6 +5404,7 @@ Set sidechain gain. Default is 1. Range is from 0.015625 to 64. This filter supports the all above options as @ref{commands}. +@anchor{silencedetect} @section silencedetect Detect silence in an audio stream. @@ -7263,6 +7264,7 @@ Filter out noisy pixels from @code{bitplane} set above. Default is disabled. @end table +@anchor{blackdetect} @section blackdetect Detect video intervals that are (almost) completely black. Can be @@ -12518,6 +12520,7 @@ Select frame after every @code{step} frames. Allowed values are positive integers higher than 0. Default value is @code{1}. @end table +@anchor{freezedetect} @section freezedetect Detect frozen video. @@ -25564,6 +25567,21 @@ missing. That basically means that an input frame is selected if its pts is within the interval set by the concat demuxer. +@item detected(kind) +Evaluates the metadata added to frames by various detection filters. +Returns -1 if the respective filter has detected what it was looking for, +0 otherwise. + +Possible values for the @var{kind} parameter: +@table @option +@item SILENCE (audio only) +Looks for metadata added by @ref{silencedetect}. +@item FREEZE (video only) +Looks for metadata added by @ref{freezedetect}. +@item BLACK (video only) +Looks for metadata added by @ref{blackdetect}. +@end table + @end table The default value of the select expression is "1". diff --git a/libavfilter/f_select.c b/libavfilter/f_select.c index f0468078e8..b1eba67876 100644 --- a/libavfilter/f_select.c +++ b/libavfilter/f_select.c @@ -86,6 +86,10 @@ static const char *const var_names[] = { "concatdec_select", ///< frame is within the interval set by the concat demuxer + "SILENCE", ///< silencedetect detected silence for this frame + "FREEZE", ///< freezedetect detected frozen frames + "BLACK", ///< blackdetect detected black frames + NULL }; @@ -138,14 +142,42 @@ enum var_name { VAR_CONCATDEC_SELECT, + VAR_SILENCE, + VAR_FREEZE, + VAR_BLACK, + VAR_VARS_NB }; +enum meta_name { + META_SILENCE, + META_FREEZE, + META_BLACK, + + META_NAMES_NB +}; + +static const char *const func1_names[] = { + "detected", ///< function to eval metadata from various detection filters (silencedetect, ...) + + NULL +}; + +static double detect_metadata(void *p, double kind); + +static double (*func1_funcs[])(void *, double) = { + detect_metadata, + + NULL +}; + typedef struct SelectContext { const AVClass *class; char *expr_str; AVExpr *expr; double var_values[VAR_VARS_NB]; + int meta_detected[META_NAMES_NB]; + AVFrame *cur_frame; ///< current frame, for use in expression parser functions ONLY int bitdepth; int nb_planes; ptrdiff_t width[4]; @@ -177,7 +209,7 @@ static av_cold int init(AVFilterContext *ctx) int i, ret; if ((ret = av_expr_parse(&select->expr, select->expr_str, - var_names, NULL, NULL, NULL, NULL, 0, ctx)) < 0) { + var_names, func1_names, func1_funcs, NULL, NULL, 0, ctx)) < 0) { av_log(ctx, AV_LOG_ERROR, "Error while parsing expression '%s'\n", select->expr_str); return ret; @@ -267,6 +299,11 @@ static int config_input(AVFilterLink *inlink) if (!select->sad) return AVERROR(EINVAL); } + + select->var_values[VAR_SILENCE] = META_SILENCE; + select->var_values[VAR_FREEZE] = META_FREEZE; + select->var_values[VAR_BLACK] = META_BLACK; + return 0; } @@ -325,6 +362,40 @@ static double get_concatdec_select(AVFrame *frame, int64_t pts) return NAN; } +static double detect_metadata(void *p, double kind) +{ + AVFilterContext *ctx = p; + SelectContext *select = ctx->priv; + AVDictionary *metadata = select->cur_frame->metadata; + int kind_i = (int)(kind + 0.5); + const char *start, *end; + + switch(kind_i) { + case META_SILENCE: + start = "lavfi.silence_start"; + end = "lavfi.silence_end"; + break; + case META_FREEZE: + start = "lavfi.freezedetect.freeze_start"; + end = "lavfi.freezedetect.freeze_end"; + break; + case META_BLACK: + start = "lavfi.black_start"; + end = "lavfi.black_end"; + break; + default: + av_log(ctx, AV_LOG_WARNING, "Invalid metadata detection kind!\n"); + return NAN; + } + + if (av_dict_get(metadata, start, NULL, 0)) + select->meta_detected[kind_i] = -1; + if (av_dict_get(metadata, end, NULL, 0)) + select->meta_detected[kind_i] = 0; + + return select->meta_detected[kind_i]; +} + static void select_frame(AVFilterContext *ctx, AVFrame *frame) { SelectContext *select = ctx->priv; @@ -363,7 +434,9 @@ static void select_frame(AVFilterContext *ctx, AVFrame *frame) break; } - select->select = res = av_expr_eval(select->expr, select->var_values, NULL); + select->cur_frame = frame; + + select->select = res = av_expr_eval(select->expr, select->var_values, ctx); av_log(inlink->dst, AV_LOG_DEBUG, "n:%f pts:%f t:%f key:%d", select->var_values[VAR_N], @@ -371,6 +444,8 @@ static void select_frame(AVFilterContext *ctx, AVFrame *frame) select->var_values[VAR_T], frame->key_frame); + select->cur_frame = NULL; + switch (inlink->type) { case AVMEDIA_TYPE_VIDEO: av_log(inlink->dst, AV_LOG_DEBUG, " interlace_type:%c pict_type:%c scene:%f", diff --git a/libavfilter/version.h b/libavfilter/version.h index 5052681653..fbb81ef31c 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #define LIBAVFILTER_VERSION_MAJOR 8 #define LIBAVFILTER_VERSION_MINOR 0 -#define LIBAVFILTER_VERSION_MICRO 102 +#define LIBAVFILTER_VERSION_MICRO 103 #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \