From patchwork Tue Oct 3 13:13:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Petrov X-Patchwork-Id: 44112 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1204:b0:15d:8365:d4b8 with SMTP id v4csp2045816pzf; Tue, 3 Oct 2023 06:14:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFZdE71oGtdtNkZffSIZeCUQVejHc/cvwrC9u07mb0X/kIfX/0Qx8/OM2IYRMT+USiIJwgw X-Received: by 2002:a05:6402:7cf:b0:536:1a05:a6d2 with SMTP id u15-20020a05640207cf00b005361a05a6d2mr2375659edy.17.1696338852858; Tue, 03 Oct 2023 06:14:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696338852; cv=none; d=google.com; s=arc-20160816; b=QG3SGwy3JNC3PfYW4i1nZFjAoUQktHAX5275VwWA7+c1fJc+fS2RVnDTezdnasK4k2 PnrJMjjQYbniUowB4y/Tlb2of0OTiEIHD2qrWZzpVLtWjKaLxL82zFgV9PoRR9282d33 2hrkxIAM5mnEIVV95IdkZPmZ7pU1LH3L+xkP0Ptunykx0Qr+nXo78U0MMYvppgs/8MnT y/G01r+qQBQcKsAo85abHXsxpGmhOlNGEBxOYL9oGdw+9RBkbwpSSCMBPfPA/+fgLQqU VPNQKLwWqmVCAkXkuRShlN8pPfzwmmMP++xIXe1Dps0fe8tePBZ6/OWneJMpzNvM4TEh T1Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:to :message-id:date:from:mime-version:dkim-signature:delivered-to; bh=W5WjuWP2pJNPrXTfQOIsICzVQZd8q3Q/xb7G7VhKbP8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=yx3uRL163rK6jcpOptGGLCgn3MfkzEV2jMDw9L6KyioHsbSFHOK3CBHDfW9J79x26L fDCtJIKo4Njyi5yDijgmEReOWtDKCWKh02HnSCaoW78+G0QStoIsVnJAsdDeU6JM3TS5 CZWOEZ8kvLByOiIGAAng7hToaM5cgQB2B65A2m4KeYSuzkowzm9Xzro0/ybB0PSWPpd9 DycKBOxVrAwJxXlOGtQYBSk3Cp2tfDPmvP5dYTaWox7somVpYhi/uqm+PXahHBDa/038 tVbdWVxm1hhsMyP71EMnSmQwWFD/7441bq1vo5Zw0hSvCHcgm2OkC4nGdX1LRGVcOtvT 22kA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=BDG77uaJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t24-20020a05640203d800b00534cca3fbf7si632075edw.190.2023.10.03.06.14.10; Tue, 03 Oct 2023 06:14:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=BDG77uaJ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 675B068CC84; Tue, 3 Oct 2023 16:14:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0B9B868CA32 for ; Tue, 3 Oct 2023 16:13:59 +0300 (EEST) Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-279013f9875so636795a91.2 for ; Tue, 03 Oct 2023 06:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696338836; x=1696943636; darn=ffmpeg.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=FqjgunXnK3H8mSAGwnM08vOkmQnDk4KkQkGA0TkvwNk=; b=BDG77uaJTbqEthzIOz63LcYYFwvnRkG2ecc4EZU9qrrP9PiIqv2SJVsk9zHafFnnSp ISlXi/Oi0dsc11usY/2Y7Ls5Ku38KFx/cpycwNFRnGGcOyU4DvtctmIJpwozGD6U/ORt K0YMa+hb3cWaLOnUhPb3L8h9hAzPJv6OSBeL3QgldBaMIN/ObGXGlkfP/Yu8FLB0bFkA fVL6yIiFzlxv2Nvwc7T93bVeLTwT3bAdM27VGTs/d+hrC/jjW0qaXKGZ/oxqRyOLO5mH pDSbG0ibdIlBwUiFqt1bT5dfxT42wp1UxaWySO4nbp5TeKaCdjoG8oNnuixWJ0PendMr y2Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696338836; x=1696943636; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=FqjgunXnK3H8mSAGwnM08vOkmQnDk4KkQkGA0TkvwNk=; b=abtxxukIK5sOdcDoEbQq7Var/17xKCojY4S0SRueI0Z/3oPflMptEti56J2bN+MBVD x2CErWwNEAscCiY3ozELrvi/yJdBSyTRkIro6qvFpcf9IMf73vPJzKlFbJQO7rw2hV7c WhTTtMKjIuewwleFpvHngxna7AOWB6KqlG59IZ3YQHfRjY5HASfy4426Gs3HyTZiRsdp xv7WRv9tt98aDHhKzvWuDO7s48kI2rYdsxMV58oS2fJEsvd6PwfwRFV+hVb5V/7LyHzs upv6bzy92Z1cZ9wa7QALZlvRmYrIc4cXaleDXLzCK1lx7+hN7GdKYTdYB5ohx4afmvD6 6awg== X-Gm-Message-State: AOJu0Yy7myoffjE9lwlL7zFRhsijLh++6p0TsjbHL2f5mLcp33fRKd+n syJ7hwYo0CsdTaDobrRf5LLGJf9ULHAjlC5o18dKfiNrRCw= X-Received: by 2002:a17:90a:b308:b0:274:6d4e:e0d9 with SMTP id d8-20020a17090ab30800b002746d4ee0d9mr14646625pjr.45.1696338836353; Tue, 03 Oct 2023 06:13:56 -0700 (PDT) MIME-Version: 1.0 From: Vladimir Petrov Date: Tue, 3 Oct 2023 16:13:45 +0300 Message-ID: To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] avfilter/vf_decimate: Improve decimation factor precision X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5t+IPYKAhQMC Currently, decimate video filter supports dropping of only single frame per cycle, limiting the range of available framerate decimation factors. Now, adding a new option 'dropcount' allows increasing of drop count, so more fractional factors could be chosen. Also added an option 'spread' to set percentage of dropped frame metrics spreading to adjanced frames. From 127bed0a4fd2ca05f43fa117ba4bf859430fc5ff Mon Sep 17 00:00:00 2001 From: Vladimir Petrov Date: Sat, 23 Sep 2023 00:37:32 +0300 Subject: [PATCH 1/2] avfilter/vf_decimate: Improve decimation factor precision Currently, decimate video filter supports dropping of only single frame per cycle, limiting the range of available framerate decimation factors. Now, adding a new option 'dropcount' allows increasing of drop count, so more fractional factors could be chosen. Also added an option 'spread' to set percentage of dropped frame metrics spreading to adjanced frames. Signed-off-by: Vladimir Petrov --- doc/filters.texi | 16 ++- libavfilter/vf_decimate.c | 269 +++++++++++++++++++++++++++++++------- 2 files changed, 237 insertions(+), 48 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index a729a08dce..eef76d71b0 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -11395,10 +11395,22 @@ The filter accepts the following options: @table @option @item cycle -Set the number of frames from which one will be dropped. Setting this to -@var{N} means one frame in every batch of @var{N} frames will be dropped. +Set the number of frames per cycle. Setting this to @var{N} and @var{dropcount} +to @var{M} means @var{M} frames in every batch of @var{N} frames will be dropped. Default is @code{5}. +@item dropcount +Set the number of frames to be dropped from each cycle. +Must be smaller than @var{cycle}. +Default is @code{1} + +@item spread +Set percentage of dropped frame metrics spreading to adjanced frames. Bigger value means +that non-duplicate frames will be distributed more evenly to output. Especially, this is +useful in case of static scenes to avoid skipping of large frame sequences, i.e. to avoid +jumpy motion. +Default is @code{75}. + @item dupthresh Set the threshold for duplicate detection. If the difference metric for a frame is less than or equal to this value, then it is declared as duplicate. Default diff --git a/libavfilter/vf_decimate.c b/libavfilter/vf_decimate.c index dbeca427f1..67896eaa0d 100644 --- a/libavfilter/vf_decimate.c +++ b/libavfilter/vf_decimate.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavutil/common.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "libavutil/timestamp.h" @@ -33,11 +34,12 @@ struct qitem { AVFrame *frame; int64_t maxbdiff; int64_t totdiff; + int drop; }; typedef struct DecimateContext { const AVClass *class; - struct qitem *queue; ///< window of cycle frames and the associated data diff + struct qitem *queue; ///< window of cycle+1 frames and the associated data diff int fid; ///< current frame id in the queue int filled; ///< 1 if the queue is filled, 0 otherwise AVFrame *last; ///< last frame from the previous queue @@ -58,6 +60,8 @@ typedef struct DecimateContext { /* options */ int cycle; + int dropcount; + int spread; double dupthresh_flt; double scthresh_flt; int64_t dupthresh; @@ -72,7 +76,9 @@ typedef struct DecimateContext { #define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM static const AVOption decimate_options[] = { - { "cycle", "set the number of frame from which one will be dropped", OFFSET(cycle), AV_OPT_TYPE_INT, {.i64 = 5}, 2, 25, FLAGS }, + { "cycle", "set the number of frames per cycle", OFFSET(cycle), AV_OPT_TYPE_INT, {.i64 = 5}, 2, 25, FLAGS }, + { "dropcount", "set the number of frames to be dropped from each cycle", OFFSET(dropcount), AV_OPT_TYPE_INT, {.i64 = 1}, 1, 24, FLAGS }, + { "spread", "set percentage of dropped frame metrics spreading to adjanced frames", OFFSET(spread), AV_OPT_TYPE_INT, {.i64 = 75}, 0, 200, FLAGS }, { "dupthresh", "set duplicate threshold", OFFSET(dupthresh_flt), AV_OPT_TYPE_DOUBLE, {.dbl = 1.1}, 0, 100, FLAGS }, { "scthresh", "set scene change threshold", OFFSET(scthresh_flt), AV_OPT_TYPE_DOUBLE, {.dbl = 15.0}, 0, 100, FLAGS }, { "blockx", "set the size of the x-axis blocks used during metric calculations", OFFSET(blockx), AV_OPT_TYPE_INT, {.i64 = 32}, 4, 1<<9, FLAGS }, @@ -150,14 +156,154 @@ static void calc_diffs(const DecimateContext *dm, struct qitem *q, q->maxbdiff = maxdiff; } +static int dup_cnt(const DecimateContext* const dm) +{ + int i; + int ret = 0; + + for (i = 0; i < dm->cycle && dm->queue[i].frame; i++) { + if (!dm->queue[i].drop && dm->queue[i].maxbdiff < dm->dupthresh) + ret++; + } + + return ret; +} + +static int q_cnt(const DecimateContext* const dm) +{ + int i; + int ret = 0; + + for (i = 0; i < dm->cycle && dm->queue[i].frame; i++) { + ret++; + } + + return ret; +} + +static int drop_cnt(const DecimateContext* const dm) +{ + int i; + int ret = 0; + + for (i = 0; i < dm->cycle && dm->queue[i].frame; i++) { + if (dm->queue[i].drop) + ret++; + } + + return ret; +} + +static int prev_frame_idx(const DecimateContext* const dm, const int cur_frame_idx) +{ + int i = 0; + + for (i = cur_frame_idx - 1; 0 <= i; i--) { + if (!dm->queue[i].drop) + break; + } + + return i; +} + +static int next_frame_idx(const DecimateContext* const dm, const int cur_frame_idx) +{ + int i = 0; + + for (i = cur_frame_idx + 1; dm->cycle > i && dm->queue[i].frame; i++) { + if (!dm->queue[i].drop) + break; + } + + if (dm->cycle == i || !dm->queue[i].frame) + i = -1; + + return i; +} + +static int best_dup_idx(const DecimateContext* const dm) +{ + int i = 0; + int ret = next_frame_idx(dm, -1); + + if (0 <= ret) { + for (i = ret + 1; dm->cycle > i && dm->queue[i].frame; i++) { + if (!dm->queue[i].drop && dm->queue[i].maxbdiff < dm->queue[ret].maxbdiff) + ret = i; + } + } + + return ret; +} + +static int best_sc_idx(const DecimateContext* const dm) +{ + int i = 0; + int ret = next_frame_idx(dm, -1); + + if (0 <= ret) { + for (i = ret + 1; dm->cycle > i && dm->queue[i].frame; i++) { + if (!dm->queue[i].drop && dm->queue[i].totdiff > dm->queue[ret].totdiff) + ret = i; + } + } + + return ret; +} + +static void update_frame_metrics(DecimateContext *dm, const int idx_p, const int idx, int idx_dr) +{ + int idx_nxt; + AVFrame *f1, *f2; + int64_t bdiff, bdiff_dr; + + idx_nxt = (0 <= idx) ? idx : dm->cycle; + f1 = (0 <= idx_p) ? dm->queue[idx_p].frame : dm->last; + f2 = dm->queue[idx_nxt].frame; + bdiff = dm->queue[idx_nxt].maxbdiff; + bdiff_dr = (0 <= idx_dr) ? dm->queue[idx_dr].maxbdiff : 0; + + if (!f1) { + dm->queue[idx_nxt].maxbdiff = INT64_MAX; + dm->queue[idx_nxt].totdiff = INT64_MAX; + } else if (dm->mixed && ((AV_FRAME_FLAG_INTERLACED & f1->flags) || (f2 && (AV_FRAME_FLAG_INTERLACED & f2->flags)))) { + dm->queue[idx_nxt].maxbdiff = INT64_MAX - 1; + dm->queue[idx_nxt].totdiff = 0; + } else if (f2) { + calc_diffs(dm, &dm->queue[idx_nxt], f1, f2); + } + + if (bdiff < dm->dupthresh || bdiff_dr < dm->dupthresh) + bdiff = bdiff_dr = 0; + dm->queue[idx_nxt].maxbdiff = av_sat_add64(dm->queue[idx_nxt].maxbdiff, av_clip64(bdiff, 0, INT64_MAX/200) * dm->spread / 100); + if (0 <= idx_p) + dm->queue[idx_p].maxbdiff = av_sat_add64(dm->queue[idx_p].maxbdiff, av_clip64(bdiff_dr, 0, INT64_MAX/200) * dm->spread / 100); +} + +static void mark_drop_frame(DecimateContext *dm, const int idx, const int drop_id) +{ + dm->queue[idx].drop = drop_id; +} + static int filter_frame(AVFilterLink *inlink, AVFrame *in) { - int scpos = -1, duppos = -1; - int drop = INT_MIN, i, lowest = 0, ret; + int i, ret; + int idx_prv = -1, idx_nxt = -1, idx_dup = -1, idx_sc = -1, idx_drop = -1; + int drop = 0, drop_case = 0, flush_drops = 0; AVFilterContext *ctx = inlink->dst; AVFilterLink *outlink = ctx->outputs[0]; DecimateContext *dm = ctx->priv; - AVFrame *prv; + + /* move already pre-read frame and clear the rest */ + if (0 == dm->fid && dm->queue[dm->cycle].frame) { + if (dm->ppsrc) { + memcpy(&dm->clean_src[0], &dm->clean_src[dm->cycle], sizeof(dm->clean_src[0])); + memset(&dm->clean_src[1], 0, sizeof(dm->clean_src[1]) * dm->cycle); + } + memcpy(&dm->queue[0], &dm->queue[dm->cycle], sizeof(dm->queue[0])); + memset(&dm->queue[1], 0, sizeof(dm->queue[1]) * dm->cycle); + dm->fid++; + } /* update frames queue(s) */ if (FF_INLINK_IDX(inlink) == INPUT_MAIN) { @@ -171,59 +317,85 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) return 0; dm->got_frame[INPUT_MAIN] = dm->got_frame[INPUT_CLEANSRC] = 0; - if (dm->ppsrc) - in = dm->queue[dm->fid].frame; - - if (in) { + if (dm->queue[dm->fid].frame) { /* update frame metrics */ - prv = dm->fid ? dm->queue[dm->fid - 1].frame : dm->last; - if (!prv) { - dm->queue[dm->fid].maxbdiff = INT64_MAX; - dm->queue[dm->fid].totdiff = INT64_MAX; - } else { - calc_diffs(dm, &dm->queue[dm->fid], prv, in); - } - if (++dm->fid != dm->cycle) + update_frame_metrics(dm, dm->fid - 1, dm->fid, -1); + if (++dm->fid <= dm->cycle) return 0; - av_frame_free(&dm->last); - dm->last = av_frame_clone(in); dm->fid = 0; - /* we have a complete cycle, select the frame to drop */ - lowest = 0; - for (i = 0; i < dm->cycle; i++) { - if (dm->queue[i].totdiff > dm->scthresh) - scpos = i; - if (dm->queue[i].maxbdiff < dm->queue[lowest].maxbdiff) - lowest = i; + /* we have a complete cycle, select frames to drop */ + if (!(dm->mixed && dup_cnt(dm) < dm->dropcount)) { + drop = 1; + while (drop_cnt(dm) < dm->dropcount) { + idx_dup = best_dup_idx(dm); + idx_sc = best_sc_idx(dm); + if (0 <= idx_dup && dm->queue[idx_dup].maxbdiff < dm->dupthresh) { + drop_case = 1; + idx_drop = idx_dup; + } else if (0 <= idx_sc && dm->queue[idx_sc].totdiff > dm->scthresh) { + drop_case = 2; + idx_drop = idx_sc; + } else { + drop_case = 3; + idx_drop = idx_dup; + } + + if (0 > idx_drop) { + break; + } else { + idx_prv = prev_frame_idx(dm, idx_drop); + idx_nxt = next_frame_idx(dm, idx_drop); + update_frame_metrics(dm, idx_prv, idx_nxt, idx_drop); + mark_drop_frame(dm, idx_drop, drop_case); + } + } } - if (dm->queue[lowest].maxbdiff < dm->dupthresh) - duppos = lowest; - - if (dm->mixed && duppos < 0) { - drop = -1; // no drop if mixed content + no frame in cycle below threshold - } else { - drop = scpos >= 0 && duppos < 0 ? scpos : lowest; + av_frame_free(&dm->last); + idx_prv = prev_frame_idx(dm, dm->cycle); + if (0 <= idx_prv && dm->queue[idx_prv].frame) + dm->last = av_frame_clone(dm->queue[idx_prv].frame); + } else { + /* prepare flushing */ + flush_drops = dm->dropcount * q_cnt(dm) / dm->cycle; + if (!(dm->mixed && dup_cnt(dm) < flush_drops)) { + drop = 1; + while (drop_cnt(dm) < flush_drops) { + idx_drop = best_dup_idx(dm); + if (0 > idx_drop) { + break; + } else { + idx_prv = prev_frame_idx(dm, idx_drop); + idx_nxt = next_frame_idx(dm, idx_drop); + update_frame_metrics(dm, idx_prv, idx_nxt, idx_drop); + mark_drop_frame(dm, idx_drop, 4); + } + } } + av_frame_free(&dm->last); } + /* metrics debug */ if (av_log_get_level() >= AV_LOG_DEBUG) { - av_log(ctx, AV_LOG_DEBUG, "1/%d frame drop:\n", dm->cycle); + av_log(ctx, AV_LOG_DEBUG, "%d/%d %s drop:\n", dm->dropcount, dm->cycle, drop ? "frame" : "no"); for (i = 0; i < dm->cycle && dm->queue[i].frame; i++) { - av_log(ctx, AV_LOG_DEBUG," #%d: totdiff=%08"PRIx64" maxbdiff=%08"PRIx64"%s%s%s%s\n", + av_log(ctx, AV_LOG_DEBUG," #%d: totdiff=%08"PRIx64" maxbdiff=%08"PRIx64"%s%s%s%s%s%s%s\n", i + 1, dm->queue[i].totdiff, dm->queue[i].maxbdiff, - i == scpos ? " sc" : "", - i == duppos ? " dup" : "", - i == lowest ? " lowest" : "", - i == drop ? " [DROP]" : ""); + dm->queue[i].totdiff > dm->scthresh ? " sc" : " ", + dm->queue[i].maxbdiff < dm->dupthresh ? " dup" : " ", + 1 == dm->queue[i].drop ? " [DROP-DUP]" : "", + 2 == dm->queue[i].drop ? " [DROP-SCN]" : "", + 3 == dm->queue[i].drop ? " [DROP-LOW]" : "", + 4 == dm->queue[i].drop ? " [DROP-FLU]" : "", + 5 <= dm->queue[i].drop ? " [DROP-UKN]" : ""); } } /* push all frames except the drop */ ret = 0; for (i = 0; i < dm->cycle && dm->queue[i].frame; i++) { - if (i == drop) { + if (drop && dm->queue[i].drop) { if (dm->ppsrc) av_frame_free(&dm->clean_src[i]); av_frame_free(&dm->queue[i].frame); @@ -243,7 +415,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) frame->pts = dm->last_duration ? dm->last_pts + dm->last_duration : (dm->start_pts == AV_NOPTS_VALUE ? 0 : dm->start_pts); - frame->duration = dm->mixed ? av_div_q(drop < 0 ? dm->nondec_tb : dm->dec_tb, outlink->time_base).num : 1; + frame->duration = dm->mixed ? av_div_q(drop ? dm->dec_tb : dm->nondec_tb, outlink->time_base).num : 1; dm->last_duration = frame->duration; dm->last_pts = frame->pts; ret = ff_filter_frame(outlink, frame); @@ -340,6 +512,11 @@ static av_cold int decimate_init(AVFilterContext *ctx) return AVERROR(EINVAL); } + if (!(dm->cycle > dm->dropcount)) { + dm->dropcount = dm->cycle - 1; + av_log(ctx, AV_LOG_WARNING, "Reducing drop count to %d\n", dm->dropcount); + } + dm->start_pts = AV_NOPTS_VALUE; dm->last_duration = 0; @@ -354,12 +531,12 @@ static av_cold void decimate_uninit(AVFilterContext *ctx) av_frame_free(&dm->last); av_freep(&dm->bdiffs); if (dm->queue) { - for (i = 0; i < dm->cycle; i++) + for (i = 0; i <= dm->cycle; i++) av_frame_free(&dm->queue[i].frame); } av_freep(&dm->queue); if (dm->clean_src) { - for (i = 0; i < dm->cycle; i++) + for (i = 0; i <= dm->cycle; i++) av_frame_free(&dm->clean_src[i]); } av_freep(&dm->clean_src); @@ -400,16 +577,16 @@ static int config_output(AVFilterLink *outlink) dm->nyblocks = (h + dm->blocky/2 - 1) / (dm->blocky/2); dm->bdiffsize = dm->nxblocks * dm->nyblocks; dm->bdiffs = av_malloc_array(dm->bdiffsize, sizeof(*dm->bdiffs)); - dm->queue = av_calloc(dm->cycle, sizeof(*dm->queue)); + dm->queue = av_calloc(dm->cycle + 1, sizeof(*dm->queue)); dm->in_tb = inlink->time_base; dm->nondec_tb = av_inv_q(fps); - dm->dec_tb = av_mul_q(dm->nondec_tb, (AVRational){dm->cycle, dm->cycle - 1}); + dm->dec_tb = av_mul_q(dm->nondec_tb, (AVRational){dm->cycle, dm->cycle - dm->dropcount}); if (!dm->bdiffs || !dm->queue) return AVERROR(ENOMEM); if (dm->ppsrc) { - dm->clean_src = av_calloc(dm->cycle, sizeof(*dm->clean_src)); + dm->clean_src = av_calloc(dm->cycle + 1, sizeof(*dm->clean_src)); if (!dm->clean_src) return AVERROR(ENOMEM); } -- 2.39.2 From patchwork Tue Oct 3 13:14:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Petrov X-Patchwork-Id: 44113 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1204:b0:15d:8365:d4b8 with SMTP id v4csp2046495pzf; Tue, 3 Oct 2023 06:15:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHvB1l39+AJdi6KJeVt5GGgHGW0QrUF1dkUeEthxxKnwHzkW7OmzZ18G/Sz1y0Qn5dwI4I6 X-Received: by 2002:a05:6402:1b1b:b0:522:2ce0:d80a with SMTP id by27-20020a0564021b1b00b005222ce0d80amr12255555edb.35.1696338909113; Tue, 03 Oct 2023 06:15:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696338909; cv=none; d=google.com; s=arc-20160816; b=ViwFYkviMV2z+Ns9BB7Y8c+Uj3Zi3o3ZJBSA6buI58mrR281x+1QKlgERdUjkDuA5J Ry4PeHQhoJ7HPgITVthdnLxSR/sDBKPmGMt0M+tVUhJ183punYoNeq9Tp9RmqHG9NvLQ PkbqasOZLceLKbNBZmslQeCw/7gs/y+37NWN3MnZT4CUZUm9yV7lszAgEjFf8W5uZlGa hIgaUe5RUAJMKgmlyy6aZByeGZKvH5t3b6q7vZgFfg96iDkuQiGfVIYBN/8+JhIG7mqU qqu8hl5p7u4kCzE/CTVTJMV0QBd8X8X+ErlFakpA+IceM2juEDvLEqTv6/rwLgNiDzKK 1C9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject:to :message-id:date:from:mime-version:dkim-signature:delivered-to; bh=RS6REl+QYcv3eAhQKDrEHsdSPOC2kgnWnQDusw+oN7Y=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=zbffepqHTFKbyapxqvBKrkoZGVNX5PRPrOPpIrabecRemntXUk8lB0lkeZVLK6qSg/ OzULxzrlVxDJctZ9KQIUjRMC+/PNyFynGAKWCwh4phI5A7nqlIKAhoISmTBG+kDk+QDR JEag4gm4MXtOEaWGQjl3UWWYezBAoUE1P0ZKlZgHkZ7OVdbbu6SmIgxnAXJTosNa89Sc YQJsI2pRyTxXQoOGbcvvq5ekSeYbay/3nsbz9EtYX1yzynzbIuaoajZNqcYhdxRE/NMQ ZfZOO5tikS/wZ1eU78NCxigV0RPo9uvKWcTGdqilmQgKamPPZx4dEoue2JdLfPf84JrV iblw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=gJs8wh2o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e18-20020a056402089200b005363066ce13si595774edy.455.2023.10.03.06.15.08; Tue, 03 Oct 2023 06:15:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=gJs8wh2o; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2082E68CC8B; Tue, 3 Oct 2023 16:15:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ABD9468CB5D for ; Tue, 3 Oct 2023 16:14:59 +0300 (EEST) Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1c434c33ec0so6406145ad.3 for ; Tue, 03 Oct 2023 06:14:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696338898; x=1696943698; darn=ffmpeg.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=fRcEqN+Hxaj/QAuAKM/3vQBEilM/62AO7z2T7a67Dn4=; b=gJs8wh2oBK2MHzXmNJtrG82ji3fKuzT4DVVJoQru30jeGe5dFqlP5CRKpcoGbTHRxQ KP5P84kzcO4xRl2NhDIDDuxnbefLzCPf+qo3J34MDnSiLppUOg5YOmu1NyfcVlyxa5d5 oNjZFv/9LGy/IojPa25rNjghlzY2X080TcPKpb4DsGbaHy7xL/Sig9VB4vk71XgfDi+h jT2N+FoK53xsXAX+fcWYQhEz/iTk/0O6Yxu6UqjXl8t43SY0nyXtSP2Dfg3fZ+UcCNwA 0NiJO56WuYseDSPjU16hwlfjZI+avvU1GHgqmBFro0GnmarX9wlTwATJigHKIpJOvrVD 4Ieg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696338898; x=1696943698; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fRcEqN+Hxaj/QAuAKM/3vQBEilM/62AO7z2T7a67Dn4=; b=rCnai+Xcu4pUmR0Q/RjEHhcNo83mxriUPb4EDw0cUm3SNqeV/4DzjiYlWWgyy61UFm Zkj7XYVvRgsiI0ZNOY+hEWnG7qwqQs6TYsg6z2gNpUYNWxAw+AzCZVw60nQfHFwX3kRO otB60HjrfU9ZK8r0R+aDL5IDtkyARP0hlFKRdKAuHNL9G5DVTr64MH8wepKLwVtNV7AD VLi/Blnq4Nhb66mdXfpgLwWFwQ/WtJW6p4OHN8ATCtJ8acAjKKTLz3Bn8QfRgfCoX/u3 5yMJ/dXxG3bIDD97Op7zNKqPAqB5ZZFIP3ZIEiYvIUBSpRkk5KMCRABR0cr1Q9W4f1sV I/VA== X-Gm-Message-State: AOJu0YzrcBnpU8ZeduGbeomtt/1RYetgHflAZqxkvCB+78pUEcn3WFRq 5+drhQJwgQwyGIIusU0Iqp5w1dFta4ODJ+Bc2VOcw0w3aw0= X-Received: by 2002:a17:90b:4a84:b0:271:addf:3c96 with SMTP id lp4-20020a17090b4a8400b00271addf3c96mr3297796pjb.46.1696338897585; Tue, 03 Oct 2023 06:14:57 -0700 (PDT) MIME-Version: 1.0 From: Vladimir Petrov Date: Tue, 3 Oct 2023 16:14:46 +0300 Message-ID: To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 2/2] avfilter/vf_decimate: Add compatibility option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RqGRAMA4ZGzm Adds compatibility with the old behaviour. Added a new option 'compat'. When set to 'true', no metrics recalculation will be done when marking a new frame to be dropped. In case of scene change detection, the last one per batch will be prioritized, instead of the best one. In mixed mode, checking for interlaced frames will be omited. From bdd5688d3c2811923895e31c886dd805e44c2ac6 Mon Sep 17 00:00:00 2001 From: Vladimir Petrov Date: Sat, 23 Sep 2023 17:41:58 +0300 Subject: [PATCH 2/2] avfilter/vf_decimate: Add compatibility option Adds compatibility with the old behaviour. Added a new option 'compat'. When set to 'true', no metrics recalculation will be done when marking a new frame to be dropped. In case of scene change detection, the last one per batch will be prioritized, instead of the best one. In mixed mode, checking for interlaced frames will be omited. Signed-off-by: Vladimir Petrov --- doc/filters.texi | 9 +++++++++ libavfilter/vf_decimate.c | 14 +++++++++----- 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index eef76d71b0..5954299713 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -11441,6 +11441,15 @@ Set whether or not chroma is considered in the metric calculations. Default is Set whether or not the input only partially contains content to be decimated. Default is @code{false}. If enabled video output stream will be in variable frame rate. + +@item compat +Set compatibility with the old behaviour. When set to @code{true}, no metrics +recalculation will be done when marking a new frame to be dropped. In case of +scene change detection, the last one per batch will be prioritized, instead of +the best one. In mixed mode, checking for interlaced frames will be omited. +In order to get smoother motion and more accurate scene change detection, +please set this to @code{false}. +Default is @code{true}. @end table @section deconvolve diff --git a/libavfilter/vf_decimate.c b/libavfilter/vf_decimate.c index 67896eaa0d..5ab953f985 100644 --- a/libavfilter/vf_decimate.c +++ b/libavfilter/vf_decimate.c @@ -70,6 +70,7 @@ typedef struct DecimateContext { int ppsrc; int chroma; int mixed; + int compat; } DecimateContext; #define OFFSET(x) offsetof(DecimateContext, x) @@ -86,6 +87,7 @@ static const AVOption decimate_options[] = { { "ppsrc", "mark main input as a pre-processed input and activate clean source input stream", OFFSET(ppsrc), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, { "chroma", "set whether or not chroma is considered in the metric calculations", OFFSET(chroma), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, FLAGS }, { "mixed", "set whether or not the input only partially contains content to be decimated", OFFSET(mixed), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS }, + { "compat", "set compatibility with old behaviour", OFFSET(compat), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, FLAGS }, { NULL } }; @@ -243,7 +245,7 @@ static int best_sc_idx(const DecimateContext* const dm) if (0 <= ret) { for (i = ret + 1; dm->cycle > i && dm->queue[i].frame; i++) { - if (!dm->queue[i].drop && dm->queue[i].totdiff > dm->queue[ret].totdiff) + if (!dm->queue[i].drop && dm->queue[i].totdiff > (dm->compat ? dm->scthresh : dm->queue[ret].totdiff)) ret = i; } } @@ -266,7 +268,7 @@ static void update_frame_metrics(DecimateContext *dm, const int idx_p, const int if (!f1) { dm->queue[idx_nxt].maxbdiff = INT64_MAX; dm->queue[idx_nxt].totdiff = INT64_MAX; - } else if (dm->mixed && ((AV_FRAME_FLAG_INTERLACED & f1->flags) || (f2 && (AV_FRAME_FLAG_INTERLACED & f2->flags)))) { + } else if (!dm->compat && dm->mixed && ((AV_FRAME_FLAG_INTERLACED & f1->flags) || (f2 && (AV_FRAME_FLAG_INTERLACED & f2->flags)))) { dm->queue[idx_nxt].maxbdiff = INT64_MAX - 1; dm->queue[idx_nxt].totdiff = 0; } else if (f2) { @@ -346,13 +348,14 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) } else { idx_prv = prev_frame_idx(dm, idx_drop); idx_nxt = next_frame_idx(dm, idx_drop); - update_frame_metrics(dm, idx_prv, idx_nxt, idx_drop); + if (!dm->compat) + update_frame_metrics(dm, idx_prv, idx_nxt, idx_drop); mark_drop_frame(dm, idx_drop, drop_case); } } } av_frame_free(&dm->last); - idx_prv = prev_frame_idx(dm, dm->cycle); + idx_prv = dm->compat ? (dm->cycle - 1) : prev_frame_idx(dm, dm->cycle); if (0 <= idx_prv && dm->queue[idx_prv].frame) dm->last = av_frame_clone(dm->queue[idx_prv].frame); } else { @@ -367,7 +370,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) } else { idx_prv = prev_frame_idx(dm, idx_drop); idx_nxt = next_frame_idx(dm, idx_drop); - update_frame_metrics(dm, idx_prv, idx_nxt, idx_drop); + if (!dm->compat) + update_frame_metrics(dm, idx_prv, idx_nxt, idx_drop); mark_drop_frame(dm, idx_drop, 4); } } -- 2.39.2