From patchwork Sun Dec 10 22:11:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marton Balint X-Patchwork-Id: 6692 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp2079148jah; Sun, 10 Dec 2017 14:12:27 -0800 (PST) X-Google-Smtp-Source: AGs4zMbow7Xj0nT4b/cp7HrWv3JgwqkfKF72aAVOALjzDh9kcZe2mUULdE+ui4hu6+DKldeJaOUF X-Received: by 10.223.138.246 with SMTP id z51mr33598131wrz.152.1512943947801; Sun, 10 Dec 2017 14:12:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512943947; cv=none; d=google.com; s=arc-20160816; b=VHQlHs6e3M0lxKogKunqVQftU2bhGypUfWkSMIlFEnMmqVtob5LxEoj5xP8AR5dTbq j7RRDY3k19EMgX6MHWcXAsQPApRcNg7Dg2TqCPuSaIfthujnBQWUGi/39VXHNI2rvBD3 fc6OZyGtOkxF+cdnH4tFlbY5SXrQaWLv7j8JxqlOlEcIW8hZqyTe8NIKhLupkVQmYNez moZqR5czydov7lIhmBUWebEqHAKY9d1L1/1J5moFHsRhkwM3ariHZROpCQAuvlgdnLGz 8tfwWpJeNZ71+tBpBvihXh3ExXytb8Shc1IrjyqXdUp8Q5CzcAqIKZV9NqHp7H6b0udI Q4Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to:arc-authentication-results; bh=tsv0FeauuoQ/f5TP60GLV5d1x/vjduZqpoAy+psH6Hg=; b=vwBllDrXCUlJZSMYJ7OTOrwarL6R/IfVrbMHS8T01ipPhaPEu1Fvq1iZi8bnbRqG/C uK27rnTgOQBg6DIgFciWcwyBrVNyDW9HJ6PosCpRcxFiubyM2NWKNs782L5Dp1cKb31X nwjuE5Ga4yn3fBnBOfA9NjB9RJfHIHwYz+89YdEPoIOxc/3I+vBARga8lII1odobAujy pAGd5phP9Mw/wF4EK5cGSPv+JzAGWXRIJAP2MEqDyhUcGaq8tvO2LH5JXLCPLWLz/ezQ KgLpjDKg4qOUCjvPhZCzu8BcPorOp+rBOK4iCwv4R0WZJVcEx3tCJc5Ws1/TkkIgfNeq 2/Dw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 81si4448581wmn.49.2017.12.10.14.12.27; Sun, 10 Dec 2017 14:12:27 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5663E68A6CD; Mon, 11 Dec 2017 00:11:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from iq.passwd.hu (iq.passwd.hu [217.27.212.140]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8B99668A6BA for ; Mon, 11 Dec 2017 00:11:34 +0200 (EET) Received: from localhost (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id DE06FE17EA; Sun, 10 Dec 2017 23:11:41 +0100 (CET) X-Virus-Scanned: amavisd-new at passwd.hu Received: from iq.passwd.hu ([127.0.0.1]) by localhost (iq.passwd.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xbPbCl1iTuPY; Sun, 10 Dec 2017 23:11:41 +0100 (CET) Received: from bluegene.passwd.hu (localhost [127.0.0.1]) by iq.passwd.hu (Postfix) with ESMTP id 017ADE183E; Sun, 10 Dec 2017 23:11:39 +0100 (CET) From: Marton Balint To: ffmpeg-devel@ffmpeg.org Date: Sun, 10 Dec 2017 23:11:21 +0100 Message-Id: <20171210221122.15674-6-cus@passwd.hu> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171210221122.15674-1-cus@passwd.hu> References: <20171210221122.15674-1-cus@passwd.hu> Subject: [FFmpeg-devel] [PATCH 6/7] avfilter/vf_framerate: do not calculate scene change score multiple times for the same frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Marton Balint MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This speeds up the filter, and also fixes scene change detection score which is reduced based on the difference of the current MAFD to the preivous MAFD. Obviously if we compare two frames twice, the difference will be 0... Signed-off-by: Marton Balint --- libavfilter/vf_framerate.c | 45 +++++++++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/libavfilter/vf_framerate.c b/libavfilter/vf_framerate.c index dd106f8e5b..5fffc2a172 100644 --- a/libavfilter/vf_framerate.c +++ b/libavfilter/vf_framerate.c @@ -71,6 +71,7 @@ typedef struct FrameRateContext { AVFrame *srce[N_SRCE]; ///< buffered source frames int64_t srce_pts_dest[N_SRCE]; ///< pts for source frames scaled to output timebase + double scre_score[N_SRCE]; ///< scene change score compared to the next scre frame int64_t pts; ///< pts of frame we are working on int max; @@ -113,9 +114,11 @@ static void next_source(AVFilterContext *ctx) for (i = s->last; i > s->frst; i--) { ff_dlog(ctx, "next_source() copy %d to %d\n", i - 1, i); s->srce[i] = s->srce[i - 1]; + s->scre_score[i] = s->scre_score[i - 1]; } ff_dlog(ctx, "next_source() make %d null\n", s->frst); s->srce[s->frst] = NULL; + s->scre_score[s->frst] = -1.0; } static av_always_inline int64_t sad_8x8_16(const uint16_t *src1, ptrdiff_t stride1, @@ -171,8 +174,7 @@ static double get_scene_score(AVFilterContext *ctx, AVFrame *crnt, AVFrame *next ff_dlog(ctx, "get_scene_score()\n"); - if (crnt && - crnt->height == next->height && + if (crnt->height == next->height && crnt->width == next->width) { int64_t sad; double mafd, diff; @@ -304,21 +306,26 @@ static int filter_slice16(AVFilterContext *ctx, void *arg, int job, int nb_jobs) } static int blend_frames(AVFilterContext *ctx, float interpolate, - AVFrame *copy_src1, AVFrame *copy_src2) + int src1, int src2) { FrameRateContext *s = ctx->priv; AVFilterLink *outlink = ctx->outputs[0]; double interpolate_scene_score = 0; - if ((s->flags & FRAMERATE_FLAG_SCD) && copy_src2) { - interpolate_scene_score = get_scene_score(ctx, copy_src1, copy_src2); + if ((s->flags & FRAMERATE_FLAG_SCD) && s->srce[src1] && s->srce[src2]) { + int i1 = src1 < src2 ? src1 : src2; + int i2 = src1 < src2 ? src2 : src1; + if (i2 == i1 + 1 && s->scre_score[i1] >= 0.0) + interpolate_scene_score = s->scre_score[i1]; + else + interpolate_scene_score = s->scre_score[i1] = get_scene_score(ctx, s->srce[i1], s->srce[i2]); ff_dlog(ctx, "blend_frames() interpolate scene score:%f\n", interpolate_scene_score); } // decide if the shot-change detection allows us to blend two frames - if (interpolate_scene_score < s->scene_score && copy_src2) { + if (interpolate_scene_score < s->scene_score && s->srce[src2]) { ThreadData td; - td.copy_src1 = copy_src1; - td.copy_src2 = copy_src2; + td.copy_src1 = s->srce[src1]; + td.copy_src2 = s->srce[src2]; td.src2_factor = fabsf(interpolate) * (1 << (s->bitdepth - 8)); td.src1_factor = s->max - td.src2_factor; @@ -340,8 +347,8 @@ static int process_work_frame(AVFilterContext *ctx, int stop) { FrameRateContext *s = ctx->priv; int64_t work_next_pts; - AVFrame *copy_src1; float interpolate; + int src1, src2; ff_dlog(ctx, "process_work_frame()\n"); @@ -385,28 +392,26 @@ static int process_work_frame(AVFilterContext *ctx, int stop) // calculate interpolation interpolate = ((s->pts - s->srce_pts_dest[s->crnt]) * 256.0 / s->average_srce_pts_dest_delta); ff_dlog(ctx, "process_work_frame() interpolate:%f/256\n", interpolate); - copy_src1 = s->srce[s->crnt]; + src1 = s->crnt; if (interpolate > s->interp_end) { ff_dlog(ctx, "process_work_frame() source is:NEXT\n"); - copy_src1 = s->srce[s->next]; + src1 = s->next; } if (s->srce[s->prev] && interpolate < -s->interp_end) { ff_dlog(ctx, "process_work_frame() source is:PREV\n"); - copy_src1 = s->srce[s->prev]; + src1 = s->prev; } // decide whether to blend two frames if ((interpolate >= s->interp_start && interpolate <= s->interp_end) || (interpolate <= -s->interp_start && interpolate >= -s->interp_end)) { - AVFrame *copy_src2; - if (interpolate > 0) { ff_dlog(ctx, "process_work_frame() interpolate source is:NEXT\n"); - copy_src2 = s->srce[s->next]; + src2 = s->next; } else { ff_dlog(ctx, "process_work_frame() interpolate source is:PREV\n"); - copy_src2 = s->srce[s->prev]; + src2 = s->prev; } - if (blend_frames(ctx, interpolate, copy_src1, copy_src2)) + if (blend_frames(ctx, interpolate, src1, src2)) goto copy_done; else ff_dlog(ctx, "process_work_frame() CUT - DON'T INTERPOLATE\n"); @@ -414,7 +419,7 @@ static int process_work_frame(AVFilterContext *ctx, int stop) ff_dlog(ctx, "process_work_frame() COPY to the work frame\n"); // copy the frame we decided is our base source - s->work = av_frame_clone(copy_src1); + s->work = av_frame_clone(s->srce[src1]); if (!s->work) return AVERROR(ENOMEM); @@ -504,6 +509,7 @@ static void set_work_frame_pts(AVFilterContext *ctx) static av_cold int init(AVFilterContext *ctx) { FrameRateContext *s = ctx->priv; + int i; s->dest_frame_num = 0; @@ -513,6 +519,9 @@ static av_cold int init(AVFilterContext *ctx) s->next = s->crnt - 1; s->prev = s->crnt + 1; + for (i = 0; i < N_SRCE; i++) + s->scre_score[i] = -1.0; + return 0; }