From patchwork Mon Jan 2 22:52:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gerion Entrup X-Patchwork-Id: 2027 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp4132917vsb; Mon, 2 Jan 2017 14:53:10 -0800 (PST) X-Received: by 10.194.58.198 with SMTP id t6mr48952635wjq.44.1483397590771; Mon, 02 Jan 2017 14:53:10 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h188si71219140wma.91.2017.01.02.14.53.09; Mon, 02 Jan 2017 14:53:10 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7692A689E12; Tue, 3 Jan 2017 00:53:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.flump.de (unknown [84.200.20.107]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B6ACC68975F for ; Tue, 3 Jan 2017 00:52:55 +0200 (EET) Received: from gump.localnet (p579313FE.dip0.t-ipconnect.de [87.147.19.254]) by mail.flump.de (Postfix) with ESMTPSA id C215A182471 for ; Mon, 2 Jan 2017 23:52:57 +0100 (CET) From: Gerion Entrup To: ffmpeg-devel@ffmpeg.org Date: Mon, 02 Jan 2017 23:52:58 +0100 Message-ID: <4126592.Myufu8ebLk@gump> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCHv3] add signature filter for MPEG7 video signature X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hi, I made a new thread because of the delay between this mail and my last one. Attached is the next iteration of the patch (rebased to current master). Main change is, that I've moved the writing and lookup code from uninit to request_frame (thank you for the suggestion). Please comment. Kind regards, Gerion diff --git a/libavfilter/signature.h b/libavfilter/signature.h index ed1c32e..195c94c 100644 --- a/libavfilter/signature.h +++ b/libavfilter/signature.h @@ -120,6 +120,8 @@ typedef struct { int coursecount; /* counter from 0 to 89 */ int midcourse; /* whether it is a coursesignature beginning from 45 + i * 90 */ uint32_t lastindex; /* helper to store amount of frames */ + + int exported; /* boolean whether stream already exported */ } StreamContext; typedef struct { diff --git a/libavfilter/vf_signature.c b/libavfilter/vf_signature.c index 2b51724..e1fff9b 100644 --- a/libavfilter/vf_signature.c +++ b/libavfilter/vf_signature.c @@ -71,6 +71,7 @@ AVFILTER_DEFINE_CLASS(signature); static int query_formats(AVFilterContext *ctx) { + /* all formats with a seperate gray value */ static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_GRAY8, AV_PIX_FMT_YUV410P, AV_PIX_FMT_YUV411P, @@ -586,21 +587,56 @@ static int export(AVFilterContext *ctx, StreamContext *sc, int input) static int request_frame(AVFilterLink *outlink) { AVFilterContext *ctx = outlink->src; - SignatureContext *sc = ctx->priv; - int i, ret; + SignatureContext *sic = ctx->priv; + StreamContext *sc, *sc2; + MatchingInfo match; + int i, j, ret; + int lookup = 1; /* indicates wheather EOF of all files is reached */ + + /* process all inputs */ + for (i = 0; i < sic->nb_inputs; i++){ + sc = &(sic->streamcontexts[i]); - for (i = 0; i < sc->nb_inputs; i++){ ret = ff_request_frame(ctx->inputs[i]); - // TODO handle this in a better way? - // Problem is the following: - // Assuming two inputs, inputA with 50 frames, inputB with 100 frames - // simply returning ret when < 0 would result in not filtering inputB - // after 50 frames anymore, not wanted - // only returning ret at the end would result in only respecting the error - // values of the last input, please comment + + /* return if unexspected error occurs in input stream */ if (ret < 0 && ret != AVERROR_EOF) return ret; + + /* export signature at EOF */ + if (ret == AVERROR_EOF){ + /* export if wanted */ + if (strlen(sic->filename) > 0) { + if (export(ctx, sc, i) < 0) + return ret; + } + sc->exported = 1; + } + lookup &= sc->exported; + } + + /* signature lookup */ + if (lookup && sic->mode != MODE_OFF) { + /* iterate over every pair */ + for (i = 0; i < sic->nb_inputs; i++) { + sc = &(sic->streamcontexts[i]); + for (j = i+1; j < sic->nb_inputs; j++) { + sc2 = &(sic->streamcontexts[j]); + match = lookup_signatures(ctx, sic, sc, sc2, sic->mode); + if (match.score != 0) { + av_log(ctx, AV_LOG_INFO, "matching of video %d at %f and %d at %f, %d frames matching\n", + i, ((double) match.first->pts * sc->time_base.num) / sc->time_base.den, + j, ((double) match.second->pts * sc2->time_base.num) / sc2->time_base.den, + match.matchframes); + if (match.whole) + av_log(ctx, AV_LOG_INFO, "whole video matching\n"); + } else { + av_log(ctx, AV_LOG_INFO, "no matching of video %d and %d\n", i, j); + } + } + } } + return ret; } @@ -663,52 +699,14 @@ static av_cold int init(AVFilterContext *ctx) static av_cold void uninit(AVFilterContext *ctx) { SignatureContext *sic = ctx->priv; - StreamContext *sc, *sc2; + StreamContext *sc; void* tmp; FineSignature* finsig; CourseSignature* cousig; - MatchingInfo match; - int i,j; - - //TODO export and especially lookup_signature can have a return value to show some error etc. - //How could this be handled in this function? + int i; - /* signature export */ - if (strlen(sic->filename) > 0) { - for (i = 0; i < sic->nb_inputs; i++) { - /* if filter frame was called at least once */ - if (&(sic->streamcontexts[i]) && (&(sic->streamcontexts[i]))->curfinesig) - export(ctx, &(sic->streamcontexts[i]), i); - } - } - - /* signature lookup */ - if (sic->mode != MODE_OFF) { - /* iterate over every pair */ - for (i = 0; i < sic->nb_inputs; i++) { - sc = &(sic->streamcontexts[i]); - if (sc && sc->curfinesig) { - for (j = i+1; j < sic->nb_inputs; j++) { - sc2 = &(sic->streamcontexts[j]); - if (sc2 && sc2->curfinesig) { - match = lookup_signatures(ctx, sic, sc, sc2, sic->mode); - if (match.score != 0) { - av_log(ctx, AV_LOG_INFO, "matching of video %d at %f and %d at %f, %d frames matching\n", - i, ((double) match.first->pts * sc->time_base.num) / sc->time_base.den, - j, ((double) match.second->pts * sc2->time_base.num) / sc2->time_base.den, - match.matchframes); - if (match.whole) - av_log(ctx, AV_LOG_INFO, "whole video matching\n"); - } else { - av_log(ctx, AV_LOG_INFO, "no matching of video %d and %d\n", i, j); - } - } - } - } - } - } - /* cleanup */ + /* free the lists */ if (sic->streamcontexts != NULL) { for (i = 0; i < sic->nb_inputs; i++) { sc = &(sic->streamcontexts[i]);