From patchwork Sat Oct 14 06:00:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ichlubna@fit.vutbr.cz X-Patchwork-Id: 44251 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4b15:b0:15d:8365:d4b8 with SMTP id fp21csp1424672pzb; Fri, 13 Oct 2023 23:01:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEhvGkMWKZ8acjszlltcCAA50iHcwdH5ECKVwvYxX7XIfR3puEjnpRxR2qqwnOeznECzyxc X-Received: by 2002:a17:906:8a71:b0:9ae:3d75:4c83 with SMTP id hy17-20020a1709068a7100b009ae3d754c83mr22592735ejc.0.1697263314129; Fri, 13 Oct 2023 23:01:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697263314; cv=none; d=google.com; s=arc-20160816; b=S/YvYdYCg0VCbqYV5wtE2JUImsrYS3zbgUQdtyn6Zh9/a0jCwclNL375DJihsaeMla KeYumoTKNXllPG8Yxt9PHppqkE1nnNzB9Ir1dyfA9vdwK51TA+3R2TVUtWQXAr2VzC+c kIHAE9pLV9FqiRJWw/46sOMIGRgGRwnK0J5R1on89aIMFlXg9+VUPyPqXqtdh/3R9nEl lR0kMNTlU8td03CBKN1AX2DTSaonSqi3URDII1MG+DajAEXVr8z3rlcNX01CjUsQysOk mhbBXOQuOSOoUmMxZcYyJWgs3HkN+pqw09q0uAP+7Zz7JEcXk+xmODCBKwxmfnobZfBe T+0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=OrBu0YsR3ila7YioZQpg9TCI/Mpi58rT/e1xvqMhgXw=; fh=wIvelk+5LC/xU/W9IeJ1fSDswWeYASI/V+yTs+XAZwg=; b=RghcET0elam4P6gJVkuZvcV5yx6uxQCIDrYahsvuPQTWTpzZFUWWzNHixF1sqXE/TJ yJHLWRgZvFHt5i8bivJNEZ3p0Zc9DlsF6Y5q6dNoMqWLhPLiTHZzSjmCXqXWKdwvH1ey DQr1+8jRylv70uQTG9U7bRYL8dmi9v7h1/24aG3lfrZZgMViJKabvU0Lo5GQnc83NhFu CFQe94dCL2h6VdqMPeeQxdLTWwr3Qv2eb836MrQNQ23EdjcRE8SmG7PknC+v4MLWDihM pHsDMMF1h96eSN+i4uBHlRNw8mYdtPXrlzzSduFmRv0g0sCFv7AVhUoljf0PyiX9Bkb3 DbTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@fit.vutbr.cz header.s=mx1 header.b=qhHGCY5r; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=fit.vutbr.cz Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gn2-20020a1709070d0200b0099bd602097bsi451835ejc.544.2023.10.13.23.01.53; Fri, 13 Oct 2023 23:01:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@fit.vutbr.cz header.s=mx1 header.b=qhHGCY5r; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=fit.vutbr.cz Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CA88968C7A6; Sat, 14 Oct 2023 09:01:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from kazi.fit.vutbr.cz (kazi.fit.vutbr.cz [147.229.8.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3BB1668C670 for ; Sat, 14 Oct 2023 09:01:42 +0300 (EEST) Received: from holygrail.fit.vutbr.cz ([IPv6:2001:67c:1220:80c:e2d5:5eff:feb5:e2a6]) (authenticated bits=0) by kazi.fit.vutbr.cz (8.17.2/8.17.2) with ESMTPSA id 39E61dmt065445 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 14 Oct 2023 08:01:42 +0200 (CEST) (envelope-from ichlubna@fit.vut.cz) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fit.vutbr.cz; s=mx1; t=1697263302; bh=XXcSPiyvW9yybIen+FXs7N17X1nFV7YDV8X448QgSlc=; h=From:To:Cc:Subject:Date; b=qhHGCY5rj1z6AI0NGh9LKanv4HlS/rx6bYnkamNqLP5Qsw4gfJfujXenzceNBRugk PK7tEZd0wuW20DJg8kTuKQGJT/R5bpMo+ZhQh1B/hcyf1nYr/luSUUK74XIhIiC54M GNLlmJHaUtJXFD05ZFVQab7M2oF7d3wO9akyHCdI= X-Authentication-Warning: kazi.fit.vutbr.cz: Host [IPv6:2001:67c:1220:80c:e2d5:5eff:feb5:e2a6] claimed to be holygrail.fit.vutbr.cz From: ichlubna@fit.vutbr.cz To: ffmpeg-devel@ffmpeg.org Date: Sat, 14 Oct 2023 08:00:42 +0200 Message-ID: <20231014060041.2252-2-ichlubna@fit.vutbr.cz> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] VMAF is now propagated to the AVFrame coming from the graph X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: ichlubna Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: muByJT5PHbj7 From: ichlubna Related to my ticket here: https://trac.ffmpeg.org/ticket/10586 VMAF score was not propagated to AVFormat like SSIM or PSNR in the result of the filter graph. I have fixed this to make the usage consistent and possible to get VMAF score per-frame in libavfilter. The only dirty thing here is the added for loop to compute the score twice. This is done to get the score in real time. Otherwise the score is delayed by one frame. Computing the score twice should not affect the final averaged result as each frame is added twice so the average does not change. --- libavfilter/vf_libvmaf.c | 76 +++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 29 deletions(-) diff --git a/libavfilter/vf_libvmaf.c b/libavfilter/vf_libvmaf.c index 2726b061ac..4c84877812 100644 --- a/libavfilter/vf_libvmaf.c +++ b/libavfilter/vf_libvmaf.c @@ -139,12 +139,40 @@ static int copy_picture_data(AVFrame *src, VmafPicture *dst, unsigned bpc) return 0; } +static enum VmafPoolingMethod pool_method_map(const char *pool_method) +{ + if (pool_method) { + if (av_stristr(pool_method, "min")) + return VMAF_POOL_METHOD_MIN; + if (av_stristr(pool_method, "mean")) + return VMAF_POOL_METHOD_MEAN; + if (av_stristr(pool_method, "harmonic_mean")) + return VMAF_POOL_METHOD_HARMONIC_MEAN; + } + + return VMAF_POOL_METHOD_MEAN; +} + +static void set_meta(AVDictionary **metadata, const char *key, char comp, float d) +{ + char value[128]; + snprintf(value, sizeof(value), "%f", d); + if (comp) { + char key2[128]; + snprintf(key2, sizeof(key2), "%s%c", key, comp); + av_dict_set(metadata, key2, value, 0); + } else { + av_dict_set(metadata, key, value, 0); + } +} + static int do_vmaf(FFFrameSync *fs) { AVFilterContext *ctx = fs->parent; LIBVMAFContext *s = ctx->priv; VmafPicture pic_ref, pic_dist; AVFrame *ref, *dist; + double vmaf_score; int err = 0; int ret = ff_framesync_dualinput_get(fs, &dist, &ref); @@ -160,25 +188,29 @@ static int do_vmaf(FFFrameSync *fs) av_color_range_name(ref->color_range)); } - err = copy_picture_data(ref, &pic_ref, s->bpc); - if (err) { - av_log(s, AV_LOG_ERROR, "problem during vmaf_picture_alloc.\n"); - return AVERROR(ENOMEM); - } + for(int i=0; i<2; i++){ + err = copy_picture_data(ref, &pic_ref, s->bpc); + if (err) { + av_log(s, AV_LOG_ERROR, "problem during vmaf_picture_alloc.\n"); + return AVERROR(ENOMEM); + } - err = copy_picture_data(dist, &pic_dist, s->bpc); - if (err) { - av_log(s, AV_LOG_ERROR, "problem during vmaf_picture_alloc.\n"); - vmaf_picture_unref(&pic_ref); - return AVERROR(ENOMEM); - } + err = copy_picture_data(dist, &pic_dist, s->bpc); + if (err) { + av_log(s, AV_LOG_ERROR, "problem during vmaf_picture_alloc.\n"); + vmaf_picture_unref(&pic_ref); + return AVERROR(ENOMEM); + } - err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt++); - if (err) { - av_log(s, AV_LOG_ERROR, "problem during vmaf_read_pictures.\n"); - return AVERROR(EINVAL); + err = vmaf_read_pictures(s->vmaf, &pic_ref, &pic_dist, s->frame_cnt++); + if (err) { + av_log(s, AV_LOG_ERROR, "problem during vmaf_read_pictures.\n"); + return AVERROR(EINVAL); + } } + vmaf_score_at_index(s->vmaf, s->model[0], &vmaf_score, s->frame_cnt - 2); + set_meta(&dist->metadata, "lavfi.vmaf", 0, vmaf_score); return ff_filter_frame(ctx->outputs[0], dist); } @@ -637,20 +669,6 @@ static enum VmafOutputFormat log_fmt_map(const char *log_fmt) return VMAF_OUTPUT_FORMAT_XML; } -static enum VmafPoolingMethod pool_method_map(const char *pool_method) -{ - if (pool_method) { - if (av_stristr(pool_method, "min")) - return VMAF_POOL_METHOD_MIN; - if (av_stristr(pool_method, "mean")) - return VMAF_POOL_METHOD_MEAN; - if (av_stristr(pool_method, "harmonic_mean")) - return VMAF_POOL_METHOD_HARMONIC_MEAN; - } - - return VMAF_POOL_METHOD_MEAN; -} - static av_cold void uninit(AVFilterContext *ctx) { LIBVMAFContext *s = ctx->priv;