From patchwork Mon Aug 19 23:36:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?UTF-8?q?Juan=20De=20Le=C3=B3n?= X-Patchwork-Id: 14602 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AE4E6447D66 for ; Tue, 20 Aug 2019 02:37:39 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9692168AB78; Tue, 20 Aug 2019 02:37:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4AE6768AAE8 for ; Tue, 20 Aug 2019 02:37:31 +0300 (EEST) Received: by mail-pl1-f201.google.com with SMTP id h3so2880041plr.22 for ; Mon, 19 Aug 2019 16:37:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=LSAt8E7ehb39yzZME0YahVdAzn67IEumEMSHpDFXxsg=; b=tecsyW7K6SZ5vP8xJQ+ueaERjHpqhQpCZoDbApus1e4iJKryfalvLeqQdndidvdlcq hpdgRrZ9K5P+p8c8dUW05VowsNKuZ+KRSNA/Y+8ud8Gg83g8qut1nYvVf2whyerP3PPQ sYXxp1hKISZ14SmNuRBDheL6zqQnK6tS8331UHF7cz1qsCXmD9fE5ZJeaI+x2qL7wXrW XYBuw+A0DsfsysBoYNeTmNGZDasPQfkGwon+VQk4LumNpWwpBoyISRM3I0zHtVLo16TZ 6Cp1RcpIR51fk/910HOTWti5/0khaedydf/brVU5jp5IwRyJxE8aiTSvL7PrSnJBitaB 3VOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=LSAt8E7ehb39yzZME0YahVdAzn67IEumEMSHpDFXxsg=; b=OHqOfWAWiYyt/Fwcr1IBkPV7lVwfEgsuzIt8j538kcLKHrlIxn6BclPGTwIj1AFnAT XI9fLQKrm3B9oRopxKmW54xM9X6X9YTcmyqxcV5dKc5yNOU8oyt6zO9cuL88kzZYGabo VgWTfoErYCdIJkfmqQTfitHW0Tuewtxp+KLbkWOGr6nUX2wB1JBOVaKZsRNyXaPOfeFS D9aJia1NBNev4iYRcRInbTROKgCu+yHA+dK2W8cvsji/mmDnT8cDSXuRJVCv0k1ozzHg +2FzvqZ6AY96LC/Nr+Vr56VTHgIIqPAr+5nNsAnxKIhb0ubaVdNYPF+/wwm7Oukr8w4K v6dg== X-Gm-Message-State: APjAAAVlT+ZCxY6Sv1NaRb9kjCKsHB7mg18C9cYLGY0MX3zIRO28vMke fykf6TkoV0Y+vG9xqU8Mwd05aO8Wor5MG2yY8XJF3LjpSFJ92F/llzF2ttkFoaqSqdLthnNqWLG NKzq+PhHNhzbJRU+GydboKz/ZWwqGK5jisQbaohOTvDVnqvIYZGbvOICjU00SQqU= X-Google-Smtp-Source: APXvYqwshUpCX4TyjExc5/DOlvzMeHEkxCHmnT5sDHAoxWZO/zpjU1oB+VmKmBIbdEswfYKhPlgEoLP42/o= X-Received: by 2002:a65:68c9:: with SMTP id k9mr21737209pgt.17.1566257849372; Mon, 19 Aug 2019 16:37:29 -0700 (PDT) Date: Mon, 19 Aug 2019 16:36:55 -0700 In-Reply-To: <20190819233655.171637-1-juandl@google.com> Message-Id: <20190819233655.171637-4-juandl@google.com> Mime-Version: 1.0 References: <20190819233655.171637-1-juandl@google.com> X-Mailer: git-send-email 2.23.0.rc1.153.gdeed80330f-goog From: "=?UTF-8?q?Juan=20De=20Le=C3=B3n?=" To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 3/3] libavfilter: extractqp filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: =?UTF-8?q?Juan=20De=20Le=C3=B3n?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Extracts quantization parameters data from AVEncodeInfoFrame side data, if available, then calculates min/max/avg and outputs the results in a log file. Signed-off-by: Juan De León --- doc/filters.texi | 40 ++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_extractqp.c | 243 +++++++++++++++++++++++++++++++++++++ 4 files changed, 285 insertions(+) create mode 100644 libavfilter/vf_extractqp.c diff --git a/doc/filters.texi b/doc/filters.texi index e081cdc7bc..46a82b147a 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -9607,6 +9607,46 @@ ffmpeg -i video.avi -filter_complex 'extractplanes=y+u+v[y][u][v]' -map '[y]' y. @end example @end itemize +@section extractqp + +Extracts Quantization Parameters from @code{AVFrameSideData} and calculates min/max/avg QP. + +QP extraction must be enabled with the option @code{-debug extractqp} before decoding the stream and calling the filter. + +The filter accepts the following options: +@table @option +@item stats_file, f +If specified, the filter will use the named file to output the QP data. If set to '-' the data is sent to standard output. +@item log +If specificed, sets the log level for the output out of three options (defaults to frame): +@table @samp +@item frame +Default log level. Outputs min/max/avg per frame. +@item block +Outputs qp data for every block in each frame. +@item all +Outputs min/max/avg and block data per frame. +@end table +@end table + +Supported decoders: +@itemize +@item x264 +@end itemize + +@subsection Examples + +@itemize +@item Get QP min/max/avg and store it in QP.log +@example +ffmpeg -debug extractqp -i -lavfi extractqp="stats_file=QP.log" -f null - +@end example +@item Output only block data +@example +ffmpeg -debug extractqp -i -lavfi extractqp="stats_file=QP.log:log=block" -f null - +@end example +@end itemize + @section elbg Apply a posterize effect using the ELBG (Enhanced LBG) algorithm. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index efc7bbb153..5944558c14 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -231,6 +231,7 @@ OBJS-$(CONFIG_EROSION_FILTER) += vf_neighbor.o OBJS-$(CONFIG_EROSION_OPENCL_FILTER) += vf_neighbor_opencl.o opencl.o \ opencl/neighbor.o OBJS-$(CONFIG_EXTRACTPLANES_FILTER) += vf_extractplanes.o +OBJS-$(CONFIG_EXTRACTQP_FILTER) += vf_extractqp.o OBJS-$(CONFIG_FADE_FILTER) += vf_fade.o OBJS-$(CONFIG_FFTDNOIZ_FILTER) += vf_fftdnoiz.o OBJS-$(CONFIG_FFTFILT_FILTER) += vf_fftfilt.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index abd726d616..93ea25401a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -217,6 +217,7 @@ extern AVFilter ff_vf_eq; extern AVFilter ff_vf_erosion; extern AVFilter ff_vf_erosion_opencl; extern AVFilter ff_vf_extractplanes; +extern AVFilter ff_vf_extractqp; extern AVFilter ff_vf_fade; extern AVFilter ff_vf_fftdnoiz; extern AVFilter ff_vf_fftfilt; diff --git a/libavfilter/vf_extractqp.c b/libavfilter/vf_extractqp.c new file mode 100644 index 0000000000..67a1770849 --- /dev/null +++ b/libavfilter/vf_extractqp.c @@ -0,0 +1,243 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include "libavutil/frame.h" +#include "libavutil/avstring.h" +#include "libavutil/opt.h" +#include "libavutil/encode_info.h" +#include "libavfilter/avfilter.h" +#include "libavfilter/internal.h" +#include "libavfilter/video.h" +#include "libavformat/avio.h" + +// Output flags +#define EXTRACTQP_LOG_FRAME (1<<0) +#define EXTRACTQP_LOG_BLOCK (1<<1) + +typedef struct ExtractQPContext { + const AVClass *class; + int log_level, nb_frames; + int frame_w, frame_h; + AVEncodeInfoFrame *frame_info; + int min_q, max_q; + double avg_q; + FILE *stats_file; + char *stats_file_str; + AVIOContext *avio_context; +} ExtractQPContext; + +#define OFFSET(x) offsetof(ExtractQPContext, x) +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM + +static const AVOption extractqp_options[] = { + { "stats_file", "Set file to store QP information", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + { "f", "Set file to store QP information", OFFSET(stats_file_str), AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, FLAGS }, + { "log", "Set output log level for filter" , OFFSET(log_level), AV_OPT_TYPE_INT, {.i64=EXTRACTQP_LOG_FRAME}, INT_MIN, INT_MAX, FLAGS, "log"}, + { "frame", "Default log level. Outputs min/max/avg per frame.", 0, AV_OPT_TYPE_CONST, {.i64=EXTRACTQP_LOG_FRAME}, INT_MIN, INT_MAX, FLAGS, "log"}, + { "block", "Outputs both calculations and blocks.", 0, AV_OPT_TYPE_CONST, {.i64=EXTRACTQP_LOG_BLOCK}, INT_MIN, INT_MAX, FLAGS, "log"}, + { "all", "Outputs both calculations and blocks.", 0, AV_OPT_TYPE_CONST, {.i64=EXTRACTQP_LOG_BLOCK+EXTRACTQP_LOG_FRAME}, INT_MIN, INT_MAX, FLAGS, "log"}, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(extractqp); + +/** + * Reset calculations to process next frame. + */ +static int resetStats(ExtractQPContext *s) +{ + s->min_q = INT_MAX; + s->max_q = INT_MIN; + s->avg_q = 0; + return 0; +} + +static int calculateStats(ExtractQPContext *s) +{ + AVEncodeInfoFrame *frame_info = s->frame_info; + + /* + * Sometimes frame_width*frame_height does not match the coded pic dimensions. + * Average is always accurate if the area of each block is summed to get the + * area of the coded frame. + */ + int pixcount = 0; + + // Calculates min, max, and avg of the deltas then adds base quantizer + if (frame_info->nb_blocks > 0) { + for(int i = 0; i < frame_info->nb_blocks; i++) { + AVEncodeInfoBlock *block = av_encode_info_get_block(frame_info, i); + s->min_q = FFMIN(block->delta_q, s->min_q); + s->max_q = FFMAX(block->delta_q, s->max_q); + + // Normalizes delta_q based on block size. + s->avg_q += block->delta_q * block->w*block->h; + pixcount += block->w * block->h; + } + + s->avg_q= s->avg_q / pixcount; + + s->min_q += frame_info->plane_q[0]; + s->max_q += frame_info->plane_q[0]; + s->avg_q += frame_info->plane_q[0]; + } + + else { + s->min_q = s->max_q = s->avg_q = frame_info->plane_q[0]; + } + + return 0; +} + +static int printStats(ExtractQPContext *s) +{ + AVEncodeInfoFrame *frame_info = s->frame_info; + + //prints for default, log=frame, or log=all + if (s->log_level & EXTRACTQP_LOG_FRAME) { + avio_printf(s->avio_context, "n:%d min_q:%d max_q:%d avg_q:%.3f ", + s->nb_frames, s->min_q, s->max_q, s->avg_q); + + avio_printf(s->avio_context, "base_q:%d ac_q:%d dc_q:%d ", + frame_info->plane_q[0], frame_info->ac_q, frame_info->dc_q); + + avio_printf(s->avio_context, "uv_q:%d ac_uv_q:%d dc_uv_q:%d ", + frame_info->plane_q[1], frame_info->ac_q, frame_info->dc_q); + + avio_printf(s->avio_context, "v_q:%d ", frame_info->plane_q[2]); + + avio_write(s->avio_context, "\n", sizeof (char)); + } + + //prints for log=block or log=all + if(s->log_level & EXTRACTQP_LOG_BLOCK) { + AVEncodeInfoBlock *block; + int qp; + for (int i=0; inb_blocks; i++) { + block = av_encode_info_get_block(frame_info, i); + qp = block->delta_q + frame_info->plane_q[0]; + + avio_printf(s->avio_context, "n:%d x:%d y:%d w:%d h:%d qp:%d\n", + s->nb_frames, block->src_x, block->src_y, block->w, block->h, qp); + } + } + + return 0; +} + +/** + * Main filtering function, checks for AV_FRAME_DATA_ENCODE_INFO type in + * AVFrameSideData for each frame and calculates min, max, and average qp + * for each frame, then prints the results in a log file. + * User manual in doc/filters.texi + */ +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + ExtractQPContext *s = ctx->priv; + + s->nb_frames++; + + if (ctx->is_disabled) { + return ff_filter_frame(outlink, in); + } + + AVFrameSideData *sd = av_frame_get_side_data(in, AV_FRAME_DATA_ENCODE_INFO); + if (!sd) { + av_log(ctx, AV_LOG_ERROR, "No encode info side data found in frame %d\n", s->nb_frames); + return ff_filter_frame(outlink, in); + } + + s->frame_info = (AVEncodeInfoFrame *)sd->data; + if (!(s->frame_info)) { + av_log(ctx, AV_LOG_ERROR, "Empty side data in frame:%d\n", s->nb_frames); + return ff_filter_frame(outlink, in); + } + + calculateStats(s); + printStats(s); + resetStats(s); + + return ff_filter_frame(outlink, in); +} + +static av_cold int init(AVFilterContext *ctx) +{ + ExtractQPContext *s = ctx->priv; + int ret; + + resetStats(s); + + s->avio_context = NULL; + if (s->stats_file_str) { + if (!strcmp("-", s->stats_file_str)) { + ret = avio_open(&s->avio_context, "pipe:1", AVIO_FLAG_WRITE); + } else { + ret = avio_open(&s->avio_context, s->stats_file_str, AVIO_FLAG_WRITE); + } + } + if (ret < 0) { + char buf[128]; + av_strerror(ret, buf, sizeof(buf)); + av_log(ctx, AV_LOG_ERROR, "Could not open %s: %s\n", + s->stats_file_str, buf); + return ret; + } + + return 0; +} + +static av_cold int uninit(AVFilterContext *ctx) { + ExtractQPContext *s = ctx->priv; + if (s->avio_context) { + avio_closep(&s->avio_context); + } + return 0; +} + +static const AVFilterPad extractqp_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad extractqp_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_extractqp = { + .name = "extractqp", + .description = NULL_IF_CONFIG_SMALL("Extract qp from AV_FRAME_DATA_ENCODE_INFO side data type."), + .init = init, + .uninit = uninit, + .priv_size = sizeof(ExtractQPContext), + .priv_class = &extractqp_class, + .inputs = extractqp_inputs, + .outputs = extractqp_outputs, +};