From patchwork Sat Apr 18 10:14:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19050 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id BE4BF44BB72 for ; Sat, 18 Apr 2020 13:15:19 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A54E268BF6D; Sat, 18 Apr 2020 13:15:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3C78A68BF3E for ; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 6D11C288ABD for ; Sat, 18 Apr 2020 12:15:06 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Yt_ungffa0Lq for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 94114288A94 for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 7803A21842 for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id QAcoUuj-wbdZ for ; Sat, 18 Apr 2020 12:14:57 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 15AC221844 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id A2F2520E003E; Sat, 18 Apr 2020 12:14:48 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:07 +0200 Message-Id: <20200418101415.26409-2-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/9] libavutil: add API for exporting video frame quantizers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Juan De León This is intended to replace the deprecated the AV_FRAME_DATA_QP_TABLE* API and extend it to a wider range of codecs. In the future, it may also be extended to support other encoding parameters such as motion vectors. Additional changes by Anton Khirnov with suggestions by Lynne . Signed-off-by: Juan De León Signed-off-by: Michael Niedermayer Signed-off-by: Anton Khirnov --- doc/APIchanges | 4 ++ libavutil/Makefile | 5 +- libavutil/frame.c | 1 + libavutil/frame.h | 5 ++ libavutil/version.h | 4 +- libavutil/video_enc_params.c | 79 ++++++++++++++++++++ libavutil/video_enc_params.h | 135 +++++++++++++++++++++++++++++++++++ 7 files changed, 230 insertions(+), 3 deletions(-) create mode 100644 libavutil/video_enc_params.c create mode 100644 libavutil/video_enc_params.h diff --git a/doc/APIchanges b/doc/APIchanges index e30148dc90..2fc9428827 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,10 @@ libavutil: 2017-10-21 API changes, most recent first: +2020-xx-xx - xxxxxxxxxx - lavu 56.43.100 - video_enc_params.h + Add a new API for exporting video encoding information. + Replaces the deprecated API for exporting QP tables from decoders. + 2020-04-15 - xxxxxxxxxx - lavc 58.79.100 - avcodec.h Add formal support for calling avcodec_flush_buffers() on encoders. Encoders that set the cap AV_CODEC_CAP_ENCODER_FLUSH will be flushed. diff --git a/libavutil/Makefile b/libavutil/Makefile index 8feb029a3a..d44b62877f 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -79,6 +79,7 @@ HEADERS = adler32.h \ tree.h \ twofish.h \ version.h \ + video_enc_params.h \ xtea.h \ tea.h \ tx.h \ @@ -165,7 +166,9 @@ OBJS = adler32.o \ tx.o \ tx_float.o \ tx_double.o \ - tx_int32.o + tx_int32.o \ + video_enc_params.o \ + OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o diff --git a/libavutil/frame.c b/libavutil/frame.c index 769851ceac..53581e4862 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -850,6 +850,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) #endif case AV_FRAME_DATA_DYNAMIC_HDR_PLUS: return "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)"; case AV_FRAME_DATA_REGIONS_OF_INTEREST: return "Regions Of Interest"; + case AV_FRAME_DATA_VIDEO_ENC_PARAMS: return "Video encoding parameters"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index b5afb58634..fc67db0f6c 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -179,6 +179,11 @@ enum AVFrameSideDataType { * array element is implied by AVFrameSideData.size / AVRegionOfInterest.self_size. */ AV_FRAME_DATA_REGIONS_OF_INTEREST, + + /** + * Encoding parameters for a video frame, as described by AVVideoEncParams. + */ + AV_FRAME_DATA_VIDEO_ENC_PARAMS, }; enum AVActiveFormatDescription { diff --git a/libavutil/version.h b/libavutil/version.h index 70836a5d52..ea9363e8e9 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 42 -#define LIBAVUTIL_VERSION_MICRO 102 +#define LIBAVUTIL_VERSION_MINOR 43 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \ diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c new file mode 100644 index 0000000000..c46c0f1dc6 --- /dev/null +++ b/libavutil/video_enc_params.c @@ -0,0 +1,79 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include + +#include "buffer.h" +#include "common.h" +#include "frame.h" +#include "mem.h" +#include "video_enc_params.h" + +AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type, + unsigned int nb_blocks, size_t *out_size) +{ + AVVideoEncParams *par; + size_t size; + + size = sizeof(*par); + if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) || + nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size) + return NULL; + size += sizeof(AVVideoBlockParams) * nb_blocks; + + par = av_mallocz(size); + if (!par) + return NULL; + + par->type = type; + par->nb_blocks = nb_blocks; + par->block_size = sizeof(AVVideoBlockParams); + par->blocks_offset = sizeof(*par); + + if (out_size) + *out_size = size; + + return par; +} + +AVVideoEncParams* +av_video_enc_params_create_side_data(AVFrame *frame, enum AVVideoEncParamsType type, + unsigned int nb_blocks) +{ + AVBufferRef *buf; + AVVideoEncParams *par; + size_t size; + + par = av_video_enc_params_alloc(type, nb_blocks, &size); + if (!par) + return NULL; + buf = av_buffer_create((uint8_t *)par, size, NULL, NULL, 0); + if (!buf) { + av_freep(&par); + return NULL; + } + + if (!av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_VIDEO_ENC_PARAMS, buf)) { + av_buffer_unref(&buf); + return NULL; + } + + return par; +} diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h new file mode 100644 index 0000000000..f64864caae --- /dev/null +++ b/libavutil/video_enc_params.h @@ -0,0 +1,135 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_VIDEO_ENC_PARAMS_H +#define AVUTIL_VIDEO_ENC_PARAMS_H + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/frame.h" + +enum AVVideoEncParamsType { + AV_VIDEO_ENC_PARAMS_NONE = -1, +}; + +/** + * Video encoding parameters for a given frame. This struct is allocated along + * with an optional array of per-block AVVideoBlockParams descriptors. + * Must be allocated with av_video_enc_params_alloc(). + */ +typedef struct AVVideoEncParams { + /** + * Number of blocks in the array. + * + * May be 0, in which case no per-block information is present. In this case + * the values of blocks_offset / block_size are unspecified and should not + * be accessed. + */ + unsigned int nb_blocks; + /** + * Offset in bytes from the beginning of this structure at which the array + * of blocks starts. + */ + size_t blocks_offset; + /* + * Size of each block in bytes. May not match sizeof(AVVideoBlockParams). + */ + size_t block_size; + + /** + * Type of the parameters (the codec they are used with). + */ + enum AVVideoEncParamsType type; + + /** + * Base quantisation parameter for the frame. The final quantiser for a + * given block in a given plane is obtained from this value, possibly + * combined with {@code delta_qp} and the per-block delta in a manner + * documented for each type. + */ + int32_t qp; + + /** + * Quantisation parameter offset from the base (per-frame) qp for a given + * plane (first index) and AC/DC coefficients (second index). + */ + int32_t delta_qp[4][2]; +} AVVideoEncParams; + +/** + * Data structure for storing block-level encoding information. + * It is allocated as a part of AVVideoEncParams and should be retrieved with + * av_video_enc_params_block(). + * + * sizeof(AVVideoBlockParams) is not a part of the ABI and new fields may be + * added to it. + */ +typedef struct AVVideoBlockParams { + /** + * Distance in luma pixels from the top-left corner of the visible frame + * to the top-left corner of the block. + * Can be negative if top/right padding is present on the coded frame. + */ + int src_x, src_y; + /** + * Width and height of the block in luma pixels. + */ + int w, h; + + /** + * Difference between this block's final quantization parameter and the + * corresponding per-frame value. + */ + int32_t delta_qp; +} AVVideoBlockParams; + +/* + * Get the block at the specified {@code idx}. Must be between 0 and nb_blocks. + */ +static av_always_inline AVVideoBlockParams* +av_video_enc_params_block(AVVideoEncParams *par, unsigned int idx) +{ + av_assert0(idx < par->nb_blocks); + return (AVVideoBlockParams *)((uint8_t *)par + par->blocks_offset + + idx * par->block_size); +} + +/** + * Allocates memory for AVVideoEncParams of the given type, plus an array of + * {@code nb_blocks} AVVideoBlockParams and initializes the variables. Can be + * freed with a normal av_free() call. + * + * @param out_size if non-NULL, the size in bytes of the resulting data array is + * written here. + */ +AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type, + unsigned int nb_blocks, size_t *out_size); + +/** + * Allocates memory for AVEncodeInfoFrame plus an array of + * {@code nb_blocks} AVEncodeInfoBlock in the given AVFrame {@code frame} + * as AVFrameSideData of type AV_FRAME_DATA_ENCODE_INFO + * and initializes the variables. + */ +AVVideoEncParams* +av_video_enc_params_create_side_data(AVFrame *frame, enum AVVideoEncParamsType type, + unsigned int nb_blocks); + +#endif /* AVUTIL_VIDEO_ENC_PARAMS_H */ From patchwork Sat Apr 18 10:14:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19044 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 73E3444BB72 for ; Sat, 18 Apr 2020 13:15:12 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 570F968BF4A; Sat, 18 Apr 2020 13:15:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CA1EA68BD9F for ; Sat, 18 Apr 2020 13:15:03 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 39CA3288174 for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Soxr7esDmVQG for ; Sat, 18 Apr 2020 12:15:02 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id B9F9D288A60 for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 15B2D213CC for ; Sat, 18 Apr 2020 12:14:59 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id AE0frxmR6fyb for ; Sat, 18 Apr 2020 12:14:57 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 13CC521843 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id C2CB620E0085; Sat, 18 Apr 2020 12:14:48 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:08 +0200 Message-Id: <20200418101415.26409-3-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/9] vf_showinfo: support AV_FRAME_DATA_VIDEO_ENC_PARAMS X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavfilter/vf_showinfo.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c index b0b0051357..5d4aee4169 100644 --- a/libavfilter/vf_showinfo.c +++ b/libavfilter/vf_showinfo.c @@ -36,6 +36,7 @@ #include "libavutil/timestamp.h" #include "libavutil/timecode.h" #include "libavutil/mastering_display_metadata.h" +#include "libavutil/video_enc_params.h" #include "avfilter.h" #include "internal.h" @@ -170,6 +171,25 @@ static void dump_content_light_metadata(AVFilterContext *ctx, AVFrameSideData *s metadata->MaxCLL, metadata->MaxFALL); } +static void dump_video_enc_params(AVFilterContext *ctx, AVFrameSideData *sd) +{ + AVVideoEncParams *par = (AVVideoEncParams*)sd->data; + int plane, acdc; + + av_log(ctx, AV_LOG_INFO, "video encoding parameters: type %d; ", par->type); + if (par->qp) + av_log(ctx, AV_LOG_INFO, "qp=%d; ", par->qp); + for (plane = 0; plane < FF_ARRAY_ELEMS(par->delta_qp); plane++) + for (acdc = 0; acdc < FF_ARRAY_ELEMS(par->delta_qp[plane]); acdc++) { + int delta_qp = par->delta_qp[plane][acdc]; + if (delta_qp) + av_log(ctx, AV_LOG_INFO, "delta_qp[%d][%d]=%d; ", + plane, acdc, delta_qp); + } + if (par->nb_blocks) + av_log(ctx, AV_LOG_INFO, "%u blocks; ", par->nb_blocks); +} + static void dump_color_property(AVFilterContext *ctx, AVFrame *frame) { const char *color_range_str = av_color_range_name(frame->color_range); @@ -352,6 +372,9 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) av_log(ctx, AV_LOG_INFO, "GOP timecode - %s", tcbuf); break; } + case AV_FRAME_DATA_VIDEO_ENC_PARAMS: + dump_video_enc_params(ctx, sd); + break; default: av_log(ctx, AV_LOG_WARNING, "unknown side data type %d (%d bytes)", sd->type, sd->size); From patchwork Sat Apr 18 10:14:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19043 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8836844BB72 for ; Sat, 18 Apr 2020 13:15:10 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7397C68BF40; Sat, 18 Apr 2020 13:15:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C796468BD8C for ; Sat, 18 Apr 2020 13:15:03 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id D1B95288A8A for ; Sat, 18 Apr 2020 12:15:02 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id E6IW5b4xIxyD for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 43860288A78 for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id C14762184C for ; Sat, 18 Apr 2020 12:14:57 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Q8msP7mDMekH for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 117AF21842 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id ED3A820E0087; Sat, 18 Apr 2020 12:14:48 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:09 +0200 Message-Id: <20200418101415.26409-4-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/9] lavc: add a flag for exporting AVVideoEncParams from decoders X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/APIchanges | 3 +++ libavcodec/avcodec.h | 5 +++++ libavcodec/options_table.h | 1 + libavcodec/version.h | 2 +- 4 files changed, 10 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index 2fc9428827..c5f56edde6 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -16,8 +16,11 @@ libavutil: 2017-10-21 API changes, most recent first: 2020-xx-xx - xxxxxxxxxx - lavu 56.43.100 - video_enc_params.h + lavc 58.81.100 - avcodec.h Add a new API for exporting video encoding information. Replaces the deprecated API for exporting QP tables from decoders. + Add AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS to request this information from + decoders. 2020-04-15 - xxxxxxxxxx - lavc 58.79.100 - avcodec.h Add formal support for calling avcodec_flush_buffers() on encoders. diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index b79b025e53..5d800dae75 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -531,6 +531,11 @@ typedef struct RcOverride{ * Export encoder Producer Reference Time through packet side data */ #define AV_CODEC_EXPORT_DATA_PRFT (1 << 1) +/** + * Decoding only. + * Export the AVVideoEncParams structure through frame side data. + */ +#define AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS (1 << 2) /** * Pan Scan area. diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index 54366747ca..695fa5c211 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -81,6 +81,7 @@ static const AVOption avcodec_options[] = { {"export_side_data", "Export metadata as side data", OFFSET(export_side_data), AV_OPT_TYPE_FLAGS, {.i64 = DEFAULT}, 0, UINT_MAX, A|V|S|D|E, "export_side_data"}, {"mvs", "export motion vectors through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_MVS}, INT_MIN, INT_MAX, V|D, "export_side_data"}, {"prft", "export Producer Reference Time through packet side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_PRFT}, INT_MIN, INT_MAX, A|V|S|E, "export_side_data"}, +{"venc_params", "export video encoding parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS}, INT_MIN, INT_MAX, A|V|S|E, "export_side_data"}, {"time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, {.dbl = 0}, 0, INT_MAX}, {"g", "set the group of picture (GOP) size", OFFSET(gop_size), AV_OPT_TYPE_INT, {.i64 = 12 }, INT_MIN, INT_MAX, V|E}, {"ar", "set audio sampling rate (in Hz)", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64 = DEFAULT }, 0, INT_MAX, A|D|E}, diff --git a/libavcodec/version.h b/libavcodec/version.h index 8cff2e855b..ad85fb15e5 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 58 -#define LIBAVCODEC_VERSION_MINOR 80 +#define LIBAVCODEC_VERSION_MINOR 81 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ From patchwork Sat Apr 18 10:14:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19042 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A3C7F44BB72 for ; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7F81A68BF34; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C524968B867 for ; Sat, 18 Apr 2020 13:15:03 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 4B4A0288A60 for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 1Z9X8X9rmRbR for ; Sat, 18 Apr 2020 12:15:02 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 3D7C0288A92 for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 88BDB21843 for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id mOIsLIkuvZ6l for ; Sat, 18 Apr 2020 12:14:59 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 17CB021845 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 12B7420E0089; Sat, 18 Apr 2020 12:14:48 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:10 +0200 Message-Id: <20200418101415.26409-5-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/9] mpegvideo: use the AVVideoEncParams API for exporting QP tables X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Do it only when requested with the AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS flag. Drop previous code using the long-deprecated AV_FRAME_DATA_QP_TABLE* API. --- libavcodec/h263dec.c | 2 ++ libavcodec/mpeg12dec.c | 1 + libavcodec/mpegpicture.c | 2 ++ libavcodec/mpegpicture.h | 1 + libavcodec/mpegvideo.c | 35 ++++++++++++++++++++++++++++------- libavcodec/mpegvideo.h | 1 + libavcodec/rv10.c | 1 + libavcodec/rv34.c | 1 + libavutil/video_enc_params.h | 8 ++++++++ 9 files changed, 45 insertions(+), 7 deletions(-) diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c index 8ee844e298..e4447d40e7 100644 --- a/libavcodec/h263dec.c +++ b/libavcodec/h263dec.c @@ -28,6 +28,8 @@ #define UNCHECKED_BITSTREAM_READER 1 #include "libavutil/cpu.h" +#include "libavutil/video_enc_params.h" + #include "avcodec.h" #include "error_resilience.h" #include "flv.h" diff --git a/libavcodec/mpeg12dec.c b/libavcodec/mpeg12dec.c index 67ce59d451..52d0eca27d 100644 --- a/libavcodec/mpeg12dec.c +++ b/libavcodec/mpeg12dec.c @@ -32,6 +32,7 @@ #include "libavutil/imgutils.h" #include "libavutil/internal.h" #include "libavutil/stereo3d.h" +#include "libavutil/video_enc_params.h" #include "avcodec.h" #include "bytestream.h" diff --git a/libavcodec/mpegpicture.c b/libavcodec/mpegpicture.c index 5fce25ec6e..86775e5c2c 100644 --- a/libavcodec/mpegpicture.c +++ b/libavcodec/mpegpicture.c @@ -225,6 +225,7 @@ static int alloc_picture_tables(AVCodecContext *avctx, Picture *pic, int encodin pic->alloc_mb_width = mb_width; pic->alloc_mb_height = mb_height; + pic->alloc_mb_stride = mb_stride; return 0; } @@ -359,6 +360,7 @@ do { \ dst->alloc_mb_width = src->alloc_mb_width; dst->alloc_mb_height = src->alloc_mb_height; + dst->alloc_mb_stride = src->alloc_mb_stride; return 0; } diff --git a/libavcodec/mpegpicture.h b/libavcodec/mpegpicture.h index 2db3d6733a..4bcd666797 100644 --- a/libavcodec/mpegpicture.h +++ b/libavcodec/mpegpicture.h @@ -69,6 +69,7 @@ typedef struct Picture { int alloc_mb_width; ///< mb_width used to allocate tables int alloc_mb_height; ///< mb_height used to allocate tables + int alloc_mb_stride; ///< mb_stride used to allocate tables AVBufferRef *mb_mean_buf; uint8_t *mb_mean; ///< Table for MB luminance diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 22cab2854b..189c2ce6b8 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -32,6 +32,8 @@ #include "libavutil/imgutils.h" #include "libavutil/internal.h" #include "libavutil/motion_vector.h" +#include "libavutil/video_enc_params.h" + #include "avcodec.h" #include "blockdsp.h" #include "h264chroma.h" @@ -1441,14 +1443,33 @@ void ff_print_debug_info(MpegEncContext *s, Picture *p, AVFrame *pict) int ff_mpv_export_qp_table(MpegEncContext *s, AVFrame *f, Picture *p, int qp_type) { - AVBufferRef *ref = av_buffer_ref(p->qscale_table_buf); - int offset = 2*s->mb_stride + 1; - if(!ref) + AVVideoEncParams *par; + int mult = (qp_type == FF_QSCALE_TYPE_MPEG1) ? 2 : 1; + unsigned int nb_mb = p->alloc_mb_height * p->alloc_mb_width; + unsigned int x, y; + + if (!(s->avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS)) + return 0; + + par = av_video_enc_params_create_side_data(f, AV_VIDEO_ENC_PARAMS_MPEG2, nb_mb); + if (!par) return AVERROR(ENOMEM); - av_assert0(ref->size >= offset + s->mb_stride * ((f->height+15)/16)); - ref->size -= offset; - ref->data += offset; - return av_frame_set_qp_table(f, ref, s->mb_stride, qp_type); + + for (y = 0; y < p->alloc_mb_height; y++) + for (x = 0; x < p->alloc_mb_width; x++) { + const unsigned int block_idx = y * p->alloc_mb_width + x; + const unsigned int mb_xy = y * p->alloc_mb_stride + x; + AVVideoBlockParams *b = av_video_enc_params_block(par, block_idx); + + b->src_x = x * 16; + b->src_y = y * 16; + b->w = 16; + b->h = 16; + + b->delta_qp = p->qscale_table[mb_xy] * mult; + } + + return 0; } static inline int hpel_motion_lowres(MpegEncContext *s, diff --git a/libavcodec/mpegvideo.h b/libavcodec/mpegvideo.h index 29e692f245..c52410623d 100644 --- a/libavcodec/mpegvideo.h +++ b/libavcodec/mpegvideo.h @@ -58,6 +58,7 @@ #include "libavutil/opt.h" #include "libavutil/timecode.h" +#include "libavutil/video_enc_params.h" #define MAX_THREADS 32 diff --git a/libavcodec/rv10.c b/libavcodec/rv10.c index 3b41d30b92..4dbbe4dd20 100644 --- a/libavcodec/rv10.c +++ b/libavcodec/rv10.c @@ -28,6 +28,7 @@ #include #include "libavutil/imgutils.h" +#include "libavutil/video_enc_params.h" #include "avcodec.h" #include "error_resilience.h" diff --git a/libavcodec/rv34.c b/libavcodec/rv34.c index ec0cd27916..44d9581ac9 100644 --- a/libavcodec/rv34.c +++ b/libavcodec/rv34.c @@ -26,6 +26,7 @@ #include "libavutil/imgutils.h" #include "libavutil/internal.h" +#include "libavutil/video_enc_params.h" #include "avcodec.h" #include "error_resilience.h" diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h index f64864caae..05dd48cec4 100644 --- a/libavutil/video_enc_params.h +++ b/libavutil/video_enc_params.h @@ -27,6 +27,14 @@ enum AVVideoEncParamsType { AV_VIDEO_ENC_PARAMS_NONE = -1, + + /** + * MPEG-2-compatible quantizer. + * + * Summing the frame-level qp with the per-block delta_qp gives the + * resulting quantizer for the block. + */ + AV_VIDEO_ENC_PARAMS_MPEG2, }; /** From patchwork Sat Apr 18 10:14:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19048 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B1EBF44BB72 for ; Sat, 18 Apr 2020 13:15:17 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9FC3168BF5F; Sat, 18 Apr 2020 13:15:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3D8E068BF50 for ; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 2BDAB288A78 for ; Sat, 18 Apr 2020 12:15:06 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id nxSCn4dwhzJK for ; Sat, 18 Apr 2020 12:15:05 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 997F0288ABD for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 3F6EA21842 for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id ZmLN3STDbwoE for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 2841721849 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 3055A20E008E; Sat, 18 Apr 2020 12:14:48 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:11 +0200 Message-Id: <20200418101415.26409-6-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/9] lavfi/vf_qp: convert to the video_enc_params API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavfilter/vf_qp.c | 65 ++++++++++++++++++++++++------------- tests/fate/filter-video.mak | 6 ++-- 2 files changed, 46 insertions(+), 25 deletions(-) diff --git a/libavfilter/vf_qp.c b/libavfilter/vf_qp.c index 33d39493bc..306e8e4594 100644 --- a/libavfilter/vf_qp.c +++ b/libavfilter/vf_qp.c @@ -23,6 +23,8 @@ #include "libavutil/imgutils.h" #include "libavutil/pixdesc.h" #include "libavutil/opt.h" +#include "libavutil/video_enc_params.h" + #include "avfilter.h" #include "formats.h" #include "internal.h" @@ -89,38 +91,59 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFilterContext *ctx = inlink->dst; AVFilterLink *outlink = ctx->outputs[0]; QPContext *s = ctx->priv; - AVBufferRef *out_qp_table_buf; AVFrame *out = NULL; - const int8_t *in_qp_table; - int type, stride, ret; + int ret; + + AVFrameSideData *sd_in; + AVVideoEncParams *par_in = NULL; + int8_t in_qp_global = 0; + + AVVideoEncParams *par_out; if (!s->qp_expr_str || ctx->is_disabled) return ff_filter_frame(outlink, in); - out_qp_table_buf = av_buffer_alloc(s->h * s->qstride); - if (!out_qp_table_buf) { - ret = AVERROR(ENOMEM); - goto fail; + sd_in = av_frame_get_side_data(in, AV_FRAME_DATA_VIDEO_ENC_PARAMS); + if (sd_in && sd_in->size >= sizeof(AVVideoEncParams)) { + par_in = (AVVideoEncParams*)sd_in->data; + + // we accept the input QP table only if it is of the MPEG2 type + // and contains either no blocks at all or 16x16 macroblocks + if (par_in->type == AV_VIDEO_ENC_PARAMS_MPEG2 && + (par_in->nb_blocks == s->h * s->qstride || !par_in->nb_blocks)) { + in_qp_global = par_in->qp; + if (!par_in->nb_blocks) + par_in = NULL; + } else + par_in = NULL; } out = av_frame_clone(in); if (!out) { - av_buffer_unref(&out_qp_table_buf); ret = AVERROR(ENOMEM); goto fail; } - in_qp_table = av_frame_get_qp_table(in, &stride, &type); - av_frame_set_qp_table(out, out_qp_table_buf, s->qstride, type); + par_out = av_video_enc_params_create_side_data(out, AV_VIDEO_ENC_PARAMS_MPEG2, + (s->evaluate_per_mb || sd_in) ? + s->h * s->qstride : 0); + if (!par_out) { + ret = AVERROR(ENOMEM); + goto fail; + } +#define BLOCK_QP_DELTA(block_idx) \ + (par_in ? av_video_enc_params_block(par_in, block_idx)->delta_qp : 0) if (s->evaluate_per_mb) { int y, x; for (y = 0; y < s->h; y++) for (x = 0; x < s->qstride; x++) { - int qp = in_qp_table ? in_qp_table[x + stride * y] : NAN; - double var_values[] = { !!in_qp_table, qp, x, y, s->qstride, s->h, 0}; + unsigned int block_idx = y * s->qstride + x; + AVVideoBlockParams *b = av_video_enc_params_block(par_out, block_idx); + int qp = sd_in ? in_qp_global + BLOCK_QP_DELTA(block_idx) : NAN; + double var_values[] = { !!sd_in, qp, x, y, s->qstride, s->h, 0}; static const char *var_names[] = { "known", "qp", "x", "y", "w", "h", NULL }; double temp_val; @@ -129,21 +152,19 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) NULL, NULL, NULL, NULL, 0, 0, ctx); if (ret < 0) goto fail; - out_qp_table_buf->data[x + s->qstride * y] = lrintf(temp_val); + b->delta_qp = lrintf(temp_val); } - } else if (in_qp_table) { + } else if (sd_in) { int y, x; for (y = 0; y < s->h; y++) - for (x = 0; x < s->qstride; x++) - out_qp_table_buf->data[x + s->qstride * y] = s->lut[129 + - ((int8_t)in_qp_table[x + stride * y])]; + for (x = 0; x < s->qstride; x++) { + unsigned int block_idx = y * s->qstride + x; + AVVideoBlockParams *b = av_video_enc_params_block(par_out, block_idx); + b->delta_qp = s->lut[129 + (int8_t)(in_qp_global + BLOCK_QP_DELTA(block_idx))]; + } } else { - int y, x, qp = s->lut[0]; - - for (y = 0; y < s->h; y++) - for (x = 0; x < s->qstride; x++) - out_qp_table_buf->data[x + s->qstride * y] = qp; + par_out->qp = s->lut[0]; } ret = ff_filter_frame(outlink, out); diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 2da27f714a..3d0d4969b8 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -537,14 +537,14 @@ $(FATE_FILTER_PP): fate-vsynth1-mpeg4-qprd fate-filter-pp: CMD = framecrc -flags bitexact -idct simple -i $(TARGET_PATH)/tests/data/fate/vsynth1-mpeg4-qprd.avi -frames:v 5 -flags +bitexact -vf "pp=be/hb/vb/tn/l5/al" fate-filter-pp1: CMD = video_filter "pp=fq|4/be/hb/vb/tn/l5/al" -fate-filter-pp2: CMD = video_filter "qp=x+y,pp=be/h1/v1/lb" -fate-filter-pp3: CMD = video_filter "qp=x+y,pp=be/ha|128|7/va/li" +fate-filter-pp2: CMD = video_filter "qp=2*(x+y),pp=be/h1/v1/lb" +fate-filter-pp3: CMD = video_filter "qp=2*(x+y),pp=be/ha|128|7/va/li" fate-filter-pp4: CMD = video_filter "pp=be/ci" fate-filter-pp5: CMD = video_filter "pp=md" fate-filter-pp6: CMD = video_filter "pp=be/fd" FATE_FILTER_VSYNTH-$(call ALLYES, QP_FILTER PP_FILTER) += fate-filter-qp -fate-filter-qp: CMD = video_filter "qp=17,pp=be/hb/vb/tn/l5/al" +fate-filter-qp: CMD = video_filter "qp=34,pp=be/hb/vb/tn/l5/al" FATE_FILTER_VSYNTH-$(CONFIG_SELECT_FILTER) += fate-filter-select fate-filter-select: CMD = framecrc -flags bitexact -idct simple -i $(SRC) -vf "select=not(eq(mod(n\,2)\,0)+eq(mod(n\,3)\,0))" -frames:v 25 -flags +bitexact From patchwork Sat Apr 18 10:14:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19047 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E1FB544BB72 for ; Sat, 18 Apr 2020 13:15:16 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CE35068BF62; Sat, 18 Apr 2020 13:15:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3D73468BF4C for ; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 602E7288AA2 for ; Sat, 18 Apr 2020 12:15:05 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id TSe6oqUIG8CH for ; Sat, 18 Apr 2020 12:15:04 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 476B7288AA3 for ; Sat, 18 Apr 2020 12:15:02 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id C3D7421842 for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id WSAJP-CnBjvB for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 2508F21847 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 79DED20E00CE; Sat, 18 Apr 2020 12:14:49 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:12 +0200 Message-Id: <20200418101415.26409-7-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/9] lavfi/vf_pp: convert to the video_enc_params API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavfilter/vf_pp.c | 54 ++++++++++++++++++++++++++++++++++--- tests/fate/filter-video.mak | 2 +- 2 files changed, 51 insertions(+), 5 deletions(-) diff --git a/libavfilter/vf_pp.c b/libavfilter/vf_pp.c index 524ef1bb0a..87eef4561f 100644 --- a/libavfilter/vf_pp.c +++ b/libavfilter/vf_pp.c @@ -26,6 +26,8 @@ #include "libavutil/avassert.h" #include "libavutil/opt.h" +#include "libavutil/video_enc_params.h" + #include "internal.h" #include "libpostproc/postprocess.h" @@ -118,6 +120,41 @@ static int pp_config_props(AVFilterLink *inlink) return 0; } +static int get_qp_table(AVFrame *in, int8_t **table, int *stride) +{ + AVFrameSideData *sd; + AVVideoEncParams *par; + unsigned int mb_h = (in->height + 15) / 16; + unsigned int mb_w = (in->width + 15) / 16; + unsigned int nb_mb = mb_h * mb_w; + unsigned int block_idx; + + sd = av_frame_get_side_data(in, AV_FRAME_DATA_VIDEO_ENC_PARAMS); + if (!sd) + return 0; + par = (AVVideoEncParams*)sd->data; + if (par->type != AV_VIDEO_ENC_PARAMS_MPEG2 || + (par->nb_blocks != 0 && par->nb_blocks != nb_mb)) + return 0; + + *table = av_malloc(nb_mb); + if (!*table) + return AVERROR(ENOMEM); + *stride = mb_w; + + if (par->nb_blocks == 0) { + memset(*table, par->qp, nb_mb); + return 0; + } + + for (block_idx = 0; block_idx < nb_mb; block_idx++) { + AVVideoBlockParams *b = av_video_enc_params_block(par, block_idx); + (*table)[block_idx] = par->qp + b->delta_qp; + } + + return 0; +} + static int pp_filter_frame(AVFilterLink *inlink, AVFrame *inbuf) { AVFilterContext *ctx = inlink->dst; @@ -126,8 +163,9 @@ static int pp_filter_frame(AVFilterLink *inlink, AVFrame *inbuf) const int aligned_w = FFALIGN(outlink->w, 8); const int aligned_h = FFALIGN(outlink->h, 8); AVFrame *outbuf; - int qstride, qp_type; - int8_t *qp_table ; + int qstride = 0; + int8_t *qp_table = NULL; + int ret; outbuf = ff_get_video_buffer(outlink, aligned_w, aligned_h); if (!outbuf) { @@ -137,7 +175,14 @@ static int pp_filter_frame(AVFilterLink *inlink, AVFrame *inbuf) av_frame_copy_props(outbuf, inbuf); outbuf->width = inbuf->width; outbuf->height = inbuf->height; - qp_table = av_frame_get_qp_table(inbuf, &qstride, &qp_type); + + ret = get_qp_table(inbuf, &qp_table, &qstride); + if (ret < 0) { + av_frame_free(&inbuf); + av_frame_free(&outbuf); + av_freep(&qp_table); + return ret; + } pp_postprocess((const uint8_t **)inbuf->data, inbuf->linesize, outbuf->data, outbuf->linesize, @@ -146,9 +191,10 @@ static int pp_filter_frame(AVFilterLink *inlink, AVFrame *inbuf) qstride, pp->modes[pp->mode_id], pp->pp_ctx, - outbuf->pict_type | (qp_type ? PP_PICT_TYPE_QP2 : 0)); + outbuf->pict_type | (qp_table ? PP_PICT_TYPE_QP2 : 0)); av_frame_free(&inbuf); + av_freep(&qp_table); return ff_filter_frame(outlink, outbuf); } diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak index 3d0d4969b8..cb7ce7a158 100644 --- a/tests/fate/filter-video.mak +++ b/tests/fate/filter-video.mak @@ -535,7 +535,7 @@ FATE_FILTER_PP = fate-filter-pp fate-filter-pp1 fate-filter-pp2 fate-filter-pp3 FATE_FILTER_VSYNTH-$(CONFIG_PP_FILTER) += $(FATE_FILTER_PP) $(FATE_FILTER_PP): fate-vsynth1-mpeg4-qprd -fate-filter-pp: CMD = framecrc -flags bitexact -idct simple -i $(TARGET_PATH)/tests/data/fate/vsynth1-mpeg4-qprd.avi -frames:v 5 -flags +bitexact -vf "pp=be/hb/vb/tn/l5/al" +fate-filter-pp: CMD = framecrc -flags bitexact -export_side_data venc_params -idct simple -i $(TARGET_PATH)/tests/data/fate/vsynth1-mpeg4-qprd.avi -frames:v 5 -flags +bitexact -vf "pp=be/hb/vb/tn/l5/al" fate-filter-pp1: CMD = video_filter "pp=fq|4/be/hb/vb/tn/l5/al" fate-filter-pp2: CMD = video_filter "qp=2*(x+y),pp=be/h1/v1/lb" fate-filter-pp3: CMD = video_filter "qp=2*(x+y),pp=be/ha|128|7/va/li" From patchwork Sat Apr 18 10:14:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19046 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4540F44BB72 for ; Sat, 18 Apr 2020 13:15:16 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 313C468BF5A; Sat, 18 Apr 2020 13:15:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3DF3068BF51 for ; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id C1563288AA3 for ; Sat, 18 Apr 2020 12:15:05 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 0PT8VOGKKXNz for ; Sat, 18 Apr 2020 12:15:05 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 7BAA3288A78 for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id EF18E213CC for ; Sat, 18 Apr 2020 12:15:02 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id C8Qtqoi_UI19 for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 2722721848 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 4CFD220E0099; Sat, 18 Apr 2020 12:14:49 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:13 +0200 Message-Id: <20200418101415.26409-8-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 7/9] h264dec: support exporting QP tables through the AVVideoEncParams API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_picture.c | 8 ++++++- libavcodec/h264_slice.c | 9 +++++++ libavcodec/h264dec.c | 46 ++++++++++++++++++++++++++++++++++++ libavcodec/h264dec.h | 6 +++++ libavutil/video_enc_params.h | 1 + 5 files changed, 69 insertions(+), 1 deletion(-) diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c index 2113947d1d..eec5e9fb9a 100644 --- a/libavcodec/h264_picture.c +++ b/libavcodec/h264_picture.c @@ -54,6 +54,7 @@ void ff_h264_unref_picture(H264Context *h, H264Picture *pic) av_buffer_unref(&pic->qscale_table_buf); av_buffer_unref(&pic->mb_type_buf); + av_buffer_unref(&pic->pps_buf); for (i = 0; i < 2; i++) { av_buffer_unref(&pic->motion_val_buf[i]); av_buffer_unref(&pic->ref_index_buf[i]); @@ -77,12 +78,14 @@ int ff_h264_ref_picture(H264Context *h, H264Picture *dst, H264Picture *src) dst->qscale_table_buf = av_buffer_ref(src->qscale_table_buf); dst->mb_type_buf = av_buffer_ref(src->mb_type_buf); - if (!dst->qscale_table_buf || !dst->mb_type_buf) { + dst->pps_buf = av_buffer_ref(src->pps_buf); + if (!dst->qscale_table_buf || !dst->mb_type_buf || !dst->pps_buf) { ret = AVERROR(ENOMEM); goto fail; } dst->qscale_table = src->qscale_table; dst->mb_type = src->mb_type; + dst->pps = src->pps; for (i = 0; i < 2; i++) { dst->motion_val_buf[i] = av_buffer_ref(src->motion_val_buf[i]); @@ -120,6 +123,9 @@ int ff_h264_ref_picture(H264Context *h, H264Picture *dst, H264Picture *src) dst->recovered = src->recovered; dst->invalid_gap = src->invalid_gap; dst->sei_recovery_frame_cnt = src->sei_recovery_frame_cnt; + dst->mb_width = src->mb_width; + dst->mb_height = src->mb_height; + dst->mb_stride = src->mb_stride; return 0; fail: diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 5a8a4a7f86..713953778a 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -243,6 +243,15 @@ static int alloc_picture(H264Context *h, H264Picture *pic) pic->ref_index[i] = pic->ref_index_buf[i]->data; } + pic->pps_buf = av_buffer_ref(h->ps.pps_ref); + if (!pic->pps_buf) + goto fail; + pic->pps = (const PPS*)pic->pps_buf->data; + + pic->mb_width = h->mb_width; + pic->mb_height = h->mb_height; + pic->mb_stride = h->mb_stride; + return 0; fail: ff_h264_unref_picture(h, pic); diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index e374f32e91..a2bd12d93f 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -32,6 +32,8 @@ #include "libavutil/imgutils.h" #include "libavutil/opt.h" #include "libavutil/stereo3d.h" +#include "libavutil/video_enc_params.h" + #include "internal.h" #include "bytestream.h" #include "cabac.h" @@ -812,6 +814,41 @@ static int get_consumed_bytes(int pos, int buf_size) return pos; } +static int h264_export_enc_params(AVFrame *f, H264Picture *p) +{ + AVVideoEncParams *par; + unsigned int nb_mb = p->mb_height * p->mb_width; + unsigned int x, y; + + par = av_video_enc_params_create_side_data(f, AV_VIDEO_ENC_PARAMS_H264, nb_mb); + if (!par) + return AVERROR(ENOMEM); + + par->qp = p->pps->init_qp; + + par->delta_qp[1][0] = p->pps->chroma_qp_index_offset[0]; + par->delta_qp[1][1] = p->pps->chroma_qp_index_offset[0]; + par->delta_qp[2][0] = p->pps->chroma_qp_index_offset[1]; + par->delta_qp[2][1] = p->pps->chroma_qp_index_offset[1]; + + for (y = 0; y < p->mb_height; y++) + for (x = 0; x < p->mb_width; x++) { + const unsigned int block_idx = y * p->mb_width + x; + const unsigned int mb_xy = y * p->mb_stride + x; + AVVideoBlockParams *b = av_video_enc_params_block(par, block_idx); + + b->src_x = x * 16; + b->src_y = y * 16; + b->w = 16; + b->h = 16; + + b->delta_qp = p->qscale_table[mb_xy] - par->qp; + } + + + return 0; +} + static int output_frame(H264Context *h, AVFrame *dst, H264Picture *srcp) { AVFrame *src = srcp->f; @@ -826,7 +863,16 @@ static int output_frame(H264Context *h, AVFrame *dst, H264Picture *srcp) if (srcp->sei_recovery_frame_cnt == 0) dst->key_frame = 1; + if (h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + ret = h264_export_enc_params(dst, srcp); + if (ret < 0) + goto fail; + } + return 0; +fail: + av_frame_unref(dst); + return ret; } static int is_extra(const uint8_t *buf, int buf_size) diff --git a/libavcodec/h264dec.h b/libavcodec/h264dec.h index 530e2d4071..29c4d4e42c 100644 --- a/libavcodec/h264dec.h +++ b/libavcodec/h264dec.h @@ -161,6 +161,12 @@ typedef struct H264Picture { int recovered; ///< picture at IDR or recovery point + recovery count int invalid_gap; int sei_recovery_frame_cnt; + + AVBufferRef *pps_buf; + const PPS *pps; + + int mb_width, mb_height; + int mb_stride; } H264Picture; typedef struct H264Ref { diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h index 05dd48cec4..057d654504 100644 --- a/libavutil/video_enc_params.h +++ b/libavutil/video_enc_params.h @@ -35,6 +35,7 @@ enum AVVideoEncParamsType { * resulting quantizer for the block. */ AV_VIDEO_ENC_PARAMS_MPEG2, + AV_VIDEO_ENC_PARAMS_H264, }; /** From patchwork Sat Apr 18 10:14:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19045 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8839644BB72 for ; Sat, 18 Apr 2020 13:15:14 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 67ADF68BF51; Sat, 18 Apr 2020 13:15:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BFC4668BF3C for ; Sat, 18 Apr 2020 13:15:04 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 604EA288A92 for ; Sat, 18 Apr 2020 12:15:04 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id pz3EPhM_rCeI for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id E7300288AA2 for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 87C5F213CC for ; Sat, 18 Apr 2020 12:15:01 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Rir2y7oykZIe for ; Sat, 18 Apr 2020 12:15:00 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 2504821846 for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 6ED1D20E00AB; Sat, 18 Apr 2020 12:14:49 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:14 +0200 Message-Id: <20200418101415.26409-9-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 8/9] vp9dec: factorise freeing per-tile allocated data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vp9.c | 36 ++++++++++++++++-------------------- 1 file changed, 16 insertions(+), 20 deletions(-) diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 2a3a4555b9..6b3807a811 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -93,6 +93,12 @@ static void vp9_free_entries(AVCodecContext *avctx) {} static int vp9_alloc_entries(AVCodecContext *avctx, int n) { return 0; } #endif +static void vp9_tile_data_free(VP9TileData *td) +{ + av_freep(&td->b_base); + av_freep(&td->block_base); +} + static void vp9_frame_unref(AVCodecContext *avctx, VP9Frame *f) { ff_thread_release_buffer(avctx, &f->tf); @@ -281,10 +287,8 @@ static int update_size(AVCodecContext *avctx, int w, int h) #undef assign if (s->td) { - for (i = 0; i < s->active_tile_cols; i++) { - av_freep(&s->td[i].b_base); - av_freep(&s->td[i].block_base); - } + for (i = 0; i < s->active_tile_cols; i++) + vp9_tile_data_free(&s->td[i]); } if (s->s.h.bpp != s->last_bpp) { @@ -306,8 +310,7 @@ static int update_block_buffers(AVCodecContext *avctx) if (td->b_base && td->block_base && s->block_alloc_using_2pass == s->s.frames[CUR_FRAME].uses_2pass) return 0; - av_free(td->b_base); - av_free(td->block_base); + vp9_tile_data_free(td); chroma_blocks = 64 * 64 >> (s->ss_h + s->ss_v); chroma_eobs = 16 * 16 >> (s->ss_h + s->ss_v); if (s->s.frames[CUR_FRAME].uses_2pass) { @@ -324,12 +327,9 @@ static int update_block_buffers(AVCodecContext *avctx) td->uveob_base[0] = td->eob_base + 16 * 16 * sbs; td->uveob_base[1] = td->uveob_base[0] + chroma_eobs * sbs; } else { - for (i = 1; i < s->active_tile_cols; i++) { - if (s->td[i].b_base && s->td[i].block_base) { - av_free(s->td[i].b_base); - av_free(s->td[i].block_base); - } - } + for (i = 1; i < s->active_tile_cols; i++) + vp9_tile_data_free(&s->td[i]); + for (i = 0; i < s->active_tile_cols; i++) { s->td[i].b_base = av_malloc(sizeof(VP9Block)); s->td[i].block_base = av_mallocz((64 * 64 + 2 * chroma_blocks) * bytesperpixel * sizeof(int16_t) + @@ -773,10 +773,8 @@ static int decode_frame_header(AVCodecContext *avctx, VP56RangeCoder *rc; if (s->td) { - for (i = 0; i < s->active_tile_cols; i++) { - av_free(s->td[i].b_base); - av_free(s->td[i].block_base); - } + for (i = 0; i < s->active_tile_cols; i++) + vp9_tile_data_free(&s->td[i]); av_free(s->td); } @@ -1204,10 +1202,8 @@ static void free_buffers(VP9Context *s) int i; av_freep(&s->intra_pred_data[0]); - for (i = 0; i < s->active_tile_cols; i++) { - av_freep(&s->td[i].b_base); - av_freep(&s->td[i].block_base); - } + for (i = 0; i < s->active_tile_cols; i++) + vp9_tile_data_free(&s->td[i]); } static av_cold int vp9_decode_free(AVCodecContext *avctx) From patchwork Sat Apr 18 10:14:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 19049 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 79A9044BB72 for ; Sat, 18 Apr 2020 13:15:18 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6165868BF61; Sat, 18 Apr 2020 13:15:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4344768BF52 for ; Sat, 18 Apr 2020 13:15:09 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 87EE6288AC9 for ; Sat, 18 Apr 2020 12:15:06 +0200 (CEST) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id un4itvfrl2fS for ; Sat, 18 Apr 2020 12:15:06 +0200 (CEST) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 1D7B0288ACA for ; Sat, 18 Apr 2020 12:15:05 +0200 (CEST) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 8684B213CC for ; Sat, 18 Apr 2020 12:15:04 +0200 (CEST) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id MHIZUdFCfhe6 for ; Sat, 18 Apr 2020 12:15:03 +0200 (CEST) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id 291012184A for ; Sat, 18 Apr 2020 12:14:55 +0200 (CEST) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 87AD520E009B; Sat, 18 Apr 2020 12:14:49 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Apr 2020 12:14:15 +0200 Message-Id: <20200418101415.26409-10-anton@khirnov.net> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200418101415.26409-1-anton@khirnov.net> References: <20200418101415.26409-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 9/9] vp9dec: support exporting QP tables through the AVVideoEncParams API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vp9.c | 73 ++++++++++++++++++++++++++++++++++++ libavcodec/vp9block.c | 8 ++++ libavcodec/vp9dec.h | 8 ++++ libavutil/video_enc_params.h | 15 ++++++++ 4 files changed, 104 insertions(+) diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index 6b3807a811..855dad2545 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -34,6 +34,7 @@ #include "vp9dec.h" #include "libavutil/avassert.h" #include "libavutil/pixdesc.h" +#include "libavutil/video_enc_params.h" #define VP9_SYNCCODE 0x498342 @@ -97,6 +98,7 @@ static void vp9_tile_data_free(VP9TileData *td) { av_freep(&td->b_base); av_freep(&td->block_base); + av_freep(&td->block_structure); } static void vp9_frame_unref(AVCodecContext *avctx, VP9Frame *f) @@ -326,6 +328,12 @@ static int update_block_buffers(AVCodecContext *avctx) td->eob_base = (uint8_t *) (td->uvblock_base[1] + sbs * chroma_blocks * bytesperpixel); td->uveob_base[0] = td->eob_base + 16 * 16 * sbs; td->uveob_base[1] = td->uveob_base[0] + chroma_eobs * sbs; + + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + td->block_structure = av_malloc_array(s->cols * s->rows, sizeof(*td->block_structure)); + if (!td->block_structure) + return AVERROR(ENOMEM); + } } else { for (i = 1; i < s->active_tile_cols; i++) vp9_tile_data_free(&s->td[i]); @@ -341,6 +349,12 @@ static int update_block_buffers(AVCodecContext *avctx) s->td[i].eob_base = (uint8_t *) (s->td[i].uvblock_base[1] + chroma_blocks * bytesperpixel); s->td[i].uveob_base[0] = s->td[i].eob_base + 16 * 16; s->td[i].uveob_base[1] = s->td[i].uveob_base[0] + chroma_eobs; + + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + s->td[i].block_structure = av_malloc_array(s->cols * s->rows, sizeof(*td->block_structure)); + if (!s->td[i].block_structure) + return AVERROR(ENOMEM); + } } } s->block_alloc_using_2pass = s->s.frames[CUR_FRAME].uses_2pass; @@ -1475,6 +1489,58 @@ int loopfilter_proc(AVCodecContext *avctx) } #endif +static int vp9_export_enc_params(VP9Context *s, VP9Frame *frame) +{ + AVVideoEncParams *par; + unsigned int tile, nb_blocks = 0; + + if (s->s.h.segmentation.enabled) { + for (tile = 0; tile < s->active_tile_cols; tile++) + nb_blocks += s->td[tile].nb_block_structure; + } + + par = av_video_enc_params_create_side_data(frame->tf.f, + AV_VIDEO_ENC_PARAMS_VP9, nb_blocks); + if (!par) + return AVERROR(ENOMEM); + + par->qp = s->s.h.yac_qi; + par->delta_qp[0][0] = s->s.h.ydc_qdelta; + par->delta_qp[1][0] = s->s.h.uvdc_qdelta; + par->delta_qp[2][0] = s->s.h.uvdc_qdelta; + par->delta_qp[1][1] = s->s.h.uvac_qdelta; + par->delta_qp[2][1] = s->s.h.uvac_qdelta; + + if (nb_blocks) { + unsigned int block = 0; + unsigned int tile, block_tile; + + for (tile = 0; tile < s->active_tile_cols; tile++) { + VP9TileData *td = &s->td[tile]; + + for (block_tile = 0; block_tile < td->nb_block_structure; block_tile++) { + AVVideoBlockParams *b = av_video_enc_params_block(par, block++); + unsigned int row = td->block_structure[block_tile].row; + unsigned int col = td->block_structure[block_tile].col; + uint8_t seg_id = frame->segmentation_map[row * 8 * s->sb_cols + col]; + + b->src_x = col * 8; + b->src_y = row * 8; + b->w = 1 << (3 + td->block_structure[block_tile].block_size_idx_x); + b->h = 1 << (3 + td->block_structure[block_tile].block_size_idx_y); + + if (s->s.h.segmentation.feat[seg_id].q_enabled) { + b->delta_qp = s->s.h.segmentation.feat[seg_id].q_val; + if (s->s.h.segmentation.absolute_vals) + b->delta_qp -= par->qp; + } + } + } + } + + return 0; +} + static int vp9_decode_frame(AVCodecContext *avctx, void *frame, int *got_frame, AVPacket *pkt) { @@ -1621,6 +1687,7 @@ FF_ENABLE_DEPRECATION_WARNINGS s->td[i].eob = s->td[i].eob_base; s->td[i].uveob[0] = s->td[i].uveob_base[0]; s->td[i].uveob[1] = s->td[i].uveob_base[1]; + s->td[i].nb_block_structure = 0; } #if HAVE_THREADS @@ -1677,6 +1744,12 @@ FF_ENABLE_DEPRECATION_WARNINGS } while (s->pass++ == 1); ff_thread_report_progress(&s->s.frames[CUR_FRAME].tf, INT_MAX, 0); + if (avctx->export_side_data & AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS) { + ret = vp9_export_enc_params(s, &s->s.frames[CUR_FRAME]); + if (ret < 0) + return ret; + } + finish: // ref frame setup for (i = 0; i < 8; i++) { diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c index 1c3f7a7225..ec16e26c69 100644 --- a/libavcodec/vp9block.c +++ b/libavcodec/vp9block.c @@ -1290,6 +1290,14 @@ void ff_vp9_decode_block(VP9TileData *td, int row, int col, b->uvtx = b->tx - ((s->ss_h && w4 * 2 == (1 << b->tx)) || (s->ss_v && h4 * 2 == (1 << b->tx))); + if (td->block_structure) { + td->block_structure[td->nb_block_structure].row = row; + td->block_structure[td->nb_block_structure].col = col; + td->block_structure[td->nb_block_structure].block_size_idx_x = av_log2(w4); + td->block_structure[td->nb_block_structure].block_size_idx_y = av_log2(h4); + td->nb_block_structure++; + } + if (!b->skip) { int has_coeffs; diff --git a/libavcodec/vp9dec.h b/libavcodec/vp9dec.h index de02b146f0..61e4140622 100644 --- a/libavcodec/vp9dec.h +++ b/libavcodec/vp9dec.h @@ -221,6 +221,14 @@ struct VP9TileData { struct { int x, y; } min_mv, max_mv; int16_t *block_base, *block, *uvblock_base[2], *uvblock[2]; uint8_t *eob_base, *uveob_base[2], *eob, *uveob[2]; + + struct { + unsigned int row:13; + unsigned int col:13; + unsigned int block_size_idx_x:2; + unsigned int block_size_idx_y:2; + } *block_structure; + unsigned int nb_block_structure; }; void ff_vp9_fill_mv(VP9TileData *td, VP56mv *mv, int mode, int sb); diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h index 057d654504..29dbc6d843 100644 --- a/libavutil/video_enc_params.h +++ b/libavutil/video_enc_params.h @@ -36,6 +36,21 @@ enum AVVideoEncParamsType { */ AV_VIDEO_ENC_PARAMS_MPEG2, AV_VIDEO_ENC_PARAMS_H264, + /** + * VP9 stores: + * - per-frame base (luma AC) quantizer index, exported as AVVideoEncParams.qp + * - deltas for luma DC, chroma AC and chroma DC, exported in the + * corresponding entries in AVVideoEncParams.delta_qp + * - per-segment delta, exported as for each block as AVVideoBlockParams.delta_qp + * + * To compute the resulting quantizer index for a block: + * - for luma AC, add the base qp and the per-block delta_qp, saturating to + * unsigned 8-bit. + * - for luma DC and chroma AC/DC, add the corresponding + * AVVideoBlockParams.delta_qp to the luma AC index, again saturating to + * unsigned 8-bit. + */ + AV_VIDEO_ENC_PARAMS_VP9, }; /**