From patchwork Tue Mar 10 17:45:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 18110 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B149044974D for ; Tue, 10 Mar 2020 19:48:51 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9248768975D; Tue, 10 Mar 2020 19:48:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.red.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CDCD687F36 for ; Tue, 10 Mar 2020 19:48:45 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail.red.khirnov.net (Postfix) with ESMTP id 3BA892956EA for ; Tue, 10 Mar 2020 18:48:44 +0100 (CET) Received: from mail.red.khirnov.net ([IPv6:::1]) by localhost (mail.red.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Bz9DdmzzkbLS for ; Tue, 10 Mar 2020 18:48:43 +0100 (CET) Received: from quelana.khirnov.net (unknown [IPv6:2a00:c500:61:23b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "quelana.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail.red.khirnov.net (Postfix) with ESMTPS id 90A862956D2 for ; Tue, 10 Mar 2020 18:48:43 +0100 (CET) Received: from localhost (quelana.khirnov.net [IPv6:::1]) by quelana.khirnov.net (Postfix) with ESMTP id 4B8EA2532C for ; Tue, 10 Mar 2020 18:48:43 +0100 (CET) Received: from quelana.khirnov.net ([IPv6:::1]) by localhost (quelana.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id TU23dIaVaSWc for ; Tue, 10 Mar 2020 18:48:41 +0100 (CET) Received: from libav.daenerys.khirnov.net (libav.daenerys.khirnov.net [IPv6:2a00:c500:561:201::7]) by quelana.khirnov.net (Postfix) with ESMTP id F3F67251FA for ; Tue, 10 Mar 2020 18:48:40 +0100 (CET) Received: by libav.daenerys.khirnov.net (Postfix, from userid 1000) id 15E5520E004A; Tue, 10 Mar 2020 18:48:36 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Tue, 10 Mar 2020 18:45:59 +0100 Message-Id: <20200310174602.10217-4-anton@khirnov.net> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310174602.10217-1-anton@khirnov.net> References: <20200310174602.10217-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/6] libavutil: add API for exporting video frame quantizers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Juan De León This is intended to replace the deprecated the AV_FRAME_DATA_QP_TABLE* API and extend it to a wider range of codecs. In the future, it may also be extended to support other encoding parameters such as motion vectors. Signed-off-by: Juan De León Signed-off-by: Michael Niedermayer Signed-off-by: Anton Khirnov --- libavutil/Makefile | 4 +- libavutil/frame.c | 1 + libavutil/frame.h | 5 ++ libavutil/video_enc_params.c | 81 ++++++++++++++++++++++++ libavutil/video_enc_params.h | 118 +++++++++++++++++++++++++++++++++++ 5 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 libavutil/video_enc_params.c create mode 100644 libavutil/video_enc_params.h diff --git a/libavutil/Makefile b/libavutil/Makefile index a2dae8e89a..2592f952af 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -78,6 +78,7 @@ HEADERS = adler32.h \ tree.h \ twofish.h \ version.h \ + video_enc_params.h \ xtea.h \ tea.h \ tx.h \ @@ -164,7 +165,8 @@ OBJS = adler32.o \ tx.o \ tx_float.o \ tx_double.o \ - tx_int32.o + tx_int32.o \ + video_enc_params.o OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o OBJS-$(CONFIG_D3D11VA) += hwcontext_d3d11va.o diff --git a/libavutil/frame.c b/libavutil/frame.c index e4038096c2..2f9ba5b6c0 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -842,6 +842,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) #endif case AV_FRAME_DATA_DYNAMIC_HDR_PLUS: return "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)"; case AV_FRAME_DATA_REGIONS_OF_INTEREST: return "Regions Of Interest"; + case AV_FRAME_DATA_VIDEO_ENC_PARAMS: return "Video encoding parameters"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index b5afb58634..fc67db0f6c 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -179,6 +179,11 @@ enum AVFrameSideDataType { * array element is implied by AVFrameSideData.size / AVRegionOfInterest.self_size. */ AV_FRAME_DATA_REGIONS_OF_INTEREST, + + /** + * Encoding parameters for a video frame, as described by AVVideoEncParams. + */ + AV_FRAME_DATA_VIDEO_ENC_PARAMS, }; enum AVActiveFormatDescription { diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c new file mode 100644 index 0000000000..4ae088939a --- /dev/null +++ b/libavutil/video_enc_params.c @@ -0,0 +1,81 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include + +#include "buffer.h" +#include "common.h" +#include "frame.h" +#include "mem.h" +#include "video_enc_params.h" + +AVVideoEncParams *av_video_enc_params_alloc(unsigned int nb_blocks, size_t *out_size) +{ + AVVideoEncParams *par; + size_t size; + int i; + + size = sizeof(*par); + if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) || + nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size) + return NULL; + size += sizeof(AVVideoBlockParams) * nb_blocks; + + par = av_mallocz(size); + if (!par) + return NULL; + + par->nb_blocks = nb_blocks; + par->block_size = sizeof(AVVideoBlockParams); + par->blocks_offset = sizeof(*par); + + par->type = AV_VIDEO_ENC_PARAMS_NONE; + + for (i = 0; i < FF_ARRAY_ELEMS(par->qp); i++) + par->qp[i] = 0; + + if (out_size) + *out_size = size; + + return par; +} + +AVVideoEncParams *av_video_enc_params_create_side_data(AVFrame *frame, unsigned nb_blocks) +{ + AVBufferRef *buf; + AVVideoEncParams *par; + size_t size; + + par = av_video_enc_params_alloc(nb_blocks, &size); + if (!par) + return NULL; + buf = av_buffer_create((uint8_t *)par, size, NULL, NULL, 0); + if (!buf) { + av_freep(&par); + return NULL; + } + + if (!av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_VIDEO_ENC_PARAMS, buf)) { + av_buffer_unref(&buf); + return NULL; + } + + return par; +} diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h new file mode 100644 index 0000000000..a7cc4210f3 --- /dev/null +++ b/libavutil/video_enc_params.h @@ -0,0 +1,118 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_VIDEO_ENC_PARAMS_H +#define AVUTIL_VIDEO_ENC_PARAMS_H + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/frame.h" + +enum AVVideoEncParamsType { + AV_VIDEO_ENC_PARAMS_NONE = -1, +}; + +/** + * Data structure for extracting block data, stored as an array in AVEncodeInfoFrame. + */ +typedef struct AVVideoBlockParams { + /** + * Distance in luma pixels from the top-left corner of the visible frame + * to the top-left corner of the block. + * Can be negative if top/right padding is present on the coded frame. + */ + int src_x, src_y; + /** + * Width and height of the block in luma pixels. + */ + int w, h; + + /** + * Difference between this block's final quantization parameter (for each + * plane) and the per-frame value. + */ + int delta_qp[4]; +} AVVideoBlockParams; + +/** + * Video encoding parameters for a given frame. This struct is allocated along + * with an array of per-block AVVideoBlockParams descriptors. + * Must be allocated with av_video_enc_params_alloc(). + */ +typedef struct AVVideoEncParams { + /** + * Number of blocks in the array. + * + * May be 0, in which case no per-block information is present. In this case + * the values of blocks_offset / block_size are unspecified and should not + * be accessed. + */ + unsigned int nb_blocks; + /** + * Offset in bytes from the beginning of this structure at which the array + * of blocks starts. + */ + size_t blocks_offset; + /* + * Size of each block in bytes. May not match sizeof(AVEncodeInfoBlock). + */ + size_t block_size; + + /** + * Type of the parameters (the codec they are used with). + */ + enum AVVideoEncParamsType type; + + /** + * Base quantisation parametere for each plane. The final quantiser is a sum + * of this value with the per-block value (if it is present). + */ + int qp[4]; +} AVVideoEncParams; + +/* + * Get the block at the specified {@code idx}. Must be between 0 and nb_blocks. + */ +static av_always_inline AVVideoBlockParams* +av_video_enc_params_block(AVVideoEncParams *par, unsigned int idx) +{ + av_assert0(idx < par->nb_blocks); + return (AVVideoBlockParams *)((uint8_t *)par + par->blocks_offset + + idx * par->block_size); +} + +/** + * Allocates memory for AVVideoEncParams plus an array of + * {@code nb_blocks} AVVideoBlockParams and initializes the variables. + * Can be freed with a normal av_free() call. + * + * @param out_size if non-NULL, the size of the resulting data array is written here. + */ +AVVideoEncParams *av_video_enc_params_alloc(unsigned int nb_blocks, size_t *out_size); + +/** + * Allocates memory for AVEncodeInfoFrame plus an array of + * {@code nb_blocks} AVEncodeInfoBlock in the given AVFrame {@code frame} + * as AVFrameSideData of type AV_FRAME_DATA_ENCODE_INFO + * and initializes the variables. + */ +AVVideoEncParams *av_video_enc_params_create_side_data(AVFrame *frame, unsigned int nb_blocks); + +#endif /* AVUTIL_VIDEO_ENC_PARAMS_H */