From patchwork Thu Nov 12 17:42:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 23601 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2AB98444747 for ; Thu, 12 Nov 2020 19:42:25 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EFCC968C1A4; Thu, 12 Nov 2020 19:42:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from w4.tutanota.de (w4.tutanota.de [81.3.6.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C9AEF68BA51 for ; Thu, 12 Nov 2020 19:42:18 +0200 (EET) Received: from w3.tutanota.de (unknown [192.168.1.164]) by w4.tutanota.de (Postfix) with ESMTP id 67D671060154 for ; Thu, 12 Nov 2020 17:42:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1605202938; s=s1; d=lynne.ee; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender; bh=wChshrhlyv0XQpVOhddeBczr7FHdhQfvTWHVKxe3lBg=; b=ut6HEYT/iKrMRgvgD/lMWJ267k4U00HgD1hKS5JFYtA9J9vCDRuSDeIfuugYqIpe O+KkKzfmJz09AZ7P1KqcwCTD8bPmWZ6jFpIz6fd7DW1eU21aqFtQZCr2wrm1QuAe500 9xTE43YCSS07eLnKL/u8OJfzC6Dqi+gaZkUwkU1AwkxWy3zMzagyeMlvxPynDvMZhAO jRKLfEX2ewycArQugT4ZZWh5B5c5fLYywIyTVxm7q/KGFq/xg264w3qNXgK0iOzeRrl M2GKCevDv+96ARlEwvXhaQzdvsZLIv26VmtDW6IzSEzk5C/QC42OERZj/2DPuVUvKWh TZfsTo7CYQ== Date: Thu, 12 Nov 2020 18:42:18 +0100 (CET) From: Lynne To: Ffmpeg Devel Message-ID: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/3] libavutil: introduce AVFilmGrainParams side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch introduces a new frame side data type AVFilmGrainParams for use with video codecs which are able to use it. It is generalized rather than being AV1 specific as AV2 is expected to carry the same data, as well as the fact there already exist rarely-used specifications for both H.264 and HEVC. Patch attached. Subject: [PATCH v2 1/3] libavutil: introduce AVFilmGrainParams side data This patch introduces a new frame side data type AVFilmGrainParams for use with video codecs which are able to use it. It is generalized rather than being AV1 specific as AV2 is expected to carry the same data, as well as the fact there already exist rarely-used specifications for both H.264 and HEVC. --- doc/APIchanges | 4 + libavutil/Makefile | 2 + libavutil/film_grain_params.c | 42 +++++++++ libavutil/film_grain_params.h | 159 ++++++++++++++++++++++++++++++++++ libavutil/frame.c | 1 + libavutil/frame.h | 6 ++ libavutil/version.h | 2 +- 7 files changed, 215 insertions(+), 1 deletion(-) create mode 100644 libavutil/film_grain_params.c create mode 100644 libavutil/film_grain_params.h diff --git a/doc/APIchanges b/doc/APIchanges index b70c78a483..41248724d9 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,10 @@ libavutil: 2017-10-21 API changes, most recent first: +2020-xx-xx - xxxxxxxxxx - lavu 56.61.100 - film_grain_params.h + Adds a new API for extracting codec film grain parameters as side data. + Adds a new AVFrameSideDataType entry AV_FRAME_DATA_FILM_GRAIN_PARAMS for it. + 2020-xx-xx - xxxxxxxxxx - lavf 58.64.100 - avformat.h Add AVSTREAM_EVENT_FLAG_NEW_PACKETS. diff --git a/libavutil/Makefile b/libavutil/Makefile index 9b08372eb2..27bafe9e12 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -84,6 +84,7 @@ HEADERS = adler32.h \ xtea.h \ tea.h \ tx.h \ + film_grain_params.h \ HEADERS-$(CONFIG_LZO) += lzo.h @@ -170,6 +171,7 @@ OBJS = adler32.o \ tx_double.o \ tx_int32.o \ video_enc_params.o \ + film_grain_params.o \ OBJS-$(CONFIG_CUDA) += hwcontext_cuda.o diff --git a/libavutil/film_grain_params.c b/libavutil/film_grain_params.c new file mode 100644 index 0000000000..930d23c7fe --- /dev/null +++ b/libavutil/film_grain_params.c @@ -0,0 +1,42 @@ +/** + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "film_grain_params.h" + +AVFilmGrainParams *av_film_grain_params_alloc(size_t *size) +{ + AVFilmGrainParams *params = av_mallocz(sizeof(AVFilmGrainParams)); + + if (size) + *size = sizeof(*params); + + return params; +} + +AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame) +{ + AVFrameSideData *side_data = av_frame_new_side_data(frame, + AV_FRAME_DATA_FILM_GRAIN_PARAMS, + sizeof(AVFilmGrainParams)); + if (!side_data) + return NULL; + + memset(side_data->data, 0, sizeof(AVFilmGrainParams)); + + return (AVFilmGrainParams *)side_data->data; +} diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h new file mode 100644 index 0000000000..ba20fe7fa6 --- /dev/null +++ b/libavutil/film_grain_params.h @@ -0,0 +1,159 @@ +/* + * Copyright (c) 2016 Neil Birkbeck + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVUTIL_FILM_GRAIN_PARAMS_H +#define AVUTIL_FILM_GRAIN_PARAMS_H + +#include "frame.h" + +enum AVFilmGrainParamsType { + AV_FILM_GRAM_PARAMS_NONE = 0, + + /** + * The union is valid when interpreted as AVFilmGrainAOMParams (codec.aom) + */ + AV_FILM_GRAM_PARAMS_AV1, +}; + +typedef struct AVFilmGrainAOMParams { + /** + * Number of points for the piecewise linear scaling function of the + * luma plane. + */ + int num_y_points; + + /** + * Piecewise scaling function values and scale for each point. + */ + uint8_t y_points[14][2 /* value, scaling */]; + + /** + * Signals whether to derive the chroma scaling function from the luma. + * Not equivalent to copying the luma values and scales. + */ + int chroma_scaling_from_luma; + + /** + * If chroma_scaling_from_luma is set to 0, signals the chroma scaling + * function parameters. + */ + int num_uv_points[2]; + uint8_t uv_points[2][10][2 /* value, scaling */]; + + /** + * Specifies the shift applied to the chroma components. For AV1, its within + * [8; 11] and determines the range and quantization of the film grain. + */ + int scaling_shift; + + /** + * Specifies the auto-regression coefficients for both luma and chroma. + */ + int ar_coeff_lag; + + /** + * Luma auto-regression coefficients. + */ + int8_t ar_coeffs_y[24]; + + /** + * Chroma auto-regression coefficients. Padded to help with SIMD, last + * 3 values of each chroma coeff are not used. + */ + int8_t ar_coeffs_uv[2][25 + 3 /* padding for alignment purposes */]; + + /** + * Specifies the range of the auto-regressive coefficients. Values of 6, + * 7, 8 and so on represent a range of [-2, 2), [-1, 1), [-0.5, 0.5) and + * so on. For AV1 must be between 6 and 9. + */ + uint64_t ar_coeff_shift; + + /** + * Signals the down shift applied to the generated gaussian numbers during + * synthesis. + */ + int grain_scale_shift; + + /** + * Specifies the luma/chroma multipliers for the index to the component + * scaling function. + */ + int uv_mult[2]; + int uv_mult_luma[2]; + + /** + * Offset used for component scaling function. For AV1 its a 9-bit value + * with a range [-256, 255] */ + int uv_offset[2]; + + /** + * Signals whether to overlap film grain blocks. + */ + int overlap_flag; +} AVFilmGrainAOMParams; + +typedef struct AVFilmGrainParams { + /** + * Specifies the codec for which this structure is valid. + */ + enum AVFilmGrainParamsType type; + + /** + * Seed to use for the synthesis process, if the codec allows for it. + */ + uint32_t seed; + + /** + * Signals to clip to limited color levels after film grain application + * for AV1. + */ + int limit_output_range; + + /** + * Additional fields may be added both here and in any structure included. + * If a codec's film grain structure differs slightly over another + * codec's, fields within may change meaning depending on the type. + */ + union { + AVFilmGrainAOMParams aom; + } codec; +} AVFilmGrainParams; + +/** + * Allocate an AVFilmGrainParams structure and set its fields to + * default values. The resulting struct can be freed using av_freep(). + * If size is not NULL it will be set to the number of bytes allocated. + * + * @return An AVFilmGrainParams filled with default values or NULL + * on failure. + */ +AVFilmGrainParams *av_film_grain_params_alloc(size_t *size); + +/** + * Allocate a complete AVFilmGrainParams and add it to the frame. + * + * @param frame The frame which side data is added to. + * + * @return The AVFilmGrainParams structure to be filled by caller. + */ +AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame); + +#endif /* AVUTIL_FILM_GRAIN_PARAMS_H */ diff --git a/libavutil/frame.c b/libavutil/frame.c index 3ab1aa3242..668a0e9e3c 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -864,6 +864,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_REGIONS_OF_INTEREST: return "Regions Of Interest"; case AV_FRAME_DATA_VIDEO_ENC_PARAMS: return "Video encoding parameters"; case AV_FRAME_DATA_SEI_UNREGISTERED: return "H.26[45] User Data Unregistered SEI message"; + case AV_FRAME_DATA_FILM_GRAIN_PARAMS: return "Film grain parameters"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index 3bd240fc97..392315f40f 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -192,6 +192,12 @@ enum AVFrameSideDataType { * uuid_iso_iec_11578 followed by AVFrameSideData.size - 16 bytes of user_data_payload_byte. */ AV_FRAME_DATA_SEI_UNREGISTERED, + + /** + * Film grain parameters for a frame, described by AVFilmGrainParameters. + * Must be present for every frame which should have film grain applied. + */ + AV_FRAME_DATA_FILM_GRAIN_PARAMS, }; enum AVActiveFormatDescription { diff --git a/libavutil/version.h b/libavutil/version.h index 1aca823650..55a59d6c58 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 60 +#define LIBAVUTIL_VERSION_MINOR 61 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Thu Nov 12 17:42:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 23602 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 51C0F444747 for ; Thu, 12 Nov 2020 19:42:29 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 363F268C1AE; Thu, 12 Nov 2020 19:42:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from w4.tutanota.de (w4.tutanota.de [81.3.6.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B654B68C1A3 for ; Thu, 12 Nov 2020 19:42:22 +0200 (EET) Received: from w3.tutanota.de (unknown [192.168.1.164]) by w4.tutanota.de (Postfix) with ESMTP id 625DF106027A for ; Thu, 12 Nov 2020 17:42:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1605202942; s=s1; d=lynne.ee; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender; bh=TswkgkuiMliCP5BbH/ncne4KV3Rw5aJDxwX+tufyAO4=; b=JPH4eVEd3fzRKfjH/ptvsi9dGEjehWGWfVbS4+RhPN9nLP8zDX59/UEhb2QaHg9H I4hcFX1eANOT3+6V5SXOWL8xUvWLyWm7G1i9j6IzQ16Bv+kbabnQi9JsI2spQKMDhsa rq2FA2l/XgiadU2HcXITfwSDwBsnxU85wJLMqwgS1uj+5h57UPWk6WwSG8ennsCZilp eYY1ews2Ff/WMIcDz0o9wDrNtU4zrohPBKyr62xS5MKgzM9wfDXIIPGMYOSVk1xMn3H HmffCz54JNGxfMxHnYlb14NmJniPfmqrFN+lDlRiAujvk33dMI+BE4CofnhAnYidbB1 Zd6SZRUPVw== Date: Thu, 12 Nov 2020 18:42:22 +0100 (CET) From: Lynne To: Ffmpeg Devel Message-ID: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/3] libavcodec: add a new AV_CODEC_EXPORT_DATA_FILM_GRAIN flag and option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This introduces a new field to allow decoders to export their film grain parameters. Will be used by the next patch. Patch attached. Subject: [PATCH v2 2/3] libavcodec: add a new AV_CODEC_EXPORT_DATA_FILM_GRAIN flag and option This introduces a new field to allow decoders to export their film grain parameters. Will be used by the next patch. --- doc/APIchanges | 3 +++ libavcodec/avcodec.h | 5 +++++ libavcodec/options_table.h | 1 + libavcodec/version.h | 4 ++-- 4 files changed, 11 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 41248724d9..9d0ddb4ff6 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,9 @@ libavutil: 2017-10-21 API changes, most recent first: +2020-xx-xx - xxxxxxxxxx - lavc 58.113.100 - avcodec.h + Adds a new flag AV_CODEC_EXPORT_DATA_FILM_GRAIN for export_side_data. + 2020-xx-xx - xxxxxxxxxx - lavu 56.61.100 - film_grain_params.h Adds a new API for extracting codec film grain parameters as side data. Adds a new AVFrameSideDataType entry AV_FRAME_DATA_FILM_GRAIN_PARAMS for it. diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 20af3ef00d..5047da0f6a 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -410,6 +410,11 @@ typedef struct RcOverride{ * Export the AVVideoEncParams structure through frame side data. */ #define AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS (1 << 2) +/** + * Decoding only. + * Do not apply film grain, export it instead. + */ +#define AV_CODEC_EXPORT_DATA_FILM_GRAIN (1 << 3) /** * Pan Scan area. diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index 66bda42663..77e1211ae0 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -83,6 +83,7 @@ static const AVOption avcodec_options[] = { {"mvs", "export motion vectors through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_MVS}, INT_MIN, INT_MAX, V|D, "export_side_data"}, {"prft", "export Producer Reference Time through packet side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_PRFT}, INT_MIN, INT_MAX, A|V|S|E, "export_side_data"}, {"venc_params", "export video encoding parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS}, INT_MIN, INT_MAX, V|D, "export_side_data"}, +{"export_film_grain", "export film grain parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_FILM_GRAIN}, INT_MIN, INT_MAX, V|D, "export_side_data"}, {"time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, {.dbl = 0}, 0, INT_MAX}, {"g", "set the group of picture (GOP) size", OFFSET(gop_size), AV_OPT_TYPE_INT, {.i64 = 12 }, INT_MIN, INT_MAX, V|E}, {"ar", "set audio sampling rate (in Hz)", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64 = DEFAULT }, 0, INT_MAX, A|D|E}, diff --git a/libavcodec/version.h b/libavcodec/version.h index 1585d95777..37fa426f40 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,8 +28,8 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 58 -#define LIBAVCODEC_VERSION_MINOR 112 -#define LIBAVCODEC_VERSION_MICRO 103 +#define LIBAVCODEC_VERSION_MINOR 113 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ From patchwork Thu Nov 12 17:42:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 23603 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 57B7F444747 for ; Thu, 12 Nov 2020 19:42:41 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3D0C368C1B9; Thu, 12 Nov 2020 19:42:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from w4.tutanota.de (w4.tutanota.de [81.3.6.165]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0C4A568C1A5 for ; Thu, 12 Nov 2020 19:42:35 +0200 (EET) Received: from w3.tutanota.de (unknown [192.168.1.164]) by w4.tutanota.de (Postfix) with ESMTP id AA60A106029E for ; Thu, 12 Nov 2020 17:42:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1605202954; s=s1; d=lynne.ee; h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender; bh=5LKr5ziVZqJWojtwtxrRF44bR7n++qMzY5htNDOLRNw=; b=gyv6Jg9w9QeDI68zmTC7599BGjyIHyD1sjI0eCy2H39Q8jl/ssQvAHDLx1JyGMat PXnSNg+0Lf52aUkDQzRQB2KIeLvAgIdMSZO1B3RAiCThX/QmKubF6iFg44+YWWG3+7N D5b2sWRYqhDpBIidEPQb3HfNeHIYiuAEuvnXadXYOL3PDGJlnydZQ6br1BSj+WtsY0p 2h7xjJCjbKyqIsxH2JTRoqvHP1fvarB8jvRCl2w0TMYINNGhwxLjcgYmNqxAGusDHE0 FTGl3joWbm47//ZZxVfs8vof9H7dTH3ySFYixs/9y7wC5OtJV2QsdWuaFZHoh9gcbY4 +my4G8dxfA== Date: Thu, 12 Nov 2020 18:42:34 +0100 (CET) From: Lynne To: Ffmpeg Devel Message-ID: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 3/3] libdav1d: use film grain export flag to export AVFilmGrainParams side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This patch is relatively straightforward with one exception: the decoder option flag. The option was introduced to troubleshoot but its existence is conflicting and redundant now that we have a codec-generic flag. Hence this patch deprecates it. The way it interacts with AV_CODEC_EXPORT_DATA_FILM_GRAIN is as follows: If filmgrain is unset and AV_CODEC_EXPORT_DATA_FILM_GRAIN is present, disable film grain application and export side data. If filmgrain is set to 0, disable film grain and export side data. If filmgrain is set to 1, apply film grain but export side data if the AV_CODEC_EXPORT_DATA_FILM_GRAIN flag is set. This may result in double film grain application, but the user has requested it by setting both. Patch attached. Subject: [PATCH v2 3/3] libdav1d: use film grain export flag to export AVFilmGrainParams side data This patch is relatively straightforward with one exception: the decoder option flag. The option was introduced to troubleshoot but its existence is conflicting and redundant now that we have a codec-generic flag. Hence this patch deprecates it. The way it interacts with AV_CODEC_EXPORT_DATA_FILM_GRAIN is as follows: If filmgrain is unset and AV_CODEC_EXPORT_DATA_FILM_GRAIN is present, disable film grain application and export side data. If filmgrain is set to 0, disable film grain and export side data. If filmgrain is set to 1, apply film grain but export side data if the AV_CODEC_EXPORT_DATA_FILM_GRAIN flag is set. This may result in double film grain application, but the user has requested it by setting both. --- libavcodec/libdav1d.c | 82 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c index 3af7ef4edc..319a9f6b47 100644 --- a/libavcodec/libdav1d.c +++ b/libavcodec/libdav1d.c @@ -22,6 +22,7 @@ #include #include "libavutil/avassert.h" +#include "libavutil/film_grain_params.h" #include "libavutil/mastering_display_metadata.h" #include "libavutil/imgutils.h" #include "libavutil/opt.h" @@ -37,6 +38,7 @@ typedef struct Libdav1dContext { Dav1dContext *c; AVBufferPool *pool; int pool_size; + AVBufferRef *film_grain_params; Dav1dData data; int tile_threads; @@ -137,6 +139,8 @@ static av_cold int libdav1d_init(AVCodecContext *c) s.frame_size_limit = c->max_pixels; if (dav1d->apply_grain >= 0) s.apply_grain = dav1d->apply_grain; + else if (c->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) + s.apply_grain = 0; s.all_layers = dav1d->all_layers; if (dav1d->operating_point >= 0) @@ -162,6 +166,7 @@ static void libdav1d_flush(AVCodecContext *c) { Libdav1dContext *dav1d = c->priv_data; + av_buffer_unref(&dav1d->film_grain_params); dav1d_data_unref(&dav1d->data); dav1d_flush(dav1d->c); } @@ -395,6 +400,80 @@ FF_ENABLE_DEPRECATION_WARNINGS break; } } + if (p->frame_hdr->film_grain.present && (!dav1d->apply_grain || + (c->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN))) { + AVFilmGrainParams *fgp; + if (p->frame_hdr->film_grain.update) { + size_t fgp_size; + fgp = av_film_grain_params_alloc(&fgp_size); + if (!fgp) { + res = AVERROR(ENOMEM); + goto fail; + } + + AVBufferRef *buf = av_buffer_create((uint8_t *)fgp, fgp_size, + av_buffer_default_free, NULL, 0); + if (!buf) { + res = AVERROR(ENOMEM); + goto fail; + } + + av_buffer_unref(&dav1d->film_grain_params); + dav1d->film_grain_params = buf; + + fgp->type = AV_FILM_GRAM_PARAMS_AV1; + fgp->seed = p->frame_hdr->film_grain.data.seed; + fgp->limit_output_range = p->frame_hdr->film_grain.data.clip_to_restricted_range; + fgp->codec.aom.num_y_points = p->frame_hdr->film_grain.data.num_y_points; + fgp->codec.aom.chroma_scaling_from_luma = p->frame_hdr->film_grain.data.chroma_scaling_from_luma; + fgp->codec.aom.scaling_shift = p->frame_hdr->film_grain.data.scaling_shift; + fgp->codec.aom.ar_coeff_lag = p->frame_hdr->film_grain.data.ar_coeff_lag; + fgp->codec.aom.ar_coeff_shift = p->frame_hdr->film_grain.data.ar_coeff_shift; + fgp->codec.aom.grain_scale_shift = p->frame_hdr->film_grain.data.grain_scale_shift; + fgp->codec.aom.overlap_flag = p->frame_hdr->film_grain.data.overlap_flag; + + memcpy(&fgp->codec.aom.y_points, &p->frame_hdr->film_grain.data.y_points, + sizeof(fgp->codec.aom.y_points)); + memcpy(&fgp->codec.aom.num_uv_points, &p->frame_hdr->film_grain.data.num_uv_points, + sizeof(fgp->codec.aom.num_uv_points)); + memcpy(&fgp->codec.aom.uv_points, &p->frame_hdr->film_grain.data.uv_points, + sizeof(fgp->codec.aom.uv_points)); + memcpy(&fgp->codec.aom.ar_coeffs_y, &p->frame_hdr->film_grain.data.ar_coeffs_y, + sizeof(fgp->codec.aom.ar_coeffs_y)); + memcpy(&fgp->codec.aom.ar_coeffs_uv, &p->frame_hdr->film_grain.data.ar_coeffs_uv, + sizeof(fgp->codec.aom.ar_coeffs_uv)); + memcpy(&fgp->codec.aom.uv_mult, &p->frame_hdr->film_grain.data.uv_mult, + sizeof(fgp->codec.aom.uv_mult)); + memcpy(&fgp->codec.aom.uv_mult_luma, &p->frame_hdr->film_grain.data.uv_luma_mult, + sizeof(fgp->codec.aom.uv_mult_luma)); + memcpy(&fgp->codec.aom.uv_offset, &p->frame_hdr->film_grain.data.uv_offset, + sizeof(fgp->codec.aom.uv_offset)); + } else if (dav1d->film_grain_params) { + if (av_buffer_make_writable(&dav1d->film_grain_params) < 0) { + res = AVERROR(ENOMEM); + goto fail; + } + + /* The RNG seed is always updated even if update == 0 */ + fgp = (AVFilmGrainParams *)dav1d->film_grain_params->data; + fgp->seed = p->frame_hdr->film_grain.data.seed; + } + + if (dav1d->film_grain_params) { + AVBufferRef *fgp_ref = av_buffer_ref(dav1d->film_grain_params); + if (!fgp_ref) { + res = AVERROR(ENOMEM); + goto fail; + } + if (!av_frame_new_side_data_from_buf(frame, + AV_FRAME_DATA_FILM_GRAIN_PARAMS, + fgp_ref)) { + av_buffer_unref(&fgp_ref); + res = AVERROR(ENOMEM); + goto fail; + } + } + } res = 0; fail: @@ -408,6 +487,7 @@ static av_cold int libdav1d_close(AVCodecContext *c) { Libdav1dContext *dav1d = c->priv_data; + av_buffer_unref(&dav1d->film_grain_params); av_buffer_pool_uninit(&dav1d->pool); dav1d_data_unref(&dav1d->data); dav1d_close(&dav1d->c); @@ -420,7 +500,7 @@ static av_cold int libdav1d_close(AVCodecContext *c) static const AVOption libdav1d_options[] = { { "tilethreads", "Tile threads", OFFSET(tile_threads), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, DAV1D_MAX_TILE_THREADS, VD }, { "framethreads", "Frame threads", OFFSET(frame_threads), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, DAV1D_MAX_FRAME_THREADS, VD }, - { "filmgrain", "Apply Film Grain", OFFSET(apply_grain), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VD }, + { "filmgrain", "Apply Film Grain", OFFSET(apply_grain), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VD | AV_OPT_FLAG_DEPRECATED }, { "oppoint", "Select an operating point of the scalable bitstream", OFFSET(operating_point), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 31, VD }, { "alllayers", "Output all spatial layers", OFFSET(all_layers), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VD }, { NULL }