From patchwork Mon Mar 11 20:58:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 46964 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:dc95:b0:1a1:738b:6bc0 with SMTP id ky21csp1469248pzb; Mon, 11 Mar 2024 13:58:57 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU/BrgVdsBZGOHYM6P77TkbKPDYMXHFOTINOc16vhwfUPJN38sC0tdFi3UXL6QzYtDqB8hVO4s3yDPLguSyu/Ko/vjLuhmISvCUMA== X-Google-Smtp-Source: AGHT+IGloZWa0MqBRHOFzrz8QGgPNhe2LDeYn3YmM9pk119QZO4/9HYgr06rV7wFwP9yT6pMn0TP X-Received: by 2002:a17:906:2c51:b0:a44:b91e:315b with SMTP id f17-20020a1709062c5100b00a44b91e315bmr4151287ejh.68.1710190737027; Mon, 11 Mar 2024 13:58:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710190737; cv=none; d=google.com; s=arc-20160816; b=lU1qnDWBg34NdDnI0iqwEStY6kINfJQ565e7ZgJjs65+q+0QgrbGAtHDxmHH3otQEX ECDcX3l5LnoqoEfuBAv7Cuu7FiQoxMuDceCvPuUIh/TumePiwhejCYo4ZmIP+xSVxGbK rVu6llMvTDUa7s8+gBRjpuEUC79q85acUoRKq+xdBmquOORF8L56/z1BC/SAJLJywfno wqGBSu2WtWeHSscYf3LN/Vx6gcfnT2xh2opzqBwCb6d4mOItLsdHnTGjk+WRq4EpEMyS Lbf9W+8bNeNz8DZIrkrk1Ov4jb6yuzSLcwAVlh6cJ2MaCb6Wt3n4y+bYijXc5dap8Ov5 voWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=m6Ov+I2zOsHvorq+PnzHRBml5QfrT2U95+C+4rKXTrI=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=aXJyvAazODH71K++Yrrwor1bREoSbC4Smj7RIJvT9ywmRH6X6PbITw1wHLDf6QDtrF ybrgvC90iMg2BrrXX6sVtfn6k0jZL6wZsf+9pX6YwoJrFPUt2tsoA/qxw/FRivQzXw1L lQHcbr3aEuHG9nvXA4ibjw1/efF8m1oQccYoxqQn3xlIXMHbqpJDhjpcl4u2ga6X323B D7g2yNyLoP2zC5jRWiOnvL8BAlR+OQlEh0iXa/B7EcESdFjnjSwfn4I4BnD7kK0KeK9y Jwj01BV8BNP2oFJddeaLinq9CsSRmYODXhimGo9wgerUsL8FfztPZMnGqUPTDIATTtup B0yQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=IpRElawj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id rv25-20020a17090710d900b00a42f6f4615asi2716967ejb.1044.2024.03.11.13.58.56; Mon, 11 Mar 2024 13:58:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=IpRElawj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C0B1968D09F; Mon, 11 Mar 2024 22:58:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 451C668C27A for ; Mon, 11 Mar 2024 22:58:46 +0200 (EET) Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2d21cdbc85bso77998401fa.2 for ; Mon, 11 Mar 2024 13:58:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710190725; x=1710795525; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=UcNo8oWFLhK4F8Rv0W2+fcYCK7k8vpsAJC9gi2vQ7v8=; b=IpRElawjoVlxsXwDhTTPxqR1bA2bZ1D6lk95hpPuoLkvMY6kp2koDgFD2pnRKMo8RH RCmOGbgAAHIvQR9hgsDb0l8qe3EyZWg2sxHPqrpQkTCPCAfQ2PFxfzxqityvB/YHTGxz HeKbbauAIVo+BO9vX7LB6nNtE6jd6s1CS1/jWnR3Ng/my/GvHQMn3+JwGWt12lzFrQlT 6fJGMVmcXrRwBxaHC6aKE4I2yBHkgRA43EQyFy1Xryitkipopvk/G5zGPRL7/krD8PAI jtiMhxiApHUMyoi8bse1yfxWKVGxFpxfwzP2PbgZFn/JPeE/t3wib3BHQZIDoRLlCOJt zlNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710190725; x=1710795525; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UcNo8oWFLhK4F8Rv0W2+fcYCK7k8vpsAJC9gi2vQ7v8=; b=TLyYxM4uXtiqJ/jWCmpihfE1MpCnYbyfOoDVgVGscx8nBzMcdHDt5ejtfwNdCjsC8Z d7FOaCfFj8vMd4D9gRBPueEbKs7oxDItb6sehWjqLtQiqRUpr90m2Q8HXoJXHtVQpTb0 cp5xaeoryrQM7huI9Us1ybMkfx1iI9AXrKwTjvcfFzlvVnW9PuXrRP1BwymA5HvFQg15 FwR/Metsx9lqt4vLJc5DlBgv/jEhVS9sSMhVCMmHuR2p8tAeX62+OeKNyb4KaV/7gkg2 +Q9ASVTpPjmuDM2j0pX8ARKZq/mDruQQEfGcfvK2Lz/gpfV0N2iUveiOrlHHGkCjNf7r CxFg== X-Gm-Message-State: AOJu0YwBev1r5aElJIXVIzLLHx+DpZs//F1IOUCcfOgxP/J8+9q6iEMz zXO/UiPsYpe4QUlBaaAgC8h1IyhCyefYGCstJnrE2MyA3SwyX15S1sgG73mJ X-Received: by 2002:a05:6512:3612:b0:513:ad46:dc7d with SMTP id f18-20020a056512361200b00513ad46dc7dmr1751461lfs.61.1710190725103; Mon, 11 Mar 2024 13:58:45 -0700 (PDT) Received: from maomao.mshome.net (91-153-198-187.elisa-laajakaista.fi. [91.153.198.187]) by smtp.gmail.com with ESMTPSA id c17-20020ac25f71000000b0051333104075sm1249247lfc.259.2024.03.11.13.58.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Mar 2024 13:58:44 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 11 Mar 2024 22:58:23 +0200 Message-ID: <20240311205844.3569895-1-jeebjp@gmail.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v8 00/14] encoder AVCodecContext configuration side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +8llwFuPquyh Differences to v7: 1. rebased on top of current master 2. Applied various review comments. Comparison URL (mostly configure and wrappers, avutil/frame.c): https://github.com/jeeb/ffmpeg/compare/avcodec_cll_mdcv_side_data_v7..avcodec_cll_mdcv_side_data_v8 This patch set I've now been working for a while since I felt like it was weird we couldn't pass through information such as static HDR metadata to encoders from decoded input. This initial version adds the necessary framework, as well as adds static HDR metadata support for libsvtav1, libx264 as well as libx265 wrappers. An alternative to this would be to make encoders only properly initialize when they receive the first AVFrame, but that seems to be a bigger, nastier change than introducing an AVFrameSideDataSet in avctx as everything seems to presume that extradata etc are available after opening the encoder. Note: Any opinions on whether FFCodec or AVCodec should have handled_side_data list, so that if format specific generic logic is added, it could be checked whether the codec itself handles this side data? This could also be utilized to list handled side data from f.ex. `ffmpeg -h encoder=libsvtav1`. Jan Jan Ekström (14): avutil/frame: split side data list wiping out to non-AVFrame function avutil/frame: add helper for freeing arrays of side data avutil/frame: split side_data_from_buf to base and AVFrame func avutil/frame: split side data removal out to non-AVFrame function avutil/frame: add helper for adding side data to array avutil/frame: add helper for adding existing side data to array avutil/frame: add helper for adding side data w/ AVBufferRef to array avutil/frame: add helper for getting side data from array avcodec: add frame side data array to AVCodecContext avcodec: add helper for configuring AVCodecContext's frame side data ffmpeg: pass first video AVFrame's side data to encoder avcodec/libsvtav1: add support for writing out CLL and MDCV avcodec/libx264: add support for writing out CLL and MDCV avcodec/libx265: add support for writing out CLL and MDCV configure | 2 + fftools/ffmpeg_enc.c | 10 ++ libavcodec/avcodec.c | 30 ++++++ libavcodec/avcodec.h | 29 +++++ libavcodec/libsvtav1.c | 68 ++++++++++++ libavcodec/libx264.c | 79 ++++++++++++++ libavcodec/libx265.c | 88 ++++++++++++++++ libavcodec/options.c | 2 + libavutil/Makefile | 1 + libavutil/frame.c | 180 ++++++++++++++++++++++++++------ libavutil/frame.h | 88 ++++++++++++++++ libavutil/tests/side_data_set.c | 103 ++++++++++++++++++ tests/fate/enc_external.mak | 15 +++ tests/fate/libavutil.mak | 4 + tests/ref/fate/libsvtav1-hdr10 | 14 +++ tests/ref/fate/libx264-hdr10 | 15 +++ tests/ref/fate/libx265-hdr10 | 16 +++ tests/ref/fate/side_data_set | 14 +++ 18 files changed, 728 insertions(+), 30 deletions(-) create mode 100644 libavutil/tests/side_data_set.c create mode 100644 tests/ref/fate/libsvtav1-hdr10 create mode 100644 tests/ref/fate/libx264-hdr10 create mode 100644 tests/ref/fate/libx265-hdr10 create mode 100644 tests/ref/fate/side_data_set Full diff based on the same base hash for v7->v8: diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index e39ba05b3b..5ec6ee17de 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -248,7 +248,7 @@ int enc_open(void *opaque, const AVFrame *frame) ret = avcodec_configure_side_data( enc_ctx, (const AVFrameSideData **)frame->side_data, frame->nb_side_data, - AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES); + AV_FRAME_SIDE_DATA_FLAG_UNIQUE); if (ret < 0) { av_log(NULL, AV_LOG_ERROR, "failed to configure video encoder: %s!\n", av_err2str(ret)); diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index f8df3137e0..0ced87b946 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -705,7 +705,7 @@ int avcodec_configure_side_data(AVCodecContext *avctx, return AVERROR(EINVAL); for (int i = 0; i < nb_sd; i++) { - int ret = av_frame_side_data_from_sd( + int ret = av_frame_side_data_clone( &avctx->frame_side_data, &avctx->nb_frame_side_data, sd[i], flags); if (ret < 0) { av_frame_side_data_free( diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 460452d591..6de4dc0f7b 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -3076,7 +3076,7 @@ void av_fast_padded_mallocz(void *ptr, unsigned int *size, size_t min_size); int avcodec_is_open(AVCodecContext *s); /** - * Add multiple side data entries to an AVCodecContext set in one go, for + * Add multiple side data entries to an AVCodecContext's array in one go, for * example from an AVFrame. * * In case the function fails to add a side data entry, it will clear the @@ -3090,11 +3090,12 @@ int avcodec_is_open(AVCodecContext *s); * * @return negative error code on failure, >=0 on success. * - * @see av_frame_side_data_set_new_entry regarding the flags. + * @see av_frame_side_data_new regarding the flags. */ int avcodec_configure_side_data(AVCodecContext *avctx, const AVFrameSideData **sd, const int nb_sd, unsigned int flags); + /** * @} */ diff --git a/libavutil/frame.c b/libavutil/frame.c index d516c1f443..47ecd964b8 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -771,7 +771,7 @@ AVFrameSideData *av_frame_side_data_new(AVFrameSideData ***sd, int *nb_sd, AVBufferRef *buf = av_buffer_alloc(size); AVFrameSideData *ret = NULL; - if (flags & AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES) + if (flags & AV_FRAME_SIDE_DATA_FLAG_UNIQUE) remove_side_data(sd, nb_sd, type); ret = add_side_data_to_set_from_buf(sd, nb_sd, type, buf); @@ -781,45 +781,43 @@ AVFrameSideData *av_frame_side_data_new(AVFrameSideData ***sd, int *nb_sd, return ret; } -AVFrameSideData *av_frame_side_data_from_buf(AVFrameSideData ***sd, int *nb_sd, - enum AVFrameSideDataType type, - const AVBufferRef *buf, - unsigned int flags) +AVFrameSideData *av_frame_side_data_add(AVFrameSideData ***sd, int *nb_sd, + enum AVFrameSideDataType type, + const AVBufferRef *buf, + unsigned int flags) { + AVBufferRef *new_buf = NULL; + AVFrameSideData *sd_dst = NULL; + if (!sd || !buf || !nb_sd || (*nb_sd && !*sd)) return NULL; - if (flags & AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES) - remove_side_data(sd, nb_sd, type); - - { - AVBufferRef *new_buf = av_buffer_ref(buf); - AVFrameSideData *sd_dst = NULL; - - if (flags & AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES) - remove_side_data(sd, nb_sd, type); + new_buf = av_buffer_ref(buf); + if (!buf) + return NULL; - sd_dst = add_side_data_to_set_from_buf(sd, nb_sd, type, new_buf); - if (!sd_dst) { - av_buffer_unref(&new_buf); - return NULL; - } + if (flags & AV_FRAME_SIDE_DATA_FLAG_UNIQUE) + remove_side_data(sd, nb_sd, type); - return sd_dst; + sd_dst = add_side_data_to_set_from_buf(sd, nb_sd, type, new_buf); + if (!sd_dst) { + av_buffer_unref(&new_buf); + return NULL; } + + return sd_dst; } -int av_frame_side_data_from_sd(AVFrameSideData ***sd, int *nb_sd, - const AVFrameSideData *src, - unsigned int flags) +int av_frame_side_data_clone(AVFrameSideData ***sd, int *nb_sd, + const AVFrameSideData *src, unsigned int flags) { AVFrameSideData *sd_dst = NULL; - int ret = AVERROR_BUG; + int ret = AVERROR_BUG; + if (!src) return AVERROR(EINVAL); - sd_dst = - av_frame_side_data_from_buf(sd, nb_sd, src->type, src->buf, flags); + sd_dst = av_frame_side_data_add(sd, nb_sd, src->type, src->buf, flags); if (!sd_dst) return AVERROR(ENOMEM); diff --git a/libavutil/frame.h b/libavutil/frame.h index 6e1409a928..e59f033cce 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -990,7 +990,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type); /** * Free all side data entries and their contents, then zeroes out the - * struct values. + * values which the pointers are pointing to. * * @param sd pointer to array of side data to free. Will be set to NULL * upon return. @@ -999,68 +999,71 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type); */ void av_frame_side_data_free(AVFrameSideData ***sd, int *nb_sd); -#define AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES (1 << 0) +#define AV_FRAME_SIDE_DATA_FLAG_UNIQUE (1 << 0) /** - * Add a new side data entry to a set. + * Add new side data entry to an array. * - * @param sd pointer to array of side data to which to add another entry. + * @param sd pointer to array of side data to which to add another entry, + * or to NULL in order to start a new array. * @param nb_sd pointer to an integer containing the number of entries in * the array. * @param type type of the added side data * @param size size of the side data - * @param flags Some combination of AV_FRAME_SIDE_DATA_SET_FLAG_* flags, or 0. + * @param flags Some combination of AV_FRAME_SIDE_DATA_FLAG_* flags, or 0. * * @return newly added side data on success, NULL on error. In case of - * AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES being set, entries - * of matching AVFrameSideDataType will be removed before the - * addition is attempted. + * AV_FRAME_SIDE_DATA_FLAG_UNIQUE being set, entries of matching + * AVFrameSideDataType will be removed before the addition is + * attempted. */ AVFrameSideData *av_frame_side_data_new(AVFrameSideData ***sd, int *nb_sd, enum AVFrameSideDataType type, size_t size, unsigned int flags); /** - * Add a new side data entry to a set from an existing AVBufferRef. + * Add a new side data entry to an array from an existing AVBufferRef. * - * @param sd pointer to array of side data to which to add another entry. + * @param sd pointer to array of side data to which to add another entry, + * or to NULL in order to start a new array. * @param nb_sd pointer to an integer containing the number of entries in * the array. * @param type type of the added side data * @param buf AVBufferRef for which a new reference will be made - * @param flags Some combination of AV_FRAME_SIDE_DATA_SET_FLAG_* flags, or 0. + * @param flags Some combination of AV_FRAME_SIDE_DATA_FLAG_* flags, or 0. * * @return newly added side data on success, NULL on error. In case of - * AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES being set, entries - * of matching AVFrameSideDataType will be removed before the - * addition is attempted. + * AV_FRAME_SIDE_DATA_FLAG_UNIQUE being set, entries of matching + * AVFrameSideDataType will be removed before the addition is + * attempted. */ -AVFrameSideData *av_frame_side_data_from_buf(AVFrameSideData ***sd, int *nb_sd, - enum AVFrameSideDataType type, - const AVBufferRef *buf, - unsigned int flags); +AVFrameSideData *av_frame_side_data_add(AVFrameSideData ***sd, int *nb_sd, + enum AVFrameSideDataType type, + const AVBufferRef *buf, + unsigned int flags); /** - * Add a new side data entry to a set based on existing side data, taking + * Add a new side data entry to an array based on existing side data, taking * a reference towards the contained AVBufferRef. * - * @param sd pointer to array of side data to which to add another entry. + * @param sd pointer to array of side data to which to add another entry, + * or to NULL in order to start a new array. * @param nb_sd pointer to an integer containing the number of entries in * the array. - * @param src side data which should be added to the set - * @param flags Some combination of AV_FRAME_SIDE_DATA_SET_FLAG_* flags, or 0. + * @param src side data to be cloned, with a new reference utilized + * for the buffer. + * @param flags Some combination of AV_FRAME_SIDE_DATA_FLAG_* flags, or 0. * * @return negative error code on failure, >=0 on success. In case of - * AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES being set, entries - * of matching AVFrameSideDataType will be removed before the - * addition is attempted. + * AV_FRAME_SIDE_DATA_FLAG_UNIQUE being set, entries of matching + * AVFrameSideDataType will be removed before the addition is + * attempted. */ -int av_frame_side_data_from_sd(AVFrameSideData ***sd, int *nb_sd, - const AVFrameSideData *src, - unsigned int flags); +int av_frame_side_data_clone(AVFrameSideData ***sd, int *nb_sd, + const AVFrameSideData *src, unsigned int flags); /** - * Get a side data entry of a specific type from a set. + * Get a side data entry of a specific type from an array. * * @param sd array of side data. * @param nb_sd integer containing the number of entries in the array. diff --git a/libavutil/tests/side_data_set.c b/libavutil/tests/side_data_set.c index 7ef99db3d2..793a62c009 100644 --- a/libavutil/tests/side_data_set.c +++ b/libavutil/tests/side_data_set.c @@ -87,7 +87,7 @@ int main(void) AVFrameSideData *sd = av_frame_side_data_new( &set.sd, &set.nb_sd, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL, sizeof(AVContentLightMetadata), - AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES); + AV_FRAME_SIDE_DATA_FLAG_UNIQUE); av_assert0(sd);