From patchwork Thu Feb 29 16:42:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 46659 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a919:b0:19e:cdac:8cce with SMTP id cd25csp719236pzb; Thu, 29 Feb 2024 08:43:28 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWdod7pNhLNehB2JULopbaU8CLwo3/XpBwWlNIyP2kpRiK+nmWQVjv0HscvprJFtNNx1TdJ8mTkCmdVccYKNq2AwhEygOkiXEWuhg== X-Google-Smtp-Source: AGHT+IFv/6qhgDEPhr83ANnxGYca9aEKWJUcGAYRVLY+7LBoj+BsuaRW+994Sp6pZXTBgpndJsOx X-Received: by 2002:a17:907:7750:b0:a3f:1e63:8ac with SMTP id kx16-20020a170907775000b00a3f1e6308acmr1858888ejc.3.1709225008275; Thu, 29 Feb 2024 08:43:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1709225008; cv=none; d=google.com; s=arc-20160816; b=SElHyKfn+dLucKVgtnhYmpbgQuLqxOtB3HfDw/Stbc0kp2zLTYrSKXevEer05qwboL 4+TsemsFU8tA+pT6Xof+i4cwY8C+cpctczOI4ki9d4sfyd4Z028WFjddzUV11rs8lkBM KtiiAuMh3qgG9HJtXlxtbC0WC+LEVpO9l/ukyObGl64i07wPYCk81Jn440VedKAZJ429 xnpIDrfyYuDG8nZZ0ZNM+GcVhJNtbTzDSNf436HG+RfFP6gXmkcO2q0kdegFzQWh0iXl 40DbkVG059LxYH1cblrQv7uEHkKR/A1PL0fI8jFFk3Dj956x8rbR/4V96jJN1pP8yzy8 5Unw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=vC2AoZ12w+fIbg+v4sFfzHOGOZSnpZu2i9vO12zoMQg=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=Thzo6gFaiJtS2efOb8Wz48cpNUt6mc4BwTT7Gto/C0zLOBnDRdn8C4dABfm4j62Soe vK3BOZhAg/BoizRcPtdU+YoWxOi6JxEi/XU2QbfsagpBaglrTDFJjkVrLhHZdhAgI/1R uOSlKx61jMOwiYeAPbPqGSIEfdo+UmktBV+vNykdBC1/zE/8bGQheoXR2bk6Qcl4uLot bgVrTQ2nS73qwNGH50CtqE5EIs0jVSnRP7VzArY+QWErv4m1dF9zY8NU1mjH8nYt6jJg QNMKAAJtGsAbjzsAEolEeTP9aD45qJsUSD+adSDuiaAwaWJMlUYfdfljULgesoHlq2Fr NNkw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ThTaxFS6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v17-20020a170906181100b00a3e92a7b1eesi724717eje.917.2024.02.29.08.43.27; Thu, 29 Feb 2024 08:43:28 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=ThTaxFS6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1FFAD68D1BC; Thu, 29 Feb 2024 18:43:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5A79268D131 for ; Thu, 29 Feb 2024 18:43:17 +0200 (EET) Received: by mail-lf1-f41.google.com with SMTP id 2adb3069b0e04-512d19e2cb8so1354289e87.0 for ; Thu, 29 Feb 2024 08:43:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709224996; x=1709829796; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=QKRlitY4IvqqXufsemQ1jxRb0h/i/gi7sDa5IMCKeaI=; b=ThTaxFS6LEPOB66gFiPhYO9WcvCjK8htFxZWQV6moKh27JKBzmuBZbQMswIaTT7P9N iJkL1xlhCdACtWDTVWj9S6K8djJBi1G9Ti6UxNAfWSNujyvahBcX+xiPnK8Yvo8Dt2UK YJu9k2TkRx3C5lvGJ1KSibHFrwKk8OLMjHADoNZjCtsDn9IJOF2VeTn82b+WuQV/HNTk aPF2Ypzf2YPR+8ZTs/xmnW7u4WmkVRP+vcATnXud9P5+6DzF5MAmlJB4oFB4TLEeOhhJ lcGTUcvziv29/1kC4x2YtbwEuaVlgElv10iRErgOoGIM1gZAT8oZ2dDHy1ZhVcPu5vMu pH1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709224996; x=1709829796; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QKRlitY4IvqqXufsemQ1jxRb0h/i/gi7sDa5IMCKeaI=; b=HcfOTWmxgEpZkSd4FBc0h6+QLVLRAWt+NfXn6zZyIsnSULexNHp3l2bw9o3ftPeRSz aWHEhGmXFu35ZkJkkTXe4jfIDZ2FWAtiLUeH2klGtZpkeJu2Qn1z/R9UdkD0UHSL8xK2 lwpjnJd+OcuNd+IMmPXP++QH4JJrynK5fkKrW5XMl4dfQo1qHR+9HL137hsY2ZKeWmEx J3rtj1ooy7NoPUDkvyUDlMAAifhD17EWFkzsfxgQr9NJUYv6nwkuUsU7doi1TVqF6gYK /gxs+d2p2uS3t9O1UMrC0o02hAGQnHDgznQLb8lYlhXVQSi/zrSEGD41muZGMSicWsv7 aUHw== X-Gm-Message-State: AOJu0YwjOFDs0vhD7vEe1LpBKrhZMScC7Qzw8YXtZmipD2FMN41J3LjX +zOLHQ+4JB+ozNUgeHnfgQVMgpvgcUfVNtRCLcU0Yq92h3jtDUddDDSGIHLT X-Received: by 2002:ac2:4829:0:b0:513:1829:9483 with SMTP id 9-20020ac24829000000b0051318299483mr1786254lft.69.1709224996076; Thu, 29 Feb 2024 08:43:16 -0800 (PST) Received: from localhost.localdomain (91-153-198-187.elisa-laajakaista.fi. [91.153.198.187]) by smtp.gmail.com with ESMTPSA id k6-20020ac24566000000b00513290734cbsm206248lfm.292.2024.02.29.08.43.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 08:43:15 -0800 (PST) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Thu, 29 Feb 2024 18:42:47 +0200 Message-ID: <20240229164307.3535613-1-jeebjp@gmail.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 00/14] encoder AVCodecContext configuration side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +/qflB9d0aV3 Differences to v5: 1. rebased on top of current master 2. as requested by James, there is now a function to add new frame side data by means of providing a AVBufferRef. Comparison URL (mostly configure and wrappers, avutil/frame.c): https://github.com/jeeb/ffmpeg/compare/avcodec_cll_mdcv_side_data_v6..avcodec_cll_mdcv_side_data_v7 This patch set I've now been working for a while since I felt like it was weird we couldn't pass through information such as static HDR metadata to encoders from decoded input. This initial version adds the necessary framework, as well as adds static HDR metadata support for libsvtav1, libx264 as well as libx265 wrappers. An alternative to this would be to make encoders only properly initialize when they receive the first AVFrame, but that seems to be a bigger, nastier change than introducing an AVFrameSideDataSet in avctx as everything seems to presume that extradata etc are available after opening the encoder. Note: Any opinions on whether FFCodec or AVCodec should have handled_side_data list, so that if format specific generic logic is added, it could be checked whether the codec itself handles this side data? This could also be utilized to list handled side data from f.ex. `ffmpeg -h encoder=libsvtav1`. Jan Jan Ekström (14): avutil/frame: split side data list wiping out to non-AVFrame function avutil/frame: add helper for freeing arrays of side data avutil/frame: split side_data_from_buf to base and AVFrame func avutil/frame: split side data removal out to non-AVFrame function avutil/frame: add helper for adding side data to array avutil/frame: add helper for adding existing side data to array avutil/frame: add helper for adding side data w/ AVBufferRef to array avutil/frame: add helper for getting side data from array avcodec: add frame side data array to AVCodecContext avcodec: add helper for configuring AVCodecContext's frame side data ffmpeg: pass first video AVFrame's side data to encoder avcodec/libsvtav1: add support for writing out CLL and MDCV avcodec/libx264: add support for writing out CLL and MDCV avcodec/libx265: add support for writing out CLL and MDCV configure | 2 + fftools/ffmpeg_enc.c | 10 ++ libavcodec/avcodec.c | 30 ++++++ libavcodec/avcodec.h | 28 +++++ libavcodec/libsvtav1.c | 68 ++++++++++++ libavcodec/libx264.c | 79 ++++++++++++++ libavcodec/libx265.c | 88 +++++++++++++++ libavcodec/options.c | 2 + libavutil/Makefile | 1 + libavutil/frame.c | 182 ++++++++++++++++++++++++++------ libavutil/frame.h | 85 +++++++++++++++ libavutil/tests/side_data_set.c | 103 ++++++++++++++++++ tests/fate/enc_external.mak | 15 +++ tests/fate/libavutil.mak | 4 + tests/ref/fate/libsvtav1-hdr10 | 14 +++ tests/ref/fate/libx264-hdr10 | 15 +++ tests/ref/fate/libx265-hdr10 | 16 +++ tests/ref/fate/side_data_set | 14 +++ 18 files changed, 726 insertions(+), 30 deletions(-) create mode 100644 libavutil/tests/side_data_set.c create mode 100644 tests/ref/fate/libsvtav1-hdr10 create mode 100644 tests/ref/fate/libx264-hdr10 create mode 100644 tests/ref/fate/libx265-hdr10 create mode 100644 tests/ref/fate/side_data_set Full diff based on the same base hash for v6->v7: diff --git a/libavutil/frame.c b/libavutil/frame.c index e7679bf34d..fd4b53f9ff 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -889,36 +889,55 @@ AVFrameSideData *av_frame_side_data_new(AVFrameSideData ***sd, int *nb_sd, return ret; } -int av_frame_side_data_from_sd(AVFrameSideData ***sd, int *nb_sd, - const AVFrameSideData *src, - unsigned int flags) +AVFrameSideData *av_frame_side_data_from_buf(AVFrameSideData ***sd, int *nb_sd, + enum AVFrameSideDataType type, + const AVBufferRef *buf, + unsigned int flags) { - if (!sd || !src || !nb_sd || (*nb_sd && !*sd)) - return AVERROR(EINVAL); + if (!sd || !buf || !nb_sd || (*nb_sd && !*sd)) + return NULL; + + if (flags & AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES) + remove_side_data(sd, nb_sd, type); { - AVBufferRef *buf = av_buffer_ref(src->buf); - AVFrameSideData *sd_dst = NULL; + AVBufferRef *new_buf = av_buffer_ref(buf); + AVFrameSideData *sd_dst = NULL; if (flags & AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES) - remove_side_data(sd, nb_sd, src->type); + remove_side_data(sd, nb_sd, type); - sd_dst = add_side_data_to_set_from_buf(sd, nb_sd, src->type, buf); + sd_dst = add_side_data_to_set_from_buf(sd, nb_sd, type, new_buf); if (!sd_dst) { - av_buffer_unref(&buf); - return AVERROR(ENOMEM); + av_buffer_unref(&new_buf); + return NULL; } - { - int ret = av_dict_copy(&sd_dst->metadata, src->metadata, 0); - if (ret < 0) { - remove_side_data_by_entry(sd, nb_sd, sd_dst); - return ret; - } - } + return sd_dst; + } +} - return 0; +int av_frame_side_data_from_sd(AVFrameSideData ***sd, int *nb_sd, + const AVFrameSideData *src, + unsigned int flags) +{ + AVFrameSideData *sd_dst = NULL; + int ret = AVERROR_BUG; + if (!src) + return AVERROR(EINVAL); + + sd_dst = + av_frame_side_data_from_buf(sd, nb_sd, src->type, src->buf, flags); + if (!sd_dst) + return AVERROR(ENOMEM); + + ret = av_dict_copy(&sd_dst->metadata, src->metadata, 0); + if (ret < 0) { + remove_side_data_by_entry(sd, nb_sd, sd_dst); + return ret; } + + return 0; } const AVFrameSideData *av_frame_side_data_get(const AVFrameSideData **sd, diff --git a/libavutil/frame.h b/libavutil/frame.h index 751f7a67d6..01311e68eb 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -1081,6 +1081,26 @@ AVFrameSideData *av_frame_side_data_new(AVFrameSideData ***sd, int *nb_sd, enum AVFrameSideDataType type, size_t size, unsigned int flags); +/** + * Add a new side data entry to a set from an existing AVBufferRef. + * + * @param sd pointer to array of side data to which to add another entry. + * @param nb_sd pointer to an integer containing the number of entries in + * the array. + * @param type type of the added side data + * @param buf AVBufferRef for which a new reference will be made + * @param flags Some combination of AV_FRAME_SIDE_DATA_SET_FLAG_* flags, or 0. + * + * @return newly added side data on success, NULL on error. In case of + * AV_FRAME_SIDE_DATA_SET_FLAG_NO_DUPLICATES being set, entries + * of matching AVFrameSideDataType will be removed before the + * addition is attempted. + */ +AVFrameSideData *av_frame_side_data_from_buf(AVFrameSideData ***sd, int *nb_sd, + enum AVFrameSideDataType type, + const AVBufferRef *buf, + unsigned int flags); + /** * Add a new side data entry to a set based on existing side data, taking * a reference towards the contained AVBufferRef.