From patchwork Fri Apr 26 12:27:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 48269 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1509:b0:1a9:af23:56c1 with SMTP id nq9csp333456pzb; Fri, 26 Apr 2024 05:28:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW7QVKjGeLtRakaHmX6Rd7E5WGIii7NiOO1Y7pz1lUJd4tt020XlIjohK+/jSe7ymJ4sol6u6CZgAzB2KaJocv9eY88CKK4nfNY2A== X-Google-Smtp-Source: AGHT+IHzcVVIve4fR2kqoMnHkIIIINrSZsO9uTa6LG+NtliUHFKNvZx/l2QtfOju2dhL8I5QuyMq X-Received: by 2002:a17:906:f199:b0:a51:d7f3:324b with SMTP id gs25-20020a170906f19900b00a51d7f3324bmr1857782ejb.66.1714134497398; Fri, 26 Apr 2024 05:28:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1714134497; cv=none; d=google.com; s=arc-20160816; b=s1JAKVm9K2N2yO1FvsH/foY1M2qDPyMEtfdJq1au7MqmZm1PaAe9JVBoBZQyg7BUdi i4nZuw1rBCUVRlSqGJIkj4aJvqCJBhuYKxe6QHnz45R2ZBL9GDuE/miVuVjDwnlxeU5O Zf3X2+vAuxQ3KmHtxun0udPGY+3uLrJEFM3j3Gg9R5eOahihMcH6qrJAWCIy8YcswTLc qgI12ZAI7bEfeo5yC0rnw6Ph+i6I3cNvGwRA/UHMFyj5GUb1+Cgeay9GWzbJylW6D5ZZ MDYRBoju7dWJLWxsVhwRMu5730wU51viIHuKoGH3YyKOOY4mINuTi9BBgWAVYAyHdwlo rVFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=Ni8nG7+FFLYgh+UVnTDs+S2qwj5+rMq8fb8KdSjGJ3o=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=LHGLPZJs6Dd3fXBEoRzNIhBCB8VX+NFCGi53uK+7abNp3r2RuX2SypkvSHGuqXJa0q 0qhzcKsISppLbg3XstZzgVMfU72z270cTPQa3BnQkK5Hrp0T2fqzsj5Ux5c3P9bqhtyY wRMlZUJGtKNzQ70IRvb7iWX9/ZdzCOrLF9N4N3bZ+U7Eu3jcXhPsGi26btSCamngZDfY d6jBvxiKxla7OB5joLk7Qu5E+/l0GlIwdtQE2tdjkdhLAlNuNWaEBlfDsguAivzAR26J IzRKRMKSvUA6dKqMrKxHw7xouc/ltLni169V4KvrrXcg231dXyUzMLX5UbOGCkEWyhbv ipwQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=odyi6+5k; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id me22-20020a170906aed600b00a5250f9fcf4si10973339ejb.898.2024.04.26.05.28.16; Fri, 26 Apr 2024 05:28:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=odyi6+5k; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8600B68D438; Fri, 26 Apr 2024 15:28:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C8BCD68D281 for ; Fri, 26 Apr 2024 15:28:06 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1714134486; bh=ZYpM89APXSnj87yfUIJLCykmsQAYo84naHBe5SSwCeE=; h=From:To:Cc:Subject:Date:From; b=odyi6+5kWtJ3as4L2BVDERV4h+RmYhse6D9ZdCAYtT31xhlHZCI8v87p9S7nxfUfq 7ggw7Qp37e8hmOoDFrSQ/MNFHvnMso0EMmXo5EyBFhAWFDg3KxiX/3B38HzPV8alPc RURZX16fEwTpSbJW+/ePnk735wVcX76m9gydNFEk= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 3212340044; Fri, 26 Apr 2024 14:28:06 +0200 (CEST) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 26 Apr 2024 14:27:58 +0200 Message-ID: <20240426122803.19967-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/6] avutil/frame: add av_frame_remove_side_data_changed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: y27Ddu4aSmmY From: Niklas Haas Many filters modify certain aspects of frame data, e.g. through resizing (vf_*scale* family), color volume mapping (vf_lut*, vf_tonemap*), or possibly others. When this happens, we should strip all frame side data that will no longer be correct/relevant after the operation. For example, changing the image size should invalidate AV_FRAME_DATA_PANSCAN because the crop window (given in pixels) no longer corresponds to the actual image size. For another example, tone-mapping filters (e.g. from HDR to SDR) should strip all of the dynamic HDR related metadata. Since there are a lot of similar with basically similar operations, it make sense to consolidate this stripping logic into a common helper function. I decided to put it into libavutil as it may be useful for API users as well, who often have their own internal processing and filtering. --- doc/APIchanges | 3 +++ libavutil/frame.c | 31 +++++++++++++++++++++++++++++++ libavutil/frame.h | 14 ++++++++++++++ libavutil/version.h | 2 +- 4 files changed, 49 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index 0566fcdcc5..26af725528 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-04-xx - xxxxxxxxxx - lavu 59.17.100 - frame.h + Add av_frame_remove_side_data_changed(). + 2024-04-24 - 8616cfe0890 - lavu 59.16.100 - opt.h Add AV_OPT_SERIALIZE_SEARCH_CHILDREN. diff --git a/libavutil/frame.c b/libavutil/frame.c index 0775e2abd9..d5482c258e 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -1015,6 +1015,37 @@ void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) remove_side_data(&frame->side_data, &frame->nb_side_data, type); } +static const struct { + enum AVFrameSideDataType type; + int aspect; +} side_data_aspects[] = { + { AV_FRAME_DATA_PANSCAN, AV_FRAME_CHANGED_SIZE }, + { AV_FRAME_DATA_MOTION_VECTORS, AV_FRAME_CHANGED_SIZE }, + { AV_FRAME_DATA_MASTERING_DISPLAY_METADATA, AV_FRAME_CHANGED_COLOR_VOLUME }, + + { AV_FRAME_DATA_SPHERICAL, AV_FRAME_CHANGED_SIZE }, + { AV_FRAME_DATA_CONTENT_LIGHT_LEVEL, AV_FRAME_CHANGED_COLOR_VOLUME }, + { AV_FRAME_DATA_ICC_PROFILE, AV_FRAME_CHANGED_COLOR_VOLUME }, + { AV_FRAME_DATA_DYNAMIC_HDR_PLUS, AV_FRAME_CHANGED_COLOR_VOLUME }, + { AV_FRAME_DATA_REGIONS_OF_INTEREST, AV_FRAME_CHANGED_SIZE }, + { AV_FRAME_DATA_DETECTION_BBOXES, AV_FRAME_CHANGED_SIZE }, + { AV_FRAME_DATA_DOVI_RPU_BUFFER, AV_FRAME_CHANGED_COLOR_VOLUME }, + { AV_FRAME_DATA_DOVI_METADATA, AV_FRAME_CHANGED_COLOR_VOLUME }, + { AV_FRAME_DATA_DYNAMIC_HDR_VIVID, AV_FRAME_CHANGED_COLOR_VOLUME }, + { AV_FRAME_DATA_VIDEO_HINT, AV_FRAME_CHANGED_SIZE }, +}; + +void av_frame_remove_side_data_changed(AVFrame *frame, int changed_aspects) +{ + if (!changed_aspects) + return; + + for (int i = 0; i < FF_ARRAY_ELEMS(side_data_aspects); i++) { + if (changed_aspects & side_data_aspects[i].aspect) + av_frame_remove_side_data(frame, side_data_aspects[i].type); + } +} + const AVSideDataDescriptor *av_frame_side_data_desc(enum AVFrameSideDataType type) { unsigned t = type; diff --git a/libavutil/frame.h b/libavutil/frame.h index 60bb966f8b..7e07ecf91f 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -983,6 +983,20 @@ AVFrameSideData *av_frame_get_side_data(const AVFrame *frame, */ void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type); +/** + * Flags for stripping side data based on changed aspects. + */ +enum { + /* Video only */ + AV_FRAME_CHANGED_SIZE = 1 << 0, ///< changed dimensions / crop + AV_FRAME_CHANGED_COLOR_VOLUME = 1 << 1, ///< changed color volume +}; + +/** + * Remove all relevant side data after a corresponding change to the frame + * data, based on the given bitset of frame aspects. + */ +void av_frame_remove_side_data_changed(AVFrame *frame, int changed_aspects); /** * Flags for frame cropping. diff --git a/libavutil/version.h b/libavutil/version.h index ea289c406f..3b5a2e7aaa 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 16 +#define LIBAVUTIL_VERSION_MINOR 17 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \