From patchwork Wed Mar 6 13:58:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 46853 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d90e:b0:19e:cdac:8cce with SMTP id jd14csp400948pzb; Wed, 6 Mar 2024 05:59:26 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWGS0rqmydj1Gy8vnaKNH+g49rFSSeR5Mr4AAH0hFpz0sxT4f0d4jfDf4qK9zGOV1hL02Qy6Y6xCTMIUctyon3PFNAEB5z988kdEg== X-Google-Smtp-Source: AGHT+IGni2E0HEpeqigDp7+zbHr81ekGkfuM1+aYFP/hSjFzV/7L9Ies90amaG0V/Fz4w9w2vHpx X-Received: by 2002:a17:906:2448:b0:a45:3308:560f with SMTP id a8-20020a170906244800b00a453308560fmr6834482ejb.64.1709733566586; Wed, 06 Mar 2024 05:59:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1709733566; cv=none; d=google.com; s=arc-20160816; b=N/ZZp24t/pc+WYIL6wA3qhisex+dalc9ZwWl89BU8knzVE+RfREnVgFk7el8i8sSRG CCk+DToisWbNtMJX3b4degJCaTDwggV04uIUUA/Yr6yVWax+MSJMl/UjE0UYN+KN1liP WHkOW5N/a9CkqQlFpVDN9bcgwmxwFJFGBOhSy9NAMsVNXdJCGKIRGGzYGxoXZ18FIaCO 8WhzU3SZZGRG2CeJ3FnSEEeAplOZN2WGfLT1JnhlmHwreGmDpMZewsl2/FmiQGET17kS XGahn4MfBnZcfWr7PqMTcDieGrEDzE4F3ypDiRaWnfzG+NvOVE8/1mLkYb21hlo3csJf TPaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=GLjnezPamNdOlzaz6KMn3Fg48Q+rDSjzNTyIMBssq/U=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=sA5P46ABiYehMKRhQJy8tmM36RO2W/ObhSKsQn3fEwno2nfybVggBYQZ26JBcQWBw3 qf3ZmZYWE8z0tz3jN7dxaErbv+FkyoFbyRrKQtAZV5LEWxxRqTdgLvlo0bO5/5JL/uV0 nZT5qsDzhyPCXjF2NUF0mb4w3zkSPzY/EGhnxbb/3r8ijkDo/sl6zcSFfbK80zB7aVfW BKSkJ2NtEuUwbchwfbHGPsFjfB/pHSMdQgb3NGXb1qO8/BiGv/U7ca23vnRSLyADLKtu fnPVXffz7YkRz81bHlGah1d131PNs9CgTuP4TiswfTWh6WL6Lasy4MdGdwlq4l0we+P5 cQ0g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=jZAvjVm6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m16-20020a1709066d1000b00a4588098c52si2116337ejr.235.2024.03.06.05.59.26; Wed, 06 Mar 2024 05:59:26 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=jZAvjVm6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1532668C2B5; Wed, 6 Mar 2024 15:59:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7D37468C2B5 for ; Wed, 6 Mar 2024 15:59:16 +0200 (EET) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=jZAvjVm6; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id D8BF04D41 for ; Wed, 6 Mar 2024 14:59:15 +0100 (CET) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id MxYxVNueoZsz for ; Wed, 6 Mar 2024 14:59:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1709733553; bh=NkCscfWCKc0k4HXqh/ZV961ZV8OT6SMkiR+b43f48Fc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=jZAvjVm6yXoaCdP8tTB9RVa0P9vqUX8iRfgcMdfxP19QvvoVZcDlpt76LvKOJSaja +8muFGyo0E824HA/jB4gyVCfPgQg/qB4IM7Ylg+QDGXUakFnNQEJiTs7FMD74g/io3 kxzMGYgw+J8os6PpTFaO1Yu1TdxL1L/w/eSDnXOHVoZM/Hhh/rLlH9D6EQwIAmehxF 3QndfBG7qj8+GF9vTqbA+6a8SVpWZxhovaXA8NmMYTkq8TcMZ45sSsy6w5POjEwlBI NCg50kIE1QwVOe5pzPQywnSO3tpwCV9hIRP6Eagb+SAt3kxctEbgbBaQd/gq54PLs1 U8SG6IKt1nVMA== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 604284482 for ; Wed, 6 Mar 2024 14:59:13 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 341763A0357 for ; Wed, 6 Mar 2024 14:59:13 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Mar 2024 14:58:15 +0100 Message-ID: <20240306135845.21011-2-anton@khirnov.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240304130657.30631-7-anton@khirnov.net> References: <20240304130657.30631-7-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/29] lavc: add a decoder option for configuring side data preference X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: r4OTahi4+v+l This and the following commits fix #10857 --- Renamed the option to 'side_data_prefer_packet', as preferred by James and Derek on IRC. --- doc/APIchanges | 3 +++ libavcodec/avcodec.h | 31 +++++++++++++++++++++++++++++++ libavcodec/decode.c | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/options_table.h | 12 ++++++++++++ 4 files changed, 82 insertions(+) diff --git a/doc/APIchanges b/doc/APIchanges index a0898dd701..83b24e5b37 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09 API changes, most recent first: +2024-02-xx - xxxxxxxxxx - lavc 60.xx.100 - avcodec.h + Add AVCodecContext.[nb_]side_data_prefer_packet. + 2024-02-xx - xxxxxxxxxx - lavu 58.xx.100 - opt.h Add AV_OPT_TYPE_FLAG_ARRAY and AVOptionArrayDef. diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 8aa1356b8c..2e61bef19b 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -2120,6 +2120,37 @@ typedef struct AVCodecContext { * an error. */ int64_t frame_num; + + /** + * Decoding only. May be set by the caller before avcodec_open2() to an + * av_malloc()'ed array (or via AVOptions). Owned and freed by the decoder + * afterwards. + * + * Side data attached to decoded frames may come from several sources: + * - coded_side_data, which the decoder will for certain types translate + * from packet-type to frame-type and attach to frames; + * - user-supplied side data attached to an AVPacket sent for decoding (same + * considerations as above); + * - extracted from the coded bytestream. + * The first two cases are supplied by the caller and typically come from a + * container. + * + * This array configures decoder behaviour in case when some side data type + * is present both in the coded bytestream and in the user-supplied side + * data. In all cases, at most one instance of each side data type will be + * attached to output frames. By default it will be the bytestream side + * data. Adding a AVPacketSideDataType value to this array will flip the + * preference for this type, thus making the decoder prefer user-supplied + * side data over bytestream. + * + * The array may also contain a single -1, in which case the preference is + * switched for all side data types. + */ + int *side_data_prefer_packet; + /** + * Number of entries in side_data_prefer_packet. + */ + unsigned nb_side_data_prefer_packet; } AVCodecContext; /** diff --git a/libavcodec/decode.c b/libavcodec/decode.c index cd483528c5..29fe840db7 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -60,6 +60,12 @@ typedef struct DecodeContext { * The caller has submitted a NULL packet on input. */ int draining_started; + + /** + * Bitmask indicating for which side data types we prefer user-supplied + * (global or attached to packets) side data over bytestream. + */ + uint64_t side_data_pref_mask; } DecodeContext; static DecodeContext *decode_ctx(AVCodecInternal *avci) @@ -1744,6 +1750,7 @@ int ff_reget_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) int ff_decode_preinit(AVCodecContext *avctx) { AVCodecInternal *avci = avctx->internal; + DecodeContext *dc = decode_ctx(avci); int ret = 0; /* if the decoder init function was already called previously, @@ -1804,6 +1811,35 @@ int ff_decode_preinit(AVCodecContext *avctx) avctx->export_side_data |= AV_CODEC_EXPORT_DATA_MVS; } + if (avctx->nb_side_data_prefer_packet == 1 && + avctx->side_data_prefer_packet[0] == -1) + dc->side_data_pref_mask = ~0ULL; + else { + for (unsigned i = 0; i < avctx->nb_side_data_prefer_packet; i++) { + int val = avctx->side_data_prefer_packet[i]; + + if (val < 0 || val >= AV_PKT_DATA_NB) { + av_log(avctx, AV_LOG_ERROR, "Invalid side data type: %d\n", val); + return AVERROR(EINVAL); + } + + for (unsigned j = 0; j < FF_ARRAY_ELEMS(sd_global_map); j++) { + if (sd_global_map[j].packet == val) { + val = sd_global_map[j].frame; + + // this code will need to be changed when we have more than + // 64 frame side data types + if (val >= 64) { + av_log(avctx, AV_LOG_ERROR, "Side data type too big\n"); + return AVERROR_BUG; + } + + dc->side_data_pref_mask |= 1ULL << val; + } + } + } + } + avci->in_pkt = av_packet_alloc(); avci->last_pkt_props = av_packet_alloc(); if (!avci->in_pkt || !avci->last_pkt_props) diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index ac32d8928a..55088bd2e4 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -42,6 +42,8 @@ #define D AV_OPT_FLAG_DECODING_PARAM #define CC AV_OPT_FLAG_CHILD_CONSTS +#define AR AV_OPT_TYPE_FLAG_ARRAY + #define AV_CODEC_DEFAULT_BITRATE 200*1000 static const AVOption avcodec_options[] = { @@ -405,6 +407,16 @@ static const AVOption avcodec_options[] = { {"unsafe_output", "allow potentially unsafe hwaccel frame output that might require special care to process successfully", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_UNSAFE_OUTPUT }, INT_MIN, INT_MAX, V | D, .unit = "hwaccel_flags"}, {"extra_hw_frames", "Number of extra hardware frames to allocate for the user", OFFSET(extra_hw_frames), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, V|D }, {"discard_damaged_percentage", "Percentage of damaged samples to discard a frame", OFFSET(discard_damaged_percentage), AV_OPT_TYPE_INT, {.i64 = 95 }, 0, 100, V|D }, +{"side_data_prefer_packet", "Comma-separated list of side data types for which user-supplied (container) data is preferred over coded bytestream", + OFFSET(side_data_prefer_packet), AV_OPT_TYPE_INT | AR, .min = -1, .max = INT_MAX, .flags = V|A|S|D, .unit = "side_data_pkt" }, + {"replaygain", .default_val.i64 = AV_PKT_DATA_REPLAYGAIN, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"displaymatrix", .default_val.i64 = AV_PKT_DATA_DISPLAYMATRIX, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"spherical", .default_val.i64 = AV_PKT_DATA_SPHERICAL, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"stereo3d", .default_val.i64 = AV_PKT_DATA_STEREO3D, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"audio_service_type", .default_val.i64 = AV_PKT_DATA_AUDIO_SERVICE_TYPE, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"mastering_display_metadata", .default_val.i64 = AV_PKT_DATA_MASTERING_DISPLAY_METADATA, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"content_light_level", .default_val.i64 = AV_PKT_DATA_CONTENT_LIGHT_LEVEL, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, + {"icc_profile", .default_val.i64 = AV_PKT_DATA_ICC_PROFILE, .type = AV_OPT_TYPE_CONST, .flags = A|D, .unit = "side_data_pkt" }, {NULL}, };