From patchwork Sun Feb 21 19:51:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25866 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AF726449DB5 for ; Sun, 21 Feb 2021 21:53:42 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9899968A87F; Sun, 21 Feb 2021 21:53:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 20A01688050 for ; Sun, 21 Feb 2021 21:53:33 +0200 (EET) Received: by mail-wr1-f45.google.com with SMTP id v15so16911762wrx.4 for ; Sun, 21 Feb 2021 11:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XIaGDBhNrSWCctxGM2T0020/FVJGHMEV90lsGnxNKwM=; b=dkeqzDUNH5sabX4fZqYp81wLMw0dsD2bnQuNNqw+nn4JVn5TECzJfb0vlhbBKqFAL4 7uXejzquVLLpMbfglBjQzFL7nyteeYT2JhoG+9/U9Xsz0NL/aDRrU1FBAIWPB+YKgiiF TdL4UDNX7lUPTKoPSHzCYtByJCvNXIqg+ccd9b9Gy+yAsWSHGdjcMJHO5Lvw8NySzUik sG7n9mFRpXLk+7fCcbLJylsPWR5HVrsYpwOFR3/Q5zbAVlFLFnfU+N9i1Jb/wgw2kq9L RSJJhRHjVSqzN1zxV4cogzXkICcQ50yF6gsfkhK6gps2aIti09fOGO5kwUVmgjuGJMf6 lDlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XIaGDBhNrSWCctxGM2T0020/FVJGHMEV90lsGnxNKwM=; b=r+2CrJfaZSQIEo4dBd2wuxj/TQuniLH7ZoXB7cjTLwp7tKuT1xkhYaTQjZtMwzcIiu IKOVF4468MQWkyq9Xl4Rs5oM1iruHNsHuSiiOlpT5SbeHjczEUuktMqEC2v0/WMJf9rP bDm1QFr6ubszxk6kpCfsz6XRr4jo2AUHdrpjgU12jBXvx964B8otBHPU42xumXpkssMW HjWRJKTzgMnJNH36DRLk371BsyR6kMpUrPiu7krVMyd0bBJPbQ2+Q9G2ly6KZzjvxf67 Ahp29IlDjp+9t4fC0i1wsUJ7evqh/ybEfDo6ZUUq0vpkYKW1lBTB6TnLPh+muTu27J7G b5fQ== X-Gm-Message-State: AOAM533D0N61ivAiU+5xyqVZiSYok2QbYLI12OCAxjLK8BkZzUUOrq9D TGp2KwY9nkfcvFAKk5CrGxYH9dfYrq7VqQ== X-Google-Smtp-Source: ABdhPJxV+Rykw3vw0k3HuynNEeD5xkfJ7CsSj2yVPVpKdW6Y076xD2qcWMPASTHyTDpN1mj60WfE1Q== X-Received: by 2002:a5d:4842:: with SMTP id n2mr6715966wrs.181.1613937212454; Sun, 21 Feb 2021 11:53:32 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:32 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:11 +0000 Message-Id: <20210221195125.1901683-4-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/18] cbs_sei: Implement metadata manipulation for SEI codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 8 ++++ libavcodec/cbs_sei.c | 94 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.h | 35 ++++++++++++++++ 3 files changed, 137 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 0c591871d4..cb0005cc1b 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1487,6 +1487,10 @@ const CodedBitstreamType ff_cbs_type_h264 = { .flush = &cbs_h264_flush, .close = &cbs_h264_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; const CodedBitstreamType ff_cbs_type_h265 = { @@ -1503,6 +1507,10 @@ const CodedBitstreamType ff_cbs_type_h265 = { .flush = &cbs_h265_flush, .close = &cbs_h265_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 14f1cae506..83bb5cca55 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -21,6 +21,7 @@ #include "cbs_h264.h" #include "cbs_h265.h" #include "cbs_sei.h" +#include "cbs_metadata.h" static void cbs_free_user_data_registered(void *opaque, uint8_t *data) { @@ -372,3 +373,96 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, } } } + +typedef struct SEIMetadata { + enum CBSMetadataType cbs_type; + int sei_type; +} SEIMetadata; + +static const SEIMetadata cbs_sei_metadata[] = { +}; + +static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata + (CodedBitstreamContext *ctx, enum CBSMetadataType metadata_type) +{ + const SEIMetadata *metadata; + + metadata = NULL; + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_sei_metadata); i++) { + if (metadata_type == cbs_sei_metadata[i].cbs_type) { + metadata = &cbs_sei_metadata[i]; + break; + } + } + if (metadata) + return ff_cbs_sei_find_type(ctx, metadata->sei_type); + else + return NULL; +} + +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + const void *data) +{ + const SEIMessageTypeDescriptor *desc; + AVBufferRef *payload_buf; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->fill) + return AVERROR(EINVAL); + + payload_buf = av_buffer_alloc(desc->size); + if (!payload_buf) + return AVERROR(ENOMEM); + + desc->fill(payload_buf->data, data); + + // All the metadata SEI messages must be unique in an access unit, + // so delete any existing ones of the given type. + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + + err = ff_cbs_sei_add_message(ctx, au, desc->prefix, desc->type, + payload_buf->data, payload_buf); + av_buffer_unref(&payload_buf); + return err; +} + +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type) +{ + const SEIMessageTypeDescriptor *desc; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc) + return AVERROR(EINVAL); + + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + return 0; +} + +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + void *data) +{ + const SEIMessageTypeDescriptor *desc; + SEIRawMessage *message; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->extract) + return AVERROR(EINVAL); + + message = NULL; + err = ff_cbs_sei_find_message(ctx, au, desc->type, &message); + if (err < 0) { + // No message of the given type found. + return err; + } + + desc->extract(data, message->payload); + return 0; +} diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index c7a7a95be0..4fbcb6bfc0 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -110,6 +110,8 @@ typedef int (*SEIMessageWriteFunction)(CodedBitstreamContext *ctx, void *current, SEIMessageState *sei); +typedef void (*SEIConvertFunction)(void *dst, const void *src); + typedef struct SEIMessageTypeDescriptor { // Payload type for the message. (-1 in this field ends a list.) int type; @@ -123,6 +125,10 @@ typedef struct SEIMessageTypeDescriptor { SEIMessageReadFunction read; // Write bitstream from SEI message. SEIMessageWriteFunction write; + // Fill SEI structure from AV metadata. + SEIConvertFunction fill; + // Extract SEI metadata to AV structure. + SEIConvertFunction extract; } SEIMessageTypeDescriptor; // Macro for the read/write pair. The clumsy cast is needed because the @@ -132,6 +138,10 @@ typedef struct SEIMessageTypeDescriptor { .read = (SEIMessageReadFunction) cbs_ ## codec ## _read_ ## name, \ .write = (SEIMessageWriteFunction)cbs_ ## codec ## _write_ ## name +#define SEI_MESSAGE_FE(codec, name) \ + .fill = (SEIConvertFunction)cbs_ ## codec ## _fill_ ## name, \ + .extract = (SEIConvertFunction)cbs_ ## codec ## _extract_ ## name + // End-of-list sentinel element. #define SEI_MESSAGE_TYPE_END { .type = -1 } @@ -189,6 +199,8 @@ int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, uint32_t payload_type, SEIRawMessage **message); +enum CBSMetadataType; + /** * Delete all messages with the given payload type from an access unit. */ @@ -196,4 +208,27 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, CodedBitstreamFragment *au, uint32_t payload_type); +/** + * Insert metadata into an access unit. + */ +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + const void *data); + +/** + * Remove metadata from an access unit. + */ +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type); + +/** + * Extract metadata from an access unit. + */ +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + void *data); + #endif /* AVCODEC_CBS_SEI_H */