From patchwork Fri Jan 1 21:35:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24760 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B28F6449523 for ; Sat, 2 Jan 2021 00:01:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8277F68A957; Sat, 2 Jan 2021 00:01:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D93A068A898 for ; Sat, 2 Jan 2021 00:01:04 +0200 (EET) Received: by mail-lf1-f46.google.com with SMTP id x20so50723278lfe.12 for ; Fri, 01 Jan 2021 14:01:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=avIye70uFKl+snYbaSpMYeGExgBSvQ14bmlClqH4gj0=; b=kMUgvwmN8SBF4yT3k3N/j6YwbKgWADnUG4UozHho9Ef3cegmCTYBPGKydekjH5O+zl 3FQKuaKbhxiPIxELHYNW9ziU2yHIc6g2iVkky/tYQY0OKhfrT0JZT5j6SAw4CO6Yzopb ODMfe0Ezx2aclbMNC4ie4YT2ZAxl/eKywK1i0uJaV4zPrhcG5IkfyPDY90OTRlqbl6+5 Xd9kGSQPVuy80HcAEBjzoNLx2MAZutJ0VY6AxTCiBr1FI9RAbxXQHv3gGP/s6BHG4nOZ QhOdUX8744uRGsx18x0e4zNm7owmFM8tTH0Gw50jTfD88zqASjRm4w26Q8icpOe7d/Ve UvSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=avIye70uFKl+snYbaSpMYeGExgBSvQ14bmlClqH4gj0=; b=nJILgGeKhGj4SZjEwhAfxlsBF1Oun0da6tbyb48EZ827jMA+wWH05qQYvndqI7xEzo gQE9dbDX+8Cg6DsinTd+siJdPhSoi6jmylY9143zxJxbG0m6z77MVHSe0JPEHvsFcZ0Y SqKB6qmRmxtVY2bhbbWDq5RX5rKikwS8jMH2H/7hheZ3AG+p66Oe5oxyZe8oRzn3ExsW 5AZHwncrg31Fcix6xoiICopp6FsULh710UAE/EywPOd7zeCC4oBOjijLQWJL/J1i/FL2 jG7qyfKVYW0Iv5dre8rYXBcI7QZNKX/az7P5XeLSaJw4QqkJZO1LEJlgkqEq3+cpS2IC 0JYQ== X-Gm-Message-State: AOAM533a11bJe8eBs73T9ZEfGNperC36Rkmy06sMU+A6CMJn7DsChbOc y9Dt4jAA8azmwMGTi6nISHxEw6e3jo8tog== X-Google-Smtp-Source: ABdhPJwEsbMUeKqKIDlZ4ecDEeQ+lmu004qKvAWMIf4/m6BAnq/N/wtFOlQnZ1AAYps0KRDJI4/ckg== X-Received: by 2002:adf:f6c9:: with SMTP id y9mr70604258wrp.121.1609537084933; Fri, 01 Jan 2021 13:38:04 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:04 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:25 +0000 Message-Id: <20210101213537.169546-16-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 15/27] cbs_sei: Implement metadata manipulation for SEI codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 8 ++++ libavcodec/cbs_sei.c | 94 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.h | 35 ++++++++++++++++ 3 files changed, 137 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index a00bc27370..42b614034e 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1478,6 +1478,10 @@ const CodedBitstreamType ff_cbs_type_h264 = { .flush = &cbs_h264_flush, .close = &cbs_h264_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; const CodedBitstreamType ff_cbs_type_h265 = { @@ -1494,6 +1498,10 @@ const CodedBitstreamType ff_cbs_type_h265 = { .flush = &cbs_h265_flush, .close = &cbs_h265_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 323997b600..e5f9e3e403 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -21,6 +21,7 @@ #include "cbs_h264.h" #include "cbs_h265.h" #include "cbs_sei.h" +#include "cbs_metadata.h" static void cbs_free_user_data_registered(void *opaque, uint8_t *data) { @@ -367,3 +368,96 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, } } } + +typedef struct SEIMetadata { + enum CBSMetadataType cbs_type; + int sei_type; +} SEIMetadata; + +static const SEIMetadata cbs_sei_metadata[] = { +}; + +static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata + (CodedBitstreamContext *ctx, enum CBSMetadataType metadata_type) +{ + const SEIMetadata *metadata; + + metadata = NULL; + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_sei_metadata); i++) { + if (metadata_type == cbs_sei_metadata[i].cbs_type) { + metadata = &cbs_sei_metadata[i]; + break; + } + } + if (metadata) + return ff_cbs_sei_find_type(ctx, metadata->sei_type); + else + return NULL; +} + +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + const void *data) +{ + const SEIMessageTypeDescriptor *desc; + AVBufferRef *payload_buf; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->fill) + return AVERROR(EINVAL); + + payload_buf = av_buffer_alloc(desc->size); + if (!payload_buf) + return AVERROR(ENOMEM); + + desc->fill(payload_buf->data, data); + + // All the metadata SEI messages must be unique in an access unit, + // so delete any existing ones of the given type. + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + + err = ff_cbs_sei_add_message(ctx, au, desc->prefix, desc->type, + payload_buf->data, payload_buf); + av_buffer_unref(&payload_buf); + return err; +} + +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type) +{ + const SEIMessageTypeDescriptor *desc; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc) + return AVERROR(EINVAL); + + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + return 0; +} + +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + void *data) +{ + const SEIMessageTypeDescriptor *desc; + SEIRawMessage *message; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->extract) + return AVERROR(EINVAL); + + message = NULL; + err = ff_cbs_sei_find_message(ctx, au, desc->type, &message); + if (err < 0) { + // No message of the given type found. + return err; + } + + desc->extract(data, message->payload); + return 0; +} diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index 5ce4ad3ccd..16fd29265e 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -226,6 +226,8 @@ typedef int (*SEIMessageWriteFunction)(CodedBitstreamContext *ctx, void *current, SEIMessageState *sei); +typedef void (*SEIConvertFunction)(void *dst, const void *src); + typedef struct SEIMessageTypeDescriptor { // Payload type for the message. (-1 in this field ends a list.) int type; @@ -239,6 +241,10 @@ typedef struct SEIMessageTypeDescriptor { SEIMessageReadFunction read; // Write bitstream from SEI message. SEIMessageWriteFunction write; + // Fill SEI structure from AV metadata. + SEIConvertFunction fill; + // Extract SEI metadata to AV structure. + SEIConvertFunction extract; } SEIMessageTypeDescriptor; // Macro for the read/write pair. The clumsy cast is needed because the @@ -248,6 +254,10 @@ typedef struct SEIMessageTypeDescriptor { .read = (SEIMessageReadFunction) cbs_ ## codec ## _read_ ## name, \ .write = (SEIMessageWriteFunction)cbs_ ## codec ## _write_ ## name +#define SEI_MESSAGE_FE(codec, name) \ + .fill = (SEIConvertFunction)cbs_ ## codec ## _fill_ ## name, \ + .extract = (SEIConvertFunction)cbs_ ## codec ## _extract_ ## name + // End-of-list sentinel element. #define SEI_MESSAGE_TYPE_END { .type = -1 } @@ -305,6 +315,8 @@ int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, uint32_t payload_type, SEIRawMessage **message); +enum CBSMetadataType; + /** * Delete all messages with the given payload type from an access unit. */ @@ -312,4 +324,27 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, CodedBitstreamFragment *au, uint32_t payload_type); +/** + * Insert metadata into an access unit. + */ +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + const void *data); + +/** + * Remove metadata from an access unit. + */ +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type); + +/** + * Extract metadata from an access unit. + */ +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + void *data); + #endif /* AVCODEC_CBS_SEI_H */