From patchwork Fri Jan 1 21:35:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24734 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4BC9D44BCD1 for ; Fri, 1 Jan 2021 23:38:06 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2DFDC68A82B; Fri, 1 Jan 2021 23:38:06 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCA8F68A7D4 for ; Fri, 1 Jan 2021 23:38:04 +0200 (EET) Received: by mail-wm1-f41.google.com with SMTP id 3so9960878wmg.4 for ; Fri, 01 Jan 2021 13:38:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=INX/BBJLWJ8elaKRVuhYcemS35q1drgBGFU+eNmt+uw=; b=PYmDFoZ3+3JZGR1m+Lhz+5JnXmJypWwfuzzaJ8CsDNHdkXyadFriZnaTk0Z4b7pAvi 9J8chLL3sbuyHNgnUhgpq+YnrJNDyxHYEoPB9fBscWsbn5Y8yKpTq+fDsmm9ND7AXe60 LNb2vFt05ElEdv0lx7GB/QILjSAztoIo4AY4+S7/wOFfulS+SKH/6z0VKEVA/NbT16E3 XgS87EO+i3SOP1+PkVb/riagE3I7hR29OS8VrDJtpA828S68EMdTvQzuz6eN2MTmn0a4 /Hgxr4DEtCMKaVog9Iioc12tdf80oREr6dG9z5ciwlrHpvRTVGRm50TX96M1orBwhZGr OgLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=INX/BBJLWJ8elaKRVuhYcemS35q1drgBGFU+eNmt+uw=; b=aHl5bNVGB3ICFQ4K1n1zyUbjITTPzPHqxzCM40Es2+CbXUEHNooSnqVkkwiPv6CJgX ghkjQFFXBMvLc395M2LcsfsULmGVNnQGBsQxaxMqjirGCOWGB6k8VJG8q2ijMPQ5aGzM wVaU6Sj9lgrDpmsiVcHMW2YdTjhuheR228G6t4EjGOu9KXA68qMRIzjWyPwspx8VFSCx zGfCmlTzdlps8JnGW+Zy3GiFUKqmTONjpcHTGYNxgG446YbJyvUG5DlZVAQlgr9Ebrx+ PuQ86M5MzeF1E/G5dPxJC8ZwtWHcmxr7rll7IOu/MM28wf2CncZApYkOEtUq72bmpEl9 Dk7A== X-Gm-Message-State: AOAM53130HprDA9kZjUuvBwF+OTZCR+6BNMo+Q08SEDFbMQ+xSydDLkM aYPHI1t4LpZTHYOPrTSO6SAog5Lks3gVWQ== X-Google-Smtp-Source: ABdhPJxPovK1mRU7DJZjttCa+d1bSx8bYr300mIP6mD8A5hDP0CUgycflps+pLp+pI8h9IPPS6QIiQ== X-Received: by 2002:a7b:c4d5:: with SMTP id g21mr17207614wmk.92.1609537084046; Fri, 01 Jan 2021 13:38:04 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:03 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:24 +0000 Message-Id: <20210101213537.169546-15-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 14/27] cbs: Add interface for manipulating metadata in fragments X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/Makefile | 2 +- libavcodec/cbs_internal.h | 17 ++++++ libavcodec/cbs_metadata.c | 107 ++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_metadata.h | 94 +++++++++++++++++++++++++++++++++ 4 files changed, 219 insertions(+), 1 deletion(-) create mode 100644 libavcodec/cbs_metadata.c create mode 100644 libavcodec/cbs_metadata.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 8f50217ad4..9b3f3ecdf7 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -69,7 +69,7 @@ OBJS-$(CONFIG_AUDIODSP) += audiodsp.o OBJS-$(CONFIG_BLOCKDSP) += blockdsp.o OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o -OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o +OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o cbs_metadata.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index faa847aad3..a505080ccf 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -83,6 +83,8 @@ typedef const struct CodedBitstreamUnitTypeDescriptor { int (*content_clone)(AVBufferRef **ref, CodedBitstreamUnit *unit); } CodedBitstreamUnitTypeDescriptor; +enum CBSMetadataType; + typedef struct CodedBitstreamType { enum AVCodecID codec_id; @@ -122,6 +124,21 @@ typedef struct CodedBitstreamType { // Free the codec internal state. void (*close)(CodedBitstreamContext *ctx); + + // Insert metadata to fragment. + int (*insert_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data); + + // Remove metadata from fragment. + int (*remove_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type); + + // Extract metadata from fragment. + int (*extract_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data); } CodedBitstreamType; diff --git a/libavcodec/cbs_metadata.c b/libavcodec/cbs_metadata.c new file mode 100644 index 0000000000..140f5bcd9c --- /dev/null +++ b/libavcodec/cbs_metadata.c @@ -0,0 +1,107 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/film_grain_params.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/stereo3d.h" + +#include "cbs.h" +#include "cbs_internal.h" +#include "cbs_metadata.h" + +static const CBSMetadataTypeDescriptor cbs_metadata_types[] = { + { + CBS_METADATA_MASTERING_DISPLAY, + "Mastering Display", + 1, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA, + 1, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, + }, + { + CBS_METADATA_CONTENT_LIGHT_LEVEL, + "Content Light Level", + 1, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL, + 1, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, + }, + { + CBS_METADATA_DISPLAY_MATRIX, + "Display Matrix", + 1, AV_FRAME_DATA_DISPLAYMATRIX, + 1, AV_PKT_DATA_DISPLAYMATRIX, + }, + { + CBS_METADATA_STEREO3D, + "Stereo 3D", + 1, AV_FRAME_DATA_STEREO3D, + 1, AV_PKT_DATA_STEREO3D, + }, +}; + +const CBSMetadataTypeDescriptor *ff_cbs_metadata_find_type(enum CBSMetadataType type) +{ + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_metadata_types); i++) { + if (cbs_metadata_types[i].type == type) + return &cbs_metadata_types[i]; + } + return NULL; +} + +void *ff_cbs_alloc_metadata(enum CBSMetadataType type, size_t *size) +{ + switch (type) { + case CBS_METADATA_MASTERING_DISPLAY: + *size = sizeof(AVMasteringDisplayMetadata); + return av_mastering_display_metadata_alloc(); + case CBS_METADATA_CONTENT_LIGHT_LEVEL: + return av_content_light_metadata_alloc(size); + case CBS_METADATA_DISPLAY_MATRIX: + *size = 9 * sizeof(int32_t); + return av_mallocz(*size); + case CBS_METADATA_STEREO3D: + *size = sizeof(AVStereo3D); + return av_stereo3d_alloc(); + default: + return NULL; + } +} + +int ff_cbs_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data) +{ + if (!ctx->codec->insert_metadata) + return AVERROR(ENOSYS); + return ctx->codec->insert_metadata(ctx, frag, type, data); +} + +int ff_cbs_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type) +{ + if (!ctx->codec->remove_metadata) + return AVERROR(ENOSYS); + return ctx->codec->remove_metadata(ctx, frag, type); +} + +int ff_cbs_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data) +{ + if (!ctx->codec->extract_metadata) + return AVERROR(ENOSYS); + return ctx->codec->extract_metadata(ctx, frag, type, data); +} diff --git a/libavcodec/cbs_metadata.h b/libavcodec/cbs_metadata.h new file mode 100644 index 0000000000..79ba6f4e26 --- /dev/null +++ b/libavcodec/cbs_metadata.h @@ -0,0 +1,94 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_METADATA_H +#define AVCODEC_CBS_METADATA_H + +#include "libavutil/frame.h" + +#include "cbs.h" +#include "packet.h" + + +enum CBSMetadataType { + // List sentinel element. + CBS_METADATA_NONE, + // AVMasteringDisplayMetadata. + CBS_METADATA_MASTERING_DISPLAY, + // AVContentLightMetadata. + CBS_METADATA_CONTENT_LIGHT_LEVEL, + // int32_t matrix[9]. + CBS_METADATA_DISPLAY_MATRIX, + // AVStereo3D. + CBS_METADATA_STEREO3D, +}; + +typedef struct CBSMetadataTypeDescriptor { + enum CBSMetadataType type; + const char *name; + + // Matching frame and packet side data types, if available. + int has_frame_side_data; + enum AVFrameSideDataType frame_side_data_type; + int has_packet_side_data; + enum AVPacketSideDataType packet_side_data_type; +} CBSMetadataTypeDescriptor; + +/** + * Retrieve the descriptor for the given metadata type. + */ +const CBSMetadataTypeDescriptor *ff_cbs_metadata_find_type(enum CBSMetadataType type); + +/** + * Allocates an object for the given metadata type. + * + * If not null, size is written with the size of the object returned. + */ +void *ff_cbs_alloc_metadata(enum CBSMetadataType type, size_t *size); + +/** + * Insert metadata into a fragment. + * + * type is a CBS_METADATA_* value. data must be a pointer to a filled + * structure matching the specified type. + */ +int ff_cbs_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data); + +/** + * Remove metadata from a fragment. + * + * Remove all metadata of the given CBS_METADATA_* type. + */ +int ff_cbs_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type); + +/** + * Extract metadata from a fragment. + * + * type is a CBS_METADATA_* value. data must be a pointer to an empty + * structure matching the specified type, which will be filled on + * success. + */ +int ff_cbs_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data); + +#endif /* AVCODEC_CBS_METADATA_H */