From patchwork Mon Jan 18 22:44:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25021 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id BDB4E44B852 for ; Tue, 19 Jan 2021 00:53:46 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A507A688291; Tue, 19 Jan 2021 00:53:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 51C166881EE for ; Tue, 19 Jan 2021 00:53:43 +0200 (EET) Received: by mail-lf1-f50.google.com with SMTP id o17so26381848lfg.4 for ; Mon, 18 Jan 2021 14:53:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=N8DAta2XqXoQJxU1SOGu+wrVvmG+chfz0W0nKcP52es=; b=Lascv03O0unJexno/DlJ19BxYZr05mG3aFcL/yJPDevWDqfOums1o4ZyFlaG7ym/LL 9lb66qeBIjpCgJo1Zi8wTw04cGWiikX4vebnNHPyT53piub4Eb0oV2rCb2y19qGhX8Rc XFWDNysKKbFWiihaoF/2HmxU/g7tmiUGdbP30ddT76xdasMd7lLpZgjQrZqK4TpAgTHP 7tF78AkhZoZgGndAH+Pgp20kIKItpAQoetZtuQUlPE6u/LCNxeuXzG9+92+sUccI/8VZ 2aXMi9MosVQKMZg3LulMJRz7vLEOo32XuvXAtuem/X7UsnPjVSY/rPEhR51vCeuQsed8 TTNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N8DAta2XqXoQJxU1SOGu+wrVvmG+chfz0W0nKcP52es=; b=Qm3uJZpZtJBf/MIuDIhNJGYTB7im6yeh6W0aPwaymFgUVyu6bRTKcmshZoXNgvdIIi QTyMwwvX9paXOTOW92zLJPfcIcvYXTdzHwn6kaUHGX+M0mPU5FHiWMhfPJD2P6Dis7P8 FYb8Z1l4gf6koVnmEKVshIqdUlh5IZZ4LD6JooMh7PkpheJToQ5cvoE70yxxInfoGh8G T7g01ihpedDOrG0hUHp4HN/+YUOzaXJWQM59kc4cD1euCSazqqvAG5r+0GRsBLcfs0uF WvziLOJpCx8d+VcDVCL/DXiwr7r1xGtk05B9a4cREDik/R6sZwqD0VkzrA/p3Yf2/4ix s3Yg== X-Gm-Message-State: AOAM530X2sSd8byfMwZs7y+LuVz5uLfbCHVyViTZEdPlqw/Z7na3TsYq bC0lcFRgB3QHpVldOSqACXMPFfbwYGdNuw== X-Google-Smtp-Source: ABdhPJwDgf2SMnSX7LzulEiDNu7AnRqMl4nqBFEaiw1kbPuIJDB0IWcaAnZrzR9O36aKDAqluSPThg== X-Received: by 2002:a5d:69cf:: with SMTP id s15mr1464531wrw.372.1611010025578; Mon, 18 Jan 2021 14:47:05 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id b3sm1525898wme.32.2021.01.18.14.47.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jan 2021 14:47:04 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Mon, 18 Jan 2021 22:44:45 +0000 Message-Id: <20210118224455.750030-4-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210118224455.750030-1-sw@jkqxz.net> References: <20210118224455.750030-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/13] cbs_h2645: Merge SEI message handling in common between codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/Makefile | 4 +- libavcodec/cbs_h264.h | 50 +--- libavcodec/cbs_h2645.c | 300 +++++++++++---------- libavcodec/cbs_h264_syntax_template.c | 173 +----------- libavcodec/cbs_h265.h | 33 +-- libavcodec/cbs_h265_syntax_template.c | 271 +++---------------- libavcodec/cbs_sei.c | 369 ++++++++++++++++++++++++++ libavcodec/cbs_sei.h | 255 ++++++++++++++++++ libavcodec/cbs_sei_syntax_template.c | 215 +++++++++++++-- libavcodec/h264_metadata_bsf.c | 113 ++++---- libavcodec/vaapi_encode_h264.c | 51 ++-- libavcodec/vaapi_encode_h265.c | 38 +-- 12 files changed, 1107 insertions(+), 765 deletions(-) create mode 100644 libavcodec/cbs_sei.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 446e6e6b3b..1fcb94fa48 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -71,8 +71,8 @@ OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o OBJS-$(CONFIG_CBS) += cbs.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o -OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o h2645_parse.o -OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o h2645_parse.o +OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o +OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index 81113f1ad0..9eb97eae24 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -291,34 +291,9 @@ typedef struct H264RawSEIDisplayOrientation { uint8_t display_orientation_extension_flag; } H264RawSEIDisplayOrientation; -typedef struct H264RawSEIPayload { - uint32_t payload_type; - uint32_t payload_size; - union { - H264RawSEIBufferingPeriod buffering_period; - H264RawSEIPicTiming pic_timing; - H264RawSEIPanScanRect pan_scan_rect; - // H264RawSEIFiller filler -> no fields. - SEIRawUserDataRegistered user_data_registered; - SEIRawUserDataUnregistered user_data_unregistered; - H264RawSEIRecoveryPoint recovery_point; - H264RawSEIDisplayOrientation display_orientation; - SEIRawMasteringDisplayColourVolume mastering_display_colour_volume; - SEIRawAlternativeTransferCharacteristics - alternative_transfer_characteristics; - struct { - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; - } other; - } payload; -} H264RawSEIPayload; - typedef struct H264RawSEI { H264RawNALUnitHeader nal_unit_header; - - H264RawSEIPayload payload[H264_MAX_SEI_PAYLOADS]; - uint8_t payload_count; + SEIRawMessageList message_list; } H264RawSEI; typedef struct H264RawSliceHeader { @@ -438,27 +413,4 @@ typedef struct CodedBitstreamH264Context { uint8_t last_slice_nal_unit_type; } CodedBitstreamH264Context; - -/** - * Add an SEI message to an access unit. - * - * On success, the payload will be owned by a unit in access_unit; - * on failure, the content of the payload will be freed. - */ -int ff_cbs_h264_add_sei_message(CodedBitstreamFragment *access_unit, - H264RawSEIPayload *payload); - -/** - * Delete an SEI message from an access unit. - * - * Deletes from nal_unit, which must be an SEI NAL unit. If this is the - * last message in nal_unit, also deletes it from access_unit. - * - * Requires nal_unit to be a unit in access_unit and position to be >= 0 - * and < the payload count of the SEI nal_unit. - */ -void ff_cbs_h264_delete_sei_message(CodedBitstreamFragment *access_unit, - CodedBitstreamUnit *nal_unit, - int position); - #endif /* AVCODEC_CBS_H264_H */ diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 9e210abba4..6005d46e0d 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -348,6 +348,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #define more_rbsp_data(var) ((var) = cbs_h2645_read_more_rbsp_data(rw)) +#define bit_position(rw) (get_bits_count(rw)) #define byte_alignment(rw) (get_bits_count(rw) % 8) #define allocate(name, size) do { \ @@ -379,6 +380,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef xse #undef infer #undef more_rbsp_data +#undef bit_position #undef byte_alignment #undef allocate @@ -424,6 +426,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #define more_rbsp_data(var) (var) +#define bit_position(rw) (put_bits_count(rw)) #define byte_alignment(rw) (put_bits_count(rw) % 8) #define allocate(name, size) do { \ @@ -460,6 +463,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef se #undef infer #undef more_rbsp_data +#undef bit_position #undef byte_alignment #undef allocate @@ -1381,36 +1385,11 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } -static void cbs_h264_free_sei_payload(H264RawSEIPayload *payload) -{ - switch (payload->payload_type) { - case H264_SEI_TYPE_BUFFERING_PERIOD: - case H264_SEI_TYPE_PIC_TIMING: - case H264_SEI_TYPE_PAN_SCAN_RECT: - case H264_SEI_TYPE_RECOVERY_POINT: - case H264_SEI_TYPE_DISPLAY_ORIENTATION: - case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: - case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: - break; - case H264_SEI_TYPE_USER_DATA_REGISTERED: - av_buffer_unref(&payload->payload.user_data_registered.data_ref); - break; - case H264_SEI_TYPE_USER_DATA_UNREGISTERED: - av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); - break; - default: - av_buffer_unref(&payload->payload.other.data_ref); - break; - } -} - static void cbs_h264_free_sei(void *opaque, uint8_t *content) { H264RawSEI *sei = (H264RawSEI*)content; - int i; - for (i = 0; i < sei->payload_count; i++) - cbs_h264_free_sei_payload(&sei->payload[i]); - av_freep(&content); + ff_cbs_sei_free_message_list(&sei->message_list); + av_free(content); } static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { @@ -1442,42 +1421,11 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; -static void cbs_h265_free_sei_payload(H265RawSEIPayload *payload) -{ - switch (payload->payload_type) { - case HEVC_SEI_TYPE_BUFFERING_PERIOD: - case HEVC_SEI_TYPE_PICTURE_TIMING: - case HEVC_SEI_TYPE_PAN_SCAN_RECT: - case HEVC_SEI_TYPE_RECOVERY_POINT: - case HEVC_SEI_TYPE_DISPLAY_ORIENTATION: - case HEVC_SEI_TYPE_ACTIVE_PARAMETER_SETS: - case HEVC_SEI_TYPE_DECODED_PICTURE_HASH: - case HEVC_SEI_TYPE_TIME_CODE: - case HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO: - case HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO: - case HEVC_SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS: - case HEVC_SEI_TYPE_ALPHA_CHANNEL_INFO: - break; - case HEVC_SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35: - av_buffer_unref(&payload->payload.user_data_registered.data_ref); - break; - case HEVC_SEI_TYPE_USER_DATA_UNREGISTERED: - av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); - break; - default: - av_buffer_unref(&payload->payload.other.data_ref); - break; - } - av_buffer_unref(&payload->extension_data.data_ref); -} - static void cbs_h265_free_sei(void *opaque, uint8_t *content) { H265RawSEI *sei = (H265RawSEI*)content; - int i; - for (i = 0; i < sei->payload_count; i++) - cbs_h265_free_sei_payload(&sei->payload[i]); - av_freep(&content); + ff_cbs_sei_free_message_list(&sei->message_list); + av_free(content); } static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { @@ -1557,92 +1505,164 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; -int ff_cbs_h264_add_sei_message(CodedBitstreamFragment *au, - H264RawSEIPayload *payload) -{ - H264RawSEI *sei = NULL; - int err, i; - - // Find an existing SEI NAL unit to add to. - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SEI) { - sei = au->units[i].content; - if (sei->payload_count < H264_MAX_SEI_PAYLOADS) - break; - - sei = NULL; - } - } - - if (!sei) { - // Need to make a new SEI NAL unit. Insert it before the first - // slice data NAL unit; if no slice data, add at the end. - AVBufferRef *sei_ref; - - sei = av_mallocz(sizeof(*sei)); - if (!sei) { - err = AVERROR(ENOMEM); - goto fail; - } - - sei->nal_unit_header.nal_unit_type = H264_NAL_SEI; - sei->nal_unit_header.nal_ref_idc = 0; - - sei_ref = av_buffer_create((uint8_t*)sei, sizeof(*sei), - &cbs_h264_free_sei, NULL, 0); - if (!sei_ref) { - av_freep(&sei); - err = AVERROR(ENOMEM); - goto fail; - } - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SLICE || - au->units[i].type == H264_NAL_IDR_SLICE) - break; - } - - err = ff_cbs_insert_unit_content(au, i, H264_NAL_SEI, - sei, sei_ref); - av_buffer_unref(&sei_ref); - if (err < 0) - goto fail; - } +static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { + { + SEI_TYPE_FILLER_PAYLOAD, + 1, 1, + sizeof(SEIRawFillerPayload), + SEI_MESSAGE_RW(sei, filler_payload), + }, + { + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + 1, 1, + sizeof(SEIRawUserDataRegistered), + SEI_MESSAGE_RW(sei, user_data_registered), + }, + { + SEI_TYPE_USER_DATA_UNREGISTERED, + 1, 1, + sizeof(SEIRawUserDataUnregistered), + SEI_MESSAGE_RW(sei, user_data_unregistered), + }, + { + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME, + 1, 0, + sizeof(SEIRawMasteringDisplayColourVolume), + SEI_MESSAGE_RW(sei, mastering_display_colour_volume), + }, + { + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + 1, 0, + sizeof(SEIRawContentLightLevelInfo), + SEI_MESSAGE_RW(sei, content_light_level_info), + }, + { + SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS, + 1, 0, + sizeof(SEIRawAlternativeTransferCharacteristics), + SEI_MESSAGE_RW(sei, alternative_transfer_characteristics), + }, + SEI_MESSAGE_TYPE_END, +}; - memcpy(&sei->payload[sei->payload_count], payload, sizeof(*payload)); - ++sei->payload_count; +static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { + { + SEI_TYPE_BUFFERING_PERIOD, + 1, 0, + sizeof(H264RawSEIBufferingPeriod), + SEI_MESSAGE_RW(h264, sei_buffering_period), + }, + { + SEI_TYPE_PIC_TIMING, + 1, 0, + sizeof(H264RawSEIPicTiming), + SEI_MESSAGE_RW(h264, sei_pic_timing), + }, + { + SEI_TYPE_PAN_SCAN_RECT, + 1, 0, + sizeof(H264RawSEIPanScanRect), + SEI_MESSAGE_RW(h264, sei_pan_scan_rect), + }, + { + SEI_TYPE_RECOVERY_POINT, + 1, 0, + sizeof(H264RawSEIRecoveryPoint), + SEI_MESSAGE_RW(h264, sei_recovery_point), + }, + { + SEI_TYPE_DISPLAY_ORIENTATION, + 1, 0, + sizeof(H264RawSEIDisplayOrientation), + SEI_MESSAGE_RW(h264, sei_display_orientation), + }, + SEI_MESSAGE_TYPE_END +}; - return 0; -fail: - cbs_h264_free_sei_payload(payload); - return err; -} +static const SEIMessageTypeDescriptor cbs_sei_h265_types[] = { + { + SEI_TYPE_BUFFERING_PERIOD, + 1, 0, + sizeof(H265RawSEIBufferingPeriod), + SEI_MESSAGE_RW(h265, sei_buffering_period), + }, + { + SEI_TYPE_PIC_TIMING, + 1, 0, + sizeof(H265RawSEIPicTiming), + SEI_MESSAGE_RW(h265, sei_pic_timing), + }, + { + SEI_TYPE_PAN_SCAN_RECT, + 1, 0, + sizeof(H265RawSEIPanScanRect), + SEI_MESSAGE_RW(h265, sei_pan_scan_rect), + }, + { + SEI_TYPE_RECOVERY_POINT, + 1, 0, + sizeof(H265RawSEIRecoveryPoint), + SEI_MESSAGE_RW(h265, sei_recovery_point), + }, + { + SEI_TYPE_DISPLAY_ORIENTATION, + 1, 0, + sizeof(H265RawSEIDisplayOrientation), + SEI_MESSAGE_RW(h265, sei_display_orientation), + }, + { + SEI_TYPE_ACTIVE_PARAMETER_SETS, + 1, 0, + sizeof(H265RawSEIActiveParameterSets), + SEI_MESSAGE_RW(h265, sei_active_parameter_sets), + }, + { + SEI_TYPE_DECODED_PICTURE_HASH, + 0, 1, + sizeof(H265RawSEIDecodedPictureHash), + SEI_MESSAGE_RW(h265, sei_decoded_picture_hash), + }, + { + SEI_TYPE_TIME_CODE, + 1, 0, + sizeof(H265RawSEITimeCode), + SEI_MESSAGE_RW(h265, sei_time_code), + }, + { + SEI_TYPE_ALPHA_CHANNEL_INFO, + 1, 0, + sizeof(H265RawSEIAlphaChannelInfo), + SEI_MESSAGE_RW(h265, sei_alpha_channel_info), + }, + SEI_MESSAGE_TYPE_END +}; -void ff_cbs_h264_delete_sei_message(CodedBitstreamFragment *au, - CodedBitstreamUnit *nal, - int position) +const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, + int payload_type) { - H264RawSEI *sei = nal->content; - - av_assert0(nal->type == H264_NAL_SEI); - av_assert0(position >= 0 && position < sei->payload_count); - - if (position == 0 && sei->payload_count == 1) { - // Deleting NAL unit entirely. - int i; + const SEIMessageTypeDescriptor *codec_list; + int i; - for (i = 0; i < au->nb_units; i++) { - if (&au->units[i] == nal) - break; - } + for (i = 0; cbs_sei_common_types[i].type >= 0; i++) { + if (cbs_sei_common_types[i].type == payload_type) + return &cbs_sei_common_types[i]; + } - ff_cbs_delete_unit(au, i); - } else { - cbs_h264_free_sei_payload(&sei->payload[position]); + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + codec_list = cbs_sei_h264_types; + break; + case AV_CODEC_ID_H265: + codec_list = cbs_sei_h265_types; + break; + default: + return NULL; + } - --sei->payload_count; - memmove(sei->payload + position, - sei->payload + position + 1, - (sei->payload_count - position) * sizeof(*sei->payload)); + for (i = 0; codec_list[i].type >= 0; i++) { + if (codec_list[i].type == payload_type) + return &codec_list[i]; } + + return NULL; } diff --git a/libavcodec/cbs_h264_syntax_template.c b/libavcodec/cbs_h264_syntax_template.c index 76ed51cc7b..9587f33985 100644 --- a/libavcodec/cbs_h264_syntax_template.c +++ b/libavcodec/cbs_h264_syntax_template.c @@ -511,7 +511,8 @@ static int FUNC(pps)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIBufferingPeriod *current) + H264RawSEIBufferingPeriod *current, + SEIMessageState *sei) { CodedBitstreamH264Context *h264 = ctx->priv_data; const H264RawSPS *sps; @@ -604,7 +605,8 @@ static int FUNC(sei_pic_timestamp)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIPicTiming *current) + H264RawSEIPicTiming *current, + SEIMessageState *sei) { CodedBitstreamH264Context *h264 = ctx->priv_data; const H264RawSPS *sps; @@ -675,7 +677,8 @@ static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIPanScanRect *current) + H264RawSEIPanScanRect *current, + SEIMessageState *sei) { int err, i; @@ -701,7 +704,8 @@ static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIRecoveryPoint *current) + H264RawSEIRecoveryPoint *current, + SEIMessageState *sei) { int err; @@ -716,7 +720,8 @@ static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIDisplayOrientation *current) + H264RawSEIDisplayOrientation *current, + SEIMessageState *sei) { int err; @@ -734,171 +739,17 @@ static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext * return 0; } -static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIPayload *current) -{ - int err, i; - int start_position, end_position; - -#ifdef READ - start_position = get_bits_count(rw); -#else - start_position = put_bits_count(rw); -#endif - - switch (current->payload_type) { - case H264_SEI_TYPE_BUFFERING_PERIOD: - CHECK(FUNC(sei_buffering_period) - (ctx, rw, ¤t->payload.buffering_period)); - break; - case H264_SEI_TYPE_PIC_TIMING: - CHECK(FUNC(sei_pic_timing) - (ctx, rw, ¤t->payload.pic_timing)); - break; - case H264_SEI_TYPE_PAN_SCAN_RECT: - CHECK(FUNC(sei_pan_scan_rect) - (ctx, rw, ¤t->payload.pan_scan_rect)); - break; - case H264_SEI_TYPE_FILLER_PAYLOAD: - { - for (i = 0; i < current->payload_size; i++) - fixed(8, ff_byte, 0xff); - } - break; - case H264_SEI_TYPE_USER_DATA_REGISTERED: - CHECK(FUNC_SEI(sei_user_data_registered) - (ctx, rw, ¤t->payload.user_data_registered, ¤t->payload_size)); - break; - case H264_SEI_TYPE_USER_DATA_UNREGISTERED: - CHECK(FUNC_SEI(sei_user_data_unregistered) - (ctx, rw, ¤t->payload.user_data_unregistered, ¤t->payload_size)); - break; - case H264_SEI_TYPE_RECOVERY_POINT: - CHECK(FUNC(sei_recovery_point) - (ctx, rw, ¤t->payload.recovery_point)); - break; - case H264_SEI_TYPE_DISPLAY_ORIENTATION: - CHECK(FUNC(sei_display_orientation) - (ctx, rw, ¤t->payload.display_orientation)); - break; - case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: - CHECK(FUNC_SEI(sei_mastering_display_colour_volume) - (ctx, rw, ¤t->payload.mastering_display_colour_volume)); - break; - case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: - CHECK(FUNC_SEI(sei_alternative_transfer_characteristics) - (ctx, rw, ¤t->payload.alternative_transfer_characteristics)); - break; - default: - { -#ifdef READ - current->payload.other.data_length = current->payload_size; -#endif - allocate(current->payload.other.data, current->payload.other.data_length); - for (i = 0; i < current->payload.other.data_length; i++) - xu(8, payload_byte[i], current->payload.other.data[i], 0, 255, 1, i); - } - } - - if (byte_alignment(rw)) { - fixed(1, bit_equal_to_one, 1); - while (byte_alignment(rw)) - fixed(1, bit_equal_to_zero, 0); - } - -#ifdef READ - end_position = get_bits_count(rw); - if (end_position < start_position + 8 * current->payload_size) { - av_log(ctx->log_ctx, AV_LOG_ERROR, "Incorrect SEI payload length: " - "header %"PRIu32" bits, actually %d bits.\n", - 8 * current->payload_size, - end_position - start_position); - return AVERROR_INVALIDDATA; - } -#else - end_position = put_bits_count(rw); - current->payload_size = (end_position - start_position) / 8; -#endif - - return 0; -} - static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, H264RawSEI *current) { - int err, k; + int err; HEADER("Supplemental Enhancement Information"); CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, 1 << H264_NAL_SEI)); -#ifdef READ - for (k = 0; k < H264_MAX_SEI_PAYLOADS; k++) { - uint32_t payload_type = 0; - uint32_t payload_size = 0; - uint32_t tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_type += 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - payload_type += tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_size += 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - payload_size += tmp; - - current->payload[k].payload_type = payload_type; - current->payload[k].payload_size = payload_size; - - current->payload_count++; - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k])); - - if (!cbs_h2645_read_more_rbsp_data(rw)) - break; - } - if (k >= H264_MAX_SEI_PAYLOADS) { - av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many payloads in " - "SEI message: found %d.\n", k); - return AVERROR_INVALIDDATA; - } -#else - for (k = 0; k < current->payload_count; k++) { - PutBitContext start_state; - uint32_t tmp; - int need_size, i; - - // Somewhat clumsy: we write the payload twice when - // we don't know the size in advance. This will mess - // with trace output, but is otherwise harmless. - start_state = *rw; - need_size = !current->payload[k].payload_size; - for (i = 0; i < 1 + need_size; i++) { - *rw = start_state; - - tmp = current->payload[k].payload_type; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - - tmp = current->payload[k].payload_size; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k])); - } - } -#endif + CHECK(FUNC_SEI(message_list)(ctx, rw, ¤t->message_list, 1)); CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index d8e93e3bb8..738cbeec2c 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -658,40 +658,9 @@ typedef struct H265RawSEIAlphaChannelInfo { uint8_t alpha_channel_clip_type_flag; } H265RawSEIAlphaChannelInfo; -typedef struct H265RawSEIPayload { - uint32_t payload_type; - uint32_t payload_size; - union { - H265RawSEIBufferingPeriod buffering_period; - H265RawSEIPicTiming pic_timing; - H265RawSEIPanScanRect pan_scan_rect; - SEIRawUserDataRegistered user_data_registered; - SEIRawUserDataUnregistered user_data_unregistered; - H265RawSEIRecoveryPoint recovery_point; - H265RawSEIDisplayOrientation display_orientation; - H265RawSEIActiveParameterSets active_parameter_sets; - H265RawSEIDecodedPictureHash decoded_picture_hash; - H265RawSEITimeCode time_code; - SEIRawMasteringDisplayColourVolume - mastering_display_colour_volume; - SEIRawContentLightLevelInfo content_light_level; - SEIRawAlternativeTransferCharacteristics - alternative_transfer_characteristics; - H265RawSEIAlphaChannelInfo alpha_channel_info; - struct { - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; - } other; - } payload; - H265RawExtensionData extension_data; -} H265RawSEIPayload; - typedef struct H265RawSEI { H265RawNALUnitHeader nal_unit_header; - - H265RawSEIPayload payload[H265_MAX_SEI_PAYLOADS]; - uint8_t payload_count; + SEIRawMessageList message_list; } H265RawSEI; typedef struct CodedBitstreamH265Context { diff --git a/libavcodec/cbs_h265_syntax_template.c b/libavcodec/cbs_h265_syntax_template.c index b792ec688a..d09934cfeb 100644 --- a/libavcodec/cbs_h265_syntax_template.c +++ b/libavcodec/cbs_h265_syntax_template.c @@ -1596,10 +1596,9 @@ static int FUNC(slice_segment_header)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIBufferingPeriod *current, - uint32_t *payload_size, - int *more_data) +static int FUNC(sei_buffering_period) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIBufferingPeriod *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawSPS *sps; @@ -1687,7 +1686,7 @@ static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, #ifdef READ end_pos = get_bits_count(rw); - if (cbs_h265_payload_extension_present(rw, *payload_size, + if (cbs_h265_payload_extension_present(rw, sei->payload_size, end_pos - start_pos)) flag(use_alt_cpb_params_flag); else @@ -1695,20 +1694,21 @@ static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, #else // If unknown extension data exists, then use_alt_cpb_params_flag is // coded in the bitstream and must be written even if it's 0. - if (current->use_alt_cpb_params_flag || *more_data) { + if (current->use_alt_cpb_params_flag || sei->extension_present) { flag(use_alt_cpb_params_flag); // Ensure this bit is not the last in the payload by making the // more_data_in_payload() check evaluate to true, so it may not // be mistaken as something else by decoders. - *more_data = 1; + sei->extension_present = 1; } #endif return 0; } -static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIPicTiming *current) +static int FUNC(sei_pic_timing) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIPicTiming *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawSPS *sps; @@ -1782,8 +1782,9 @@ static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIPanScanRect *current) +static int FUNC(sei_pan_scan_rect) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIPanScanRect *current, SEIMessageState *sei) { int err, i; @@ -1808,8 +1809,9 @@ static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIRecoveryPoint *current) +static int FUNC(sei_recovery_point) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIRecoveryPoint *current, SEIMessageState *sei) { int err; @@ -1823,8 +1825,9 @@ static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIDisplayOrientation *current) +static int FUNC(sei_display_orientation) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIDisplayOrientation *current, SEIMessageState *sei) { int err; @@ -1841,8 +1844,9 @@ static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext * return 0; } -static int FUNC(sei_active_parameter_sets)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIActiveParameterSets *current) +static int FUNC(sei_active_parameter_sets) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIActiveParameterSets *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawVPS *vps; @@ -1877,8 +1881,9 @@ static int FUNC(sei_active_parameter_sets)(CodedBitstreamContext *ctx, RWContext return 0; } -static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIDecodedPictureHash *current) +static int FUNC(sei_decoded_picture_hash) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIDecodedPictureHash *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawSPS *sps = h265->active_sps; @@ -1908,8 +1913,9 @@ static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext return 0; } -static int FUNC(sei_time_code)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEITimeCode *current) +static int FUNC(sei_time_code) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEITimeCode *current, SEIMessageState *sei) { int err, i; @@ -1958,9 +1964,9 @@ static int FUNC(sei_time_code)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_alpha_channel_info)(CodedBitstreamContext *ctx, - RWContext *rw, - H265RawSEIAlphaChannelInfo *current) +static int FUNC(sei_alpha_channel_info) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIAlphaChannelInfo *current, SEIMessageState *sei) { int err, length; @@ -1986,158 +1992,10 @@ static int FUNC(sei_alpha_channel_info)(CodedBitstreamContext *ctx, return 0; } -static int FUNC(payload_extension)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawExtensionData *current, uint32_t payload_size, - int cur_pos) -{ - int err; - size_t byte_length, k; - -#ifdef READ - GetBitContext tmp; - int bits_left, payload_zero_bits; - - if (!cbs_h265_payload_extension_present(rw, payload_size, cur_pos)) - return 0; - - bits_left = 8 * payload_size - cur_pos; - tmp = *rw; - if (bits_left > 8) - skip_bits_long(&tmp, bits_left - 8); - payload_zero_bits = get_bits(&tmp, FFMIN(bits_left, 8)); - if (!payload_zero_bits) - return AVERROR_INVALIDDATA; - payload_zero_bits = ff_ctz(payload_zero_bits); - current->bit_length = bits_left - payload_zero_bits - 1; - allocate(current->data, (current->bit_length + 7) / 8); -#endif - - byte_length = (current->bit_length + 7) / 8; - for (k = 0; k < byte_length; k++) { - int length = FFMIN(current->bit_length - k * 8, 8); - xu(length, reserved_payload_extension_data, current->data[k], - 0, MAX_UINT_BITS(length), 0); - } - - return 0; -} - -static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIPayload *current, int prefix) -{ - int err, i; - int start_position, current_position; - int more_data = !!current->extension_data.bit_length; - -#ifdef READ - start_position = get_bits_count(rw); -#else - start_position = put_bits_count(rw); -#endif - - switch (current->payload_type) { -#define SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid) do { \ - if (prefix && !prefix_valid) { \ - av_log(ctx->log_ctx, AV_LOG_ERROR, "SEI type %s invalid " \ - "as prefix SEI!\n", #name); \ - return AVERROR_INVALIDDATA; \ - } \ - if (!prefix && !suffix_valid) { \ - av_log(ctx->log_ctx, AV_LOG_ERROR, "SEI type %s invalid " \ - "as suffix SEI!\n", #name); \ - return AVERROR_INVALIDDATA; \ - } \ - } while (0) -#define SEI_TYPE_N(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name)); \ - break -#define SEI_TYPE_S(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name, \ - ¤t->payload_size)); \ - break -#define SEI_TYPE_E(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name, \ - ¤t->payload_size, \ - &more_data)); \ - break - -#define SEI_TYPE_N2(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC_SEI(sei_ ## name)(ctx, rw, ¤t->payload.name)); \ - break -#define SEI_TYPE_S2(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC_SEI(sei_ ## name)(ctx, rw, ¤t->payload.name, \ - ¤t->payload_size)); \ - break - - SEI_TYPE_E(BUFFERING_PERIOD, 1, 0, buffering_period); - SEI_TYPE_N(PICTURE_TIMING, 1, 0, pic_timing); - SEI_TYPE_N(PAN_SCAN_RECT, 1, 0, pan_scan_rect); - SEI_TYPE_S2(USER_DATA_REGISTERED_ITU_T_T35, - 1, 1, user_data_registered); - SEI_TYPE_S2(USER_DATA_UNREGISTERED, 1, 1, user_data_unregistered); - SEI_TYPE_N(RECOVERY_POINT, 1, 0, recovery_point); - SEI_TYPE_N(DISPLAY_ORIENTATION, 1, 0, display_orientation); - SEI_TYPE_N(ACTIVE_PARAMETER_SETS, 1, 0, active_parameter_sets); - SEI_TYPE_N(DECODED_PICTURE_HASH, 0, 1, decoded_picture_hash); - SEI_TYPE_N(TIME_CODE, 1, 0, time_code); - SEI_TYPE_N2(MASTERING_DISPLAY_INFO, 1, 0, mastering_display_colour_volume); - SEI_TYPE_N2(CONTENT_LIGHT_LEVEL_INFO,1, 0, content_light_level); - SEI_TYPE_N2(ALTERNATIVE_TRANSFER_CHARACTERISTICS, - 1, 0, alternative_transfer_characteristics); - SEI_TYPE_N(ALPHA_CHANNEL_INFO, 1, 0, alpha_channel_info); - -#undef SEI_TYPE_N -#undef SEI_TYPE_S -#undef SEI_TYPE_E - default: - { -#ifdef READ - current->payload.other.data_length = current->payload_size; -#endif - allocate(current->payload.other.data, current->payload.other.data_length); - - for (i = 0; i < current->payload_size; i++) - xu(8, payload_byte[i], current->payload.other.data[i], 0, 255, - 1, i); - } - } - - // more_data_in_payload() -#ifdef READ - current_position = get_bits_count(rw) - start_position; - if (current_position < 8 * current->payload_size) { -#else - current_position = put_bits_count(rw) - start_position; - if (byte_alignment(rw) || more_data) { -#endif - CHECK(FUNC(payload_extension)(ctx, rw, ¤t->extension_data, - current->payload_size, current_position)); - fixed(1, bit_equal_to_one, 1); - while (byte_alignment(rw)) - fixed(1, bit_equal_to_zero, 0); - } - -#ifdef WRITE - current->payload_size = (put_bits_count(rw) - start_position) >> 3; -#endif - - return 0; -} - static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, H265RawSEI *current, int prefix) { - int err, k; + int err; if (prefix) HEADER("Prefix Supplemental Enhancement Information"); @@ -2148,72 +2006,7 @@ static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, prefix ? HEVC_NAL_SEI_PREFIX : HEVC_NAL_SEI_SUFFIX)); -#ifdef READ - for (k = 0; k < H265_MAX_SEI_PAYLOADS; k++) { - uint32_t payload_type = 0; - uint32_t payload_size = 0; - uint32_t tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_type += 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - payload_type += tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_size += 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - payload_size += tmp; - - current->payload[k].payload_type = payload_type; - current->payload[k].payload_size = payload_size; - - current->payload_count++; - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k], prefix)); - - if (!cbs_h2645_read_more_rbsp_data(rw)) - break; - } - if (k >= H265_MAX_SEI_PAYLOADS) { - av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many payloads in " - "SEI message: found %d.\n", k); - return AVERROR_INVALIDDATA; - } -#else - for (k = 0; k < current->payload_count; k++) { - PutBitContext start_state; - uint32_t tmp; - int need_size, i; - - // Somewhat clumsy: we write the payload twice when - // we don't know the size in advance. This will mess - // with trace output, but is otherwise harmless. - start_state = *rw; - need_size = !current->payload[k].payload_size; - for (i = 0; i < 1 + need_size; i++) { - *rw = start_state; - - tmp = current->payload[k].payload_type; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - - tmp = current->payload[k].payload_size; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k], prefix)); - } - } -#endif + CHECK(FUNC_SEI(message_list)(ctx, rw, ¤t->message_list, prefix)); CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c new file mode 100644 index 0000000000..c49830ad77 --- /dev/null +++ b/libavcodec/cbs_sei.c @@ -0,0 +1,369 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_internal.h" +#include "cbs_h264.h" +#include "cbs_h265.h" +#include "cbs_sei.h" + +static void cbs_free_user_data_registered(void *opaque, uint8_t *data) +{ + SEIRawUserDataRegistered *udr = (SEIRawUserDataRegistered*)data; + av_buffer_unref(&udr->data_ref); + av_free(udr); +} + +static void cbs_free_user_data_unregistered(void *opaque, uint8_t *data) +{ + SEIRawUserDataUnregistered *udu = (SEIRawUserDataUnregistered*)data; + av_buffer_unref(&udu->data_ref); + av_free(udu); +} + +int ff_cbs_sei_alloc_message_payload(SEIRawMessage *message, + const SEIMessageTypeDescriptor *desc) +{ + void (*free_func)(void*, uint8_t*); + + av_assert0(message->payload == NULL && + message->payload_ref == NULL); + message->payload_type = desc->type; + + if (desc->type == SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35) + free_func = &cbs_free_user_data_registered; + else if (desc->type == SEI_TYPE_USER_DATA_UNREGISTERED) + free_func = &cbs_free_user_data_unregistered; + else + free_func = NULL; + + if (free_func) { + message->payload = av_mallocz(desc->size); + if (!message->payload) + return AVERROR(ENOMEM); + message->payload_ref = + av_buffer_create(message->payload, desc->size, + free_func, NULL, 0); + } else { + message->payload_ref = av_buffer_alloc(desc->size); + } + if (!message->payload_ref) { + av_freep(&message->payload); + return AVERROR(ENOMEM); + } + message->payload = message->payload_ref->data; + + return 0; +} + +int ff_cbs_sei_list_add(SEIRawMessageList *list) +{ + void *ptr; + int old_count = list->nb_messages_allocated; + + av_assert0(list->nb_messages <= old_count); + if (list->nb_messages + 1 > old_count) { + int new_count = 2 * old_count + 1; + + ptr = av_realloc_array(list->messages, + new_count, sizeof(*list->messages)); + if (!ptr) + return AVERROR(ENOMEM); + + list->messages = ptr; + list->nb_messages_allocated = new_count; + + // Zero the newly-added entries. + memset(list->messages + old_count, 0, + (new_count - old_count) * sizeof(*list->messages)); + } + ++list->nb_messages; + return 0; +} + +void ff_cbs_sei_free_message_list(SEIRawMessageList *list) +{ + for (int i = 0; i < list->nb_messages; i++) { + SEIRawMessage *message = &list->messages[i]; + av_buffer_unref(&message->payload_ref); + av_buffer_unref(&message->extension_data_ref); + } + av_free(list->messages); +} + +static int cbs_sei_get_unit(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + int prefix, + CodedBitstreamUnit **sei_unit) +{ + CodedBitstreamUnit *unit; + int sei_type, highest_vcl_type, err, i, position; + + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + // (We can ignore auxiliary slices because we only have prefix + // SEI in H.264 and an auxiliary picture must always follow a + // primary picture.) + highest_vcl_type = H264_NAL_IDR_SLICE; + if (prefix) + sei_type = H264_NAL_SEI; + else + return AVERROR(EINVAL); + break; + case AV_CODEC_ID_H265: + highest_vcl_type = HEVC_NAL_RSV_VCL31; + if (prefix) + sei_type = HEVC_NAL_SEI_PREFIX; + else + sei_type = HEVC_NAL_SEI_SUFFIX; + break; + default: + return AVERROR(EINVAL); + } + + // Find an existing SEI NAL unit of the right type. + unit = NULL; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == sei_type) { + unit = &au->units[i]; + break; + } + } + + if (unit) { + *sei_unit = unit; + return 0; + } + + // Need to add a new SEI NAL unit ... + if (prefix) { + // ... before the first VCL NAL unit. + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type < highest_vcl_type) + break; + } + position = i; + } else { + // ... after the last VCL NAL unit. + for (i = au->nb_units - 1; i >= 0; i--) { + if (au->units[i].type < highest_vcl_type) + break; + } + if (i < 0) { + // No VCL units; just put it at the end. + position = -1; + } else { + position = i + 1; + } + } + + err = ff_cbs_insert_unit_content(au, position, sei_type, + NULL, NULL); + if (err < 0) + return err; + unit = &au->units[position]; + unit->type = sei_type; + + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) + return err; + + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + { + H264RawSEI sei = { + .nal_unit_header = { + .nal_ref_idc = 0, + .nal_unit_type = sei_type, + }, + }; + memcpy(unit->content, &sei, sizeof(sei)); + } + break; + case AV_CODEC_ID_H265: + { + H265RawSEI sei = { + .nal_unit_header = { + .nal_unit_type = sei_type, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }, + }; + memcpy(unit->content, &sei, sizeof(sei)); + } + break; + default: + av_assert0(0); + } + + *sei_unit = unit; + return 0; +} + +static int cbs_sei_get_message_list(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit, + SEIRawMessageList **list) +{ + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + { + H264RawSEI *sei = unit->content; + if (unit->type != H264_NAL_SEI) + return AVERROR(EINVAL); + *list = &sei->message_list; + } + break; + case AV_CODEC_ID_H265: + { + H265RawSEI *sei = unit->content; + if (unit->type != HEVC_NAL_SEI_PREFIX && + unit->type != HEVC_NAL_SEI_SUFFIX) + return AVERROR(EINVAL); + *list = &sei->message_list; + } + break; + default: + return AVERROR(EINVAL); + } + + return 0; +} + +int ff_cbs_sei_add_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + int prefix, + uint32_t payload_type, + void *payload_data, + AVBufferRef *payload_buf) +{ + const SEIMessageTypeDescriptor *desc; + CodedBitstreamUnit *unit; + SEIRawMessageList *list; + SEIRawMessage *message; + AVBufferRef *payload_ref; + int err; + + desc = ff_cbs_sei_find_type(ctx, payload_type); + if (!desc) + return AVERROR(EINVAL); + + if (payload_buf) { + payload_ref = av_buffer_ref(payload_buf); + if (!payload_ref) + return AVERROR(ENOMEM); + } else { + payload_ref = NULL; + } + + // Find an existing SEI unit or make a new one to add to. + err = cbs_sei_get_unit(ctx, au, prefix, &unit); + if (err < 0) + return err; + + // Find the message list inside the codec-dependent unit. + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + return err; + + // Add a new message to the message list. + err = ff_cbs_sei_list_add(list); + if (err < 0) + return err; + + message = &list->messages[list->nb_messages - 1]; + + message->payload_type = payload_type; + message->payload = payload_data; + message->payload_ref = payload_ref; + + return 0; +} + +int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type, + SEIRawMessage **iter) +{ + int err, i, j, found; + + found = 0; + for (i = 0; i < au->nb_units; i++) { + CodedBitstreamUnit *unit = &au->units[i]; + SEIRawMessageList *list; + + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + continue; + + for (j = 0; j < list->nb_messages; j++) { + SEIRawMessage *message = &list->messages[j]; + + if (message->payload_type == payload_type) { + if (!*iter || found) { + *iter = message; + return 0; + } + if (message == *iter) + found = 1; + } + } + } + + return AVERROR(ENOENT); +} + +static void cbs_sei_delete_message(SEIRawMessageList *list, + int position) +{ + SEIRawMessage *message; + + av_assert0(0 <= position && position < list->nb_messages); + + message = &list->messages[position]; + av_buffer_unref(&message->payload_ref); + av_buffer_unref(&message->extension_data_ref); + + --list->nb_messages; + + if (list->nb_messages > 0) { + memmove(list->messages + position, + list->messages + position + 1, + (list->nb_messages - position) * sizeof(*list->messages)); + } +} + +void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type) +{ + int err, i, j; + + for (i = 0; i < au->nb_units; i++) { + CodedBitstreamUnit *unit = &au->units[i]; + SEIRawMessageList *list; + + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + continue; + + for (j = list->nb_messages - 1; j >= 0; j--) { + if (list->messages[j].payload_type == payload_type) + cbs_sei_delete_message(list, j); + } + } +} diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index 95beabf4d7..5ce4ad3ccd 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -21,8 +21,132 @@ #include #include + #include "libavutil/buffer.h" +#include "cbs.h" + +// SEI payload types form a common namespace between the H.264, H.265 +// and H.266 standards. A given payload type always has the same +// meaning, but some names have different payload types in different +// standards (e.g. scalable-nesting is 30 in H.264 but 133 in H.265). +// The content of the payload data depends on the standard, though +// many generic parts have the same interpretation everywhere (such as +// mastering-display-colour-volume and user-data-unregistered). +enum { + SEI_TYPE_BUFFERING_PERIOD = 0, + SEI_TYPE_PIC_TIMING = 1, + SEI_TYPE_PAN_SCAN_RECT = 2, + SEI_TYPE_FILLER_PAYLOAD = 3, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35 = 4, + SEI_TYPE_USER_DATA_UNREGISTERED = 5, + SEI_TYPE_RECOVERY_POINT = 6, + SEI_TYPE_DEC_REF_PIC_MARKING_REPETITION = 7, + SEI_TYPE_SPARE_PIC = 8, + SEI_TYPE_SCENE_INFO = 9, + SEI_TYPE_SUB_SEQ_INFO = 10, + SEI_TYPE_SUB_SEQ_LAYER_CHARACTERISTICS = 11, + SEI_TYPE_SUB_SEQ_CHARACTERISTICS = 12, + SEI_TYPE_FULL_FRAME_FREEZE = 13, + SEI_TYPE_FULL_FRAME_FREEZE_RELEASE = 14, + SEI_TYPE_FULL_FRAME_SNAPSHOT = 15, + SEI_TYPE_PROGRESSIVE_REFINEMENT_SEGMENT_START = 16, + SEI_TYPE_PROGRESSIVE_REFINEMENT_SEGMENT_END = 17, + SEI_TYPE_MOTION_CONSTRAINED_SLICE_GROUP_SET = 18, + SEI_TYPE_FILM_GRAIN_CHARACTERISTICS = 19, + SEI_TYPE_DEBLOCKING_FILTER_DISPLAY_PREFERENCE = 20, + SEI_TYPE_STEREO_VIDEO_INFO = 21, + SEI_TYPE_POST_FILTER_HINT = 22, + SEI_TYPE_TONE_MAPPING_INFO = 23, + SEI_TYPE_SCALABILITY_INFO = 24, + SEI_TYPE_SUB_PIC_SCALABLE_LAYER = 25, + SEI_TYPE_NON_REQUIRED_LAYER_REP = 26, + SEI_TYPE_PRIORITY_LAYER_INFO = 27, + SEI_TYPE_LAYERS_NOT_PRESENT_4 = 28, + SEI_TYPE_LAYER_DEPENDENCY_CHANGE = 29, + SEI_TYPE_SCALABLE_NESTING_4 = 30, + SEI_TYPE_BASE_LAYER_TEMPORAL_HRD = 31, + SEI_TYPE_QUALITY_LAYER_INTEGRITY_CHECK = 32, + SEI_TYPE_REDUNDANT_PIC_PROPERTY = 33, + SEI_TYPE_TL0_DEP_REP_INDEX = 34, + SEI_TYPE_TL_SWITCHING_POINT = 35, + SEI_TYPE_PARALLEL_DECODING_INFO = 36, + SEI_TYPE_MVC_SCALABLE_NESTING = 37, + SEI_TYPE_VIEW_SCALABILITY_INFO = 38, + SEI_TYPE_MULTIVIEW_SCENE_INFO_4 = 39, + SEI_TYPE_MULTIVIEW_ACQUISITION_INFO_4 = 40, + SEI_TYPE_NON_REQUIRED_VIEW_COMPONENT = 41, + SEI_TYPE_VIEW_DEPENDENCY_CHANGE = 42, + SEI_TYPE_OPERATION_POINTS_NOT_PRESENT = 43, + SEI_TYPE_BASE_VIEW_TEMPORAL_HRD = 44, + SEI_TYPE_FRAME_PACKING_ARRANGEMENT = 45, + SEI_TYPE_MULTIVIEW_VIEW_POSITION_4 = 46, + SEI_TYPE_DISPLAY_ORIENTATION = 47, + SEI_TYPE_MVCD_SCALABLE_NESTING = 48, + SEI_TYPE_MVCD_VIEW_SCALABILITY_INFO = 49, + SEI_TYPE_DEPTH_REPRESENTATION_INFO_4 = 50, + SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO_4 = 51, + SEI_TYPE_DEPTH_TIMING = 52, + SEI_TYPE_DEPTH_SAMPLING_INFO = 53, + SEI_TYPE_CONSTRAINED_DEPTH_PARAMETER_SET_IDENTIFIER = 54, + SEI_TYPE_GREEN_METADATA = 56, + SEI_TYPE_STRUCTURE_OF_PICTURES_INFO = 128, + SEI_TYPE_ACTIVE_PARAMETER_SETS = 129, + SEI_TYPE_DECODING_UNIT_INFO = 130, + SEI_TYPE_TEMPORAL_SUB_LAYER_ZERO_IDX = 131, + SEI_TYPE_DECODED_PICTURE_HASH = 132, + SEI_TYPE_SCALABLE_NESTING_5 = 133, + SEI_TYPE_REGION_REFRESH_INFO = 134, + SEI_TYPE_NO_DISPLAY = 135, + SEI_TYPE_TIME_CODE = 136, + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME = 137, + SEI_TYPE_SEGMENTED_RECT_FRAME_PACKING_ARRANGEMENT = 138, + SEI_TYPE_TEMPORAL_MOTION_CONSTRAINED_TILE_SETS = 139, + SEI_TYPE_CHROMA_RESAMPLING_FILTER_HINT = 140, + SEI_TYPE_KNEE_FUNCTION_INFO = 141, + SEI_TYPE_COLOUR_REMAPPING_INFO = 142, + SEI_TYPE_DEINTERLACED_FIELD_IDENTIFICATION = 143, + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO = 144, + SEI_TYPE_DEPENDENT_RAP_INDICATION = 145, + SEI_TYPE_CODED_REGION_COMPLETION = 146, + SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS = 147, + SEI_TYPE_AMBIENT_VIEWING_ENVIRONMENT = 148, + SEI_TYPE_CONTENT_COLOUR_VOLUME = 149, + SEI_TYPE_EQUIRECTANGULAR_PROJECTION = 150, + SEI_TYPE_CUBEMAP_PROJECTION = 151, + SEI_TYPE_FISHEYE_VIDEO_INFO = 152, + SEI_TYPE_SPHERE_ROTATION = 154, + SEI_TYPE_REGIONWISE_PACKING = 155, + SEI_TYPE_OMNI_VIEWPORT = 156, + SEI_TYPE_REGIONAL_NESTING = 157, + SEI_TYPE_MCTS_EXTRACTION_INFO_SETS = 158, + SEI_TYPE_MCTS_EXTRACTION_INFO_NESTING = 159, + SEI_TYPE_LAYERS_NOT_PRESENT_5 = 160, + SEI_TYPE_INTER_LAYER_CONSTRAINED_TILE_SETS = 161, + SEI_TYPE_BSP_NESTING = 162, + SEI_TYPE_BSP_INITIAL_ARRIVAL_TIME = 163, + SEI_TYPE_SUB_BITSTREAM_PROPERTY = 164, + SEI_TYPE_ALPHA_CHANNEL_INFO = 165, + SEI_TYPE_OVERLAY_INFO = 166, + SEI_TYPE_TEMPORAL_MV_PREDICTION_CONSTRAINTS = 167, + SEI_TYPE_FRAME_FIELD_INFO = 168, + SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO = 176, + SEI_TYPE_DEPTH_REPRESENTATION_INFO_5 = 177, + SEI_TYPE_MULTIVIEW_SCENE_INFO_5 = 178, + SEI_TYPE_MULTIVIEW_ACQUISITION_INFO_5 = 179, + SEI_TYPE_MULTIVIEW_VIEW_POSITION_5 = 180, + SEI_TYPE_ALTERNATIVE_DEPTH_INFO = 181, + SEI_TYPE_SEI_MANIFEST = 200, + SEI_TYPE_SEI_PREFIX_INDICATION = 201, + SEI_TYPE_ANNOTATED_REGIONS = 202, + SEI_TYPE_SUBPIC_LEVEL_INFO = 203, + SEI_TYPE_SAMPLE_ASPECT_RATIO_INFO = 204, +}; + + +typedef struct SEIRawFillerPayload { + uint32_t payload_size; +} SEIRawFillerPayload; typedef struct SEIRawUserDataRegistered { uint8_t itu_t_t35_country_code; @@ -57,4 +181,135 @@ typedef struct SEIRawAlternativeTransferCharacteristics { uint8_t preferred_transfer_characteristics; } SEIRawAlternativeTransferCharacteristics; +typedef struct SEIRawMessage { + uint32_t payload_type; + uint32_t payload_size; + void *payload; + AVBufferRef *payload_ref; + uint8_t *extension_data; + AVBufferRef *extension_data_ref; + size_t extension_bit_length; +} SEIRawMessage; + +typedef struct SEIRawMessageList { + SEIRawMessage *messages; + int nb_messages; + int nb_messages_allocated; +} SEIRawMessageList; + + +typedef struct SEIMessageState { + // The type of the payload being written. + uint32_t payload_type; + // When reading, contains the size of the payload to allow finding the + // end of variable-length fields (such as user_data_payload_byte[]). + // (When writing, the size will be derived from the total number of + // bytes actually written.) + uint32_t payload_size; + // When writing, indicates that payload extension data is present so + // all extended fields must be written. May be updated by the writer + // to indicate that extended fields have been written, so the extension + // end bits must be written too. + uint8_t extension_present; +} SEIMessageState; + +struct GetBitContext; +struct PutBitContext; + +typedef int (*SEIMessageReadFunction)(CodedBitstreamContext *ctx, + struct GetBitContext *rw, + void *current, + SEIMessageState *sei); + +typedef int (*SEIMessageWriteFunction)(CodedBitstreamContext *ctx, + struct PutBitContext *rw, + void *current, + SEIMessageState *sei); + +typedef struct SEIMessageTypeDescriptor { + // Payload type for the message. (-1 in this field ends a list.) + int type; + // Valid in a prefix SEI NAL unit (always for H.264). + uint8_t prefix; + // Valid in a suffix SEI NAL unit (never for H.264). + uint8_t suffix; + // Size of the decomposed structure. + size_t size; + // Read bitstream into SEI message. + SEIMessageReadFunction read; + // Write bitstream from SEI message. + SEIMessageWriteFunction write; +} SEIMessageTypeDescriptor; + +// Macro for the read/write pair. The clumsy cast is needed because the +// current pointer is typed in all of the read/write functions but has to +// be void here to fit all cases. +#define SEI_MESSAGE_RW(codec, name) \ + .read = (SEIMessageReadFunction) cbs_ ## codec ## _read_ ## name, \ + .write = (SEIMessageWriteFunction)cbs_ ## codec ## _write_ ## name + +// End-of-list sentinel element. +#define SEI_MESSAGE_TYPE_END { .type = -1 } + + +/** + * Find the type descriptor for the given payload type. + * + * Returns NULL if the payload type is not known. + */ +const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, + int payload_type); + +/** + * Allocate a new payload for the given SEI message. + */ +int ff_cbs_sei_alloc_message_payload(SEIRawMessage *message, + const SEIMessageTypeDescriptor *desc); + +/** + * Allocate a new empty SEI message in a message list. + * + * The new message is in place nb_messages - 1. + */ +int ff_cbs_sei_list_add(SEIRawMessageList *list); + +/** + * Free all SEI messages in a message list. + */ +void ff_cbs_sei_free_message_list(SEIRawMessageList *list); + +/** + * Add an SEI message to an access unit. + * + * Will add to an existing SEI NAL unit, or create a new one for the + * message if there is no suitable existing one. + * + * Takes a new reference to payload_buf, if set. If payload_buf is + * NULL then the new message will not be reference counted. + */ +int ff_cbs_sei_add_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + int prefix, + uint32_t payload_type, + void *payload_data, + AVBufferRef *payload_buf); + +/** + * Iterate over messages with the given payload type in an access unit. + * + * Set message to NULL in the first call. Returns 0 while more messages + * are available, AVERROR(ENOENT) when all messages have been found. + */ +int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type, + SEIRawMessage **message); + +/** + * Delete all messages with the given payload type from an access unit. + */ +void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type); + #endif /* AVCODEC_CBS_SEI_H */ diff --git a/libavcodec/cbs_sei_syntax_template.c b/libavcodec/cbs_sei_syntax_template.c index 93d9fafde1..5f84246663 100644 --- a/libavcodec/cbs_sei_syntax_template.c +++ b/libavcodec/cbs_sei_syntax_template.c @@ -16,9 +16,27 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -static int FUNC(sei_user_data_registered) +static int FUNC(filler_payload) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawUserDataRegistered *current, uint32_t *payload_size) + SEIRawFillerPayload *current, SEIMessageState *state) +{ + int err, i; + + HEADER("Filler Payload"); + +#ifdef READ + current->payload_size = state->payload_size; +#endif + + for (i = 0; i < current->payload_size; i++) + fixed(8, ff_byte, 0xff); + + return 0; +} + +static int FUNC(user_data_registered) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawUserDataRegistered *current, SEIMessageState *state) { int err, i, j; @@ -33,14 +51,12 @@ static int FUNC(sei_user_data_registered) } #ifdef READ - if (*payload_size < i) { + if (state->payload_size < i) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid SEI user data registered payload.\n"); return AVERROR_INVALIDDATA; } - current->data_length = *payload_size - i; -#else - *payload_size = i + current->data_length; + current->data_length = state->payload_size - i; #endif allocate(current->data, current->data_length); @@ -50,23 +66,21 @@ static int FUNC(sei_user_data_registered) return 0; } -static int FUNC(sei_user_data_unregistered) +static int FUNC(user_data_unregistered) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawUserDataUnregistered *current, uint32_t *payload_size) + SEIRawUserDataUnregistered *current, SEIMessageState *state) { int err, i; HEADER("User Data Unregistered"); #ifdef READ - if (*payload_size < 16) { + if (state->payload_size < 16) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid SEI user data unregistered payload.\n"); return AVERROR_INVALIDDATA; } - current->data_length = *payload_size - 16; -#else - *payload_size = 16 + current->data_length; + current->data_length = state->payload_size - 16; #endif for (i = 0; i < 16; i++) @@ -80,9 +94,9 @@ static int FUNC(sei_user_data_unregistered) return 0; } -static int FUNC(sei_mastering_display_colour_volume) +static int FUNC(mastering_display_colour_volume) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawMasteringDisplayColourVolume *current) + SEIRawMasteringDisplayColourVolume *current, SEIMessageState *state) { int err, c; @@ -104,13 +118,13 @@ static int FUNC(sei_mastering_display_colour_volume) return 0; } -static int FUNC(sei_content_light_level) +static int FUNC(content_light_level_info) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawContentLightLevelInfo *current) + SEIRawContentLightLevelInfo *current, SEIMessageState *state) { int err; - HEADER("Content Light Level"); + HEADER("Content Light Level Information"); ub(16, max_content_light_level); ub(16, max_pic_average_light_level); @@ -118,9 +132,10 @@ static int FUNC(sei_content_light_level) return 0; } -static int FUNC(sei_alternative_transfer_characteristics) +static int FUNC(alternative_transfer_characteristics) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawAlternativeTransferCharacteristics *current) + SEIRawAlternativeTransferCharacteristics *current, + SEIMessageState *state) { int err; @@ -130,3 +145,165 @@ static int FUNC(sei_alternative_transfer_characteristics) return 0; } + +static int FUNC(message)(CodedBitstreamContext *ctx, RWContext *rw, + SEIRawMessage *current) +{ + const SEIMessageTypeDescriptor *desc; + int err, i; + + desc = ff_cbs_sei_find_type(ctx, current->payload_type); + if (desc) { + SEIMessageState state = { + .payload_type = current->payload_type, + .payload_size = current->payload_size, + .extension_present = current->extension_bit_length > 0, + }; + int start_position, current_position, bits_written; + +#ifdef READ + CHECK(ff_cbs_sei_alloc_message_payload(current, desc)); +#endif + + start_position = bit_position(rw); + + CHECK(desc->READWRITE(ctx, rw, current->payload, &state)); + + current_position = bit_position(rw); + bits_written = current_position - start_position; + + if (byte_alignment(rw) || state.extension_present || + bits_written < 8 * current->payload_size) { + size_t bits_left; + +#ifdef READ + GetBitContext tmp = *rw; + int trailing_bits, trailing_zero_bits; + + bits_left = 8 * current->payload_size - bits_written; + if (bits_left > 8) + skip_bits_long(&tmp, bits_left - 8); + trailing_bits = get_bits(&tmp, FFMIN(bits_left, 8)); + if (trailing_bits == 0) { + // The trailing bits must contain a bit_equal_to_one, so + // they can't all be zero. + return AVERROR_INVALIDDATA; + } + trailing_zero_bits = ff_ctz(trailing_bits); + current->extension_bit_length = + bits_left - 1 - trailing_zero_bits; +#endif + + if (current->extension_bit_length > 0) { + allocate(current->extension_data, + (current->extension_bit_length + 7) / 8); + + bits_left = current->extension_bit_length; + for (i = 0; bits_left > 0; i++) { + int length = FFMIN(bits_left, 8); + xu(length, reserved_payload_extension_data, + current->extension_data[i], + 0, MAX_UINT_BITS(length), 0); + bits_left -= length; + } + } + + fixed(1, bit_equal_to_one, 1); + while (byte_alignment(rw)) + fixed(1, bit_equal_to_zero, 0); + } + +#ifdef WRITE + current->payload_size = (put_bits_count(rw) - start_position) / 8; +#endif + } else { + uint8_t *data; + + allocate(current->payload, current->payload_size); + data = current->payload; + + for (i = 0; i < current->payload_size; i++) + xu(8, payload_byte[i], data[i], 0, 255, 1, i); + } + + return 0; +} + +static int FUNC(message_list)(CodedBitstreamContext *ctx, RWContext *rw, + SEIRawMessageList *current, int prefix) +{ + SEIRawMessage *message; + int err, k; + +#ifdef READ + for (k = 0;; k++) { + uint32_t payload_type = 0; + uint32_t payload_size = 0; + uint32_t tmp; + + while (show_bits(rw, 8) == 0xff) { + fixed(8, ff_byte, 0xff); + payload_type += 255; + } + xu(8, last_payload_type_byte, tmp, 0, 254, 0); + payload_type += tmp; + + while (show_bits(rw, 8) == 0xff) { + fixed(8, ff_byte, 0xff); + payload_size += 255; + } + xu(8, last_payload_size_byte, tmp, 0, 254, 0); + payload_size += tmp; + + CHECK(ff_cbs_sei_list_add(current)); + message = ¤t->messages[k]; + + message->payload_type = payload_type; + message->payload_size = payload_size; + + CHECK(FUNC(message)(ctx, rw, message)); + + if (!cbs_h2645_read_more_rbsp_data(rw)) + break; + } +#else + for (k = 0; k < current->nb_messages; k++) { + PutBitContext start_state; + uint32_t tmp; + int trace, i; + + message = ¤t->messages[k]; + + // We write the payload twice in order to find the size. Trace + // output is switched off for the first write. + trace = ctx->trace_enable; + ctx->trace_enable = 0; + + start_state = *rw; + for (i = 0; i < 2; i++) { + *rw = start_state; + + tmp = message->payload_type; + while (tmp >= 255) { + fixed(8, ff_byte, 0xff); + tmp -= 255; + } + xu(8, last_payload_type_byte, tmp, 0, 254, 0); + + tmp = message->payload_size; + while (tmp >= 255) { + fixed(8, ff_byte, 0xff); + tmp -= 255; + } + xu(8, last_payload_size_byte, tmp, 0, 254, 0); + + err = FUNC(message)(ctx, rw, message); + ctx->trace_enable = trace; + if (err < 0) + return err; + } + } +#endif + + return 0; +} diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index f39e649ac6..4ab97aee3a 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -78,13 +78,14 @@ typedef struct H264MetadataContext { int crop_bottom; const char *sei_user_data; - H264RawSEIPayload sei_user_data_payload; + SEIRawUserDataUnregistered sei_user_data_payload; int delete_filler; int display_orientation; double rotate; int flip; + H264RawSEIDisplayOrientation display_orientation_payload; int level; } H264MetadataContext; @@ -414,7 +415,9 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) // Only insert the SEI in access units containing SPSs, and also // unconditionally in the first access unit we ever see. if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - err = ff_cbs_h264_add_sei_message(au, &ctx->sei_user_data_payload); + err = ff_cbs_sei_add_message(ctx->output, au, 1, + H264_SEI_TYPE_USER_DATA_UNREGISTERED, + &ctx->sei_user_data_payload, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " "message to access unit.\n"); @@ -428,74 +431,54 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) ff_cbs_delete_unit(au, i); continue; } - - if (au->units[i].type == H264_NAL_SEI) { - // Filler SEI messages. - H264RawSEI *sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - if (sei->payload[j].payload_type == - H264_SEI_TYPE_FILLER_PAYLOAD) - ff_cbs_h264_delete_sei_message(au, &au->units[i], j); - } - } } + + ff_cbs_sei_delete_message_type(ctx->output, au, + H264_SEI_TYPE_FILLER_PAYLOAD); } if (ctx->display_orientation != PASS) { - for (i = au->nb_units - 1; i >= 0; i--) { - H264RawSEI *sei; - if (au->units[i].type != H264_NAL_SEI) - continue; - sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - H264RawSEIDisplayOrientation *disp; - int32_t *matrix; - - if (sei->payload[j].payload_type != - H264_SEI_TYPE_DISPLAY_ORIENTATION) - continue; - disp = &sei->payload[j].payload.display_orientation; - - if (ctx->display_orientation == REMOVE || - ctx->display_orientation == INSERT) { - ff_cbs_h264_delete_sei_message(au, &au->units[i], j); - continue; - } - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) { - err = AVERROR(ENOMEM); - goto fail; - } + SEIRawMessage *message = NULL; + while (ff_cbs_sei_find_message(ctx->output, au, + H264_SEI_TYPE_DISPLAY_ORIENTATION, + &message) == 0) { + H264RawSEIDisplayOrientation *disp = message->payload; + int32_t *matrix; + + matrix = av_malloc(9 * sizeof(int32_t)); + if (!matrix) { + err = AVERROR(ENOMEM); + goto fail; + } - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_free(matrix); - goto fail; - } + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 180.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); + + // If there are multiple display orientation messages in an + // access unit, then the last one added to the packet (i.e. + // the first one in the access unit) will prevail. + err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, + (uint8_t*)matrix, + 9 * sizeof(int32_t)); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " + "displaymatrix side data to packet.\n"); + av_free(matrix); + goto fail; } } + + if (ctx->display_orientation == REMOVE || + ctx->display_orientation == INSERT) { + ff_cbs_sei_delete_message_type(ctx->output, au, + H264_SEI_TYPE_DISPLAY_ORIENTATION); + } } if (ctx->display_orientation == INSERT) { - H264RawSEIPayload payload = { - .payload_type = H264_SEI_TYPE_DISPLAY_ORIENTATION, - }; H264RawSEIDisplayOrientation *disp = - &payload.payload.display_orientation; + &ctx->display_orientation_payload; uint8_t *data; int size; int write = 0; @@ -551,7 +534,9 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (write) { disp->display_orientation_repetition_period = 1; - err = ff_cbs_h264_add_sei_message(au, &payload); + err = ff_cbs_sei_add_message(ctx->output, au, 1, + H264_SEI_TYPE_DISPLAY_ORIENTATION, + disp, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); @@ -585,13 +570,9 @@ static int h264_metadata_init(AVBSFContext *bsf) int err, i; if (ctx->sei_user_data) { - SEIRawUserDataUnregistered *udu = - &ctx->sei_user_data_payload.payload.user_data_unregistered; + SEIRawUserDataUnregistered *udu = &ctx->sei_user_data_payload; int j; - ctx->sei_user_data_payload.payload_type = - H264_SEI_TYPE_USER_DATA_UNREGISTERED; - // Parse UUID. It must be a hex string of length 32, possibly // containing '-'s between hex digits (which we ignore). for (i = j = 0; j < 32 && i < 64 && ctx->sei_user_data[i]; i++) { diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index b577d09caf..d24462414c 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -90,7 +90,6 @@ typedef struct VAAPIEncodeH264Context { H264RawAUD raw_aud; H264RawSPS raw_sps; H264RawPPS raw_pps; - H264RawSEI raw_sei; H264RawSlice raw_slice; H264RawSEIBufferingPeriod sei_buffering_period; @@ -210,11 +209,9 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, { VAAPIEncodeH264Context *priv = avctx->priv_data; CodedBitstreamFragment *au = &priv->current_access_unit; - int err, i; + int err; if (priv->sei_needed) { - H264RawSEI *sei = &priv->raw_sei; - if (priv->aud_needed) { err = vaapi_encode_h264_add_nal(avctx, au, &priv->raw_aud); if (err < 0) @@ -222,41 +219,35 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, priv->aud_needed = 0; } - *sei = (H264RawSEI) { - .nal_unit_header = { - .nal_unit_type = H264_NAL_SEI, - }, - }; - - i = 0; - if (priv->sei_needed & SEI_IDENTIFIER) { - sei->payload[i].payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED; - sei->payload[i].payload.user_data_unregistered = priv->sei_identifier; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_USER_DATA_UNREGISTERED, + &priv->sei_identifier, NULL); + if (err < 0) + goto fail; } if (priv->sei_needed & SEI_TIMING) { if (pic->type == PICTURE_TYPE_IDR) { - sei->payload[i].payload_type = H264_SEI_TYPE_BUFFERING_PERIOD; - sei->payload[i].payload.buffering_period = priv->sei_buffering_period; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_BUFFERING_PERIOD, + &priv->sei_buffering_period, NULL); + if (err < 0) + goto fail; } - sei->payload[i].payload_type = H264_SEI_TYPE_PIC_TIMING; - sei->payload[i].payload.pic_timing = priv->sei_pic_timing; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_PIC_TIMING, + &priv->sei_pic_timing, NULL); + if (err < 0) + goto fail; } if (priv->sei_needed & SEI_RECOVERY_POINT) { - sei->payload[i].payload_type = H264_SEI_TYPE_RECOVERY_POINT; - sei->payload[i].payload.recovery_point = priv->sei_recovery_point; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_RECOVERY_POINT, + &priv->sei_recovery_point, NULL); + if (err < 0) + goto fail; } - sei->payload_count = i; - av_assert0(sei->payload_count > 0); - - err = vaapi_encode_h264_add_nal(avctx, au, sei); - if (err < 0) - goto fail; priv->sei_needed = 0; err = vaapi_encode_h264_write_access_unit(avctx, data, data_len, au); diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index a7af763ae4..2e8e772008 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -73,7 +73,6 @@ typedef struct VAAPIEncodeH265Context { H265RawVPS raw_vps; H265RawSPS raw_sps; H265RawPPS raw_pps; - H265RawSEI raw_sei; H265RawSlice raw_slice; SEIRawMasteringDisplayColourVolume sei_mastering_display; @@ -195,11 +194,9 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, { VAAPIEncodeH265Context *priv = avctx->priv_data; CodedBitstreamFragment *au = &priv->current_access_unit; - int err, i; + int err; if (priv->sei_needed) { - H265RawSEI *sei = &priv->raw_sei; - if (priv->aud_needed) { err = vaapi_encode_h265_add_nal(avctx, au, &priv->aud); if (err < 0) @@ -207,35 +204,22 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, priv->aud_needed = 0; } - *sei = (H265RawSEI) { - .nal_unit_header = { - .nal_unit_type = HEVC_NAL_SEI_PREFIX, - .nuh_layer_id = 0, - .nuh_temporal_id_plus1 = 1, - }, - }; - - i = 0; - if (priv->sei_needed & SEI_MASTERING_DISPLAY) { - sei->payload[i].payload_type = HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO; - sei->payload[i].payload.mastering_display_colour_volume = - priv->sei_mastering_display; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, + &priv->sei_mastering_display, NULL); + if (err < 0) + goto fail; } if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { - sei->payload[i].payload_type = HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO; - sei->payload[i].payload.content_light_level = priv->sei_content_light_level; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + &priv->sei_content_light_level, NULL); + if (err < 0) + goto fail; } - sei->payload_count = i; - av_assert0(sei->payload_count > 0); - - err = vaapi_encode_h265_add_nal(avctx, au, sei); - if (err < 0) - goto fail; priv->sei_needed = 0; err = vaapi_encode_h265_write_access_unit(avctx, data, data_len, au);