From patchwork Fri Jan 1 21:35:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24754 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 354BC44B645 for ; Fri, 1 Jan 2021 23:45:19 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1553D68A91B; Fri, 1 Jan 2021 23:45:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9A82B6880B2 for ; Fri, 1 Jan 2021 23:45:17 +0200 (EET) Received: by mail-wm1-f45.google.com with SMTP id g185so10014678wmf.3 for ; Fri, 01 Jan 2021 13:45:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=G30UgA53lhwsDZ4JOYs2VY9OjysWasCbR1RdN/osetA=; b=JV6+c5bEEo7iyYtGxsuOF1qLugVqb12PBW5F05+eXxYwBogohhh5cwfI7fRUn9VdN1 EUpdPGKxsqYDqtYyswPVE4lVSZk/6rTAqD3Aa+9l1E+stg64TuhUsWHvwH5dtDP/K9eG NbQgfboQps8w3ui1IAWxPIlVAe+CsyjS9KpoBevMBw51vhJjChnQYEC8KL2dEkvdq6OS yFRdw+Uh2eUt41MJSeoPsUhenYdOUuP5DNk0RcCeVSeoaUi0dXwnzWpvmFFbfEgdRBbr 3wT3W8ivompk/n7I47JhPwjdEA4RntV/LIbhJIpIhKJ6o0RkfxWb74drYDW5tefAS/Ql LGhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G30UgA53lhwsDZ4JOYs2VY9OjysWasCbR1RdN/osetA=; b=Yrx0HEiUOhpmkcc1UitqA81NueNpQf3GqAYzOPVsijMBHP5TbBb9hVj8KiMspvsBw9 ot/pT//G/p2eK/Or8vBlRzRMknnZLDQhx5ZZHHra1hS2RmXMJBsaoN4ri9npc5LDtFS2 RZeA/lCAwFKQ2whYM6P9CVY/GjsTAJ8hfGD32Z+TBG+4KKMHCdOoBcDz0JeqDSnz48yM jE+6IEI7wzj8GkRig5Pa07LYrxspIrmUgPaBZLT7/WoPrVYX7kQK9uWeWH7TzPLiq8G3 XE1ySpiUxEZ3rJHa1FChgU98bK/X15IurciugyvoHcmLe/Pkh9y1xZkx1SYwTEoUtC/Y u/6g== X-Gm-Message-State: AOAM530zGqeGU0GBWADRhORDSbmRuqmCyq7+BC/mjoj2dsslbcO3wLZ4 hYVLF9EoQpcpOy8gQD7cYVkfMk7Vb85R6w== X-Google-Smtp-Source: ABdhPJztejzO4bBGeBWcVAN+gZZTH6o7pMQozq+Iip06INBDewCFyK8L/g4zo3j9u5h9eaEZrG3juQ== X-Received: by 2002:a7b:c397:: with SMTP id s23mr16832842wmj.63.1609537069758; Fri, 01 Jan 2021 13:37:49 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:48 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:11 +0000 Message-Id: <20210101213537.169546-2-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/27] cbs_h2645: Merge SEI messages in common between codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Make a new template file for common SEI messages - this will also apply to H.266. --- libavcodec/cbs_h264.h | 37 +------ libavcodec/cbs_h2645.c | 24 +++-- libavcodec/cbs_h264_syntax_template.c | 106 +------------------- libavcodec/cbs_h265.h | 45 ++------- libavcodec/cbs_h265_syntax_template.c | 135 ++++---------------------- libavcodec/cbs_sei.h | 60 ++++++++++++ libavcodec/cbs_sei_syntax_template.c | 132 +++++++++++++++++++++++++ libavcodec/h264_metadata_bsf.c | 2 +- libavcodec/vaapi_encode_h264.c | 2 +- libavcodec/vaapi_encode_h265.c | 11 ++- 10 files changed, 250 insertions(+), 304 deletions(-) create mode 100644 libavcodec/cbs_sei.h create mode 100644 libavcodec/cbs_sei_syntax_template.c diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index a6fe0a6af2..81113f1ad0 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -24,6 +24,7 @@ #include "cbs.h" #include "cbs_h2645.h" +#include "cbs_sei.h" #include "h264.h" @@ -274,21 +275,6 @@ typedef struct H264RawSEIPanScanRect { uint16_t pan_scan_rect_repetition_period; } H264RawSEIPanScanRect; -typedef struct H264RawSEIUserDataRegistered { - uint8_t itu_t_t35_country_code; - uint8_t itu_t_t35_country_code_extension_byte; - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; -} H264RawSEIUserDataRegistered; - -typedef struct H264RawSEIUserDataUnregistered { - uint8_t uuid_iso_iec_11578[16]; - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; -} H264RawSEIUserDataUnregistered; - typedef struct H264RawSEIRecoveryPoint { uint16_t recovery_frame_cnt; uint8_t exact_match_flag; @@ -305,19 +291,6 @@ typedef struct H264RawSEIDisplayOrientation { uint8_t display_orientation_extension_flag; } H264RawSEIDisplayOrientation; -typedef struct H264RawSEIMasteringDisplayColourVolume { - uint16_t display_primaries_x[3]; - uint16_t display_primaries_y[3]; - uint16_t white_point_x; - uint16_t white_point_y; - uint32_t max_display_mastering_luminance; - uint32_t min_display_mastering_luminance; -} H264RawSEIMasteringDisplayColourVolume; - -typedef struct H264RawSEIAlternativeTransferCharacteristics { - uint8_t preferred_transfer_characteristics; -} H264RawSEIAlternativeTransferCharacteristics; - typedef struct H264RawSEIPayload { uint32_t payload_type; uint32_t payload_size; @@ -326,12 +299,12 @@ typedef struct H264RawSEIPayload { H264RawSEIPicTiming pic_timing; H264RawSEIPanScanRect pan_scan_rect; // H264RawSEIFiller filler -> no fields. - H264RawSEIUserDataRegistered user_data_registered; - H264RawSEIUserDataUnregistered user_data_unregistered; + SEIRawUserDataRegistered user_data_registered; + SEIRawUserDataUnregistered user_data_unregistered; H264RawSEIRecoveryPoint recovery_point; H264RawSEIDisplayOrientation display_orientation; - H264RawSEIMasteringDisplayColourVolume mastering_display_colour_volume; - H264RawSEIAlternativeTransferCharacteristics + SEIRawMasteringDisplayColourVolume mastering_display_colour_volume; + SEIRawAlternativeTransferCharacteristics alternative_transfer_characteristics; struct { uint8_t *data; diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 434322492c..ce58e47a36 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -253,9 +253,11 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo return err; \ } while (0) -#define FUNC_NAME(rw, codec, name) cbs_ ## codec ## _ ## rw ## _ ## name -#define FUNC_H264(rw, name) FUNC_NAME(rw, h264, name) -#define FUNC_H265(rw, name) FUNC_NAME(rw, h265, name) +#define FUNC_NAME2(rw, codec, name) cbs_ ## codec ## _ ## rw ## _ ## name +#define FUNC_NAME1(rw, codec, name) FUNC_NAME2(rw, codec, name) +#define FUNC_H264(name) FUNC_NAME1(READWRITE, h264, name) +#define FUNC_H265(name) FUNC_NAME1(READWRITE, h265, name) +#define FUNC_SEI(name) FUNC_NAME1(READWRITE, sei, name) #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL) @@ -356,11 +358,15 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) name = name ## _ref->data; \ } while (0) -#define FUNC(name) FUNC_H264(READWRITE, name) +#define FUNC(name) FUNC_SEI(name) +#include "cbs_sei_syntax_template.c" +#undef FUNC + +#define FUNC(name) FUNC_H264(name) #include "cbs_h264_syntax_template.c" #undef FUNC -#define FUNC(name) FUNC_H265(READWRITE, name) +#define FUNC(name) FUNC_H265(name) #include "cbs_h265_syntax_template.c" #undef FUNC @@ -428,11 +434,15 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) } \ } while (0) -#define FUNC(name) FUNC_H264(READWRITE, name) +#define FUNC(name) FUNC_SEI(name) +#include "cbs_sei_syntax_template.c" +#undef FUNC + +#define FUNC(name) FUNC_H264(name) #include "cbs_h264_syntax_template.c" #undef FUNC -#define FUNC(name) FUNC_H265(READWRITE, name) +#define FUNC(name) FUNC_H265(name) #include "cbs_h265_syntax_template.c" #undef FUNC diff --git a/libavcodec/cbs_h264_syntax_template.c b/libavcodec/cbs_h264_syntax_template.c index b65460996b..76ed51cc7b 100644 --- a/libavcodec/cbs_h264_syntax_template.c +++ b/libavcodec/cbs_h264_syntax_template.c @@ -700,70 +700,6 @@ static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_user_data_registered)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIUserDataRegistered *current, - uint32_t *payload_size) -{ - int err, i, j; - - HEADER("User Data Registered ITU-T T.35"); - - u(8, itu_t_t35_country_code, 0x00, 0xff); - if (current->itu_t_t35_country_code != 0xff) - i = 1; - else { - u(8, itu_t_t35_country_code_extension_byte, 0x00, 0xff); - i = 2; - } - -#ifdef READ - if (*payload_size < i) { - av_log(ctx->log_ctx, AV_LOG_ERROR, - "Invalid SEI user data registered payload.\n"); - return AVERROR_INVALIDDATA; - } - current->data_length = *payload_size - i; -#else - *payload_size = i + current->data_length; -#endif - - allocate(current->data, current->data_length); - for (j = 0; j < current->data_length; j++) - xu(8, itu_t_t35_payload_byte[i], current->data[j], 0x00, 0xff, 1, i + j); - - return 0; -} - -static int FUNC(sei_user_data_unregistered)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIUserDataUnregistered *current, - uint32_t *payload_size) -{ - int err, i; - - HEADER("User Data Unregistered"); - -#ifdef READ - if (*payload_size < 16) { - av_log(ctx->log_ctx, AV_LOG_ERROR, - "Invalid SEI user data unregistered payload.\n"); - return AVERROR_INVALIDDATA; - } - current->data_length = *payload_size - 16; -#else - *payload_size = 16 + current->data_length; -#endif - - for (i = 0; i < 16; i++) - us(8, uuid_iso_iec_11578[i], 0x00, 0xff, 1, i); - - allocate(current->data, current->data_length); - - for (i = 0; i < current->data_length; i++) - xu(8, user_data_payload_byte[i], current->data[i], 0x00, 0xff, 1, i); - - return 0; -} - static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, H264RawSEIRecoveryPoint *current) { @@ -798,40 +734,6 @@ static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext * return 0; } -static int FUNC(sei_mastering_display_colour_volume)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIMasteringDisplayColourVolume *current) -{ - int err, c; - - HEADER("Mastering Display Colour Volume"); - - for (c = 0; c < 3; c++) { - us(16, display_primaries_x[c], 0, 50000, 1, c); - us(16, display_primaries_y[c], 0, 50000, 1, c); - } - - u(16, white_point_x, 0, 50000); - u(16, white_point_y, 0, 50000); - - u(32, max_display_mastering_luminance, 1, MAX_UINT_BITS(32)); - u(32, min_display_mastering_luminance, 0, current->max_display_mastering_luminance - 1); - - return 0; -} - -static int FUNC(sei_alternative_transfer_characteristics)(CodedBitstreamContext *ctx, - RWContext *rw, - H264RawSEIAlternativeTransferCharacteristics *current) -{ - int err; - - HEADER("Alternative Transfer Characteristics"); - - ub(8, preferred_transfer_characteristics); - - return 0; -} - static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, H264RawSEIPayload *current) { @@ -864,11 +766,11 @@ static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, } break; case H264_SEI_TYPE_USER_DATA_REGISTERED: - CHECK(FUNC(sei_user_data_registered) + CHECK(FUNC_SEI(sei_user_data_registered) (ctx, rw, ¤t->payload.user_data_registered, ¤t->payload_size)); break; case H264_SEI_TYPE_USER_DATA_UNREGISTERED: - CHECK(FUNC(sei_user_data_unregistered) + CHECK(FUNC_SEI(sei_user_data_unregistered) (ctx, rw, ¤t->payload.user_data_unregistered, ¤t->payload_size)); break; case H264_SEI_TYPE_RECOVERY_POINT: @@ -880,11 +782,11 @@ static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, (ctx, rw, ¤t->payload.display_orientation)); break; case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: - CHECK(FUNC(sei_mastering_display_colour_volume) + CHECK(FUNC_SEI(sei_mastering_display_colour_volume) (ctx, rw, ¤t->payload.mastering_display_colour_volume)); break; case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: - CHECK(FUNC(sei_alternative_transfer_characteristics) + CHECK(FUNC_SEI(sei_alternative_transfer_characteristics) (ctx, rw, ¤t->payload.alternative_transfer_characteristics)); break; default: diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index 15b22bbfd4..d8e93e3bb8 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -23,6 +23,7 @@ #include #include "cbs_h2645.h" +#include "cbs_sei.h" #include "hevc.h" enum { @@ -596,21 +597,6 @@ typedef struct H265RawSEIPanScanRect { uint16_t pan_scan_rect_persistence_flag; } H265RawSEIPanScanRect; -typedef struct H265RawSEIUserDataRegistered { - uint8_t itu_t_t35_country_code; - uint8_t itu_t_t35_country_code_extension_byte; - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; -} H265RawSEIUserDataRegistered; - -typedef struct H265RawSEIUserDataUnregistered { - uint8_t uuid_iso_iec_11578[16]; - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; -} H265RawSEIUserDataUnregistered; - typedef struct H265RawSEIRecoveryPoint { int16_t recovery_poc_cnt; uint8_t exact_match_flag; @@ -661,24 +647,6 @@ typedef struct H265RawSEITimeCode { int32_t time_offset_value[3]; } H265RawSEITimeCode; -typedef struct H265RawSEIMasteringDisplayColourVolume { - uint16_t display_primaries_x[3]; - uint16_t display_primaries_y[3]; - uint16_t white_point_x; - uint16_t white_point_y; - uint32_t max_display_mastering_luminance; - uint32_t min_display_mastering_luminance; -} H265RawSEIMasteringDisplayColourVolume; - -typedef struct H265RawSEIContentLightLevelInfo { - uint16_t max_content_light_level; - uint16_t max_pic_average_light_level; -} H265RawSEIContentLightLevelInfo; - -typedef struct H265RawSEIAlternativeTransferCharacteristics { - uint8_t preferred_transfer_characteristics; -} H265RawSEIAlternativeTransferCharacteristics; - typedef struct H265RawSEIAlphaChannelInfo { uint8_t alpha_channel_cancel_flag; uint8_t alpha_channel_use_idc; @@ -697,16 +665,17 @@ typedef struct H265RawSEIPayload { H265RawSEIBufferingPeriod buffering_period; H265RawSEIPicTiming pic_timing; H265RawSEIPanScanRect pan_scan_rect; - H265RawSEIUserDataRegistered user_data_registered; - H265RawSEIUserDataUnregistered user_data_unregistered; + SEIRawUserDataRegistered user_data_registered; + SEIRawUserDataUnregistered user_data_unregistered; H265RawSEIRecoveryPoint recovery_point; H265RawSEIDisplayOrientation display_orientation; H265RawSEIActiveParameterSets active_parameter_sets; H265RawSEIDecodedPictureHash decoded_picture_hash; H265RawSEITimeCode time_code; - H265RawSEIMasteringDisplayColourVolume mastering_display; - H265RawSEIContentLightLevelInfo content_light_level; - H265RawSEIAlternativeTransferCharacteristics + SEIRawMasteringDisplayColourVolume + mastering_display_colour_volume; + SEIRawContentLightLevelInfo content_light_level; + SEIRawAlternativeTransferCharacteristics alternative_transfer_characteristics; H265RawSEIAlphaChannelInfo alpha_channel_info; struct { diff --git a/libavcodec/cbs_h265_syntax_template.c b/libavcodec/cbs_h265_syntax_template.c index 48fae82d04..e4cc1a9be8 100644 --- a/libavcodec/cbs_h265_syntax_template.c +++ b/libavcodec/cbs_h265_syntax_template.c @@ -1808,70 +1808,6 @@ static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_user_data_registered)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIUserDataRegistered *current, - uint32_t *payload_size) -{ - int err, i, j; - - HEADER("User Data Registered ITU-T T.35"); - - u(8, itu_t_t35_country_code, 0x00, 0xff); - if (current->itu_t_t35_country_code != 0xff) - i = 1; - else { - u(8, itu_t_t35_country_code_extension_byte, 0x00, 0xff); - i = 2; - } - -#ifdef READ - if (*payload_size < i) { - av_log(ctx->log_ctx, AV_LOG_ERROR, - "Invalid SEI user data registered payload.\n"); - return AVERROR_INVALIDDATA; - } - current->data_length = *payload_size - i; -#else - *payload_size = i + current->data_length; -#endif - - allocate(current->data, current->data_length); - for (j = 0; j < current->data_length; j++) - xu(8, itu_t_t35_payload_byte[i], current->data[j], 0x00, 0xff, 1, i + j); - - return 0; -} - -static int FUNC(sei_user_data_unregistered)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIUserDataUnregistered *current, - uint32_t *payload_size) -{ - int err, i; - - HEADER("User Data Unregistered"); - -#ifdef READ - if (*payload_size < 16) { - av_log(ctx->log_ctx, AV_LOG_ERROR, - "Invalid SEI user data unregistered payload.\n"); - return AVERROR_INVALIDDATA; - } - current->data_length = *payload_size - 16; -#else - *payload_size = 16 + current->data_length; -#endif - - for (i = 0; i < 16; i++) - us(8, uuid_iso_iec_11578[i], 0x00, 0xff, 1, i); - - allocate(current->data, current->data_length); - - for (i = 0; i < current->data_length; i++) - xu(8, user_data_payload_byte[i], current->data[i], 0x00, 0xff, 1, i); - - return 0; -} - static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, H265RawSEIRecoveryPoint *current) { @@ -2022,55 +1958,6 @@ static int FUNC(sei_time_code)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_mastering_display)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIMasteringDisplayColourVolume *current) -{ - int err, c; - - HEADER("Mastering Display Colour Volume"); - - for (c = 0; c < 3; c++) { - us(16, display_primaries_x[c], 0, 50000, 1, c); - us(16, display_primaries_y[c], 0, 50000, 1, c); - } - - u(16, white_point_x, 0, 50000); - u(16, white_point_y, 0, 50000); - - u(32, max_display_mastering_luminance, - 1, MAX_UINT_BITS(32)); - u(32, min_display_mastering_luminance, - 0, current->max_display_mastering_luminance - 1); - - return 0; -} - -static int FUNC(sei_content_light_level)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIContentLightLevelInfo *current) -{ - int err; - - HEADER("Content Light Level"); - - ub(16, max_content_light_level); - ub(16, max_pic_average_light_level); - - return 0; -} - -static int FUNC(sei_alternative_transfer_characteristics)(CodedBitstreamContext *ctx, - RWContext *rw, - H265RawSEIAlternativeTransferCharacteristics *current) -{ - int err; - - HEADER("Alternative Transfer Characteristics"); - - ub(8, preferred_transfer_characteristics); - - return 0; -} - static int FUNC(sei_alpha_channel_info)(CodedBitstreamContext *ctx, RWContext *rw, H265RawSEIAlphaChannelInfo *current) @@ -2180,20 +2067,32 @@ static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, &more_data)); \ break +#define SEI_TYPE_N2(type, prefix_valid, suffix_valid, name) \ + case HEVC_SEI_TYPE_ ## type: \ + SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ + CHECK(FUNC_SEI(sei_ ## name)(ctx, rw, ¤t->payload.name)); \ + break +#define SEI_TYPE_S2(type, prefix_valid, suffix_valid, name) \ + case HEVC_SEI_TYPE_ ## type: \ + SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ + CHECK(FUNC_SEI(sei_ ## name)(ctx, rw, ¤t->payload.name, \ + ¤t->payload_size)); \ + break + SEI_TYPE_E(BUFFERING_PERIOD, 1, 0, buffering_period); SEI_TYPE_N(PICTURE_TIMING, 1, 0, pic_timing); SEI_TYPE_N(PAN_SCAN_RECT, 1, 0, pan_scan_rect); - SEI_TYPE_S(USER_DATA_REGISTERED_ITU_T_T35, + SEI_TYPE_S2(USER_DATA_REGISTERED_ITU_T_T35, 1, 1, user_data_registered); - SEI_TYPE_S(USER_DATA_UNREGISTERED, 1, 1, user_data_unregistered); + SEI_TYPE_S2(USER_DATA_UNREGISTERED, 1, 1, user_data_unregistered); SEI_TYPE_N(RECOVERY_POINT, 1, 0, recovery_point); SEI_TYPE_N(DISPLAY_ORIENTATION, 1, 0, display_orientation); SEI_TYPE_N(ACTIVE_PARAMETER_SETS, 1, 0, active_parameter_sets); SEI_TYPE_N(DECODED_PICTURE_HASH, 0, 1, decoded_picture_hash); SEI_TYPE_N(TIME_CODE, 1, 0, time_code); - SEI_TYPE_N(MASTERING_DISPLAY_INFO, 1, 0, mastering_display); - SEI_TYPE_N(CONTENT_LIGHT_LEVEL_INFO, 1, 0, content_light_level); - SEI_TYPE_N(ALTERNATIVE_TRANSFER_CHARACTERISTICS, + SEI_TYPE_N2(MASTERING_DISPLAY_INFO, 1, 0, mastering_display_colour_volume); + SEI_TYPE_N2(CONTENT_LIGHT_LEVEL_INFO,1, 0, content_light_level); + SEI_TYPE_N2(ALTERNATIVE_TRANSFER_CHARACTERISTICS, 1, 0, alternative_transfer_characteristics); SEI_TYPE_N(ALPHA_CHANNEL_INFO, 1, 0, alpha_channel_info); diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h new file mode 100644 index 0000000000..95beabf4d7 --- /dev/null +++ b/libavcodec/cbs_sei.h @@ -0,0 +1,60 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_SEI_H +#define AVCODEC_CBS_SEI_H + +#include +#include +#include "libavutil/buffer.h" + + +typedef struct SEIRawUserDataRegistered { + uint8_t itu_t_t35_country_code; + uint8_t itu_t_t35_country_code_extension_byte; + uint8_t *data; + AVBufferRef *data_ref; + size_t data_length; +} SEIRawUserDataRegistered; + +typedef struct SEIRawUserDataUnregistered { + uint8_t uuid_iso_iec_11578[16]; + uint8_t *data; + AVBufferRef *data_ref; + size_t data_length; +} SEIRawUserDataUnregistered; + +typedef struct SEIRawMasteringDisplayColourVolume { + uint16_t display_primaries_x[3]; + uint16_t display_primaries_y[3]; + uint16_t white_point_x; + uint16_t white_point_y; + uint32_t max_display_mastering_luminance; + uint32_t min_display_mastering_luminance; +} SEIRawMasteringDisplayColourVolume; + +typedef struct SEIRawContentLightLevelInfo { + uint16_t max_content_light_level; + uint16_t max_pic_average_light_level; +} SEIRawContentLightLevelInfo; + +typedef struct SEIRawAlternativeTransferCharacteristics { + uint8_t preferred_transfer_characteristics; +} SEIRawAlternativeTransferCharacteristics; + +#endif /* AVCODEC_CBS_SEI_H */ diff --git a/libavcodec/cbs_sei_syntax_template.c b/libavcodec/cbs_sei_syntax_template.c new file mode 100644 index 0000000000..cc900830ae --- /dev/null +++ b/libavcodec/cbs_sei_syntax_template.c @@ -0,0 +1,132 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +static int FUNC(sei_user_data_registered) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawUserDataRegistered *current, uint32_t *payload_size) +{ + int err, i, j; + + HEADER("User Data Registered ITU-T T.35"); + + u(8, itu_t_t35_country_code, 0x00, 0xff); + if (current->itu_t_t35_country_code != 0xff) + i = 1; + else { + u(8, itu_t_t35_country_code_extension_byte, 0x00, 0xff); + i = 2; + } + +#ifdef READ + if (*payload_size < i) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid SEI user data registered payload.\n"); + return AVERROR_INVALIDDATA; + } + current->data_length = *payload_size - i; +#else + *payload_size = i + current->data_length; +#endif + + allocate(current->data, current->data_length); + for (j = 0; j < current->data_length; j++) + xu(8, itu_t_t35_payload_byte[i], current->data[j], 0x00, 0xff, 1, i + j); + + return 0; +} + +static int FUNC(sei_user_data_unregistered) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawUserDataUnregistered *current, uint32_t *payload_size) +{ + int err, i; + + HEADER("User Data Unregistered"); + +#ifdef READ + if (*payload_size < 16) { + av_log(ctx->log_ctx, AV_LOG_ERROR, + "Invalid SEI user data unregistered payload.\n"); + return AVERROR_INVALIDDATA; + } + current->data_length = *payload_size - 16; +#else + *payload_size = 16 + current->data_length; +#endif + + for (i = 0; i < 16; i++) + us(8, uuid_iso_iec_11578[i], 0x00, 0xff, 1, i); + + allocate(current->data, current->data_length); + + for (i = 0; i < current->data_length; i++) + xu(8, user_data_payload_byte[i], current->data[i], 0x00, 0xff, 1, i); + + return 0; +} + +static int FUNC(sei_mastering_display_colour_volume) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawMasteringDisplayColourVolume *current) +{ + int err, c; + + HEADER("Mastering Display Colour Volume"); + + for (c = 0; c < 3; c++) { + us(16, display_primaries_x[c], 0, 50000, 1, c); + us(16, display_primaries_y[c], 0, 50000, 1, c); + } + + u(16, white_point_x, 0, 50000); + u(16, white_point_y, 0, 50000); + + u(32, max_display_mastering_luminance, + 1, MAX_UINT_BITS(32)); + u(32, min_display_mastering_luminance, + 0, current->max_display_mastering_luminance - 1); + + return 0; +} + +static int FUNC(sei_content_light_level) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawContentLightLevelInfo *current) +{ + int err; + + HEADER("Content Light Level"); + + ub(16, max_content_light_level); + ub(16, max_pic_average_light_level); + + return 0; +} + +static int FUNC(sei_alternative_transfer_characteristics) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawAlternativeTransferCharacteristics *current) +{ + int err; + + HEADER("Alternative Transfer Characteristics"); + + ub(8, preferred_transfer_characteristics); + + return 0; +} diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index eb1503159b..d7cf58095b 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -416,7 +416,7 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) H264RawSEIPayload payload = { .payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED, }; - H264RawSEIUserDataUnregistered *udu = + SEIRawUserDataUnregistered *udu = &payload.payload.user_data_unregistered; for (i = j = 0; j < 32 && ctx->sei_user_data[i]; i++) { diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index e52a0e37a4..b577d09caf 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -96,7 +96,7 @@ typedef struct VAAPIEncodeH264Context { H264RawSEIBufferingPeriod sei_buffering_period; H264RawSEIPicTiming sei_pic_timing; H264RawSEIRecoveryPoint sei_recovery_point; - H264RawSEIUserDataUnregistered sei_identifier; + SEIRawUserDataUnregistered sei_identifier; char *sei_identifier_string; int aud_needed; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 3e0af17ea5..a7af763ae4 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -76,8 +76,8 @@ typedef struct VAAPIEncodeH265Context { H265RawSEI raw_sei; H265RawSlice raw_slice; - H265RawSEIMasteringDisplayColourVolume sei_mastering_display; - H265RawSEIContentLightLevelInfo sei_content_light_level; + SEIRawMasteringDisplayColourVolume sei_mastering_display; + SEIRawContentLightLevelInfo sei_content_light_level; CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; @@ -219,7 +219,8 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, if (priv->sei_needed & SEI_MASTERING_DISPLAY) { sei->payload[i].payload_type = HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO; - sei->payload[i].payload.mastering_display = priv->sei_mastering_display; + sei->payload[i].payload.mastering_display_colour_volume = + priv->sei_mastering_display; ++i; } @@ -781,7 +782,7 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, // SEI is needed when both the primaries and luminance are set if (mdm->has_primaries && mdm->has_luminance) { - H265RawSEIMasteringDisplayColourVolume *mdcv = + SEIRawMasteringDisplayColourVolume *mdcv = &priv->sei_mastering_display; const int mapping[3] = {1, 2, 0}; const int chroma_den = 50000; @@ -826,7 +827,7 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, if (sd) { AVContentLightMetadata *clm = (AVContentLightMetadata *)sd->data; - H265RawSEIContentLightLevelInfo *clli = + SEIRawContentLightLevelInfo *clli = &priv->sei_content_light_level; clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); From patchwork Fri Jan 1 21:35:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24756 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A9C0144B645 for ; Fri, 1 Jan 2021 23:45:59 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 86F0568A9AA; Fri, 1 Jan 2021 23:45:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AA7C068A817 for ; Fri, 1 Jan 2021 23:45:58 +0200 (EET) Received: by mail-wm1-f53.google.com with SMTP id 190so9905081wmz.0 for ; Fri, 01 Jan 2021 13:45:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wxZfjziJjWCoPwBEZPWE89U269aeQ/92IHywIpzet6k=; b=GEriUbKtMDQCyeQ5hVH3Uyc/nZ7Mop69K9MAVga79Qo0l8iqQswdaZnBrYAqIjVd4T GA3JbnbipGfGzllbsbUreAHjsZBWpvGHQcz7KSMwCTdGy9ONQlu4OzMG9KcNW+GjTnyC ZU6N0fG2AJ2R5JWnLei5WfAMyVjLlcfxr202qISkDEFEiXNdUE1dkkkcvSyFv1wKeNL+ OiarlejR8SIuKchLFI/VT4Vc26PVW1pDs3zntyMOQ79T4daMaaY5pGuvnZSLC86GL5uH z/WvCYh3BBkD+VfvJ+0K+CnB1wbcdjiinfk2/b0wshImOp/d026+uK6orYQvmpS4cZy1 PuYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wxZfjziJjWCoPwBEZPWE89U269aeQ/92IHywIpzet6k=; b=cBf5+rXmi+vxa4RRgprd41m3j0MPr7vGhlvpIbe1lQDxu2TCb9+LPnjQfzH39DIAYu RIOLw00yh1dOTHxI1R5+bwk6nxTJr+8GzPjpPR7I6QXCbsYA4XOMC6dmzK/CuuWFN1ht fMhxB6Iu1ie+GaXltFspFZ5kFHLKjb5k1/0JE75c9yAT9k9z30KNvUJZjXRclII8Ib2P xw8baBuoWiTBbYElf4xdkPaY9V9nES7t7k4h658mJ2SjEXvL1/R4eObXfZ9T3AN1f/uL 2JZbXGi39ueBcH9vabJS+B6fP1HX4e4WyZC8O49tJvIkOrtUmwI64kTVC2RIKwrYbnTu IG2g== X-Gm-Message-State: AOAM531ILOWKrAQzOmRcl3DBJk0mxmfg5ZN4ihufY9T2xRLBuJUQJDs4 mr42O608G5gtvavjw0a/IhRKQobDMALwtw== X-Google-Smtp-Source: ABdhPJxfyo2LRoczUbKyN7GmlNzD/qt0db4Lmq5bGs4vRzKaBk4Ylv/hCYHNotmsfonC8gQdIvCMPA== X-Received: by 2002:a7b:c7d3:: with SMTP id z19mr17297697wmk.31.1609537071025; Fri, 01 Jan 2021 13:37:51 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:50 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:12 +0000 Message-Id: <20210101213537.169546-3-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/27] h264_metadata_bsf: Move SEI user data parsing to init time X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 91 ++++++++++++++++------------------ 1 file changed, 43 insertions(+), 48 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index d7cf58095b..f39e649ac6 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -78,6 +78,7 @@ typedef struct H264MetadataContext { int crop_bottom; const char *sei_user_data; + H264RawSEIPayload sei_user_data_payload; int delete_filler; @@ -413,54 +414,10 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) // Only insert the SEI in access units containing SPSs, and also // unconditionally in the first access unit we ever see. if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - H264RawSEIPayload payload = { - .payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED, - }; - SEIRawUserDataUnregistered *udu = - &payload.payload.user_data_unregistered; - - for (i = j = 0; j < 32 && ctx->sei_user_data[i]; i++) { - int c, v; - c = ctx->sei_user_data[i]; - if (c == '-') { - continue; - } else if (av_isxdigit(c)) { - c = av_tolower(c); - v = (c <= '9' ? c - '0' : c - 'a' + 10); - } else { - goto invalid_user_data; - } - if (j & 1) - udu->uuid_iso_iec_11578[j / 2] |= v; - else - udu->uuid_iso_iec_11578[j / 2] = v << 4; - ++j; - } - if (j == 32 && ctx->sei_user_data[i] == '+') { - size_t len = strlen(ctx->sei_user_data + i + 1); - - udu->data_ref = av_buffer_alloc(len + 1); - if (!udu->data_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - - udu->data = udu->data_ref->data; - udu->data_length = len + 1; - memcpy(udu->data, ctx->sei_user_data + i + 1, len + 1); - - err = ff_cbs_h264_add_sei_message(au, &payload); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " - "message to access unit.\n"); - goto fail; - } - - } else { - invalid_user_data: - av_log(bsf, AV_LOG_ERROR, "Invalid user data: " - "must be \"UUID+string\".\n"); - err = AVERROR(EINVAL); + err = ff_cbs_h264_add_sei_message(au, &ctx->sei_user_data_payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " + "message to access unit.\n"); goto fail; } } @@ -627,6 +584,44 @@ static int h264_metadata_init(AVBSFContext *bsf) CodedBitstreamFragment *au = &ctx->access_unit; int err, i; + if (ctx->sei_user_data) { + SEIRawUserDataUnregistered *udu = + &ctx->sei_user_data_payload.payload.user_data_unregistered; + int j; + + ctx->sei_user_data_payload.payload_type = + H264_SEI_TYPE_USER_DATA_UNREGISTERED; + + // Parse UUID. It must be a hex string of length 32, possibly + // containing '-'s between hex digits (which we ignore). + for (i = j = 0; j < 32 && i < 64 && ctx->sei_user_data[i]; i++) { + int c, v; + c = ctx->sei_user_data[i]; + if (c == '-') { + continue; + } else if (av_isxdigit(c)) { + c = av_tolower(c); + v = (c <= '9' ? c - '0' : c - 'a' + 10); + } else { + break; + } + if (j & 1) + udu->uuid_iso_iec_11578[j / 2] |= v; + else + udu->uuid_iso_iec_11578[j / 2] = v << 4; + ++j; + } + if (j == 32 && ctx->sei_user_data[i] == '+') { + udu->data = (uint8_t*)ctx->sei_user_data + i + 1; + udu->data_length = strlen(udu->data) + 1; + } else { + av_log(bsf, AV_LOG_ERROR, "Invalid user data: " + "must be \"UUID+string\".\n"); + err = AVERROR(EINVAL); + goto fail; + } + } + err = ff_cbs_init(&ctx->input, AV_CODEC_ID_H264, bsf); if (err < 0) return err; From patchwork Fri Jan 1 21:35:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24749 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3B12044B4DF for ; Fri, 1 Jan 2021 23:44:08 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1DB4B68A987; Fri, 1 Jan 2021 23:44:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BA8F568A943 for ; Fri, 1 Jan 2021 23:44:05 +0200 (EET) Received: by mail-wr1-f49.google.com with SMTP id t30so22980972wrb.0 for ; Fri, 01 Jan 2021 13:44:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zgdIIrMOgxFVr2fWmC/g/T9780cTR7guSEv0rgUEryA=; b=nEOfYcV5yypWn0YG7+ZrfsRYgm2/0ImAqNPRDBSqPonpE2L1b+jx3STfO+nONZ9UwB 4UqnMNcnQIwGxN7Gl7O+/FNvmb9K0El6tWka71CufwWJPFA/h9scXaOA9nUta0lv3inJ QAvyrwAKZpn7mxozZ+G3l1Cpk1/QG66h1WT2RQ6HrrADUU9yqaoQsmg0BlbhPiiQde2V W94JHPoSnI3fCJcz2ZZSnbcFqhMUZ6nx0j7kGfpo+b74+xWmLWz6ulMIUEz9GmvYBEvx 3Axnk5wh72yya44xrEoxe5wX0z3hYsf16xyj8EHJa0/TKGCNC2tABB6f7BxiUZYrnhSh bytA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zgdIIrMOgxFVr2fWmC/g/T9780cTR7guSEv0rgUEryA=; b=p1JgWAbEKX1A/fYRkVm9uzmJCQ9alwd4maWcmaaWOz6ob925UrRwj+DjCUlXcw4VrH s1YPkU+L9dP9Nnxzlr9XU5oDQTGAcfM8NhPIQaNc0sjzTZnAIou2whHWtf/0EQjGy5vM PUaXk7LzhRqh7ORu5k12kyoJyJjIG/OWx7ne2/2B4J+HsTAkh+4mMCzIx9PrgOUvjoai gMcd9C0ofW18jJr1qTcIVNQMgBRq2YWLYGigSuFGIB4zQ3zrcUEWtcqGZwCJ99+Y2osH 2owz/SnNQkSIB/T8V0h/qHFQ+zj814uOFBOr9kay+WgFzZGH7nF5uhqB0XNlqDNwjmWS G1Yw== X-Gm-Message-State: AOAM531bKlDUB417ccQbpMvrBFCvQgkjLsX0fpTpwu13OMd8fQ5gpdmC wIbFcbe3UrmDakpxNOqEFzJFoimR2oi6/Q== X-Google-Smtp-Source: ABdhPJxF6STdzczMUMKN8uHUo0qC52ZrximgCFASs2ynjmw1PHZEW+n6heynv/XMiilbrXS5exRA6Q== X-Received: by 2002:a5d:43cc:: with SMTP id v12mr43488820wrr.319.1609537072396; Fri, 01 Jan 2021 13:37:52 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:51 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:13 +0000 Message-Id: <20210101213537.169546-4-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/27] cbs_h2645: Merge SEI message handling in common between codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/Makefile | 4 +- libavcodec/cbs_h264.h | 50 +--- libavcodec/cbs_h2645.c | 300 +++++++++++---------- libavcodec/cbs_h264_syntax_template.c | 173 +----------- libavcodec/cbs_h265.h | 33 +-- libavcodec/cbs_h265_syntax_template.c | 269 +++---------------- libavcodec/cbs_sei.c | 369 ++++++++++++++++++++++++++ libavcodec/cbs_sei.h | 255 ++++++++++++++++++ libavcodec/cbs_sei_syntax_template.c | 215 +++++++++++++-- libavcodec/h264_metadata_bsf.c | 113 ++++---- libavcodec/vaapi_encode_h264.c | 51 ++-- libavcodec/vaapi_encode_h265.c | 38 +-- 12 files changed, 1107 insertions(+), 763 deletions(-) create mode 100644 libavcodec/cbs_sei.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 450781886d..6e12a8171d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -71,8 +71,8 @@ OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o OBJS-$(CONFIG_CBS) += cbs.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o -OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o h2645_parse.o -OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o h2645_parse.o +OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o +OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index 81113f1ad0..9eb97eae24 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -291,34 +291,9 @@ typedef struct H264RawSEIDisplayOrientation { uint8_t display_orientation_extension_flag; } H264RawSEIDisplayOrientation; -typedef struct H264RawSEIPayload { - uint32_t payload_type; - uint32_t payload_size; - union { - H264RawSEIBufferingPeriod buffering_period; - H264RawSEIPicTiming pic_timing; - H264RawSEIPanScanRect pan_scan_rect; - // H264RawSEIFiller filler -> no fields. - SEIRawUserDataRegistered user_data_registered; - SEIRawUserDataUnregistered user_data_unregistered; - H264RawSEIRecoveryPoint recovery_point; - H264RawSEIDisplayOrientation display_orientation; - SEIRawMasteringDisplayColourVolume mastering_display_colour_volume; - SEIRawAlternativeTransferCharacteristics - alternative_transfer_characteristics; - struct { - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; - } other; - } payload; -} H264RawSEIPayload; - typedef struct H264RawSEI { H264RawNALUnitHeader nal_unit_header; - - H264RawSEIPayload payload[H264_MAX_SEI_PAYLOADS]; - uint8_t payload_count; + SEIRawMessageList message_list; } H264RawSEI; typedef struct H264RawSliceHeader { @@ -438,27 +413,4 @@ typedef struct CodedBitstreamH264Context { uint8_t last_slice_nal_unit_type; } CodedBitstreamH264Context; - -/** - * Add an SEI message to an access unit. - * - * On success, the payload will be owned by a unit in access_unit; - * on failure, the content of the payload will be freed. - */ -int ff_cbs_h264_add_sei_message(CodedBitstreamFragment *access_unit, - H264RawSEIPayload *payload); - -/** - * Delete an SEI message from an access unit. - * - * Deletes from nal_unit, which must be an SEI NAL unit. If this is the - * last message in nal_unit, also deletes it from access_unit. - * - * Requires nal_unit to be a unit in access_unit and position to be >= 0 - * and < the payload count of the SEI nal_unit. - */ -void ff_cbs_h264_delete_sei_message(CodedBitstreamFragment *access_unit, - CodedBitstreamUnit *nal_unit, - int position); - #endif /* AVCODEC_CBS_H264_H */ diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index ce58e47a36..a00bc27370 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -348,6 +348,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #define more_rbsp_data(var) ((var) = cbs_h2645_read_more_rbsp_data(rw)) +#define bit_position(rw) (get_bits_count(rw)) #define byte_alignment(rw) (get_bits_count(rw) % 8) #define allocate(name, size) do { \ @@ -379,6 +380,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef xse #undef infer #undef more_rbsp_data +#undef bit_position #undef byte_alignment #undef allocate @@ -424,6 +426,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #define more_rbsp_data(var) (var) +#define bit_position(rw) (put_bits_count(rw)) #define byte_alignment(rw) (put_bits_count(rw) % 8) #define allocate(name, size) do { \ @@ -460,6 +463,7 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef se #undef infer #undef more_rbsp_data +#undef bit_position #undef byte_alignment #undef allocate @@ -1372,36 +1376,11 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } -static void cbs_h264_free_sei_payload(H264RawSEIPayload *payload) -{ - switch (payload->payload_type) { - case H264_SEI_TYPE_BUFFERING_PERIOD: - case H264_SEI_TYPE_PIC_TIMING: - case H264_SEI_TYPE_PAN_SCAN_RECT: - case H264_SEI_TYPE_RECOVERY_POINT: - case H264_SEI_TYPE_DISPLAY_ORIENTATION: - case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: - case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: - break; - case H264_SEI_TYPE_USER_DATA_REGISTERED: - av_buffer_unref(&payload->payload.user_data_registered.data_ref); - break; - case H264_SEI_TYPE_USER_DATA_UNREGISTERED: - av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); - break; - default: - av_buffer_unref(&payload->payload.other.data_ref); - break; - } -} - static void cbs_h264_free_sei(void *opaque, uint8_t *content) { H264RawSEI *sei = (H264RawSEI*)content; - int i; - for (i = 0; i < sei->payload_count; i++) - cbs_h264_free_sei_payload(&sei->payload[i]); - av_freep(&content); + ff_cbs_sei_free_message_list(&sei->message_list); + av_free(content); } static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { @@ -1433,42 +1412,11 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; -static void cbs_h265_free_sei_payload(H265RawSEIPayload *payload) -{ - switch (payload->payload_type) { - case HEVC_SEI_TYPE_BUFFERING_PERIOD: - case HEVC_SEI_TYPE_PICTURE_TIMING: - case HEVC_SEI_TYPE_PAN_SCAN_RECT: - case HEVC_SEI_TYPE_RECOVERY_POINT: - case HEVC_SEI_TYPE_DISPLAY_ORIENTATION: - case HEVC_SEI_TYPE_ACTIVE_PARAMETER_SETS: - case HEVC_SEI_TYPE_DECODED_PICTURE_HASH: - case HEVC_SEI_TYPE_TIME_CODE: - case HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO: - case HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO: - case HEVC_SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS: - case HEVC_SEI_TYPE_ALPHA_CHANNEL_INFO: - break; - case HEVC_SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35: - av_buffer_unref(&payload->payload.user_data_registered.data_ref); - break; - case HEVC_SEI_TYPE_USER_DATA_UNREGISTERED: - av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); - break; - default: - av_buffer_unref(&payload->payload.other.data_ref); - break; - } - av_buffer_unref(&payload->extension_data.data_ref); -} - static void cbs_h265_free_sei(void *opaque, uint8_t *content) { H265RawSEI *sei = (H265RawSEI*)content; - int i; - for (i = 0; i < sei->payload_count; i++) - cbs_h265_free_sei_payload(&sei->payload[i]); - av_freep(&content); + ff_cbs_sei_free_message_list(&sei->message_list); + av_free(content); } static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { @@ -1548,92 +1496,164 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; -int ff_cbs_h264_add_sei_message(CodedBitstreamFragment *au, - H264RawSEIPayload *payload) -{ - H264RawSEI *sei = NULL; - int err, i; - - // Find an existing SEI NAL unit to add to. - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SEI) { - sei = au->units[i].content; - if (sei->payload_count < H264_MAX_SEI_PAYLOADS) - break; - - sei = NULL; - } - } - - if (!sei) { - // Need to make a new SEI NAL unit. Insert it before the first - // slice data NAL unit; if no slice data, add at the end. - AVBufferRef *sei_ref; - - sei = av_mallocz(sizeof(*sei)); - if (!sei) { - err = AVERROR(ENOMEM); - goto fail; - } - - sei->nal_unit_header.nal_unit_type = H264_NAL_SEI; - sei->nal_unit_header.nal_ref_idc = 0; - - sei_ref = av_buffer_create((uint8_t*)sei, sizeof(*sei), - &cbs_h264_free_sei, NULL, 0); - if (!sei_ref) { - av_freep(&sei); - err = AVERROR(ENOMEM); - goto fail; - } - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SLICE || - au->units[i].type == H264_NAL_IDR_SLICE) - break; - } - - err = ff_cbs_insert_unit_content(au, i, H264_NAL_SEI, - sei, sei_ref); - av_buffer_unref(&sei_ref); - if (err < 0) - goto fail; - } +static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { + { + SEI_TYPE_FILLER_PAYLOAD, + 1, 1, + sizeof(SEIRawFillerPayload), + SEI_MESSAGE_RW(sei, filler_payload), + }, + { + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + 1, 1, + sizeof(SEIRawUserDataRegistered), + SEI_MESSAGE_RW(sei, user_data_registered), + }, + { + SEI_TYPE_USER_DATA_UNREGISTERED, + 1, 1, + sizeof(SEIRawUserDataUnregistered), + SEI_MESSAGE_RW(sei, user_data_unregistered), + }, + { + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME, + 1, 0, + sizeof(SEIRawMasteringDisplayColourVolume), + SEI_MESSAGE_RW(sei, mastering_display_colour_volume), + }, + { + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + 1, 0, + sizeof(SEIRawContentLightLevelInfo), + SEI_MESSAGE_RW(sei, content_light_level_info), + }, + { + SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS, + 1, 0, + sizeof(SEIRawAlternativeTransferCharacteristics), + SEI_MESSAGE_RW(sei, alternative_transfer_characteristics), + }, + SEI_MESSAGE_TYPE_END, +}; - memcpy(&sei->payload[sei->payload_count], payload, sizeof(*payload)); - ++sei->payload_count; +static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { + { + SEI_TYPE_BUFFERING_PERIOD, + 1, 0, + sizeof(H264RawSEIBufferingPeriod), + SEI_MESSAGE_RW(h264, sei_buffering_period), + }, + { + SEI_TYPE_PIC_TIMING, + 1, 0, + sizeof(H264RawSEIPicTiming), + SEI_MESSAGE_RW(h264, sei_pic_timing), + }, + { + SEI_TYPE_PAN_SCAN_RECT, + 1, 0, + sizeof(H264RawSEIPanScanRect), + SEI_MESSAGE_RW(h264, sei_pan_scan_rect), + }, + { + SEI_TYPE_RECOVERY_POINT, + 1, 0, + sizeof(H264RawSEIRecoveryPoint), + SEI_MESSAGE_RW(h264, sei_recovery_point), + }, + { + SEI_TYPE_DISPLAY_ORIENTATION, + 1, 0, + sizeof(H264RawSEIDisplayOrientation), + SEI_MESSAGE_RW(h264, sei_display_orientation), + }, + SEI_MESSAGE_TYPE_END +}; - return 0; -fail: - cbs_h264_free_sei_payload(payload); - return err; -} +static const SEIMessageTypeDescriptor cbs_sei_h265_types[] = { + { + SEI_TYPE_BUFFERING_PERIOD, + 1, 0, + sizeof(H265RawSEIBufferingPeriod), + SEI_MESSAGE_RW(h265, sei_buffering_period), + }, + { + SEI_TYPE_PIC_TIMING, + 1, 0, + sizeof(H265RawSEIPicTiming), + SEI_MESSAGE_RW(h265, sei_pic_timing), + }, + { + SEI_TYPE_PAN_SCAN_RECT, + 1, 0, + sizeof(H265RawSEIPanScanRect), + SEI_MESSAGE_RW(h265, sei_pan_scan_rect), + }, + { + SEI_TYPE_RECOVERY_POINT, + 1, 0, + sizeof(H265RawSEIRecoveryPoint), + SEI_MESSAGE_RW(h265, sei_recovery_point), + }, + { + SEI_TYPE_DISPLAY_ORIENTATION, + 1, 0, + sizeof(H265RawSEIDisplayOrientation), + SEI_MESSAGE_RW(h265, sei_display_orientation), + }, + { + SEI_TYPE_ACTIVE_PARAMETER_SETS, + 1, 0, + sizeof(H265RawSEIActiveParameterSets), + SEI_MESSAGE_RW(h265, sei_active_parameter_sets), + }, + { + SEI_TYPE_DECODED_PICTURE_HASH, + 0, 1, + sizeof(H265RawSEIDecodedPictureHash), + SEI_MESSAGE_RW(h265, sei_decoded_picture_hash), + }, + { + SEI_TYPE_TIME_CODE, + 1, 0, + sizeof(H265RawSEITimeCode), + SEI_MESSAGE_RW(h265, sei_time_code), + }, + { + SEI_TYPE_ALPHA_CHANNEL_INFO, + 1, 0, + sizeof(H265RawSEIAlphaChannelInfo), + SEI_MESSAGE_RW(h265, sei_alpha_channel_info), + }, + SEI_MESSAGE_TYPE_END +}; -void ff_cbs_h264_delete_sei_message(CodedBitstreamFragment *au, - CodedBitstreamUnit *nal, - int position) +const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, + int payload_type) { - H264RawSEI *sei = nal->content; - - av_assert0(nal->type == H264_NAL_SEI); - av_assert0(position >= 0 && position < sei->payload_count); - - if (position == 0 && sei->payload_count == 1) { - // Deleting NAL unit entirely. - int i; + const SEIMessageTypeDescriptor *codec_list; + int i; - for (i = 0; i < au->nb_units; i++) { - if (&au->units[i] == nal) - break; - } + for (i = 0; cbs_sei_common_types[i].type >= 0; i++) { + if (cbs_sei_common_types[i].type == payload_type) + return &cbs_sei_common_types[i]; + } - ff_cbs_delete_unit(au, i); - } else { - cbs_h264_free_sei_payload(&sei->payload[position]); + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + codec_list = cbs_sei_h264_types; + break; + case AV_CODEC_ID_H265: + codec_list = cbs_sei_h265_types; + break; + default: + return NULL; + } - --sei->payload_count; - memmove(sei->payload + position, - sei->payload + position + 1, - (sei->payload_count - position) * sizeof(*sei->payload)); + for (i = 0; codec_list[i].type >= 0; i++) { + if (codec_list[i].type == payload_type) + return &codec_list[i]; } + + return NULL; } diff --git a/libavcodec/cbs_h264_syntax_template.c b/libavcodec/cbs_h264_syntax_template.c index 76ed51cc7b..9587f33985 100644 --- a/libavcodec/cbs_h264_syntax_template.c +++ b/libavcodec/cbs_h264_syntax_template.c @@ -511,7 +511,8 @@ static int FUNC(pps)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIBufferingPeriod *current) + H264RawSEIBufferingPeriod *current, + SEIMessageState *sei) { CodedBitstreamH264Context *h264 = ctx->priv_data; const H264RawSPS *sps; @@ -604,7 +605,8 @@ static int FUNC(sei_pic_timestamp)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIPicTiming *current) + H264RawSEIPicTiming *current, + SEIMessageState *sei) { CodedBitstreamH264Context *h264 = ctx->priv_data; const H264RawSPS *sps; @@ -675,7 +677,8 @@ static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIPanScanRect *current) + H264RawSEIPanScanRect *current, + SEIMessageState *sei) { int err, i; @@ -701,7 +704,8 @@ static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIRecoveryPoint *current) + H264RawSEIRecoveryPoint *current, + SEIMessageState *sei) { int err; @@ -716,7 +720,8 @@ static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, } static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIDisplayOrientation *current) + H264RawSEIDisplayOrientation *current, + SEIMessageState *sei) { int err; @@ -734,171 +739,17 @@ static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext * return 0; } -static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, - H264RawSEIPayload *current) -{ - int err, i; - int start_position, end_position; - -#ifdef READ - start_position = get_bits_count(rw); -#else - start_position = put_bits_count(rw); -#endif - - switch (current->payload_type) { - case H264_SEI_TYPE_BUFFERING_PERIOD: - CHECK(FUNC(sei_buffering_period) - (ctx, rw, ¤t->payload.buffering_period)); - break; - case H264_SEI_TYPE_PIC_TIMING: - CHECK(FUNC(sei_pic_timing) - (ctx, rw, ¤t->payload.pic_timing)); - break; - case H264_SEI_TYPE_PAN_SCAN_RECT: - CHECK(FUNC(sei_pan_scan_rect) - (ctx, rw, ¤t->payload.pan_scan_rect)); - break; - case H264_SEI_TYPE_FILLER_PAYLOAD: - { - for (i = 0; i < current->payload_size; i++) - fixed(8, ff_byte, 0xff); - } - break; - case H264_SEI_TYPE_USER_DATA_REGISTERED: - CHECK(FUNC_SEI(sei_user_data_registered) - (ctx, rw, ¤t->payload.user_data_registered, ¤t->payload_size)); - break; - case H264_SEI_TYPE_USER_DATA_UNREGISTERED: - CHECK(FUNC_SEI(sei_user_data_unregistered) - (ctx, rw, ¤t->payload.user_data_unregistered, ¤t->payload_size)); - break; - case H264_SEI_TYPE_RECOVERY_POINT: - CHECK(FUNC(sei_recovery_point) - (ctx, rw, ¤t->payload.recovery_point)); - break; - case H264_SEI_TYPE_DISPLAY_ORIENTATION: - CHECK(FUNC(sei_display_orientation) - (ctx, rw, ¤t->payload.display_orientation)); - break; - case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: - CHECK(FUNC_SEI(sei_mastering_display_colour_volume) - (ctx, rw, ¤t->payload.mastering_display_colour_volume)); - break; - case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: - CHECK(FUNC_SEI(sei_alternative_transfer_characteristics) - (ctx, rw, ¤t->payload.alternative_transfer_characteristics)); - break; - default: - { -#ifdef READ - current->payload.other.data_length = current->payload_size; -#endif - allocate(current->payload.other.data, current->payload.other.data_length); - for (i = 0; i < current->payload.other.data_length; i++) - xu(8, payload_byte[i], current->payload.other.data[i], 0, 255, 1, i); - } - } - - if (byte_alignment(rw)) { - fixed(1, bit_equal_to_one, 1); - while (byte_alignment(rw)) - fixed(1, bit_equal_to_zero, 0); - } - -#ifdef READ - end_position = get_bits_count(rw); - if (end_position < start_position + 8 * current->payload_size) { - av_log(ctx->log_ctx, AV_LOG_ERROR, "Incorrect SEI payload length: " - "header %"PRIu32" bits, actually %d bits.\n", - 8 * current->payload_size, - end_position - start_position); - return AVERROR_INVALIDDATA; - } -#else - end_position = put_bits_count(rw); - current->payload_size = (end_position - start_position) / 8; -#endif - - return 0; -} - static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, H264RawSEI *current) { - int err, k; + int err; HEADER("Supplemental Enhancement Information"); CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, 1 << H264_NAL_SEI)); -#ifdef READ - for (k = 0; k < H264_MAX_SEI_PAYLOADS; k++) { - uint32_t payload_type = 0; - uint32_t payload_size = 0; - uint32_t tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_type += 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - payload_type += tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_size += 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - payload_size += tmp; - - current->payload[k].payload_type = payload_type; - current->payload[k].payload_size = payload_size; - - current->payload_count++; - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k])); - - if (!cbs_h2645_read_more_rbsp_data(rw)) - break; - } - if (k >= H264_MAX_SEI_PAYLOADS) { - av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many payloads in " - "SEI message: found %d.\n", k); - return AVERROR_INVALIDDATA; - } -#else - for (k = 0; k < current->payload_count; k++) { - PutBitContext start_state; - uint32_t tmp; - int need_size, i; - - // Somewhat clumsy: we write the payload twice when - // we don't know the size in advance. This will mess - // with trace output, but is otherwise harmless. - start_state = *rw; - need_size = !current->payload[k].payload_size; - for (i = 0; i < 1 + need_size; i++) { - *rw = start_state; - - tmp = current->payload[k].payload_type; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - - tmp = current->payload[k].payload_size; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k])); - } - } -#endif + CHECK(FUNC_SEI(message_list)(ctx, rw, ¤t->message_list, 1)); CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index d8e93e3bb8..738cbeec2c 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -658,40 +658,9 @@ typedef struct H265RawSEIAlphaChannelInfo { uint8_t alpha_channel_clip_type_flag; } H265RawSEIAlphaChannelInfo; -typedef struct H265RawSEIPayload { - uint32_t payload_type; - uint32_t payload_size; - union { - H265RawSEIBufferingPeriod buffering_period; - H265RawSEIPicTiming pic_timing; - H265RawSEIPanScanRect pan_scan_rect; - SEIRawUserDataRegistered user_data_registered; - SEIRawUserDataUnregistered user_data_unregistered; - H265RawSEIRecoveryPoint recovery_point; - H265RawSEIDisplayOrientation display_orientation; - H265RawSEIActiveParameterSets active_parameter_sets; - H265RawSEIDecodedPictureHash decoded_picture_hash; - H265RawSEITimeCode time_code; - SEIRawMasteringDisplayColourVolume - mastering_display_colour_volume; - SEIRawContentLightLevelInfo content_light_level; - SEIRawAlternativeTransferCharacteristics - alternative_transfer_characteristics; - H265RawSEIAlphaChannelInfo alpha_channel_info; - struct { - uint8_t *data; - AVBufferRef *data_ref; - size_t data_length; - } other; - } payload; - H265RawExtensionData extension_data; -} H265RawSEIPayload; - typedef struct H265RawSEI { H265RawNALUnitHeader nal_unit_header; - - H265RawSEIPayload payload[H265_MAX_SEI_PAYLOADS]; - uint8_t payload_count; + SEIRawMessageList message_list; } H265RawSEI; typedef struct CodedBitstreamH265Context { diff --git a/libavcodec/cbs_h265_syntax_template.c b/libavcodec/cbs_h265_syntax_template.c index e4cc1a9be8..d09934cfeb 100644 --- a/libavcodec/cbs_h265_syntax_template.c +++ b/libavcodec/cbs_h265_syntax_template.c @@ -1596,10 +1596,9 @@ static int FUNC(slice_segment_header)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIBufferingPeriod *current, - uint32_t *payload_size, - int *more_data) +static int FUNC(sei_buffering_period) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIBufferingPeriod *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawSPS *sps; @@ -1687,7 +1686,7 @@ static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, #ifdef READ end_pos = get_bits_count(rw); - if (cbs_h265_payload_extension_present(rw, *payload_size, + if (cbs_h265_payload_extension_present(rw, sei->payload_size, end_pos - start_pos)) flag(use_alt_cpb_params_flag); else @@ -1695,20 +1694,21 @@ static int FUNC(sei_buffering_period)(CodedBitstreamContext *ctx, RWContext *rw, #else // If unknown extension data exists, then use_alt_cpb_params_flag is // coded in the bitstream and must be written even if it's 0. - if (current->use_alt_cpb_params_flag || *more_data) { + if (current->use_alt_cpb_params_flag || sei->extension_present) { flag(use_alt_cpb_params_flag); // Ensure this bit is not the last in the payload by making the // more_data_in_payload() check evaluate to true, so it may not // be mistaken as something else by decoders. - *more_data = 1; + sei->extension_present = 1; } #endif return 0; } -static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIPicTiming *current) +static int FUNC(sei_pic_timing) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIPicTiming *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawSPS *sps; @@ -1782,8 +1782,9 @@ static int FUNC(sei_pic_timing)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIPanScanRect *current) +static int FUNC(sei_pan_scan_rect) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIPanScanRect *current, SEIMessageState *sei) { int err, i; @@ -1808,8 +1809,9 @@ static int FUNC(sei_pan_scan_rect)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIRecoveryPoint *current) +static int FUNC(sei_recovery_point) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIRecoveryPoint *current, SEIMessageState *sei) { int err; @@ -1823,8 +1825,9 @@ static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIDisplayOrientation *current) +static int FUNC(sei_display_orientation) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIDisplayOrientation *current, SEIMessageState *sei) { int err; @@ -1841,8 +1844,9 @@ static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext * return 0; } -static int FUNC(sei_active_parameter_sets)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIActiveParameterSets *current) +static int FUNC(sei_active_parameter_sets) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIActiveParameterSets *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawVPS *vps; @@ -1877,8 +1881,9 @@ static int FUNC(sei_active_parameter_sets)(CodedBitstreamContext *ctx, RWContext return 0; } -static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIDecodedPictureHash *current) +static int FUNC(sei_decoded_picture_hash) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIDecodedPictureHash *current, SEIMessageState *sei) { CodedBitstreamH265Context *h265 = ctx->priv_data; const H265RawSPS *sps = h265->active_sps; @@ -1908,8 +1913,9 @@ static int FUNC(sei_decoded_picture_hash)(CodedBitstreamContext *ctx, RWContext return 0; } -static int FUNC(sei_time_code)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEITimeCode *current) +static int FUNC(sei_time_code) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEITimeCode *current, SEIMessageState *sei) { int err, i; @@ -1958,9 +1964,9 @@ static int FUNC(sei_time_code)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } -static int FUNC(sei_alpha_channel_info)(CodedBitstreamContext *ctx, - RWContext *rw, - H265RawSEIAlphaChannelInfo *current) +static int FUNC(sei_alpha_channel_info) + (CodedBitstreamContext *ctx, RWContext *rw, + H265RawSEIAlphaChannelInfo *current, SEIMessageState *sei) { int err, length; @@ -1986,156 +1992,10 @@ static int FUNC(sei_alpha_channel_info)(CodedBitstreamContext *ctx, return 0; } -static int FUNC(payload_extension)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawExtensionData *current, uint32_t payload_size, - int cur_pos) -{ - int err; - size_t byte_length, k; - -#ifdef READ - GetBitContext tmp; - int bits_left, payload_zero_bits; - - if (!cbs_h265_payload_extension_present(rw, payload_size, cur_pos)) - return 0; - - bits_left = 8 * payload_size - cur_pos; - tmp = *rw; - if (bits_left > 8) - skip_bits_long(&tmp, bits_left - 8); - payload_zero_bits = get_bits(&tmp, FFMIN(bits_left, 8)); - if (!payload_zero_bits) - return AVERROR_INVALIDDATA; - payload_zero_bits = ff_ctz(payload_zero_bits); - current->bit_length = bits_left - payload_zero_bits - 1; - allocate(current->data, (current->bit_length + 7) / 8); -#endif - - byte_length = (current->bit_length + 7) / 8; - for (k = 0; k < byte_length; k++) { - int length = FFMIN(current->bit_length - k * 8, 8); - xu(length, reserved_payload_extension_data, current->data[k], - 0, MAX_UINT_BITS(length), 0); - } - - return 0; -} - -static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, - H265RawSEIPayload *current, int prefix) -{ - int err, i; - int start_position, current_position; - int more_data = !!current->extension_data.bit_length; - -#ifdef READ - start_position = get_bits_count(rw); -#else - start_position = put_bits_count(rw); -#endif - - switch (current->payload_type) { -#define SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid) do { \ - if (prefix && !prefix_valid) { \ - av_log(ctx->log_ctx, AV_LOG_ERROR, "SEI type %s invalid " \ - "as prefix SEI!\n", #name); \ - return AVERROR_INVALIDDATA; \ - } \ - if (!prefix && !suffix_valid) { \ - av_log(ctx->log_ctx, AV_LOG_ERROR, "SEI type %s invalid " \ - "as suffix SEI!\n", #name); \ - return AVERROR_INVALIDDATA; \ - } \ - } while (0) -#define SEI_TYPE_N(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name)); \ - break -#define SEI_TYPE_S(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name, \ - ¤t->payload_size)); \ - break -#define SEI_TYPE_E(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC(sei_ ## name)(ctx, rw, ¤t->payload.name, \ - ¤t->payload_size, \ - &more_data)); \ - break - -#define SEI_TYPE_N2(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC_SEI(sei_ ## name)(ctx, rw, ¤t->payload.name)); \ - break -#define SEI_TYPE_S2(type, prefix_valid, suffix_valid, name) \ - case HEVC_SEI_TYPE_ ## type: \ - SEI_TYPE_CHECK_VALID(name, prefix_valid, suffix_valid); \ - CHECK(FUNC_SEI(sei_ ## name)(ctx, rw, ¤t->payload.name, \ - ¤t->payload_size)); \ - break - - SEI_TYPE_E(BUFFERING_PERIOD, 1, 0, buffering_period); - SEI_TYPE_N(PICTURE_TIMING, 1, 0, pic_timing); - SEI_TYPE_N(PAN_SCAN_RECT, 1, 0, pan_scan_rect); - SEI_TYPE_S2(USER_DATA_REGISTERED_ITU_T_T35, - 1, 1, user_data_registered); - SEI_TYPE_S2(USER_DATA_UNREGISTERED, 1, 1, user_data_unregistered); - SEI_TYPE_N(RECOVERY_POINT, 1, 0, recovery_point); - SEI_TYPE_N(DISPLAY_ORIENTATION, 1, 0, display_orientation); - SEI_TYPE_N(ACTIVE_PARAMETER_SETS, 1, 0, active_parameter_sets); - SEI_TYPE_N(DECODED_PICTURE_HASH, 0, 1, decoded_picture_hash); - SEI_TYPE_N(TIME_CODE, 1, 0, time_code); - SEI_TYPE_N2(MASTERING_DISPLAY_INFO, 1, 0, mastering_display_colour_volume); - SEI_TYPE_N2(CONTENT_LIGHT_LEVEL_INFO,1, 0, content_light_level); - SEI_TYPE_N2(ALTERNATIVE_TRANSFER_CHARACTERISTICS, - 1, 0, alternative_transfer_characteristics); - SEI_TYPE_N(ALPHA_CHANNEL_INFO, 1, 0, alpha_channel_info); - -#undef SEI_TYPE - default: - { -#ifdef READ - current->payload.other.data_length = current->payload_size; -#endif - allocate(current->payload.other.data, current->payload.other.data_length); - - for (i = 0; i < current->payload_size; i++) - xu(8, payload_byte[i], current->payload.other.data[i], 0, 255, - 1, i); - } - } - - // more_data_in_payload() -#ifdef READ - current_position = get_bits_count(rw) - start_position; - if (current_position < 8 * current->payload_size) { -#else - current_position = put_bits_count(rw) - start_position; - if (byte_alignment(rw) || more_data) { -#endif - CHECK(FUNC(payload_extension)(ctx, rw, ¤t->extension_data, - current->payload_size, current_position)); - fixed(1, bit_equal_to_one, 1); - while (byte_alignment(rw)) - fixed(1, bit_equal_to_zero, 0); - } - -#ifdef WRITE - current->payload_size = (put_bits_count(rw) - start_position) >> 3; -#endif - - return 0; -} - static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, H265RawSEI *current, int prefix) { - int err, k; + int err; if (prefix) HEADER("Prefix Supplemental Enhancement Information"); @@ -2146,72 +2006,7 @@ static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, prefix ? HEVC_NAL_SEI_PREFIX : HEVC_NAL_SEI_SUFFIX)); -#ifdef READ - for (k = 0; k < H265_MAX_SEI_PAYLOADS; k++) { - uint32_t payload_type = 0; - uint32_t payload_size = 0; - uint32_t tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_type += 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - payload_type += tmp; - - while (show_bits(rw, 8) == 0xff) { - fixed(8, ff_byte, 0xff); - payload_size += 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - payload_size += tmp; - - current->payload[k].payload_type = payload_type; - current->payload[k].payload_size = payload_size; - - current->payload_count++; - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k], prefix)); - - if (!cbs_h2645_read_more_rbsp_data(rw)) - break; - } - if (k >= H265_MAX_SEI_PAYLOADS) { - av_log(ctx->log_ctx, AV_LOG_ERROR, "Too many payloads in " - "SEI message: found %d.\n", k); - return AVERROR_INVALIDDATA; - } -#else - for (k = 0; k < current->payload_count; k++) { - PutBitContext start_state; - uint32_t tmp; - int need_size, i; - - // Somewhat clumsy: we write the payload twice when - // we don't know the size in advance. This will mess - // with trace output, but is otherwise harmless. - start_state = *rw; - need_size = !current->payload[k].payload_size; - for (i = 0; i < 1 + need_size; i++) { - *rw = start_state; - - tmp = current->payload[k].payload_type; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_type_byte, tmp, 0, 254, 0); - - tmp = current->payload[k].payload_size; - while (tmp >= 255) { - fixed(8, ff_byte, 0xff); - tmp -= 255; - } - xu(8, last_payload_size_byte, tmp, 0, 254, 0); - - CHECK(FUNC(sei_payload)(ctx, rw, ¤t->payload[k], prefix)); - } - } -#endif + CHECK(FUNC_SEI(message_list)(ctx, rw, ¤t->message_list, prefix)); CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c new file mode 100644 index 0000000000..323997b600 --- /dev/null +++ b/libavcodec/cbs_sei.c @@ -0,0 +1,369 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs.h" +#include "cbs_internal.h" +#include "cbs_h264.h" +#include "cbs_h265.h" +#include "cbs_sei.h" + +static void cbs_free_user_data_registered(void *opaque, uint8_t *data) +{ + SEIRawUserDataRegistered *udr = (SEIRawUserDataRegistered*)data; + av_buffer_unref(&udr->data_ref); + av_free(udr); +} + +static void cbs_free_user_data_unregistered(void *opaque, uint8_t *data) +{ + SEIRawUserDataUnregistered *udu = (SEIRawUserDataUnregistered*)data; + av_buffer_unref(&udu->data_ref); + av_free(udu); +} + +int ff_cbs_sei_alloc_message_payload(SEIRawMessage *message, + const SEIMessageTypeDescriptor *desc) +{ + void (*free_func)(void*, uint8_t*); + + av_assert0(message->payload == NULL && + message->payload_ref == NULL); + message->payload_type = desc->type; + + if (desc->type == SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35) + free_func = &cbs_free_user_data_registered; + else if (desc->type == SEI_TYPE_USER_DATA_UNREGISTERED) + free_func = &cbs_free_user_data_unregistered; + else + free_func = NULL; + + if (free_func) { + message->payload = av_mallocz(desc->size); + if (!message->payload) + return AVERROR(ENOMEM); + message->payload_ref = + av_buffer_create(message->payload, desc->size, + free_func, NULL, 0); + } else { + message->payload_ref = av_buffer_alloc(desc->size); + } + if (!message->payload_ref) + return AVERROR(ENOMEM); + message->payload = message->payload_ref->data; + + return 0; +} + +int ff_cbs_sei_list_add(SEIRawMessageList *list) +{ + void *ptr; + int old_count = list->nb_messages_allocated; + + av_assert0(list->nb_messages <= old_count); + if (list->nb_messages + 1 > old_count) { + int new_count = 2 * old_count + 1; + + ptr = av_realloc_array(list->messages, + new_count, sizeof(*list->messages)); + if (!ptr) + return AVERROR(ENOMEM); + + list->messages = ptr; + list->nb_messages_allocated = new_count; + + // Zero the newly-added entries. + memset(list->messages + old_count, 0, + (new_count - old_count) * sizeof(*list->messages)); + } + ++list->nb_messages; + return 0; +} + +void ff_cbs_sei_free_message_list(SEIRawMessageList *list) +{ + for (int i = 0; i < list->nb_messages; i++) { + SEIRawMessage *message = &list->messages[i]; + av_buffer_unref(&message->payload_ref); + av_buffer_unref(&message->extension_data_ref); + } + av_free(list->messages); +} + +static int cbs_sei_get_unit(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + int prefix, + CodedBitstreamUnit **sei_unit) +{ + CodedBitstreamUnit *unit; + int sei_type, highest_vcl_type, err, i, position; + + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + // (We can ignore auxiliary slices because we only have prefix + // SEI in H.264 and an auxiliary picture must always follow a + // primary picture.) + highest_vcl_type = H264_NAL_IDR_SLICE; + if (prefix) + sei_type = H264_NAL_SEI; + else + return AVERROR(EINVAL); + break; + case AV_CODEC_ID_H265: + highest_vcl_type = HEVC_NAL_RSV_VCL31; + if (prefix) + sei_type = HEVC_NAL_SEI_PREFIX; + else + sei_type = HEVC_NAL_SEI_SUFFIX; + break; + default: + return AVERROR(EINVAL); + } + + // Find an existing SEI NAL unit of the right type. + unit = NULL; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == sei_type) { + unit = &au->units[i]; + break; + } + } + + if (unit) { + *sei_unit = unit; + return 0; + } + + // Need to add a new SEI NAL unit ... + if (prefix) { + // ... before the first VCL NAL unit. + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type < highest_vcl_type) + break; + } + position = i; + } else { + // ... after the last VCL NAL unit. + for (i = au->nb_units; i >= 0; i--) { + if (au->units[i].type < highest_vcl_type) + break; + } + if (i < 0) { + // No VCL units; just put it at the end. + position = -1; + } else { + position = i + 1; + } + } + + err = ff_cbs_insert_unit_content(au, position, sei_type, + NULL, NULL); + if (err < 0) + return err; + unit = &au->units[position]; + unit->type = sei_type; + + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) + return err; + + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + { + H264RawSEI sei = { + .nal_unit_header = { + .nal_ref_idc = 0, + .nal_unit_type = sei_type, + }, + }; + memcpy(unit->content, &sei, sizeof(sei)); + } + break; + case AV_CODEC_ID_H265: + { + H265RawSEI sei = { + .nal_unit_header = { + .nal_unit_type = sei_type, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }, + }; + memcpy(unit->content, &sei, sizeof(sei)); + } + break; + default: + av_assert0(0); + } + + *sei_unit = unit; + return 0; +} + +static int cbs_sei_get_message_list(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit, + SEIRawMessageList **list) +{ + switch (ctx->codec->codec_id) { + case AV_CODEC_ID_H264: + { + H264RawSEI *sei = unit->content; + if (unit->type != H264_NAL_SEI) + return AVERROR(EINVAL); + *list = &sei->message_list; + } + break; + case AV_CODEC_ID_H265: + { + H265RawSEI *sei = unit->content; + if (unit->type != HEVC_NAL_SEI_PREFIX && + unit->type != HEVC_NAL_SEI_SUFFIX) + return AVERROR(EINVAL); + *list = &sei->message_list; + } + break; + default: + return AVERROR(EINVAL); + } + + return 0; +} + +int ff_cbs_sei_add_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + int prefix, + uint32_t payload_type, + void *payload_data, + AVBufferRef *payload_buf) +{ + const SEIMessageTypeDescriptor *desc; + CodedBitstreamUnit *unit; + SEIRawMessageList *list; + SEIRawMessage *message; + AVBufferRef *payload_ref; + int err; + + desc = ff_cbs_sei_find_type(ctx, payload_type); + if (!desc) + return AVERROR(EINVAL); + + if (payload_buf) { + payload_ref = av_buffer_ref(payload_buf); + if (!payload_ref) + return AVERROR(ENOMEM); + } else { + payload_ref = NULL; + } + + // Find an existing SEI unit or make a new one to add to. + err = cbs_sei_get_unit(ctx, au, prefix, &unit); + if (err < 0) + return err; + + // Find the message list inside the codec-dependent unit. + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + return err; + + // Add a new message to the message list. + err = ff_cbs_sei_list_add(list); + if (err < 0) + return err; + + message = &list->messages[list->nb_messages - 1]; + + message->payload_type = payload_type; + message->payload = payload_data; + message->payload_ref = payload_ref; + + return 0; +} + +int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type, + SEIRawMessage **iter) +{ + int err, i, j, found; + + found = 0; + for (i = 0; i < au->nb_units; i++) { + CodedBitstreamUnit *unit = &au->units[i]; + SEIRawMessageList *list; + + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + continue; + + for (j = 0; j < list->nb_messages; j++) { + SEIRawMessage *message = &list->messages[j]; + + if (message->payload_type == payload_type) { + if (!*iter || found) { + *iter = message; + return 0; + } + if (message == *iter) + found = 1; + } + } + } + + return AVERROR(ENOENT); +} + +static void cbs_sei_delete_message(SEIRawMessageList *list, + int position) +{ + SEIRawMessage *message; + + av_assert0(0 <= position && position < list->nb_messages); + + message = &list->messages[position]; + av_buffer_unref(&message->payload_ref); + av_buffer_unref(&message->extension_data_ref); + + --list->nb_messages; + + if (list->nb_messages > 0) { + memmove(list->messages + position, + list->messages + position + 1, + (list->nb_messages - position) * sizeof(*list->messages)); + } +} + +void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type) +{ + int err, i, j; + + for (i = 0; i < au->nb_units; i++) { + CodedBitstreamUnit *unit = &au->units[i]; + SEIRawMessageList *list; + + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + continue; + + for (j = 0; j < list->nb_messages;) { + if (list->messages[j].payload_type == payload_type) + cbs_sei_delete_message(list, j); + else + ++j; + } + } +} diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index 95beabf4d7..5ce4ad3ccd 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -21,8 +21,132 @@ #include #include + #include "libavutil/buffer.h" +#include "cbs.h" + +// SEI payload types form a common namespace between the H.264, H.265 +// and H.266 standards. A given payload type always has the same +// meaning, but some names have different payload types in different +// standards (e.g. scalable-nesting is 30 in H.264 but 133 in H.265). +// The content of the payload data depends on the standard, though +// many generic parts have the same interpretation everywhere (such as +// mastering-display-colour-volume and user-data-unregistered). +enum { + SEI_TYPE_BUFFERING_PERIOD = 0, + SEI_TYPE_PIC_TIMING = 1, + SEI_TYPE_PAN_SCAN_RECT = 2, + SEI_TYPE_FILLER_PAYLOAD = 3, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35 = 4, + SEI_TYPE_USER_DATA_UNREGISTERED = 5, + SEI_TYPE_RECOVERY_POINT = 6, + SEI_TYPE_DEC_REF_PIC_MARKING_REPETITION = 7, + SEI_TYPE_SPARE_PIC = 8, + SEI_TYPE_SCENE_INFO = 9, + SEI_TYPE_SUB_SEQ_INFO = 10, + SEI_TYPE_SUB_SEQ_LAYER_CHARACTERISTICS = 11, + SEI_TYPE_SUB_SEQ_CHARACTERISTICS = 12, + SEI_TYPE_FULL_FRAME_FREEZE = 13, + SEI_TYPE_FULL_FRAME_FREEZE_RELEASE = 14, + SEI_TYPE_FULL_FRAME_SNAPSHOT = 15, + SEI_TYPE_PROGRESSIVE_REFINEMENT_SEGMENT_START = 16, + SEI_TYPE_PROGRESSIVE_REFINEMENT_SEGMENT_END = 17, + SEI_TYPE_MOTION_CONSTRAINED_SLICE_GROUP_SET = 18, + SEI_TYPE_FILM_GRAIN_CHARACTERISTICS = 19, + SEI_TYPE_DEBLOCKING_FILTER_DISPLAY_PREFERENCE = 20, + SEI_TYPE_STEREO_VIDEO_INFO = 21, + SEI_TYPE_POST_FILTER_HINT = 22, + SEI_TYPE_TONE_MAPPING_INFO = 23, + SEI_TYPE_SCALABILITY_INFO = 24, + SEI_TYPE_SUB_PIC_SCALABLE_LAYER = 25, + SEI_TYPE_NON_REQUIRED_LAYER_REP = 26, + SEI_TYPE_PRIORITY_LAYER_INFO = 27, + SEI_TYPE_LAYERS_NOT_PRESENT_4 = 28, + SEI_TYPE_LAYER_DEPENDENCY_CHANGE = 29, + SEI_TYPE_SCALABLE_NESTING_4 = 30, + SEI_TYPE_BASE_LAYER_TEMPORAL_HRD = 31, + SEI_TYPE_QUALITY_LAYER_INTEGRITY_CHECK = 32, + SEI_TYPE_REDUNDANT_PIC_PROPERTY = 33, + SEI_TYPE_TL0_DEP_REP_INDEX = 34, + SEI_TYPE_TL_SWITCHING_POINT = 35, + SEI_TYPE_PARALLEL_DECODING_INFO = 36, + SEI_TYPE_MVC_SCALABLE_NESTING = 37, + SEI_TYPE_VIEW_SCALABILITY_INFO = 38, + SEI_TYPE_MULTIVIEW_SCENE_INFO_4 = 39, + SEI_TYPE_MULTIVIEW_ACQUISITION_INFO_4 = 40, + SEI_TYPE_NON_REQUIRED_VIEW_COMPONENT = 41, + SEI_TYPE_VIEW_DEPENDENCY_CHANGE = 42, + SEI_TYPE_OPERATION_POINTS_NOT_PRESENT = 43, + SEI_TYPE_BASE_VIEW_TEMPORAL_HRD = 44, + SEI_TYPE_FRAME_PACKING_ARRANGEMENT = 45, + SEI_TYPE_MULTIVIEW_VIEW_POSITION_4 = 46, + SEI_TYPE_DISPLAY_ORIENTATION = 47, + SEI_TYPE_MVCD_SCALABLE_NESTING = 48, + SEI_TYPE_MVCD_VIEW_SCALABILITY_INFO = 49, + SEI_TYPE_DEPTH_REPRESENTATION_INFO_4 = 50, + SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO_4 = 51, + SEI_TYPE_DEPTH_TIMING = 52, + SEI_TYPE_DEPTH_SAMPLING_INFO = 53, + SEI_TYPE_CONSTRAINED_DEPTH_PARAMETER_SET_IDENTIFIER = 54, + SEI_TYPE_GREEN_METADATA = 56, + SEI_TYPE_STRUCTURE_OF_PICTURES_INFO = 128, + SEI_TYPE_ACTIVE_PARAMETER_SETS = 129, + SEI_TYPE_DECODING_UNIT_INFO = 130, + SEI_TYPE_TEMPORAL_SUB_LAYER_ZERO_IDX = 131, + SEI_TYPE_DECODED_PICTURE_HASH = 132, + SEI_TYPE_SCALABLE_NESTING_5 = 133, + SEI_TYPE_REGION_REFRESH_INFO = 134, + SEI_TYPE_NO_DISPLAY = 135, + SEI_TYPE_TIME_CODE = 136, + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME = 137, + SEI_TYPE_SEGMENTED_RECT_FRAME_PACKING_ARRANGEMENT = 138, + SEI_TYPE_TEMPORAL_MOTION_CONSTRAINED_TILE_SETS = 139, + SEI_TYPE_CHROMA_RESAMPLING_FILTER_HINT = 140, + SEI_TYPE_KNEE_FUNCTION_INFO = 141, + SEI_TYPE_COLOUR_REMAPPING_INFO = 142, + SEI_TYPE_DEINTERLACED_FIELD_IDENTIFICATION = 143, + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO = 144, + SEI_TYPE_DEPENDENT_RAP_INDICATION = 145, + SEI_TYPE_CODED_REGION_COMPLETION = 146, + SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS = 147, + SEI_TYPE_AMBIENT_VIEWING_ENVIRONMENT = 148, + SEI_TYPE_CONTENT_COLOUR_VOLUME = 149, + SEI_TYPE_EQUIRECTANGULAR_PROJECTION = 150, + SEI_TYPE_CUBEMAP_PROJECTION = 151, + SEI_TYPE_FISHEYE_VIDEO_INFO = 152, + SEI_TYPE_SPHERE_ROTATION = 154, + SEI_TYPE_REGIONWISE_PACKING = 155, + SEI_TYPE_OMNI_VIEWPORT = 156, + SEI_TYPE_REGIONAL_NESTING = 157, + SEI_TYPE_MCTS_EXTRACTION_INFO_SETS = 158, + SEI_TYPE_MCTS_EXTRACTION_INFO_NESTING = 159, + SEI_TYPE_LAYERS_NOT_PRESENT_5 = 160, + SEI_TYPE_INTER_LAYER_CONSTRAINED_TILE_SETS = 161, + SEI_TYPE_BSP_NESTING = 162, + SEI_TYPE_BSP_INITIAL_ARRIVAL_TIME = 163, + SEI_TYPE_SUB_BITSTREAM_PROPERTY = 164, + SEI_TYPE_ALPHA_CHANNEL_INFO = 165, + SEI_TYPE_OVERLAY_INFO = 166, + SEI_TYPE_TEMPORAL_MV_PREDICTION_CONSTRAINTS = 167, + SEI_TYPE_FRAME_FIELD_INFO = 168, + SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO = 176, + SEI_TYPE_DEPTH_REPRESENTATION_INFO_5 = 177, + SEI_TYPE_MULTIVIEW_SCENE_INFO_5 = 178, + SEI_TYPE_MULTIVIEW_ACQUISITION_INFO_5 = 179, + SEI_TYPE_MULTIVIEW_VIEW_POSITION_5 = 180, + SEI_TYPE_ALTERNATIVE_DEPTH_INFO = 181, + SEI_TYPE_SEI_MANIFEST = 200, + SEI_TYPE_SEI_PREFIX_INDICATION = 201, + SEI_TYPE_ANNOTATED_REGIONS = 202, + SEI_TYPE_SUBPIC_LEVEL_INFO = 203, + SEI_TYPE_SAMPLE_ASPECT_RATIO_INFO = 204, +}; + + +typedef struct SEIRawFillerPayload { + uint32_t payload_size; +} SEIRawFillerPayload; typedef struct SEIRawUserDataRegistered { uint8_t itu_t_t35_country_code; @@ -57,4 +181,135 @@ typedef struct SEIRawAlternativeTransferCharacteristics { uint8_t preferred_transfer_characteristics; } SEIRawAlternativeTransferCharacteristics; +typedef struct SEIRawMessage { + uint32_t payload_type; + uint32_t payload_size; + void *payload; + AVBufferRef *payload_ref; + uint8_t *extension_data; + AVBufferRef *extension_data_ref; + size_t extension_bit_length; +} SEIRawMessage; + +typedef struct SEIRawMessageList { + SEIRawMessage *messages; + int nb_messages; + int nb_messages_allocated; +} SEIRawMessageList; + + +typedef struct SEIMessageState { + // The type of the payload being written. + uint32_t payload_type; + // When reading, contains the size of the payload to allow finding the + // end of variable-length fields (such as user_data_payload_byte[]). + // (When writing, the size will be derived from the total number of + // bytes actually written.) + uint32_t payload_size; + // When writing, indicates that payload extension data is present so + // all extended fields must be written. May be updated by the writer + // to indicate that extended fields have been written, so the extension + // end bits must be written too. + uint8_t extension_present; +} SEIMessageState; + +struct GetBitContext; +struct PutBitContext; + +typedef int (*SEIMessageReadFunction)(CodedBitstreamContext *ctx, + struct GetBitContext *rw, + void *current, + SEIMessageState *sei); + +typedef int (*SEIMessageWriteFunction)(CodedBitstreamContext *ctx, + struct PutBitContext *rw, + void *current, + SEIMessageState *sei); + +typedef struct SEIMessageTypeDescriptor { + // Payload type for the message. (-1 in this field ends a list.) + int type; + // Valid in a prefix SEI NAL unit (always for H.264). + uint8_t prefix; + // Valid in a suffix SEI NAL unit (never for H.264). + uint8_t suffix; + // Size of the decomposed structure. + size_t size; + // Read bitstream into SEI message. + SEIMessageReadFunction read; + // Write bitstream from SEI message. + SEIMessageWriteFunction write; +} SEIMessageTypeDescriptor; + +// Macro for the read/write pair. The clumsy cast is needed because the +// current pointer is typed in all of the read/write functions but has to +// be void here to fit all cases. +#define SEI_MESSAGE_RW(codec, name) \ + .read = (SEIMessageReadFunction) cbs_ ## codec ## _read_ ## name, \ + .write = (SEIMessageWriteFunction)cbs_ ## codec ## _write_ ## name + +// End-of-list sentinel element. +#define SEI_MESSAGE_TYPE_END { .type = -1 } + + +/** + * Find the type descriptor for the given payload type. + * + * Returns NULL if the payload type is not known. + */ +const SEIMessageTypeDescriptor *ff_cbs_sei_find_type(CodedBitstreamContext *ctx, + int payload_type); + +/** + * Allocate a new payload for the given SEI message. + */ +int ff_cbs_sei_alloc_message_payload(SEIRawMessage *message, + const SEIMessageTypeDescriptor *desc); + +/** + * Allocate a new empty SEI message in a message list. + * + * The new message is in place nb_messages - 1. + */ +int ff_cbs_sei_list_add(SEIRawMessageList *list); + +/** + * Free all SEI messages in a message list. + */ +void ff_cbs_sei_free_message_list(SEIRawMessageList *list); + +/** + * Add an SEI message to an access unit. + * + * Will add to an existing SEI NAL unit, or create a new one for the + * message if there is no suitable existing one. + * + * Takes a new reference to payload_buf, if set. If payload_buf is + * NULL then the new message will not be reference counted. + */ +int ff_cbs_sei_add_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + int prefix, + uint32_t payload_type, + void *payload_data, + AVBufferRef *payload_buf); + +/** + * Iterate over messages with the given payload type in an access unit. + * + * Set message to NULL in the first call. Returns 0 while more messages + * are available, AVERROR(ENOENT) when all messages have been found. + */ +int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type, + SEIRawMessage **message); + +/** + * Delete all messages with the given payload type from an access unit. + */ +void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + uint32_t payload_type); + #endif /* AVCODEC_CBS_SEI_H */ diff --git a/libavcodec/cbs_sei_syntax_template.c b/libavcodec/cbs_sei_syntax_template.c index cc900830ae..1a0516acce 100644 --- a/libavcodec/cbs_sei_syntax_template.c +++ b/libavcodec/cbs_sei_syntax_template.c @@ -16,9 +16,27 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -static int FUNC(sei_user_data_registered) +static int FUNC(filler_payload) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawUserDataRegistered *current, uint32_t *payload_size) + SEIRawFillerPayload *current, SEIMessageState *state) +{ + int err, i; + + HEADER("Filler Payload"); + +#ifdef READ + current->payload_size = state->payload_size; +#endif + + for (i = 0; i < current->payload_size; i++) + fixed(8, ff_byte, 0xff); + + return 0; +} + +static int FUNC(user_data_registered) + (CodedBitstreamContext *ctx, RWContext *rw, + SEIRawUserDataRegistered *current, SEIMessageState *state) { int err, i, j; @@ -33,14 +51,12 @@ static int FUNC(sei_user_data_registered) } #ifdef READ - if (*payload_size < i) { + if (state->payload_size < i) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid SEI user data registered payload.\n"); return AVERROR_INVALIDDATA; } - current->data_length = *payload_size - i; -#else - *payload_size = i + current->data_length; + current->data_length = state->payload_size - i; #endif allocate(current->data, current->data_length); @@ -50,23 +66,21 @@ static int FUNC(sei_user_data_registered) return 0; } -static int FUNC(sei_user_data_unregistered) +static int FUNC(user_data_unregistered) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawUserDataUnregistered *current, uint32_t *payload_size) + SEIRawUserDataUnregistered *current, SEIMessageState *state) { int err, i; HEADER("User Data Unregistered"); #ifdef READ - if (*payload_size < 16) { + if (state->payload_size < 16) { av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid SEI user data unregistered payload.\n"); return AVERROR_INVALIDDATA; } - current->data_length = *payload_size - 16; -#else - *payload_size = 16 + current->data_length; + current->data_length = state->payload_size - 16; #endif for (i = 0; i < 16; i++) @@ -80,9 +94,9 @@ static int FUNC(sei_user_data_unregistered) return 0; } -static int FUNC(sei_mastering_display_colour_volume) +static int FUNC(mastering_display_colour_volume) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawMasteringDisplayColourVolume *current) + SEIRawMasteringDisplayColourVolume *current, SEIMessageState *state) { int err, c; @@ -104,13 +118,13 @@ static int FUNC(sei_mastering_display_colour_volume) return 0; } -static int FUNC(sei_content_light_level) +static int FUNC(content_light_level_info) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawContentLightLevelInfo *current) + SEIRawContentLightLevelInfo *current, SEIMessageState *state) { int err; - HEADER("Content Light Level"); + HEADER("Content Light Level Information"); ub(16, max_content_light_level); ub(16, max_pic_average_light_level); @@ -118,9 +132,10 @@ static int FUNC(sei_content_light_level) return 0; } -static int FUNC(sei_alternative_transfer_characteristics) +static int FUNC(alternative_transfer_characteristics) (CodedBitstreamContext *ctx, RWContext *rw, - SEIRawAlternativeTransferCharacteristics *current) + SEIRawAlternativeTransferCharacteristics *current, + SEIMessageState *state) { int err; @@ -130,3 +145,165 @@ static int FUNC(sei_alternative_transfer_characteristics) return 0; } + +static int FUNC(message)(CodedBitstreamContext *ctx, RWContext *rw, + SEIRawMessage *current) +{ + const SEIMessageTypeDescriptor *desc; + int err, i; + + desc = ff_cbs_sei_find_type(ctx, current->payload_type); + if (desc) { + SEIMessageState state = { + .payload_type = current->payload_type, + .payload_size = current->payload_size, + .extension_present = current->extension_bit_length > 0, + }; + int start_position, current_position, bits_written; + +#ifdef READ + CHECK(ff_cbs_sei_alloc_message_payload(current, desc)); +#endif + + start_position = bit_position(rw); + + CHECK(desc->READWRITE(ctx, rw, current->payload, &state)); + + current_position = bit_position(rw); + bits_written = current_position - start_position; + + if (byte_alignment(rw) || state.extension_present || + bits_written < 8 * current->payload_size) { + size_t bits_left; + +#ifdef READ + GetBitContext tmp = *rw; + int trailing_bits, trailing_zero_bits; + + bits_left = 8 * current->payload_size - bits_written; + if (bits_left > 8) + skip_bits_long(&tmp, bits_left - 8); + trailing_bits = get_bits(&tmp, FFMIN(bits_left, 8)); + if (trailing_bits == 0) { + // The trailing bits must contain a bit_equal_to_one, so + // they can't all be zero. + return AVERROR_INVALIDDATA; + } + trailing_zero_bits = ff_ctz(trailing_bits); + current->extension_bit_length = + bits_left - 1 - trailing_zero_bits; +#endif + + if (current->extension_bit_length > 0) { + allocate(current->extension_data, + (current->extension_bit_length + 7) / 8); + + bits_left = current->extension_bit_length; + for (i = 0; bits_left > 0; i++) { + int length = FFMIN(bits_left, 8); + xu(length, reserved_payload_extension_data, + current->extension_data[i], + 0, MAX_UINT_BITS(length), 0); + bits_left -= length; + } + } + + fixed(1, bit_equal_to_one, 1); + while (byte_alignment(rw)) + fixed(1, bit_equal_to_zero, 0); + } + +#ifdef WRITE + current->payload_size = (put_bits_count(rw) - start_position) / 8; +#endif + } else { + uint8_t *data; + + allocate(current->payload, current->payload_size); + data = current->payload; + + for (i = 0; i < current->payload_size; i++) + xu(8, payload_byte[i], data[i], 0, 255, 1, i); + } + + return 0; +} + +static int FUNC(message_list)(CodedBitstreamContext *ctx, RWContext *rw, + SEIRawMessageList *current, int prefix) +{ + SEIRawMessage *message; + int err, k; + +#ifdef READ + for (k = 0;; k++) { + uint32_t payload_type = 0; + uint32_t payload_size = 0; + uint32_t tmp; + + while (show_bits(rw, 8) == 0xff) { + fixed(8, ff_byte, 0xff); + payload_type += 255; + } + xu(8, last_payload_type_byte, tmp, 0, 254, 0); + payload_type += tmp; + + while (show_bits(rw, 8) == 0xff) { + fixed(8, ff_byte, 0xff); + payload_size += 255; + } + xu(8, last_payload_size_byte, tmp, 0, 254, 0); + payload_size += tmp; + + CHECK(ff_cbs_sei_list_add(current)); + message = ¤t->messages[k]; + + message->payload_type = payload_type; + message->payload_size = payload_size; + + CHECK(FUNC(message)(ctx, rw, message)); + + if (!cbs_h2645_read_more_rbsp_data(rw)) + break; + } +#else + for (k = 0; k < current->nb_messages; k++) { + PutBitContext start_state; + uint32_t tmp; + int trace, i; + + message = ¤t->messages[k]; + + // We write the payload twice in order to find the size. Trace + // output is switched off for the first write. + trace = ctx->trace_enable; + ctx->trace_enable = 0; + + start_state = *rw; + for (i = 0; i < 2; i++) { + *rw = start_state; + + tmp = message->payload_type; + while (tmp >= 255) { + fixed(8, ff_byte, 0xff); + tmp -= 255; + } + xu(8, last_payload_type_byte, tmp, 0, 254, 0); + + tmp = message->payload_size; + while (tmp >= 255) { + fixed(8, ff_byte, 0xff); + tmp -= 255; + } + xu(8, last_payload_size_byte, tmp, 0, 254, 0); + + err = FUNC(message)(ctx, rw, message); + ctx->trace_enable = trace; + if (err < 0) + return err; + } + } +#endif + + return 0; +} diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index f39e649ac6..4ab97aee3a 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -78,13 +78,14 @@ typedef struct H264MetadataContext { int crop_bottom; const char *sei_user_data; - H264RawSEIPayload sei_user_data_payload; + SEIRawUserDataUnregistered sei_user_data_payload; int delete_filler; int display_orientation; double rotate; int flip; + H264RawSEIDisplayOrientation display_orientation_payload; int level; } H264MetadataContext; @@ -414,7 +415,9 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) // Only insert the SEI in access units containing SPSs, and also // unconditionally in the first access unit we ever see. if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - err = ff_cbs_h264_add_sei_message(au, &ctx->sei_user_data_payload); + err = ff_cbs_sei_add_message(ctx->output, au, 1, + H264_SEI_TYPE_USER_DATA_UNREGISTERED, + &ctx->sei_user_data_payload, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " "message to access unit.\n"); @@ -428,74 +431,54 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) ff_cbs_delete_unit(au, i); continue; } - - if (au->units[i].type == H264_NAL_SEI) { - // Filler SEI messages. - H264RawSEI *sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - if (sei->payload[j].payload_type == - H264_SEI_TYPE_FILLER_PAYLOAD) - ff_cbs_h264_delete_sei_message(au, &au->units[i], j); - } - } } + + ff_cbs_sei_delete_message_type(ctx->output, au, + H264_SEI_TYPE_FILLER_PAYLOAD); } if (ctx->display_orientation != PASS) { - for (i = au->nb_units - 1; i >= 0; i--) { - H264RawSEI *sei; - if (au->units[i].type != H264_NAL_SEI) - continue; - sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - H264RawSEIDisplayOrientation *disp; - int32_t *matrix; - - if (sei->payload[j].payload_type != - H264_SEI_TYPE_DISPLAY_ORIENTATION) - continue; - disp = &sei->payload[j].payload.display_orientation; - - if (ctx->display_orientation == REMOVE || - ctx->display_orientation == INSERT) { - ff_cbs_h264_delete_sei_message(au, &au->units[i], j); - continue; - } - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) { - err = AVERROR(ENOMEM); - goto fail; - } + SEIRawMessage *message = NULL; + while (ff_cbs_sei_find_message(ctx->output, au, + H264_SEI_TYPE_DISPLAY_ORIENTATION, + &message) == 0) { + H264RawSEIDisplayOrientation *disp = message->payload; + int32_t *matrix; + + matrix = av_malloc(9 * sizeof(int32_t)); + if (!matrix) { + err = AVERROR(ENOMEM); + goto fail; + } - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_free(matrix); - goto fail; - } + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 180.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); + + // If there are multiple display orientation messages in an + // access unit, then the last one added to the packet (i.e. + // the first one in the access unit) will prevail. + err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, + (uint8_t*)matrix, + 9 * sizeof(int32_t)); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " + "displaymatrix side data to packet.\n"); + av_free(matrix); + goto fail; } } + + if (ctx->display_orientation == REMOVE || + ctx->display_orientation == INSERT) { + ff_cbs_sei_delete_message_type(ctx->output, au, + H264_SEI_TYPE_DISPLAY_ORIENTATION); + } } if (ctx->display_orientation == INSERT) { - H264RawSEIPayload payload = { - .payload_type = H264_SEI_TYPE_DISPLAY_ORIENTATION, - }; H264RawSEIDisplayOrientation *disp = - &payload.payload.display_orientation; + &ctx->display_orientation_payload; uint8_t *data; int size; int write = 0; @@ -551,7 +534,9 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (write) { disp->display_orientation_repetition_period = 1; - err = ff_cbs_h264_add_sei_message(au, &payload); + err = ff_cbs_sei_add_message(ctx->output, au, 1, + H264_SEI_TYPE_DISPLAY_ORIENTATION, + disp, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); @@ -585,13 +570,9 @@ static int h264_metadata_init(AVBSFContext *bsf) int err, i; if (ctx->sei_user_data) { - SEIRawUserDataUnregistered *udu = - &ctx->sei_user_data_payload.payload.user_data_unregistered; + SEIRawUserDataUnregistered *udu = &ctx->sei_user_data_payload; int j; - ctx->sei_user_data_payload.payload_type = - H264_SEI_TYPE_USER_DATA_UNREGISTERED; - // Parse UUID. It must be a hex string of length 32, possibly // containing '-'s between hex digits (which we ignore). for (i = j = 0; j < 32 && i < 64 && ctx->sei_user_data[i]; i++) { diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index b577d09caf..d24462414c 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -90,7 +90,6 @@ typedef struct VAAPIEncodeH264Context { H264RawAUD raw_aud; H264RawSPS raw_sps; H264RawPPS raw_pps; - H264RawSEI raw_sei; H264RawSlice raw_slice; H264RawSEIBufferingPeriod sei_buffering_period; @@ -210,11 +209,9 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, { VAAPIEncodeH264Context *priv = avctx->priv_data; CodedBitstreamFragment *au = &priv->current_access_unit; - int err, i; + int err; if (priv->sei_needed) { - H264RawSEI *sei = &priv->raw_sei; - if (priv->aud_needed) { err = vaapi_encode_h264_add_nal(avctx, au, &priv->raw_aud); if (err < 0) @@ -222,41 +219,35 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, priv->aud_needed = 0; } - *sei = (H264RawSEI) { - .nal_unit_header = { - .nal_unit_type = H264_NAL_SEI, - }, - }; - - i = 0; - if (priv->sei_needed & SEI_IDENTIFIER) { - sei->payload[i].payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED; - sei->payload[i].payload.user_data_unregistered = priv->sei_identifier; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_USER_DATA_UNREGISTERED, + &priv->sei_identifier, NULL); + if (err < 0) + goto fail; } if (priv->sei_needed & SEI_TIMING) { if (pic->type == PICTURE_TYPE_IDR) { - sei->payload[i].payload_type = H264_SEI_TYPE_BUFFERING_PERIOD; - sei->payload[i].payload.buffering_period = priv->sei_buffering_period; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_BUFFERING_PERIOD, + &priv->sei_buffering_period, NULL); + if (err < 0) + goto fail; } - sei->payload[i].payload_type = H264_SEI_TYPE_PIC_TIMING; - sei->payload[i].payload.pic_timing = priv->sei_pic_timing; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_PIC_TIMING, + &priv->sei_pic_timing, NULL); + if (err < 0) + goto fail; } if (priv->sei_needed & SEI_RECOVERY_POINT) { - sei->payload[i].payload_type = H264_SEI_TYPE_RECOVERY_POINT; - sei->payload[i].payload.recovery_point = priv->sei_recovery_point; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + H264_SEI_TYPE_RECOVERY_POINT, + &priv->sei_recovery_point, NULL); + if (err < 0) + goto fail; } - sei->payload_count = i; - av_assert0(sei->payload_count > 0); - - err = vaapi_encode_h264_add_nal(avctx, au, sei); - if (err < 0) - goto fail; priv->sei_needed = 0; err = vaapi_encode_h264_write_access_unit(avctx, data, data_len, au); diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index a7af763ae4..2e8e772008 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -73,7 +73,6 @@ typedef struct VAAPIEncodeH265Context { H265RawVPS raw_vps; H265RawSPS raw_sps; H265RawPPS raw_pps; - H265RawSEI raw_sei; H265RawSlice raw_slice; SEIRawMasteringDisplayColourVolume sei_mastering_display; @@ -195,11 +194,9 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, { VAAPIEncodeH265Context *priv = avctx->priv_data; CodedBitstreamFragment *au = &priv->current_access_unit; - int err, i; + int err; if (priv->sei_needed) { - H265RawSEI *sei = &priv->raw_sei; - if (priv->aud_needed) { err = vaapi_encode_h265_add_nal(avctx, au, &priv->aud); if (err < 0) @@ -207,35 +204,22 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, priv->aud_needed = 0; } - *sei = (H265RawSEI) { - .nal_unit_header = { - .nal_unit_type = HEVC_NAL_SEI_PREFIX, - .nuh_layer_id = 0, - .nuh_temporal_id_plus1 = 1, - }, - }; - - i = 0; - if (priv->sei_needed & SEI_MASTERING_DISPLAY) { - sei->payload[i].payload_type = HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO; - sei->payload[i].payload.mastering_display_colour_volume = - priv->sei_mastering_display; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, + &priv->sei_mastering_display, NULL); + if (err < 0) + goto fail; } if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { - sei->payload[i].payload_type = HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO; - sei->payload[i].payload.content_light_level = priv->sei_content_light_level; - ++i; + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + &priv->sei_content_light_level, NULL); + if (err < 0) + goto fail; } - sei->payload_count = i; - av_assert0(sei->payload_count > 0); - - err = vaapi_encode_h265_add_nal(avctx, au, sei); - if (err < 0) - goto fail; priv->sei_needed = 0; err = vaapi_encode_h265_write_access_unit(avctx, data, data_len, au); From patchwork Fri Jan 1 21:35:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24758 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3782D44878B for ; Fri, 1 Jan 2021 23:58:51 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0CC3268A878; Fri, 1 Jan 2021 23:58:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2CAFC68A878 for ; Fri, 1 Jan 2021 23:58:44 +0200 (EET) Received: by mail-lf1-f46.google.com with SMTP id a12so50769695lfl.6 for ; Fri, 01 Jan 2021 13:58:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=LoSx2tiF4NodIRE2/FNkQgvJepeenfc1KnVyjGpZyvM=; b=toz8L5nZtQcfP8uHYVP76qmqKnwtipOQS4WYctR5ffvPDyfb3tZqyEvd+YPkCRotEi oQXKY69bCR4bXqfQ6/9DycshSi3jynayVHjDStC/ejtgcbW1kxjn/XDidqPOqJk/8y+C SyoPRvnCvAOiVSe+9jtQ9p1lpIQqczU2aMAznsrCZoXWbaBH3V38jy0B0QPI0xD7D8bY 3/UnmHxAzJbUGGDiWzzdSQWZn73jC7OX+MVyWfmxX6/WlE6jTVV1vhr6rgrqxo5DljVS jm2D7SGzk+bQY2mYMacsiNcg3FrGYyv9smQx2mQlMG/tEqpvzfrs2oiTSLB2qJxahs6s y0wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LoSx2tiF4NodIRE2/FNkQgvJepeenfc1KnVyjGpZyvM=; b=aB+ER2WEp2u0YZS5zp3Zw+6oIkdpitkK/z2dJM/OXN1Fm8AAQescrrUmJ3NejfoDFI g1VTVgQ9dGzlAD6nD8/4rz9lEKuRRwyfGE2MA5y94hvX8LxsT7535S1YXfDUknKDYTaZ rYsF/o73GXYp0kH72W0/5OSp8Ayo+6tfRniUNhcufPiA9GwVLkWowXpNnnkM62zk637w YhPidBOpLlndjsjjOANqPcAeF2j5pVNakUb9A2CL9gLGOU8i9VFFQQfshrNawFZWNm6i 56Asxnybo4f3bgrBeYfe9Q7SkP0SRdli9ZVd3NpED47QbumiSOTkZokXYyypVO0YeKqq wvVg== X-Gm-Message-State: AOAM5338y+oeSz9oLhhIlq6VtaNn0kaqmdKCo1Dc8SVZbU1LW2MaChoO 3pBdoKbqIwM2dU/cnOpPWj1xxm++BueMuQ== X-Google-Smtp-Source: ABdhPJwevnOa1oig6MA2kHYPm0ED4Vx3b1p53o56JUEwdmDwbsLHRKryaY2vAVMU+VWJrvt8+JkwJQ== X-Received: by 2002:adf:ded1:: with SMTP id i17mr69376598wrn.190.1609537073283; Fri, 01 Jan 2021 13:37:53 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:52 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:14 +0000 Message-Id: <20210101213537.169546-5-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/27] cbs_sei: Remove restrictions on MDCV values X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Since this was originally written the standards have changed to allow arbitrary values here, but leaves their meaning unspecified. --- libavcodec/cbs_sei_syntax_template.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/libavcodec/cbs_sei_syntax_template.c b/libavcodec/cbs_sei_syntax_template.c index 1a0516acce..55377ea5a9 100644 --- a/libavcodec/cbs_sei_syntax_template.c +++ b/libavcodec/cbs_sei_syntax_template.c @@ -103,17 +103,15 @@ static int FUNC(mastering_display_colour_volume) HEADER("Mastering Display Colour Volume"); for (c = 0; c < 3; c++) { - us(16, display_primaries_x[c], 0, 50000, 1, c); - us(16, display_primaries_y[c], 0, 50000, 1, c); + ubs(16, display_primaries_x[c], 1, c); + ubs(16, display_primaries_y[c], 1, c); } - u(16, white_point_x, 0, 50000); - u(16, white_point_y, 0, 50000); + ub(16, white_point_x); + ub(16, white_point_y); - u(32, max_display_mastering_luminance, - 1, MAX_UINT_BITS(32)); - u(32, min_display_mastering_luminance, - 0, current->max_display_mastering_luminance - 1); + ub(32, max_display_mastering_luminance); + ub(32, min_display_mastering_luminance); return 0; } From patchwork Fri Jan 1 21:35:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24752 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9645E44B4DF for ; Fri, 1 Jan 2021 23:44:15 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7925968A957; Fri, 1 Jan 2021 23:44:15 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BB49468A817 for ; Fri, 1 Jan 2021 23:44:14 +0200 (EET) Received: by mail-wm1-f44.google.com with SMTP id c133so9882830wme.4 for ; Fri, 01 Jan 2021 13:44:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=A1XHS4w3PJnIiz8KyMgW+v3rprEXZ3MgkRSPdC2kgc4=; b=pe5zPNxV5W9dS9IMDNGJEi0oP8iksQ0xth3k3vwqg7BwlThH1DhviicVBEErkz33nI QqlP2mbuI5sN4cEyUzGllzAhuFHdGbW6qP4RjLEOGFUxziJEv0/KRLuAXNxssnCDdCZd IM+CK9L5MS4S9MAe5adCHN6AvijsMQimw7XZuIFGHclAAf74cxEYqhewQepYSnzmHc2Y 4jW84RaN/lNdv+GYxIErv733KmVdaGhdrB99uoYRxuQIwZU64SSVDgM1LYQ2X9VpJftv jGhUrQtP0753aIQ4FT2eoMpnOjAbpHAu9YcG7QLowA+SuIekEY2DTKHZamsZSJz4r7BR 2+fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A1XHS4w3PJnIiz8KyMgW+v3rprEXZ3MgkRSPdC2kgc4=; b=flOdr0hPdEUoShlE+LxAv5Vw9GYLvAF+k6pDvNt+G8o9jw9GxVvYWdh8m8wu0KvdeR uzjWuSh6vemnZT8mHiUImIWPRYywJJARvUocEUAxNcXFoIUeQwaSXgb7gvZz8Eramn6K LrpjURPyBXs2HyUXwl/yTbGLekRHwx2zz8u2hidTmwD/vvL0xrOPwTKwBWVFvhVVzp/2 E0KxwRY7B9kbxv7FZnjvSoNqbSnX8QoNcQCnQ36dCBlmm5VJe/r9cLmvI4CwkqvWz3q+ FGVjJlnHi2gPFZjMHkW6EVxK+CcKJ3CAVd7NONidB9iNJkWfGOCd8gPZ4fWVoAu0vOfD 8v0g== X-Gm-Message-State: AOAM531INeFKG6BAqq0+gmk2ijkRpnFpPP1TW3a392SFydNf0/AQjp2D PzaZ2jis7y5F//ks8FyGzW4CjSa5eN2wPw== X-Google-Smtp-Source: ABdhPJzB6GTFRgLdIXbLhW4DmL6khMD7auijzl4zhJEMrCbJ8T4zujmWGo2qnX5Xua2FHrMjKHgq2Q== X-Received: by 2002:a1c:a785:: with SMTP id q127mr16737077wme.25.1609537074375; Fri, 01 Jan 2021 13:37:54 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:53 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:15 +0000 Message-Id: <20210101213537.169546-6-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/27] h264_metadata_bsf: Refactor the filter function into smaller parts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 326 ++++++++++++++++++--------------- 1 file changed, 180 insertions(+), 146 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 4ab97aee3a..36f8aab245 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -56,6 +56,7 @@ typedef struct H264MetadataContext { int done_first_au; int aud; + H264RawAUD aud_nal; AVRational sample_aspect_ratio; @@ -91,6 +92,59 @@ typedef struct H264MetadataContext { } H264MetadataContext; +static int h264_metadata_insert_aud(AVBSFContext *bsf, + CodedBitstreamFragment *au) +{ + H264MetadataContext *ctx = bsf->priv_data; + int primary_pic_type_mask = 0xff; + int err, i, j; + + static const int primary_pic_type_table[] = { + 0x084, // 2, 7 + 0x0a5, // 0, 2, 5, 7 + 0x0e7, // 0, 1, 2, 5, 6, 7 + 0x210, // 4, 9 + 0x318, // 3, 4, 8, 9 + 0x294, // 2, 4, 7, 9 + 0x3bd, // 0, 2, 3, 4, 5, 7, 8, 9 + 0x3ff, // 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 + }; + + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SLICE || + au->units[i].type == H264_NAL_IDR_SLICE) { + H264RawSlice *slice = au->units[i].content; + for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) { + if (!(primary_pic_type_table[j] & + (1 << slice->header.slice_type))) + primary_pic_type_mask &= ~(1 << j); + } + } + } + for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) + if (primary_pic_type_mask & (1 << j)) + break; + if (j >= FF_ARRAY_ELEMS(primary_pic_type_table)) { + av_log(bsf, AV_LOG_ERROR, "No usable primary_pic_type: " + "invalid slice types?\n"); + return AVERROR_INVALIDDATA; + } + + ctx->aud_nal = (H264RawAUD) { + .nal_unit_header.nal_unit_type = H264_NAL_AUD, + .primary_pic_type = j, + }; + + err = ff_cbs_insert_unit_content(au, 0, H264_NAL_AUD, + &ctx->aud_nal, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + return err; + } + + return 0; +} + static int h264_metadata_update_sps(AVBSFContext *bsf, H264RawSPS *sps) { @@ -322,160 +376,51 @@ static int h264_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) return 0; } -static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *au, + int seek_point) { H264MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - int err, i, j, has_sps; - H264RawAUD aud; - - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = h264_metadata_update_side_data(bsf, pkt); - if (err < 0) - goto fail; - - err = ff_cbs_read_packet(ctx->input, au, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } - - if (au->nb_units == 0) { - av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - - // If an AUD is present, it must be the first NAL unit. - if (au->units[0].type == H264_NAL_AUD) { - if (ctx->aud == REMOVE) - ff_cbs_delete_unit(au, 0); - } else { - if (ctx->aud == INSERT) { - static const int primary_pic_type_table[] = { - 0x084, // 2, 7 - 0x0a5, // 0, 2, 5, 7 - 0x0e7, // 0, 1, 2, 5, 6, 7 - 0x210, // 4, 9 - 0x318, // 3, 4, 8, 9 - 0x294, // 2, 4, 7, 9 - 0x3bd, // 0, 2, 3, 4, 5, 7, 8, 9 - 0x3ff, // 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 - }; - int primary_pic_type_mask = 0xff; - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SLICE || - au->units[i].type == H264_NAL_IDR_SLICE) { - H264RawSlice *slice = au->units[i].content; - for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) { - if (!(primary_pic_type_table[j] & - (1 << slice->header.slice_type))) - primary_pic_type_mask &= ~(1 << j); - } - } - } - for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) - if (primary_pic_type_mask & (1 << j)) - break; - if (j >= FF_ARRAY_ELEMS(primary_pic_type_table)) { - av_log(bsf, AV_LOG_ERROR, "No usable primary_pic_type: " - "invalid slice types?\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - - aud = (H264RawAUD) { - .nal_unit_header.nal_unit_type = H264_NAL_AUD, - .primary_pic_type = j, - }; - - err = ff_cbs_insert_unit_content(au, - 0, H264_NAL_AUD, &aud, NULL); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); - goto fail; - } - } - } - - has_sps = 0; - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SPS) { - err = h264_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - goto fail; - has_sps = 1; - } - } - - // Only insert the SEI in access units containing SPSs, and also - // unconditionally in the first access unit we ever see. - if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - err = ff_cbs_sei_add_message(ctx->output, au, 1, - H264_SEI_TYPE_USER_DATA_UNREGISTERED, - &ctx->sei_user_data_payload, NULL); + SEIRawMessage *message; + int err; + + message = NULL; + while (ff_cbs_sei_find_message(ctx->output, au, + H264_SEI_TYPE_DISPLAY_ORIENTATION, + &message) == 0) { + H264RawSEIDisplayOrientation *disp = message->payload; + int32_t *matrix; + + matrix = av_malloc(9 * sizeof(int32_t)); + if (!matrix) + return AVERROR(ENOMEM); + + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 180.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); + + // If there are multiple display orientation messages in an + // access unit, then the last one added to the packet (i.e. + // the first one in the access unit) will prevail. + err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, + (uint8_t*)matrix, + 9 * sizeof(int32_t)); if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " - "message to access unit.\n"); - goto fail; + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " + "displaymatrix side data to packet.\n"); + av_free(matrix); + return AVERROR(ENOMEM); } } - if (ctx->delete_filler) { - for (i = au->nb_units - 1; i >= 0; i--) { - if (au->units[i].type == H264_NAL_FILLER_DATA) { - ff_cbs_delete_unit(au, i); - continue; - } - } - + if (ctx->display_orientation == REMOVE || + ctx->display_orientation == INSERT) { ff_cbs_sei_delete_message_type(ctx->output, au, - H264_SEI_TYPE_FILLER_PAYLOAD); + H264_SEI_TYPE_DISPLAY_ORIENTATION); } - if (ctx->display_orientation != PASS) { - SEIRawMessage *message = NULL; - while (ff_cbs_sei_find_message(ctx->output, au, - H264_SEI_TYPE_DISPLAY_ORIENTATION, - &message) == 0) { - H264RawSEIDisplayOrientation *disp = message->payload; - int32_t *matrix; - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) { - err = AVERROR(ENOMEM); - goto fail; - } - - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_free(matrix); - goto fail; - } - } - - if (ctx->display_orientation == REMOVE || - ctx->display_orientation == INSERT) { - ff_cbs_sei_delete_message_type(ctx->output, au, - H264_SEI_TYPE_DISPLAY_ORIENTATION); - } - } if (ctx->display_orientation == INSERT) { H264RawSEIDisplayOrientation *disp = &ctx->display_orientation_payload; @@ -516,7 +461,7 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) } } - if (has_sps || !ctx->done_first_au) { + if (seek_point) { if (!isnan(ctx->rotate)) { disp->anticlockwise_rotation = (uint16_t)rint((ctx->rotate >= 0.0 ? ctx->rotate @@ -540,9 +485,98 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); + return err; + } + } + } + + return 0; +} + +static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +{ + H264MetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err, i, has_sps, seek_point; + + err = ff_bsf_get_packet_ref(bsf, pkt); + if (err < 0) + return err; + + err = h264_metadata_update_side_data(bsf, pkt); + if (err < 0) + goto fail; + + err = ff_cbs_read_packet(ctx->input, au, pkt); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (au->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + // If an AUD is present, it must be the first NAL unit. + if (au->units[0].type == H264_NAL_AUD) { + if (ctx->aud == REMOVE) + ff_cbs_delete_unit(au, 0); + } else { + if (ctx->aud == INSERT) { + err = h264_metadata_insert_aud(bsf, au); + if (err < 0) goto fail; + } + } + + has_sps = 0; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SPS) { + err = h264_metadata_update_sps(bsf, au->units[i].content); + if (err < 0) + goto fail; + has_sps = 1; + } + } + + // The current packet should be treated as a seek point for metadata + // insertion if any of: + // - It is the first packet in the stream. + // - It contains an SPS, indicating that a sequence might start here. + // - It is marked as containing a key frame. + seek_point = !ctx->done_first_au || has_sps || + (pkt->flags & AV_PKT_FLAG_KEY); + + if (ctx->sei_user_data && seek_point) { + err = ff_cbs_sei_add_message(ctx->output, au, 1, + H264_SEI_TYPE_USER_DATA_UNREGISTERED, + &ctx->sei_user_data_payload, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " + "message to access unit.\n"); + goto fail; + } + } + + if (ctx->delete_filler) { + for (i = au->nb_units - 1; i >= 0; i--) { + if (au->units[i].type == H264_NAL_FILLER_DATA) { + ff_cbs_delete_unit(au, i); + continue; } } + + ff_cbs_sei_delete_message_type(ctx->output, au, + H264_SEI_TYPE_FILLER_PAYLOAD); + } + + if (ctx->display_orientation != PASS) { + err = h264_metadata_handle_display_orientation(bsf, pkt, au, + seek_point); + if (err < 0) + goto fail; } err = ff_cbs_write_packet(ctx->output, pkt, au); From patchwork Fri Jan 1 21:35:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24755 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 673D244B645 for ; Fri, 1 Jan 2021 23:45:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4E6ED68AA13; Fri, 1 Jan 2021 23:45:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B93C06880B2 for ; Fri, 1 Jan 2021 23:45:16 +0200 (EET) Received: by mail-lf1-f41.google.com with SMTP id s26so50655786lfc.8 for ; Fri, 01 Jan 2021 13:45:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=aR9uOOoh3oK/aQLNaF4a7C2DYm1gIiIvrFWx3+McuuQ=; b=OSUtaL1fBPYAK+Zf1dK+KUIdAjiPrCK8U8aCb+HLYPeaiGcLWyqtGeh589MZla/+gc 415yp0P//wK/D1bv9o39FCQjzexy/QQhsItwaE3zI8Nt3Eo41vRRa4MHUhaZdbrbs5W2 7xv9TIdYoNLQ+a7UrrPcvdveOeMf8yzTRDzdhy2tUUtU8UqL4qhzBaZHwXvrN7jesdTY 8iiNl5MCympjjI+BENFHW/1YKKBgPiN0T4yrf1vINLHQ6dfg43nkpLxEHMWeEnx6tcIA F6Vzwycaqjz+M4wx/h6roHqxkFAfRXn+CoBTWNl6EWxVMPmtOXCrBHVrBV4cqQYxmo1h bQXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aR9uOOoh3oK/aQLNaF4a7C2DYm1gIiIvrFWx3+McuuQ=; b=KLtwmeD2j2FN8jWdpsNc0eMqa6xlKP5NaIRnecUjsgmOleGNDiZP9DjECaLI/iQHCC 72OU6HUDvD5uMfRAXMHGfS6kjawS3s1gCclgmewi8a6OIy/Us7iqXAjP77O1hyhStUNN RTbisAv8ggLErGql31vOyZ9dY3Or2TCdiuKsmkZ2hwInGudWZOzhEY0R13Y/jjqoZ1LU ObI2G8a5m8cuVT3acNCqjNC7MLf8T9hJpRX61Yp6GcBg9v5lo7/Jf81yb/aoKmDh7vWr Cqnhh/8tKyU4i5lkWAOv9hU6VFFg0MFpFVN5pf/Dzj4vjtq19xfDxAVc424eBffV1nUL ef3A== X-Gm-Message-State: AOAM532aES94wJunWXZTdI1GEMCOd3rxJVGMVshsjt9V/FML49JXw48V FV8nAv2lOX4GwVXhsxpeC0/hVH4R847K6Q== X-Google-Smtp-Source: ABdhPJx3q5zxHJg2HKC/HYAdPqk3BLNpI+56vJ7puyjV8m/UhrL6hF6UTunzafCoEXFk+p+wka0efg== X-Received: by 2002:a5d:6a83:: with SMTP id s3mr71659909wru.334.1609537075762; Fri, 01 Jan 2021 13:37:55 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:54 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:16 +0000 Message-Id: <20210101213537.169546-7-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/27] h264_metadata_bsf: Improve interpretation of input display matrices X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The previous code here only worked in more limited cases. --- libavcodec/h264_metadata_bsf.c | 44 +++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 36f8aab245..71b49696cd 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -431,23 +431,39 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, data = av_packet_get_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, &size); if (data && size >= 9 * sizeof(int32_t)) { int32_t matrix[9]; - int hflip, vflip; - double angle; + double dmatrix[9]; + int hflip, vflip, i; + double scale_x, scale_y, angle; memcpy(matrix, data, sizeof(matrix)); - hflip = vflip = 0; - if (matrix[0] < 0 && matrix[4] > 0) - hflip = 1; - else if (matrix[0] > 0 && matrix[4] < 0) - vflip = 1; - av_display_matrix_flip(matrix, hflip, vflip); + for (i = 0; i < 9; i++) + dmatrix[i] = matrix[i] / 65536.0; + + // Extract scale factors. + scale_x = hypot(dmatrix[0], dmatrix[3]); + scale_y = hypot(dmatrix[1], dmatrix[4]); + + // Select flips to make the main diagonal positive. + hflip = dmatrix[0] < 0.0; + vflip = dmatrix[4] < 0.0; + if (hflip) + scale_x = -scale_x; + if (vflip) + scale_y = -scale_y; + + // Rescale. + for (i = 0; i < 9; i += 3) { + dmatrix[i] /= scale_x; + dmatrix[i + 1] /= scale_y; + } - angle = av_display_rotation_get(matrix); + // Extract rotation. + angle = atan2(dmatrix[3], dmatrix[0]); - if (!(angle >= -180.0 && angle <= 180.0 /* also excludes NaN */) || - matrix[2] != 0 || matrix[5] != 0 || - matrix[6] != 0 || matrix[7] != 0) { + if (!(angle >= -M_PI && angle <= M_PI) || + matrix[2] != 0.0 || matrix[5] != 0.0 || + matrix[6] != 0.0 || matrix[7] != 0.0) { av_log(bsf, AV_LOG_WARNING, "Input display matrix is not " "representable in H.264 parameters.\n"); } else { @@ -455,8 +471,8 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, disp->ver_flip = vflip; disp->anticlockwise_rotation = (uint16_t)rint((angle >= 0.0 ? angle - : angle + 360.0) * - 65536.0 / 360.0); + : angle + 2 * M_PI) * + 32768.0 / M_PI); write = 1; } } From patchwork Fri Jan 1 21:35:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24748 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EEB4144B18F for ; Fri, 1 Jan 2021 23:43:51 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D1A9768A8E6; Fri, 1 Jan 2021 23:43:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3BF4668A8E6 for ; Fri, 1 Jan 2021 23:43:45 +0200 (EET) Received: by mail-wm1-f51.google.com with SMTP id g185so10005839wmf.3 for ; Fri, 01 Jan 2021 13:43:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4hK83d7VL1wN/wl1jqGkXbMYrtGu4Forbt4zLd3ya9o=; b=ysjaObRNHN2wsFYpzqylcbR/eSPmkL3oGh7jBTp8DM48ja8cZ2almUpMpkHdEbEMaF I2Z2Yn3WLK8SqJNvHw33/dV52iyumcdVBCoWwJcGjBpg4xKOo4N5JcyU38imxuD1kr/O ktdnMAha3k8qZMz58McLmJwnsEd3BDRpRjJJEgcfNZPzVGIfyJHmplh27+gPr6cfhVoa a943hUgR0qRUA0jQhzzPqjnpO5SWKHEFlKLtGlh7YG179ltKABLShYhrh/U2Ek4Jo5SR P5bPFFjLcVshUnACFun9igzuHvYR19Gl6DK0oNNsUU/x9KfZ0pcv0j7sregHUxtCbVZ6 mvZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4hK83d7VL1wN/wl1jqGkXbMYrtGu4Forbt4zLd3ya9o=; b=bGYLGMQw9684ZAEkh4X8JYTOLuOm2JBU7NLbCSEWpDuUFg2sCSnUzr4ZixCMABf/+n IWuD6Y1D/01OJln5ZL9Wrb6xz1LKX1SWcfPniyPMQkD17AoUrHei1FINzCP7lb9Z4vEx q4wujWUaHxrU6rsuwt0kOj9LnjHJHXENyC9QG7cSifbZuSHpNFZshabcIaS65fJu5xhQ /8H8bL8Uup7heTG8z/3FeFomgUox26M04wsLoPfPeGdKzxfr7+BvTa0oWIceARfOFE8Z /Sm9N+8+nCZkRlFE3G5299GpQpUcXK5+W05e+POTRpvRrH+BT+iZgCc7KHgXoLPt44Q5 jkyQ== X-Gm-Message-State: AOAM531y8YAbF1KqCXbp9xuY+OXLBI2Cvnsp154VladD/xha2OKA9Caz cfsqwDOT7LoWQXLQuZ4wRbbLMsaFQ91+zQ== X-Google-Smtp-Source: ABdhPJwMLIYctjE2EA0h5LLL4wtsGmLo+MI8YruuMv5bTxHadDom2aEfVS/ZQE2GaZdIeV7op7lZbQ== X-Received: by 2002:a7b:cf0d:: with SMTP id l13mr17425470wmg.168.1609537076998; Fri, 01 Jan 2021 13:37:56 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:56 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:17 +0000 Message-Id: <20210101213537.169546-8-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/27] cbs: Implement common parts of cbs-based bitstream filters separately X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This allows removal of a lot of duplicated code between BSFs. --- libavcodec/Makefile | 2 +- libavcodec/cbs_bsf.c | 161 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_bsf.h | 100 +++++++++++++++++++++++++++ 3 files changed, 262 insertions(+), 1 deletion(-) create mode 100644 libavcodec/cbs_bsf.c create mode 100644 libavcodec/cbs_bsf.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 6e12a8171d..8f50217ad4 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -69,7 +69,7 @@ OBJS-$(CONFIG_AUDIODSP) += audiodsp.o OBJS-$(CONFIG_BLOCKDSP) += blockdsp.o OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o -OBJS-$(CONFIG_CBS) += cbs.o +OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o diff --git a/libavcodec/cbs_bsf.c b/libavcodec/cbs_bsf.c new file mode 100644 index 0000000000..429f360014 --- /dev/null +++ b/libavcodec/cbs_bsf.c @@ -0,0 +1,161 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bsf_internal.h" +#include "cbs_bsf.h" + +static int ff_cbs_bsf_update_side_data(AVBSFContext *bsf, AVPacket *pkt) +{ + CBSBSFContext *ctx = bsf->priv_data; + CodedBitstreamFragment *frag = &ctx->fragment; + uint8_t *side_data; + int side_data_size; + int err; + + side_data = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + &side_data_size); + if (!side_data_size) + return 0; + + err = ff_cbs_read(ctx->input, frag, side_data, side_data_size); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, + "Failed to read extradata from packet side data.\n"); + return err; + } + + err = ctx->type->update_fragment(bsf, NULL, frag); + if (err < 0) + goto fail; + + err = ff_cbs_write_fragment_data(ctx->output, frag); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, + "Failed to write extradata into packet side data.\n"); + return err; + } + + side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, + frag->data_size); + if (!side_data) + return AVERROR(ENOMEM); + memcpy(side_data, frag->data, frag->data_size); + + err = 0; +fail: + ff_cbs_fragment_reset(frag); + return err; +} + +int ff_cbs_bsf_filter(AVBSFContext *bsf, AVPacket *pkt) +{ + CBSBSFContext *ctx = bsf->priv_data; + CodedBitstreamFragment *frag = &ctx->fragment; + int err; + + err = ff_bsf_get_packet_ref(bsf, pkt); + if (err < 0) + return err; + + err = ff_cbs_bsf_update_side_data(bsf, pkt); + if (err < 0) + goto fail; + + err = ff_cbs_read_packet(ctx->input, frag, pkt); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read %s from packet.\n", + ctx->type->fragment_name); + goto fail; + } + + if (frag->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No %s found in packet.\n", + ctx->type->unit_name); + err = AVERROR_INVALIDDATA; + goto fail; + } + + err = ctx->type->update_fragment(bsf, pkt, frag); + if (err < 0) + goto fail; + + err = ff_cbs_write_packet(ctx->output, pkt, frag); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write %s into packet.\n", + ctx->type->fragment_name); + goto fail; + } + + err = 0; +fail: + ff_cbs_fragment_reset(frag); + + if (err < 0) + av_packet_unref(pkt); + + return err; +} + +int ff_cbs_bsf_init(AVBSFContext *bsf, const CBSBSFType *type) +{ + CBSBSFContext *ctx = bsf->priv_data; + CodedBitstreamFragment *frag = &ctx->fragment; + int err; + + ctx->type = type; + + err = ff_cbs_init(&ctx->input, type->codec_id, bsf); + if (err < 0) + return err; + + err = ff_cbs_init(&ctx->output, type->codec_id, bsf); + if (err < 0) + return err; + + if (bsf->par_in->extradata) { + err = ff_cbs_read_extradata(ctx->input, frag, bsf->par_in); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); + goto fail; + } + + err = type->update_fragment(bsf, NULL, frag); + if (err < 0) + goto fail; + + err = ff_cbs_write_extradata(ctx->output, bsf->par_out, frag); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); + goto fail; + } + } + + err = 0; +fail: + ff_cbs_fragment_reset(frag); + return err; +} + +void ff_cbs_bsf_close(AVBSFContext *bsf) +{ + CBSBSFContext *ctx = bsf->priv_data; + + ff_cbs_fragment_free(&ctx->fragment); + ff_cbs_close(&ctx->input); + ff_cbs_close(&ctx->output); +} diff --git a/libavcodec/cbs_bsf.h b/libavcodec/cbs_bsf.h new file mode 100644 index 0000000000..8cab3f144c --- /dev/null +++ b/libavcodec/cbs_bsf.h @@ -0,0 +1,100 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_BSF_H +#define AVCODEC_CBS_BSF_H + +#include "cbs.h" + + +typedef struct CBSBSFType { + enum AVCodecID codec_id; + + // Name of a frame fragment in this codec (e.g. "access unit", + // "temporal unit"). + const char *fragment_name; + + // Name of a unit for this BSF, for use in error messages (e.g. + // "NAL unit", "OBU"). + const char *unit_name; + + // Update the content of a fragment with whatever metadata changes + // are desired. The associated AVPacket is provided so that any side + // data associated with the fragment can be inspected or edited. If + // pkt is NULL, then an extradata header fragment is being updated. + int (*update_fragment)(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *frag); +} CBSBSFType; + +typedef struct CBSBSFContext { + const AVClass *class; + const CBSBSFType *type; + + CodedBitstreamContext *input; + CodedBitstreamContext *output; + CodedBitstreamFragment fragment; +} CBSBSFContext; + + +int ff_cbs_bsf_filter(AVBSFContext *bsf, AVPacket *pkt); +int ff_cbs_bsf_init(AVBSFContext *bsf, const CBSBSFType *type); +void ff_cbs_bsf_close(AVBSFContext *bsf); + + +// Options for element manipulation. +enum { + // Pass this element through unchanged. + BSF_ELEMENT_PASS, + // Insert this element, replacing any existing instances of it. + // Associated values may be provided explicitly (as addtional options) + // or implicitly (either as side data or deduced from other parts of + // the stream). + BSF_ELEMENT_INSERT, + // Remove this element if it appears in the stream. + BSF_ELEMENT_REMOVE, + // Extract this element to side data, so that further manipulation + // can happen elsewhere. + BSF_ELEMENT_EXTRACT, +}; + +#define BSF_ELEMENT_OPTIONS_PIR(name, help, field, unit_name) \ + { name, help, OFFSET(field), AV_OPT_TYPE_INT, \ + { .i64 = BSF_ELEMENT_PASS }, \ + BSF_ELEMENT_PASS, BSF_ELEMENT_REMOVE, FLAGS, unit_name }, \ + { "pass", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_PASS }, .flags = FLAGS, .unit = unit_name }, \ + { "insert", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_INSERT }, .flags = FLAGS, .unit = unit_name }, \ + { "remove", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_REMOVE }, .flags = FLAGS, .unit = unit_name } + +#define BSF_ELEMENT_OPTIONS_PIRE(name, help, field, unit_name) \ + { name, help, OFFSET(field), AV_OPT_TYPE_INT, \ + { .i64 = BSF_ELEMENT_PASS }, \ + BSF_ELEMENT_PASS, BSF_ELEMENT_EXTRACT, FLAGS, unit_name }, \ + { "pass", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_PASS }, .flags = FLAGS, .unit = unit_name }, \ + { "insert", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_INSERT }, .flags = FLAGS, .unit = unit_name }, \ + { "remove", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_REMOVE }, .flags = FLAGS, .unit = unit_name }, \ + { "extract", NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = BSF_ELEMENT_EXTRACT }, .flags = FLAGS, .unit = unit_name } \ + + +#endif /* AVCODEC_CBS_BSF_H */ From patchwork Fri Jan 1 21:35:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24751 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 7269644B4DF for ; Fri, 1 Jan 2021 23:44:10 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5475968AA35; Fri, 1 Jan 2021 23:44:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4210B68A9D9 for ; Fri, 1 Jan 2021 23:44:03 +0200 (EET) Received: by mail-wm1-f50.google.com with SMTP id e25so10026302wme.0 for ; Fri, 01 Jan 2021 13:44:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=85Lfrna4j+4RFIRAVil03RcyjgPHcBgqvw1cpk6QIjM=; b=FHHbzzhj7m6lDzQPiZQknaHHh8JMqZRqV1qvpgqex2yubBid6riq5Fys8z9tTDmfB+ /Ms3tV7uNBjIzzVgVQt2KQqhsPQ3Z+au6bn+em2uPUHGSmR0foYC4S4eMNRVe5W1qJ2i ltQbVO4cnozAXAp1H+A7Y63vvm/aqKSoTa000piJ2cWluBztqZUTitpD1afQbSpWDh90 pNlI7Rw3oZNzK3xMuozhS6GP3GHjwztYowxyboAihAUm6L5pxyetvorW87Cj7pQ1rtNj 1U8N4udOTceJnhnBE3og3J+ZSBXCn0qv8azqZ5CaXkZb4WX/xTVnynpTKuqJOzXsTxLi 6yig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=85Lfrna4j+4RFIRAVil03RcyjgPHcBgqvw1cpk6QIjM=; b=k1eS4bsIua4AxWPLVCiXVyZihkzNHQNs/GsKW1zbKWSv3M3wHElghSJc6f610iWu8Q tM+gwu38JU1cMJPL34cMyNQiVPmyL8M7+MrjhgIOaYE0pKOorkLUverxJH5E4T2bjArg lp12MBQXdutVSf9pgAnykCSajAzKuAxHGIJUgG0I3YtnCepJiD06dvM9q2hdp+L05Gi9 fIH0pK5qbxepMoHBTmkHkJXNBOH7gTVmAqToTh/3/WhwmZPWtx4tHQB1JG6a2FMkHC2a IgWK90G/yR+a9gLJFv84m9h3sIS5NP3PS/wcpM9Zb8noIWlPCRWvZUB3XQwmslA30yPx 2NFA== X-Gm-Message-State: AOAM533f5427Q/jIQwgcAb0Z8KrfIUjWSaWsdweLKZaeJgIzVlam1ql4 E9ah5NgOIijQ3kmniqTPrv6iKzrMniBwgw== X-Google-Smtp-Source: ABdhPJx5/5I7brfi8wdDwd4Z8djdtXKFh+QQcpmy49vu5v0BmazZwxrSBNC3UZo3l58LUlA3+o0z9Q== X-Received: by 2002:a1c:2605:: with SMTP id m5mr17351612wmm.111.1609537078007; Fri, 01 Jan 2021 13:37:58 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:57 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:18 +0000 Message-Id: <20210101213537.169546-9-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/27] h264_metadata_bsf: Use common cbs bsf implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 232 +++++++-------------------------- 1 file changed, 49 insertions(+), 183 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 71b49696cd..f5f439018d 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -22,20 +22,13 @@ #include "libavutil/opt.h" #include "bsf.h" -#include "bsf_internal.h" #include "cbs.h" +#include "cbs_bsf.h" #include "cbs_h264.h" #include "h264.h" #include "h264_levels.h" #include "h264_sei.h" -enum { - PASS, - INSERT, - REMOVE, - EXTRACT, -}; - enum { FLIP_HORIZONTAL = 1, FLIP_VERTICAL = 2, @@ -47,11 +40,7 @@ enum { }; typedef struct H264MetadataContext { - const AVClass *class; - - CodedBitstreamContext *input; - CodedBitstreamContext *output; - CodedBitstreamFragment access_unit; + CBSBSFContext common; int done_first_au; @@ -333,49 +322,6 @@ static int h264_metadata_update_sps(AVBSFContext *bsf, return 0; } -static int h264_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) -{ - H264MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - uint8_t *side_data; - int side_data_size; - int err, i; - - side_data = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, - &side_data_size); - if (!side_data_size) - return 0; - - err = ff_cbs_read(ctx->input, au, side_data, side_data_size); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata from packet side data.\n"); - return err; - } - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SPS) { - err = h264_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - return err; - } - } - - err = ff_cbs_write_fragment_data(ctx->output, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata into packet side data.\n"); - return err; - } - - side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, au->data_size); - if (!side_data) - return AVERROR(ENOMEM); - memcpy(side_data, au->data, au->data_size); - - ff_cbs_fragment_reset(au); - - return 0; -} - static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, AVPacket *pkt, CodedBitstreamFragment *au, @@ -386,7 +332,7 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, int err; message = NULL; - while (ff_cbs_sei_find_message(ctx->output, au, + while (ff_cbs_sei_find_message(ctx->common.output, au, H264_SEI_TYPE_DISPLAY_ORIENTATION, &message) == 0) { H264RawSEIDisplayOrientation *disp = message->payload; @@ -415,13 +361,13 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, } } - if (ctx->display_orientation == REMOVE || - ctx->display_orientation == INSERT) { - ff_cbs_sei_delete_message_type(ctx->output, au, + if (ctx->display_orientation == BSF_ELEMENT_REMOVE || + ctx->display_orientation == BSF_ELEMENT_INSERT) { + ff_cbs_sei_delete_message_type(ctx->common.output, au, H264_SEI_TYPE_DISPLAY_ORIENTATION); } - if (ctx->display_orientation == INSERT) { + if (ctx->display_orientation == BSF_ELEMENT_INSERT) { H264RawSEIDisplayOrientation *disp = &ctx->display_orientation_payload; uint8_t *data; @@ -495,7 +441,7 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, if (write) { disp->display_orientation_repetition_period = 1; - err = ff_cbs_sei_add_message(ctx->output, au, 1, + err = ff_cbs_sei_add_message(ctx->common.output, au, 1, H264_SEI_TYPE_DISPLAY_ORIENTATION, disp, NULL); if (err < 0) { @@ -509,41 +455,21 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, return 0; } -static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +static int h264_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au) { H264MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; int err, i, has_sps, seek_point; - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = h264_metadata_update_side_data(bsf, pkt); - if (err < 0) - goto fail; - - err = ff_cbs_read_packet(ctx->input, au, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } - - if (au->nb_units == 0) { - av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - // If an AUD is present, it must be the first NAL unit. if (au->units[0].type == H264_NAL_AUD) { - if (ctx->aud == REMOVE) + if (ctx->aud == BSF_ELEMENT_REMOVE) ff_cbs_delete_unit(au, 0); } else { - if (ctx->aud == INSERT) { + if (ctx->aud == BSF_ELEMENT_INSERT) { err = h264_metadata_insert_aud(bsf, au); if (err < 0) - goto fail; + return err; } } @@ -552,27 +478,31 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (au->units[i].type == H264_NAL_SPS) { err = h264_metadata_update_sps(bsf, au->units[i].content); if (err < 0) - goto fail; + return err; has_sps = 1; } } - // The current packet should be treated as a seek point for metadata - // insertion if any of: - // - It is the first packet in the stream. - // - It contains an SPS, indicating that a sequence might start here. - // - It is marked as containing a key frame. - seek_point = !ctx->done_first_au || has_sps || - (pkt->flags & AV_PKT_FLAG_KEY); + if (pkt) { + // The current packet should be treated as a seek point for metadata + // insertion if any of: + // - It is the first packet in the stream. + // - It contains an SPS, indicating that a sequence might start here. + // - It is marked as containing a key frame. + seek_point = !ctx->done_first_au || has_sps || + (pkt->flags & AV_PKT_FLAG_KEY); + } else { + seek_point = 0; + } if (ctx->sei_user_data && seek_point) { - err = ff_cbs_sei_add_message(ctx->output, au, 1, + err = ff_cbs_sei_add_message(ctx->common.output, au, 1, H264_SEI_TYPE_USER_DATA_UNREGISTERED, &ctx->sei_user_data_payload, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " "message to access unit.\n"); - goto fail; + return err; } } @@ -584,44 +514,37 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) } } - ff_cbs_sei_delete_message_type(ctx->output, au, + ff_cbs_sei_delete_message_type(ctx->common.output, au, H264_SEI_TYPE_FILLER_PAYLOAD); } - if (ctx->display_orientation != PASS) { + if (pkt && ctx->display_orientation != BSF_ELEMENT_PASS) { err = h264_metadata_handle_display_orientation(bsf, pkt, au, seek_point); if (err < 0) - goto fail; + return err; } - err = ff_cbs_write_packet(ctx->output, pkt, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); - goto fail; - } - - ctx->done_first_au = 1; - - err = 0; -fail: - ff_cbs_fragment_reset(au); - - if (err < 0) - av_packet_unref(pkt); + if (pkt) + ctx->done_first_au = 1; return err; } +static const CBSBSFType h264_metadata_type = { + .codec_id = AV_CODEC_ID_H264, + .fragment_name = "access unit", + .unit_name = "NAL unit", + .update_fragment = &h264_metadata_update_fragment, +}; + static int h264_metadata_init(AVBSFContext *bsf) { H264MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - int err, i; if (ctx->sei_user_data) { SEIRawUserDataUnregistered *udu = &ctx->sei_user_data_payload; - int j; + int i, j; // Parse UUID. It must be a hex string of length 32, possibly // containing '-'s between hex digits (which we ignore). @@ -648,67 +571,18 @@ static int h264_metadata_init(AVBSFContext *bsf) } else { av_log(bsf, AV_LOG_ERROR, "Invalid user data: " "must be \"UUID+string\".\n"); - err = AVERROR(EINVAL); - goto fail; + return AVERROR(EINVAL); } } - err = ff_cbs_init(&ctx->input, AV_CODEC_ID_H264, bsf); - if (err < 0) - return err; - err = ff_cbs_init(&ctx->output, AV_CODEC_ID_H264, bsf); - if (err < 0) - return err; - - if (bsf->par_in->extradata) { - err = ff_cbs_read_extradata(ctx->input, au, bsf->par_in); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); - goto fail; - } - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SPS) { - err = h264_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - goto fail; - } - } - - err = ff_cbs_write_extradata(ctx->output, bsf->par_out, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); - goto fail; - } - } - - err = 0; -fail: - ff_cbs_fragment_reset(au); - return err; -} - -static void h264_metadata_close(AVBSFContext *bsf) -{ - H264MetadataContext *ctx = bsf->priv_data; - - ff_cbs_fragment_free(&ctx->access_unit); - ff_cbs_close(&ctx->input); - ff_cbs_close(&ctx->output); + return ff_cbs_bsf_init(bsf, &h264_metadata_type); } #define OFFSET(x) offsetof(H264MetadataContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) static const AVOption h264_metadata_options[] = { - { "aud", "Access Unit Delimiter NAL units", - OFFSET(aud), AV_OPT_TYPE_INT, - { .i64 = PASS }, PASS, REMOVE, FLAGS, "aud" }, - { "pass", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = PASS }, .flags = FLAGS, .unit = "aud" }, - { "insert", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = INSERT }, .flags = FLAGS, .unit = "aud" }, - { "remove", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = REMOVE }, .flags = FLAGS, .unit = "aud" }, + BSF_ELEMENT_OPTIONS_PIR("aud", "Access Unit Delimiter NAL units", + aud, "aud"), { "sample_aspect_ratio", "Set sample aspect ratio (table E-1)", OFFSET(sample_aspect_ratio), AV_OPT_TYPE_RATIONAL, @@ -764,17 +638,9 @@ static const AVOption h264_metadata_options[] = { { "delete_filler", "Delete all filler (both NAL and SEI)", OFFSET(delete_filler), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS}, - { "display_orientation", "Display orientation SEI", - OFFSET(display_orientation), AV_OPT_TYPE_INT, - { .i64 = PASS }, PASS, EXTRACT, FLAGS, "disp_or" }, - { "pass", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = PASS }, .flags = FLAGS, .unit = "disp_or" }, - { "insert", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = INSERT }, .flags = FLAGS, .unit = "disp_or" }, - { "remove", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = REMOVE }, .flags = FLAGS, .unit = "disp_or" }, - { "extract", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = EXTRACT }, .flags = FLAGS, .unit = "disp_or" }, + BSF_ELEMENT_OPTIONS_PIRE("display_orientation", + "Display orientation SEI", + display_orientation, "disp_or"), { "rotate", "Set rotation in display orientation SEI (anticlockwise angle in degrees)", OFFSET(rotate), AV_OPT_TYPE_DOUBLE, @@ -838,7 +704,7 @@ const AVBitStreamFilter ff_h264_metadata_bsf = { .priv_data_size = sizeof(H264MetadataContext), .priv_class = &h264_metadata_class, .init = &h264_metadata_init, - .close = &h264_metadata_close, - .filter = &h264_metadata_filter, + .close = &ff_cbs_bsf_close, + .filter = &ff_cbs_bsf_filter, .codec_ids = h264_metadata_codec_ids, }; From patchwork Fri Jan 1 21:35:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24750 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4BECC44B4DF for ; Fri, 1 Jan 2021 23:44:09 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 29BE368A9A5; Fri, 1 Jan 2021 23:44:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0CDA468A957 for ; Fri, 1 Jan 2021 23:44:06 +0200 (EET) Received: by mail-wm1-f52.google.com with SMTP id y23so10010955wmi.1 for ; Fri, 01 Jan 2021 13:44:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bhjBEg9CIZYBPSfqod9jhVP9H+4+sTnjmvPwg3+/1HM=; b=b4ApRcrmPz6x9a6y+LsMmZmWAJ3uc4tMUd7VagJ/WDFcEujM9ZaGLXji7yt8/RWvfX EnMTz3GKy2q40lkuf/NMYeXelKwHXSbi/UD8WHsqpKl27S5RoJXyi4zJTUi9KLjoZRxA zcyrg3g9ekkS0J2jX7Z4pZ7xQMEyBkgUKqOd3pLqTkz/GSg/ZpNdciN/WgCfVgjJ246b qeOa8R9l5FL8+vvGWZOoo49a+BJPgk/3Vj+HG8U8CnNyfG8EG59UYh7zWwp7IvCvz6fb wOlYbVFCh6Dl79TJRz8FjYkLmAQCuldUsk4auDZZiLCO0juUCPavtOQUzaVdOtnlOlFI wTHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bhjBEg9CIZYBPSfqod9jhVP9H+4+sTnjmvPwg3+/1HM=; b=AMVbWj29qs5GYTYWr5K7ozcRH6cA/TCOMAMrTZ5BRkIMseInMeZotsPMJbJTN9zivf EFx10Dt50XWrfrXJMMj/qW1gcvdvtJrsXBUn1zehj50PzZ/m420Tc+hpwRuUpoCfmFHt zqxTiJ0H4d3uedapMD/JRQi+Gl3w3i+m7AWCs4RV5nH+RAi+B7gBZkD3/XJGMqCxpgvk 9nRWXa/+9uoylOsdZh/nnc8q/1l+M6a0owJ//TdmKLTs3KYaVs94CzCbGWxsxCUdqynv qtvewXJp/APR41w/R6IPLg6w1OdamribpwF1dEm1oiTO567kOnKm7qPDJINVjGDwVwAw CBVw== X-Gm-Message-State: AOAM531ddzvWqurSvuCs/cYPXipRBSJrEr0gN/z+jyTKWrjJB+JiJrlF HlwaVg3tjGyn+tsTH6AfkpLaGtjl9Unayg== X-Google-Smtp-Source: ABdhPJwBXM8DC8ZVaFv3HCaGza61AWYFQS8FrzNMYnlJf4yYTfoXhpYP9RUOobIeu9pzmBqwt5QUjQ== X-Received: by 2002:a7b:c04c:: with SMTP id u12mr17630561wmc.185.1609537079070; Fri, 01 Jan 2021 13:37:59 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:58 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:19 +0000 Message-Id: <20210101213537.169546-10-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/27] h265_metadata_bsf: Use common cbs bsf implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h265_metadata_bsf.c | 191 ++++----------------------------- 1 file changed, 22 insertions(+), 169 deletions(-) diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index 504a75dac2..c3eadee92b 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -20,29 +20,19 @@ #include "libavutil/opt.h" #include "bsf.h" -#include "bsf_internal.h" #include "cbs.h" +#include "cbs_bsf.h" #include "cbs_h265.h" #include "hevc.h" #include "h265_profile_level.h" -enum { - PASS, - INSERT, - REMOVE, -}; - enum { LEVEL_UNSET = -2, LEVEL_AUTO = -1, }; typedef struct H265MetadataContext { - const AVClass *class; - - CodedBitstreamContext *input; - CodedBitstreamContext *output; - CodedBitstreamFragment access_unit; + CBSBSFContext common; H265RawAUD aud_nal; @@ -338,89 +328,18 @@ static int h265_metadata_update_sps(AVBSFContext *bsf, return 0; } -static int h265_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) -{ - H265MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - uint8_t *side_data; - int side_data_size; - int err, i; - - side_data = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, - &side_data_size); - if (!side_data_size) - return 0; - - err = ff_cbs_read(ctx->input, au, side_data, side_data_size); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata from packet side data.\n"); - return err; - } - - if (ctx->level == LEVEL_AUTO && !ctx->level_guess) - h265_metadata_guess_level(bsf, au); - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == HEVC_NAL_VPS) { - err = h265_metadata_update_vps(bsf, au->units[i].content); - if (err < 0) - return err; - } - if (au->units[i].type == HEVC_NAL_SPS) { - err = h265_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - return err; - } - } - - err = ff_cbs_write_fragment_data(ctx->output, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata into packet side data.\n"); - return err; - } - - side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, au->data_size); - if (!side_data) - return AVERROR(ENOMEM); - memcpy(side_data, au->data, au->data_size); - - ff_cbs_fragment_reset(au); - - return 0; -} - -static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au) { H265MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; int err, i; - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = h265_metadata_update_side_data(bsf, pkt); - if (err < 0) - goto fail; - - err = ff_cbs_read_packet(ctx->input, au, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } - - if (au->nb_units == 0) { - av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - // If an AUD is present, it must be the first NAL unit. if (au->units[0].type == HEVC_NAL_AUD) { - if (ctx->aud == REMOVE) + if (ctx->aud == BSF_ELEMENT_REMOVE) ff_cbs_delete_unit(au, 0); } else { - if (ctx->aud == INSERT) { + if (pkt && ctx->aud == BSF_ELEMENT_INSERT) { H265RawAUD *aud = &ctx->aud_nal; int pic_type = 0, temporal_id = 8, layer_id = 0; @@ -453,7 +372,7 @@ static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) err = ff_cbs_insert_unit_content(au, 0, HEVC_NAL_AUD, aud, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); - goto fail; + return err; } } } @@ -465,101 +384,35 @@ static int h265_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (au->units[i].type == HEVC_NAL_VPS) { err = h265_metadata_update_vps(bsf, au->units[i].content); if (err < 0) - goto fail; + return err; } if (au->units[i].type == HEVC_NAL_SPS) { err = h265_metadata_update_sps(bsf, au->units[i].content); if (err < 0) - goto fail; + return err; } } - err = ff_cbs_write_packet(ctx->output, pkt, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); - goto fail; - } - - err = 0; -fail: - ff_cbs_fragment_reset(au); - - if (err < 0) - av_packet_unref(pkt); - - return err; + return 0; } -static int h265_metadata_init(AVBSFContext *bsf) -{ - H265MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - int err, i; - - err = ff_cbs_init(&ctx->input, AV_CODEC_ID_HEVC, bsf); - if (err < 0) - return err; - err = ff_cbs_init(&ctx->output, AV_CODEC_ID_HEVC, bsf); - if (err < 0) - return err; - - if (bsf->par_in->extradata) { - err = ff_cbs_read_extradata(ctx->input, au, bsf->par_in); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); - goto fail; - } - - if (ctx->level == LEVEL_AUTO) - h265_metadata_guess_level(bsf, au); - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == HEVC_NAL_VPS) { - err = h265_metadata_update_vps(bsf, au->units[i].content); - if (err < 0) - goto fail; - } - if (au->units[i].type == HEVC_NAL_SPS) { - err = h265_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - goto fail; - } - } - - err = ff_cbs_write_extradata(ctx->output, bsf->par_out, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); - goto fail; - } - } - - err = 0; -fail: - ff_cbs_fragment_reset(au); - return err; -} +static const CBSBSFType h265_metadata_type = { + .codec_id = AV_CODEC_ID_HEVC, + .fragment_name = "access unit", + .unit_name = "NAL unit", + .update_fragment = &h265_metadata_update_fragment, +}; -static void h265_metadata_close(AVBSFContext *bsf) +static int h265_metadata_init(AVBSFContext *bsf) { - H265MetadataContext *ctx = bsf->priv_data; - - ff_cbs_fragment_free(&ctx->access_unit); - ff_cbs_close(&ctx->input); - ff_cbs_close(&ctx->output); + return ff_cbs_bsf_init(bsf, &h265_metadata_type); } #define OFFSET(x) offsetof(H265MetadataContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) static const AVOption h265_metadata_options[] = { - { "aud", "Access Unit Delimiter NAL units", - OFFSET(aud), AV_OPT_TYPE_INT, - { .i64 = PASS }, PASS, REMOVE, FLAGS, "aud" }, - { "pass", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = PASS }, .flags = FLAGS, .unit = "aud" }, - { "insert", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = INSERT }, .flags = FLAGS, .unit = "aud" }, - { "remove", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = REMOVE }, .flags = FLAGS, .unit = "aud" }, + BSF_ELEMENT_OPTIONS_PIR("aud", "Access Unit Delimiter NAL units", + aud, "aud"), { "sample_aspect_ratio", "Set sample aspect ratio (table E-1)", OFFSET(sample_aspect_ratio), AV_OPT_TYPE_RATIONAL, @@ -650,7 +503,7 @@ const AVBitStreamFilter ff_hevc_metadata_bsf = { .priv_data_size = sizeof(H265MetadataContext), .priv_class = &h265_metadata_class, .init = &h265_metadata_init, - .close = &h265_metadata_close, - .filter = &h265_metadata_filter, + .close = &ff_cbs_bsf_close, + .filter = &ff_cbs_bsf_filter, .codec_ids = h265_metadata_codec_ids, }; From patchwork Fri Jan 1 21:35:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24732 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2CD0D44BCD1 for ; Fri, 1 Jan 2021 23:38:03 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DE7D268A7B8; Fri, 1 Jan 2021 23:38:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2EF6C68A591 for ; Fri, 1 Jan 2021 23:38:01 +0200 (EET) Received: by mail-wr1-f50.google.com with SMTP id d13so22887411wrc.13 for ; Fri, 01 Jan 2021 13:38:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PDNJD9I+Al/tMuvAn7jsDuE+iObe7Jjo6th6PnAOW8w=; b=gJvB1xmIVncjwzSa5D3WZP9tISiVll1cGg5BVDMUcz6e/iVW5IMOD/hrme7Fxc7NwK GYCo4tikHLY2fW7a2CWui00lFsm+tWIoD0nloSGr1hsN7JVkyvmMIyU97RwbafzlYYtF xccZ1s2xcv3r8M5dmRrlnkTT84YxbbwaZve137PRgrpf70Mxoidedo2VwQxIK725pJz7 pNfPy+Zh4sDgXKVDBDH29EnGLQBFy4KDm6CfVVxpA4GdsiMeg1C41lcbN6Gzydw3IeTN OTbmAN11Cu9LN9A0Q+QX2PcN4RaZBnm9kF7TkFG5QtX8gAeRT3n/B5ZaEpBPVqLYj+/s IFyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PDNJD9I+Al/tMuvAn7jsDuE+iObe7Jjo6th6PnAOW8w=; b=MfrTdNS2E5CQF+qAtkRyGS6qjOkWL19qxCKwxG+1wsIvwKlZoeacemMS1DBnF8eAhg V+nCEzFtgn1QtV/6n4f2GK7HXVpnxPk2Du2aQSNYmg2KH1Z5Cf0XdFQudXNLauK0/Bn4 VqT/NR3lzcra+gmN7IyalwCsbuf0JaeyoLyRvX5R+B35WJV3zcWLeSsrUyse2q21ESrq B5YR+rssh+dikckviGbApmHTeGOq57OEboxsBvptNxjd0piMZdJxdF6mMpbu0ZnM0P3c fYbOII9ew9YFExBADe0A4rbzSIWog3kktJZUJzK74IBoX8A88bAEXu/CR7fpI1zzQi4h gSXw== X-Gm-Message-State: AOAM530TkpNSMyGzqDHWjd1Pz4uVg8NgrrcTS3xvFaCNXLKe8DN+Es/u itAY/jcdr+gMrC7KKyh7mM2xAYzO+iORZQ== X-Google-Smtp-Source: ABdhPJwTmLtvRhbs2fc+eKHJTo9UXfMUtf6ZG0pHBoWsKHhmu+iQURp2amhx99KSd+QDW4P/7AY7/Q== X-Received: by 2002:adf:e348:: with SMTP id n8mr70988010wrj.148.1609537080256; Fri, 01 Jan 2021 13:38:00 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.37.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:37:59 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:20 +0000 Message-Id: <20210101213537.169546-11-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/27] mpeg2_metadata_bsf: Use common cbs bsf implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This also adds support for updating new extradata. --- libavcodec/mpeg2_metadata_bsf.c | 94 +++++---------------------------- 1 file changed, 12 insertions(+), 82 deletions(-) diff --git a/libavcodec/mpeg2_metadata_bsf.c b/libavcodec/mpeg2_metadata_bsf.c index d0048c0e25..367aa1f9f5 100644 --- a/libavcodec/mpeg2_metadata_bsf.c +++ b/libavcodec/mpeg2_metadata_bsf.c @@ -21,16 +21,13 @@ #include "libavutil/opt.h" #include "bsf.h" -#include "bsf_internal.h" #include "cbs.h" +#include "cbs_bsf.h" #include "cbs_mpeg2.h" #include "mpeg12.h" typedef struct MPEG2MetadataContext { - const AVClass *class; - - CodedBitstreamContext *cbc; - CodedBitstreamFragment fragment; + CBSBSFContext common; MPEG2RawExtensionData sequence_display_extension; @@ -48,6 +45,7 @@ typedef struct MPEG2MetadataContext { static int mpeg2_metadata_update_fragment(AVBSFContext *bsf, + AVPacket *pkt, CodedBitstreamFragment *frag) { MPEG2MetadataContext *ctx = bsf->priv_data; @@ -170,49 +168,16 @@ static int mpeg2_metadata_update_fragment(AVBSFContext *bsf, return 0; } -static int mpeg2_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) -{ - MPEG2MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *frag = &ctx->fragment; - int err; - - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = ff_cbs_read_packet(ctx->cbc, frag, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } - - err = mpeg2_metadata_update_fragment(bsf, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to update frame fragment.\n"); - goto fail; - } - - err = ff_cbs_write_packet(ctx->cbc, pkt, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); - goto fail; - } - - err = 0; -fail: - ff_cbs_fragment_reset(frag); - - if (err < 0) - av_packet_unref(pkt); - - return err; -} +static const CBSBSFType mpeg2_metadata_type = { + .codec_id = AV_CODEC_ID_MPEG2VIDEO, + .fragment_name = "frame", + .unit_name = "start code", + .update_fragment = &mpeg2_metadata_update_fragment, +}; static int mpeg2_metadata_init(AVBSFContext *bsf) { MPEG2MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *frag = &ctx->fragment; - int err; #define VALIDITY_CHECK(name) do { \ if (!ctx->name) { \ @@ -226,42 +191,7 @@ static int mpeg2_metadata_init(AVBSFContext *bsf) VALIDITY_CHECK(matrix_coefficients); #undef VALIDITY_CHECK - err = ff_cbs_init(&ctx->cbc, AV_CODEC_ID_MPEG2VIDEO, bsf); - if (err < 0) - return err; - - if (bsf->par_in->extradata) { - err = ff_cbs_read_extradata(ctx->cbc, frag, bsf->par_in); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); - goto fail; - } - - err = mpeg2_metadata_update_fragment(bsf, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to update metadata fragment.\n"); - goto fail; - } - - err = ff_cbs_write_extradata(ctx->cbc, bsf->par_out, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); - goto fail; - } - } - - err = 0; -fail: - ff_cbs_fragment_reset(frag); - return err; -} - -static void mpeg2_metadata_close(AVBSFContext *bsf) -{ - MPEG2MetadataContext *ctx = bsf->priv_data; - - ff_cbs_fragment_free(&ctx->fragment); - ff_cbs_close(&ctx->cbc); + return ff_cbs_bsf_init(bsf, &mpeg2_metadata_type); } #define OFFSET(x) offsetof(MPEG2MetadataContext, x) @@ -307,7 +237,7 @@ const AVBitStreamFilter ff_mpeg2_metadata_bsf = { .priv_data_size = sizeof(MPEG2MetadataContext), .priv_class = &mpeg2_metadata_class, .init = &mpeg2_metadata_init, - .close = &mpeg2_metadata_close, - .filter = &mpeg2_metadata_filter, + .close = &ff_cbs_bsf_close, + .filter = &ff_cbs_bsf_filter, .codec_ids = mpeg2_metadata_codec_ids, }; From patchwork Fri Jan 1 21:35:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24759 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 63D8C448818 for ; Fri, 1 Jan 2021 23:59:17 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4BAD968A9F3; Fri, 1 Jan 2021 23:59:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D7BA568A9F3 for ; Fri, 1 Jan 2021 23:59:10 +0200 (EET) Received: by mail-lf1-f53.google.com with SMTP id l11so50829600lfg.0 for ; Fri, 01 Jan 2021 13:59:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RLGTFnRy8X/tiBZ4AGPUA+lYaYyisBzfDTMVQ9DQCGo=; b=xSYcoX7S/8MtjHrpzBYlQ8wHJZj1j8qj3r5UfsTD+JeQE/3osfVXDv2EX1vT990Jkd EF0rLj7GSP4TTUiSBDh7kUuINrrbwyGOvPX8AubSRocb0LUryM8HdKtkbEqj+q7b9eSs 8JuCOJXpC9JUdSPilfxbblJxS7+tTvJrk/LNK/i1vmgXvkdBBjwUdEdcdrQZER2odu1m mFV7FM4G6a/IJBUKz96oLe7N48M+aApDpCgQaqrIhxwNhwQiDQpf9r9t9DqJGi8VRXz0 h6ydiozBy1JkXGmhHXA7Y/Zgd0xtaO/AAUkpJdeo+eR5JdujhcawicPZhPiSitI11Us7 vi0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RLGTFnRy8X/tiBZ4AGPUA+lYaYyisBzfDTMVQ9DQCGo=; b=s0Edbj8K/En/yUtSA4JtGHFI9z2672vDlP3QJSXH/1TINyWzN7W1oLWZt2oR6TubjE ey/vhRe2woEGUOdn0cx+Un4kS59aAw2kiEjS/Suc65ymCNU/RTG3IHx2mTobzBXtZ2gM KAI+XH+JHjLBS9XxCiEn2iJAPr4WjWMixDzXusl2graGTZ3OENY5nraBt3zr+kbU67hm 4SdIaZBf0XUZGUCKiSgq2UsX92ByZkZ0kDsiBJ9s+xm3Zvg0h4sZU52bBLfNidaFbR8a D+shrMuxF4n3FiFFjHAGpUqq56LxG99vT0wcGb8wG6el5+wHK5sjvWNJ8KtwtIU8FyYd v96g== X-Gm-Message-State: AOAM5307DGOA+0m4rom2YC9IX2DgjCkrJCJJDM2JLv/pAb7f5eHT9f0y sZFSzyT2Z0WssrrQ06hiw5rh5vjH3BXj0A== X-Google-Smtp-Source: ABdhPJxsIUbnRyK6xSqB65RwKTNfBCiC7WjK5Q9Lpt5WYA23MOexWPJGa36XNFEWIqylXyMl4nGbgA== X-Received: by 2002:a05:6000:101:: with SMTP id o1mr69531990wrx.211.1609537081192; Fri, 01 Jan 2021 13:38:01 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:00 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:21 +0000 Message-Id: <20210101213537.169546-12-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 11/27] vp9_metadata_bsf: Use common cbs bsf implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vp9_metadata_bsf.c | 61 +++++++++-------------------------- 1 file changed, 16 insertions(+), 45 deletions(-) diff --git a/libavcodec/vp9_metadata_bsf.c b/libavcodec/vp9_metadata_bsf.c index 00a5580c4d..1d82a401d5 100644 --- a/libavcodec/vp9_metadata_bsf.c +++ b/libavcodec/vp9_metadata_bsf.c @@ -21,15 +21,12 @@ #include "libavutil/opt.h" #include "bsf.h" -#include "bsf_internal.h" #include "cbs.h" +#include "cbs_bsf.h" #include "cbs_vp9.h" typedef struct VP9MetadataContext { - const AVClass *class; - - CodedBitstreamContext *cbc; - CodedBitstreamFragment fragment; + CBSBSFContext common; int color_space; int color_range; @@ -38,21 +35,11 @@ typedef struct VP9MetadataContext { } VP9MetadataContext; -static int vp9_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +static int vp9_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *frag) { VP9MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *frag = &ctx->fragment; - int err, i; - - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = ff_cbs_read_packet(ctx->cbc, frag, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } + int i; for (i = 0; i < frag->nb_units; i++) { VP9RawFrame *frame = frag->units[i].content; @@ -90,35 +77,19 @@ static int vp9_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) } } - err = ff_cbs_write_packet(ctx->cbc, pkt, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); - goto fail; - } - - err = 0; -fail: - ff_cbs_fragment_reset(frag); - - if (err < 0) - av_packet_unref(pkt); - - return err; + return 0; } -static int vp9_metadata_init(AVBSFContext *bsf) -{ - VP9MetadataContext *ctx = bsf->priv_data; - - return ff_cbs_init(&ctx->cbc, AV_CODEC_ID_VP9, bsf); -} +static const CBSBSFType vp9_metadata_type = { + .codec_id = AV_CODEC_ID_VP9, + .fragment_name = "superframe", + .unit_name = "frame", + .update_fragment = &vp9_metadata_update_fragment, +}; -static void vp9_metadata_close(AVBSFContext *bsf) +static int vp9_metadata_init(AVBSFContext *bsf) { - VP9MetadataContext *ctx = bsf->priv_data; - - ff_cbs_fragment_free(&ctx->fragment); - ff_cbs_close(&ctx->cbc); + return ff_cbs_bsf_init(bsf, &vp9_metadata_type); } #define OFFSET(x) offsetof(VP9MetadataContext, x) @@ -169,7 +140,7 @@ const AVBitStreamFilter ff_vp9_metadata_bsf = { .priv_data_size = sizeof(VP9MetadataContext), .priv_class = &vp9_metadata_class, .init = &vp9_metadata_init, - .close = &vp9_metadata_close, - .filter = &vp9_metadata_filter, + .close = &ff_cbs_bsf_close, + .filter = &ff_cbs_bsf_filter, .codec_ids = vp9_metadata_codec_ids, }; From patchwork Fri Jan 1 21:35:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24733 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 24E2F44BCD1 for ; Fri, 1 Jan 2021 23:38:05 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 089A368A82A; Fri, 1 Jan 2021 23:38:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0512968A7E0 for ; Fri, 1 Jan 2021 23:38:02 +0200 (EET) Received: by mail-wm1-f41.google.com with SMTP id r4so9958007wmh.5 for ; Fri, 01 Jan 2021 13:38:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=y0piYZ56aZ4gk4HjlPT5iVR2lPSpUK8A5omvL8G4ZqI=; b=qmOYLvvasn0HDub2dmAiL/lBmHE+3fYMIDgp+NdcdjvigObSXUtTwHGXBdRAoUZ4Vj zV9QYQSgYHlpYN9HOvRvOuRhDd5Rm1uol+XbB7CqNxVYIadqtyfkdfh2mfOY7U3QXvJW BLYSHA/4BTyn4txgqbLddXj4S9Tpb7kEUvOdpvbwDB3Grr7q0TItOMqobRFtKqcS/z9v DzAQVAO+NQ+jH6Ca1zT1wl95jJdRwyKG/DnkTdTVUiI5Z55wnCvHOO2lxBHNca9O4CqM 3gdWGDOvh6a0H0H9TA//NcRLehc9L3SP+0FVnzJb/nh1WG2cQMoVoAXr0n6s4WCDO8gr I+lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y0piYZ56aZ4gk4HjlPT5iVR2lPSpUK8A5omvL8G4ZqI=; b=otez0GRfTSy+UqVp6kMKzVtoJdvs/AWfGFyBMh2tsGga1dzhjlI+wwuKzVMf+dzpo/ G57HXZgroxHfD6tHRYBAaIT994ARXnQlf+30v4HIRA3LSYtIDoIkWbdFAlG+16bCsALQ +fZ25EU+lL4n2U52sLeKqFZ1KpGyy5uHmEVt7hB/TcmMuOA8NtsT/55uAXzFksBnlE/a F/dpzw4LalaAbmqma2MDVORYgrDNNlnp04HGiWCJIwgaFekqrm8zDrBmL7zmNo2tHk5R DtufI5YjLyviP6Wilc2hD36kjAdm69wwPK6lqk7o5g9s7O4hvcJZqR54hpEZCxSBOC03 DgFA== X-Gm-Message-State: AOAM533G+NVkRICBuPtz86IrQvdIDr9v05A/lVWIax6G+qRBxbKvFsXy FuFxjFmRWEV8wz7i71B4+WpbXtD3LX7FNA== X-Google-Smtp-Source: ABdhPJziRbA5r1/EJs0HvAELmGPS2fp7HlpbN4Jy3J8c/RSk/qKSw7A9YJAF+q3tP/cv931bI8/UEw== X-Received: by 2002:a1c:4d05:: with SMTP id o5mr17582985wmh.85.1609537082149; Fri, 01 Jan 2021 13:38:02 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:01 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:22 +0000 Message-Id: <20210101213537.169546-13-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 12/27] av1_metadata_bsf: Use common cbs bsf implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/av1_metadata_bsf.c | 176 ++++------------------------------ 1 file changed, 21 insertions(+), 155 deletions(-) diff --git a/libavcodec/av1_metadata_bsf.c b/libavcodec/av1_metadata_bsf.c index 3158ba995b..31fd4f0f41 100644 --- a/libavcodec/av1_metadata_bsf.c +++ b/libavcodec/av1_metadata_bsf.c @@ -20,22 +20,12 @@ #include "libavutil/opt.h" #include "bsf.h" -#include "bsf_internal.h" #include "cbs.h" +#include "cbs_bsf.h" #include "cbs_av1.h" -enum { - PASS, - INSERT, - REMOVE, -}; - typedef struct AV1MetadataContext { - const AVClass *class; - - CodedBitstreamContext *input; - CodedBitstreamContext *output; - CodedBitstreamFragment access_unit; + CBSBSFContext common; int td; @@ -113,91 +103,27 @@ static int av1_metadata_update_sequence_header(AVBSFContext *bsf, return 0; } -static int av1_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) +static int av1_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *frag) { AV1MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *frag = &ctx->access_unit; - uint8_t *side_data; - int side_data_size; - int err, i; - - side_data = av_packet_get_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, - &side_data_size); - if (!side_data_size) - return 0; - - err = ff_cbs_read(ctx->input, frag, side_data, side_data_size); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata from packet side data.\n"); - return err; - } - - for (i = 0; i < frag->nb_units; i++) { - if (frag->units[i].type == AV1_OBU_SEQUENCE_HEADER) { - AV1RawOBU *obu = frag->units[i].content; - err = av1_metadata_update_sequence_header(bsf, &obu->obu.sequence_header); - if (err < 0) - return err; - } - } - - err = ff_cbs_write_fragment_data(ctx->output, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata into packet side data.\n"); - return err; - } - - side_data = av_packet_new_side_data(pkt, AV_PKT_DATA_NEW_EXTRADATA, frag->data_size); - if (!side_data) - return AVERROR(ENOMEM); - memcpy(side_data, frag->data, frag->data_size); - - ff_cbs_fragment_reset(frag); - - return 0; -} - -static int av1_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) -{ - AV1MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *frag = &ctx->access_unit; AV1RawOBU td, *obu; int err, i; - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = av1_metadata_update_side_data(bsf, pkt); - if (err < 0) - goto fail; - - err = ff_cbs_read_packet(ctx->input, frag, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } - - if (frag->nb_units == 0) { - av_log(bsf, AV_LOG_ERROR, "No OBU in packet.\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - for (i = 0; i < frag->nb_units; i++) { if (frag->units[i].type == AV1_OBU_SEQUENCE_HEADER) { obu = frag->units[i].content; err = av1_metadata_update_sequence_header(bsf, &obu->obu.sequence_header); if (err < 0) - goto fail; + return err; } } // If a Temporal Delimiter is present, it must be the first OBU. if (frag->units[0].type == AV1_OBU_TEMPORAL_DELIMITER) { - if (ctx->td == REMOVE) + if (ctx->td == BSF_ELEMENT_REMOVE) ff_cbs_delete_unit(frag, 0); - } else if (ctx->td == INSERT) { + } else if (pkt && ctx->td == BSF_ELEMENT_INSERT) { td = (AV1RawOBU) { .header.obu_type = AV1_OBU_TEMPORAL_DELIMITER, }; @@ -206,7 +132,7 @@ static int av1_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) &td, NULL); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to insert Temporal Delimiter.\n"); - goto fail; + return err; } } @@ -217,86 +143,26 @@ static int av1_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) } } - err = ff_cbs_write_packet(ctx->output, pkt, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); - goto fail; - } - - err = 0; -fail: - ff_cbs_fragment_reset(frag); - - if (err < 0) - av_packet_unref(pkt); - - return err; + return 0; } -static int av1_metadata_init(AVBSFContext *bsf) -{ - AV1MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *frag = &ctx->access_unit; - AV1RawOBU *obu; - int err, i; - - err = ff_cbs_init(&ctx->input, AV_CODEC_ID_AV1, bsf); - if (err < 0) - return err; - err = ff_cbs_init(&ctx->output, AV_CODEC_ID_AV1, bsf); - if (err < 0) - return err; - - if (bsf->par_in->extradata) { - err = ff_cbs_read_extradata(ctx->input, frag, bsf->par_in); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); - goto fail; - } - - for (i = 0; i < frag->nb_units; i++) { - if (frag->units[i].type == AV1_OBU_SEQUENCE_HEADER) { - obu = frag->units[i].content; - err = av1_metadata_update_sequence_header(bsf, &obu->obu.sequence_header); - if (err < 0) - goto fail; - } - } - - err = ff_cbs_write_extradata(ctx->output, bsf->par_out, frag); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); - goto fail; - } - } - - err = 0; -fail: - ff_cbs_fragment_reset(frag); - return err; -} +static const CBSBSFType av1_metadata_type = { + .codec_id = AV_CODEC_ID_AV1, + .fragment_name = "temporal unit", + .unit_name = "OBU", + .update_fragment = &av1_metadata_update_fragment, +}; -static void av1_metadata_close(AVBSFContext *bsf) +static int av1_metadata_init(AVBSFContext *bsf) { - AV1MetadataContext *ctx = bsf->priv_data; - - ff_cbs_fragment_free(&ctx->access_unit); - ff_cbs_close(&ctx->input); - ff_cbs_close(&ctx->output); + return ff_cbs_bsf_init(bsf, &av1_metadata_type); } #define OFFSET(x) offsetof(AV1MetadataContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) static const AVOption av1_metadata_options[] = { - { "td", "Temporal Delimiter OBU", - OFFSET(td), AV_OPT_TYPE_INT, - { .i64 = PASS }, PASS, REMOVE, FLAGS, "td" }, - { "pass", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = PASS }, .flags = FLAGS, .unit = "td" }, - { "insert", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = INSERT }, .flags = FLAGS, .unit = "td" }, - { "remove", NULL, 0, AV_OPT_TYPE_CONST, - { .i64 = REMOVE }, .flags = FLAGS, .unit = "td" }, + BSF_ELEMENT_OPTIONS_PIR("td", "Temporal Delimiter OBU", + td, "td"), { "color_primaries", "Set color primaries (section 6.4.2)", OFFSET(color_primaries), AV_OPT_TYPE_INT, @@ -356,7 +222,7 @@ const AVBitStreamFilter ff_av1_metadata_bsf = { .priv_data_size = sizeof(AV1MetadataContext), .priv_class = &av1_metadata_class, .init = &av1_metadata_init, - .close = &av1_metadata_close, - .filter = &av1_metadata_filter, + .close = &ff_cbs_bsf_close, + .filter = &ff_cbs_bsf_filter, .codec_ids = av1_metadata_codec_ids, }; From patchwork Fri Jan 1 21:35:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24753 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id F0DBF44B645 for ; Fri, 1 Jan 2021 23:45:07 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D0EDC68A912; Fri, 1 Jan 2021 23:45:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C56C568A817 for ; Fri, 1 Jan 2021 23:45:06 +0200 (EET) Received: by mail-wm1-f53.google.com with SMTP id 3so9998801wmg.4 for ; Fri, 01 Jan 2021 13:45:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Qongk7hXDxU+xX+/kWG1hd2GP64NBh4fvFBuyX66mIA=; b=vRek27oHHW9bdl459ZTCYrMUJHsPKf7iajR0XNNcLfJXQkxq/UEOJ/STcSD4V4eLmN 2wHV7CCGRBtpnAIFinfMbYi0LgYCyAOY8AY+GkgCKfPxXz+q8Wnpnwvb6kANkbLic5RV mTK1bW1h6tIIJgNZlMVKiL61JoxpHHY5tPio346KW4s/oErZMJljuaGVWVvJnup0Alnd YYqawiuhSZKoSgKBLnKwxf7qjESR2d16zxeL6kuByqLkJsTS1yjVhxJ84VZxu7EZ20T+ EmSjgGfqRlBGQAMp5YAxLMZAd9FjVKQXxem6HUYchIbBD0YyeuT3RDGJ77eOWsGErMgO Ao7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qongk7hXDxU+xX+/kWG1hd2GP64NBh4fvFBuyX66mIA=; b=PAAG5px50iEIhaQG6rZTUSrzcZiRnKgcS5GxkK8Gn1s+xHiVSuALaFsKBqgcCV+fUS xCgO6C/ikRokkXsRNakFAFJcPA2Cl5+ddMnkMUCJZ7fybYVhSNGdkHWXhcG4x76l6ys2 Zp09/EZgsvMbOhr/ntb3EtP3FfoGmSPV2YCzf5fW+ecywfcvD6xDEK9q+0+A/k/q7aT2 0m2y68FzZYYys/3lg++jiRMOLWXBpNo2rOGSFwyQLdkDtNYX4Ysw2E4wdTzLwU6WXa8w gemec9jcjsw1ex6LLFpq1dwqOZlKA/Y+RZDDmOOSfYn3Bc2UOoiCuYrCBc8FPt1X9iOv BNpw== X-Gm-Message-State: AOAM531C4wME0j8RF8ykiKE20VVN1kwRYJWo8DoTqWVeEA9ivJTn1iC3 tdMeislFHTl7tbbU2Sm1YvwHdqzYTIMxbg== X-Google-Smtp-Source: ABdhPJwke0z75Z0C+IF5M+5hNYcAQvpKB3GB878XFI3QTlehMkZWLYJEKujFXsHJEftzEfs3rofm5Q== X-Received: by 2002:a1c:9ac6:: with SMTP id c189mr17155645wme.137.1609537083035; Fri, 01 Jan 2021 13:38:03 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:02 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:23 +0000 Message-Id: <20210101213537.169546-14-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 13/27] h264_redundant_pps_bsf: Use common cbs bsf implementation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_redundant_pps_bsf.c | 99 ++++++----------------------- 1 file changed, 21 insertions(+), 78 deletions(-) diff --git a/libavcodec/h264_redundant_pps_bsf.c b/libavcodec/h264_redundant_pps_bsf.c index cf41abe96c..fb91227eec 100644 --- a/libavcodec/h264_redundant_pps_bsf.c +++ b/libavcodec/h264_redundant_pps_bsf.c @@ -24,15 +24,13 @@ #include "bsf.h" #include "bsf_internal.h" #include "cbs.h" +#include "cbs_bsf.h" #include "cbs_h264.h" #include "h264.h" typedef struct H264RedundantPPSContext { - CodedBitstreamContext *input; - CodedBitstreamContext *output; - - CodedBitstreamFragment access_unit; + CBSBSFContext common; int global_pic_init_qp; int current_pic_init_qp; @@ -49,7 +47,7 @@ static int h264_redundant_pps_fixup_pps(H264RedundantPPSContext *ctx, // The changes we are about to perform affect the parsing process, // so we must make sure that the PPS is writable, otherwise the // parsing of future slices will be incorrect and even raise errors. - err = ff_cbs_make_unit_writable(ctx->input, unit); + err = ff_cbs_make_unit_writable(ctx->common.input, unit); if (err < 0) return err; pps = unit->content; @@ -78,21 +76,14 @@ static int h264_redundant_pps_fixup_slice(H264RedundantPPSContext *ctx, return 0; } -static int h264_redundant_pps_filter(AVBSFContext *bsf, AVPacket *pkt) +static int h264_redundant_pps_update_fragment(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *au) { H264RedundantPPSContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; int au_has_sps; int err, i; - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = ff_cbs_read_packet(ctx->input, au, pkt); - if (err < 0) - goto fail; - au_has_sps = 0; for (i = 0; i < au->nb_units; i++) { CodedBitstreamUnit *nal = &au->units[i]; @@ -102,7 +93,7 @@ static int h264_redundant_pps_filter(AVBSFContext *bsf, AVPacket *pkt) if (nal->type == H264_NAL_PPS) { err = h264_redundant_pps_fixup_pps(ctx, nal); if (err < 0) - goto fail; + return err; if (!au_has_sps) { av_log(bsf, AV_LOG_VERBOSE, "Deleting redundant PPS " "at %"PRId64".\n", pkt->pts); @@ -118,62 +109,7 @@ static int h264_redundant_pps_filter(AVBSFContext *bsf, AVPacket *pkt) } } - err = ff_cbs_write_packet(ctx->output, pkt, au); - if (err < 0) - goto fail; - - err = 0; -fail: - ff_cbs_fragment_reset(au); - if (err < 0) - av_packet_unref(pkt); - - return err; -} - -static int h264_redundant_pps_init(AVBSFContext *bsf) -{ - H264RedundantPPSContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - int err, i; - - err = ff_cbs_init(&ctx->input, AV_CODEC_ID_H264, bsf); - if (err < 0) - return err; - - err = ff_cbs_init(&ctx->output, AV_CODEC_ID_H264, bsf); - if (err < 0) - return err; - - ctx->global_pic_init_qp = 26; - - if (bsf->par_in->extradata) { - err = ff_cbs_read_extradata(ctx->input, au, bsf->par_in); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read extradata.\n"); - goto fail; - } - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_PPS) { - err = h264_redundant_pps_fixup_pps(ctx, &au->units[i]); - if (err < 0) - goto fail; - } - } - - ctx->extradata_pic_init_qp = ctx->current_pic_init_qp; - err = ff_cbs_write_extradata(ctx->output, bsf->par_out, au); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to write extradata.\n"); - goto fail; - } - } - - err = 0; -fail: - ff_cbs_fragment_reset(au); - return err; + return 0; } static void h264_redundant_pps_flush(AVBSFContext *bsf) @@ -182,13 +118,20 @@ static void h264_redundant_pps_flush(AVBSFContext *bsf) ctx->current_pic_init_qp = ctx->extradata_pic_init_qp; } -static void h264_redundant_pps_close(AVBSFContext *bsf) +static const CBSBSFType h264_redundant_pps_type = { + .codec_id = AV_CODEC_ID_H264, + .fragment_name = "access unit", + .unit_name = "NAL unit", + .update_fragment = &h264_redundant_pps_update_fragment, +}; + +static int h264_redundant_pps_init(AVBSFContext *bsf) { H264RedundantPPSContext *ctx = bsf->priv_data; - ff_cbs_fragment_free(&ctx->access_unit); - ff_cbs_close(&ctx->input); - ff_cbs_close(&ctx->output); + ctx->global_pic_init_qp = 26; + + return ff_cbs_bsf_init(bsf, &h264_redundant_pps_type); } static const enum AVCodecID h264_redundant_pps_codec_ids[] = { @@ -200,7 +143,7 @@ const AVBitStreamFilter ff_h264_redundant_pps_bsf = { .priv_data_size = sizeof(H264RedundantPPSContext), .init = &h264_redundant_pps_init, .flush = &h264_redundant_pps_flush, - .close = &h264_redundant_pps_close, - .filter = &h264_redundant_pps_filter, + .close = &ff_cbs_bsf_close, + .filter = &ff_cbs_bsf_filter, .codec_ids = h264_redundant_pps_codec_ids, }; From patchwork Fri Jan 1 21:35:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24734 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4BC9D44BCD1 for ; Fri, 1 Jan 2021 23:38:06 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2DFDC68A82B; Fri, 1 Jan 2021 23:38:06 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCA8F68A7D4 for ; Fri, 1 Jan 2021 23:38:04 +0200 (EET) Received: by mail-wm1-f41.google.com with SMTP id 3so9960878wmg.4 for ; Fri, 01 Jan 2021 13:38:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=INX/BBJLWJ8elaKRVuhYcemS35q1drgBGFU+eNmt+uw=; b=PYmDFoZ3+3JZGR1m+Lhz+5JnXmJypWwfuzzaJ8CsDNHdkXyadFriZnaTk0Z4b7pAvi 9J8chLL3sbuyHNgnUhgpq+YnrJNDyxHYEoPB9fBscWsbn5Y8yKpTq+fDsmm9ND7AXe60 LNb2vFt05ElEdv0lx7GB/QILjSAztoIo4AY4+S7/wOFfulS+SKH/6z0VKEVA/NbT16E3 XgS87EO+i3SOP1+PkVb/riagE3I7hR29OS8VrDJtpA828S68EMdTvQzuz6eN2MTmn0a4 /Hgxr4DEtCMKaVog9Iioc12tdf80oREr6dG9z5ciwlrHpvRTVGRm50TX96M1orBwhZGr OgLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=INX/BBJLWJ8elaKRVuhYcemS35q1drgBGFU+eNmt+uw=; b=aHl5bNVGB3ICFQ4K1n1zyUbjITTPzPHqxzCM40Es2+CbXUEHNooSnqVkkwiPv6CJgX ghkjQFFXBMvLc395M2LcsfsULmGVNnQGBsQxaxMqjirGCOWGB6k8VJG8q2ijMPQ5aGzM wVaU6Sj9lgrDpmsiVcHMW2YdTjhuheR228G6t4EjGOu9KXA68qMRIzjWyPwspx8VFSCx zGfCmlTzdlps8JnGW+Zy3GiFUKqmTONjpcHTGYNxgG446YbJyvUG5DlZVAQlgr9Ebrx+ PuQ86M5MzeF1E/G5dPxJC8ZwtWHcmxr7rll7IOu/MM28wf2CncZApYkOEtUq72bmpEl9 Dk7A== X-Gm-Message-State: AOAM53130HprDA9kZjUuvBwF+OTZCR+6BNMo+Q08SEDFbMQ+xSydDLkM aYPHI1t4LpZTHYOPrTSO6SAog5Lks3gVWQ== X-Google-Smtp-Source: ABdhPJxPovK1mRU7DJZjttCa+d1bSx8bYr300mIP6mD8A5hDP0CUgycflps+pLp+pI8h9IPPS6QIiQ== X-Received: by 2002:a7b:c4d5:: with SMTP id g21mr17207614wmk.92.1609537084046; Fri, 01 Jan 2021 13:38:04 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:03 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:24 +0000 Message-Id: <20210101213537.169546-15-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 14/27] cbs: Add interface for manipulating metadata in fragments X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/Makefile | 2 +- libavcodec/cbs_internal.h | 17 ++++++ libavcodec/cbs_metadata.c | 107 ++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_metadata.h | 94 +++++++++++++++++++++++++++++++++ 4 files changed, 219 insertions(+), 1 deletion(-) create mode 100644 libavcodec/cbs_metadata.c create mode 100644 libavcodec/cbs_metadata.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 8f50217ad4..9b3f3ecdf7 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -69,7 +69,7 @@ OBJS-$(CONFIG_AUDIODSP) += audiodsp.o OBJS-$(CONFIG_BLOCKDSP) += blockdsp.o OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o -OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o +OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o cbs_metadata.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index faa847aad3..a505080ccf 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -83,6 +83,8 @@ typedef const struct CodedBitstreamUnitTypeDescriptor { int (*content_clone)(AVBufferRef **ref, CodedBitstreamUnit *unit); } CodedBitstreamUnitTypeDescriptor; +enum CBSMetadataType; + typedef struct CodedBitstreamType { enum AVCodecID codec_id; @@ -122,6 +124,21 @@ typedef struct CodedBitstreamType { // Free the codec internal state. void (*close)(CodedBitstreamContext *ctx); + + // Insert metadata to fragment. + int (*insert_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data); + + // Remove metadata from fragment. + int (*remove_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type); + + // Extract metadata from fragment. + int (*extract_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data); } CodedBitstreamType; diff --git a/libavcodec/cbs_metadata.c b/libavcodec/cbs_metadata.c new file mode 100644 index 0000000000..140f5bcd9c --- /dev/null +++ b/libavcodec/cbs_metadata.c @@ -0,0 +1,107 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/film_grain_params.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/stereo3d.h" + +#include "cbs.h" +#include "cbs_internal.h" +#include "cbs_metadata.h" + +static const CBSMetadataTypeDescriptor cbs_metadata_types[] = { + { + CBS_METADATA_MASTERING_DISPLAY, + "Mastering Display", + 1, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA, + 1, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, + }, + { + CBS_METADATA_CONTENT_LIGHT_LEVEL, + "Content Light Level", + 1, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL, + 1, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, + }, + { + CBS_METADATA_DISPLAY_MATRIX, + "Display Matrix", + 1, AV_FRAME_DATA_DISPLAYMATRIX, + 1, AV_PKT_DATA_DISPLAYMATRIX, + }, + { + CBS_METADATA_STEREO3D, + "Stereo 3D", + 1, AV_FRAME_DATA_STEREO3D, + 1, AV_PKT_DATA_STEREO3D, + }, +}; + +const CBSMetadataTypeDescriptor *ff_cbs_metadata_find_type(enum CBSMetadataType type) +{ + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_metadata_types); i++) { + if (cbs_metadata_types[i].type == type) + return &cbs_metadata_types[i]; + } + return NULL; +} + +void *ff_cbs_alloc_metadata(enum CBSMetadataType type, size_t *size) +{ + switch (type) { + case CBS_METADATA_MASTERING_DISPLAY: + *size = sizeof(AVMasteringDisplayMetadata); + return av_mastering_display_metadata_alloc(); + case CBS_METADATA_CONTENT_LIGHT_LEVEL: + return av_content_light_metadata_alloc(size); + case CBS_METADATA_DISPLAY_MATRIX: + *size = 9 * sizeof(int32_t); + return av_mallocz(*size); + case CBS_METADATA_STEREO3D: + *size = sizeof(AVStereo3D); + return av_stereo3d_alloc(); + default: + return NULL; + } +} + +int ff_cbs_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data) +{ + if (!ctx->codec->insert_metadata) + return AVERROR(ENOSYS); + return ctx->codec->insert_metadata(ctx, frag, type, data); +} + +int ff_cbs_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type) +{ + if (!ctx->codec->remove_metadata) + return AVERROR(ENOSYS); + return ctx->codec->remove_metadata(ctx, frag, type); +} + +int ff_cbs_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data) +{ + if (!ctx->codec->extract_metadata) + return AVERROR(ENOSYS); + return ctx->codec->extract_metadata(ctx, frag, type, data); +} diff --git a/libavcodec/cbs_metadata.h b/libavcodec/cbs_metadata.h new file mode 100644 index 0000000000..79ba6f4e26 --- /dev/null +++ b/libavcodec/cbs_metadata.h @@ -0,0 +1,94 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_METADATA_H +#define AVCODEC_CBS_METADATA_H + +#include "libavutil/frame.h" + +#include "cbs.h" +#include "packet.h" + + +enum CBSMetadataType { + // List sentinel element. + CBS_METADATA_NONE, + // AVMasteringDisplayMetadata. + CBS_METADATA_MASTERING_DISPLAY, + // AVContentLightMetadata. + CBS_METADATA_CONTENT_LIGHT_LEVEL, + // int32_t matrix[9]. + CBS_METADATA_DISPLAY_MATRIX, + // AVStereo3D. + CBS_METADATA_STEREO3D, +}; + +typedef struct CBSMetadataTypeDescriptor { + enum CBSMetadataType type; + const char *name; + + // Matching frame and packet side data types, if available. + int has_frame_side_data; + enum AVFrameSideDataType frame_side_data_type; + int has_packet_side_data; + enum AVPacketSideDataType packet_side_data_type; +} CBSMetadataTypeDescriptor; + +/** + * Retrieve the descriptor for the given metadata type. + */ +const CBSMetadataTypeDescriptor *ff_cbs_metadata_find_type(enum CBSMetadataType type); + +/** + * Allocates an object for the given metadata type. + * + * If not null, size is written with the size of the object returned. + */ +void *ff_cbs_alloc_metadata(enum CBSMetadataType type, size_t *size); + +/** + * Insert metadata into a fragment. + * + * type is a CBS_METADATA_* value. data must be a pointer to a filled + * structure matching the specified type. + */ +int ff_cbs_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data); + +/** + * Remove metadata from a fragment. + * + * Remove all metadata of the given CBS_METADATA_* type. + */ +int ff_cbs_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type); + +/** + * Extract metadata from a fragment. + * + * type is a CBS_METADATA_* value. data must be a pointer to an empty + * structure matching the specified type, which will be filled on + * success. + */ +int ff_cbs_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data); + +#endif /* AVCODEC_CBS_METADATA_H */ From patchwork Fri Jan 1 21:35:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24760 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B28F6449523 for ; Sat, 2 Jan 2021 00:01:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8277F68A957; Sat, 2 Jan 2021 00:01:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D93A068A898 for ; Sat, 2 Jan 2021 00:01:04 +0200 (EET) Received: by mail-lf1-f46.google.com with SMTP id x20so50723278lfe.12 for ; Fri, 01 Jan 2021 14:01:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=avIye70uFKl+snYbaSpMYeGExgBSvQ14bmlClqH4gj0=; b=kMUgvwmN8SBF4yT3k3N/j6YwbKgWADnUG4UozHho9Ef3cegmCTYBPGKydekjH5O+zl 3FQKuaKbhxiPIxELHYNW9ziU2yHIc6g2iVkky/tYQY0OKhfrT0JZT5j6SAw4CO6Yzopb ODMfe0Ezx2aclbMNC4ie4YT2ZAxl/eKywK1i0uJaV4zPrhcG5IkfyPDY90OTRlqbl6+5 Xd9kGSQPVuy80HcAEBjzoNLx2MAZutJ0VY6AxTCiBr1FI9RAbxXQHv3gGP/s6BHG4nOZ QhOdUX8744uRGsx18x0e4zNm7owmFM8tTH0Gw50jTfD88zqASjRm4w26Q8icpOe7d/Ve UvSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=avIye70uFKl+snYbaSpMYeGExgBSvQ14bmlClqH4gj0=; b=nJILgGeKhGj4SZjEwhAfxlsBF1Oun0da6tbyb48EZ827jMA+wWH05qQYvndqI7xEzo gQE9dbDX+8Cg6DsinTd+siJdPhSoi6jmylY9143zxJxbG0m6z77MVHSe0JPEHvsFcZ0Y SqKB6qmRmxtVY2bhbbWDq5RX5rKikwS8jMH2H/7hheZ3AG+p66Oe5oxyZe8oRzn3ExsW 5AZHwncrg31Fcix6xoiICopp6FsULh710UAE/EywPOd7zeCC4oBOjijLQWJL/J1i/FL2 jG7qyfKVYW0Iv5dre8rYXBcI7QZNKX/az7P5XeLSaJw4QqkJZO1LEJlgkqEq3+cpS2IC 0JYQ== X-Gm-Message-State: AOAM533a11bJe8eBs73T9ZEfGNperC36Rkmy06sMU+A6CMJn7DsChbOc y9Dt4jAA8azmwMGTi6nISHxEw6e3jo8tog== X-Google-Smtp-Source: ABdhPJwEsbMUeKqKIDlZ4ecDEeQ+lmu004qKvAWMIf4/m6BAnq/N/wtFOlQnZ1AAYps0KRDJI4/ckg== X-Received: by 2002:adf:f6c9:: with SMTP id y9mr70604258wrp.121.1609537084933; Fri, 01 Jan 2021 13:38:04 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:04 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:25 +0000 Message-Id: <20210101213537.169546-16-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 15/27] cbs_sei: Implement metadata manipulation for SEI codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 8 ++++ libavcodec/cbs_sei.c | 94 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.h | 35 ++++++++++++++++ 3 files changed, 137 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index a00bc27370..42b614034e 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1478,6 +1478,10 @@ const CodedBitstreamType ff_cbs_type_h264 = { .flush = &cbs_h264_flush, .close = &cbs_h264_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; const CodedBitstreamType ff_cbs_type_h265 = { @@ -1494,6 +1498,10 @@ const CodedBitstreamType ff_cbs_type_h265 = { .flush = &cbs_h265_flush, .close = &cbs_h265_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 323997b600..e5f9e3e403 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -21,6 +21,7 @@ #include "cbs_h264.h" #include "cbs_h265.h" #include "cbs_sei.h" +#include "cbs_metadata.h" static void cbs_free_user_data_registered(void *opaque, uint8_t *data) { @@ -367,3 +368,96 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, } } } + +typedef struct SEIMetadata { + enum CBSMetadataType cbs_type; + int sei_type; +} SEIMetadata; + +static const SEIMetadata cbs_sei_metadata[] = { +}; + +static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata + (CodedBitstreamContext *ctx, enum CBSMetadataType metadata_type) +{ + const SEIMetadata *metadata; + + metadata = NULL; + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_sei_metadata); i++) { + if (metadata_type == cbs_sei_metadata[i].cbs_type) { + metadata = &cbs_sei_metadata[i]; + break; + } + } + if (metadata) + return ff_cbs_sei_find_type(ctx, metadata->sei_type); + else + return NULL; +} + +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + const void *data) +{ + const SEIMessageTypeDescriptor *desc; + AVBufferRef *payload_buf; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->fill) + return AVERROR(EINVAL); + + payload_buf = av_buffer_alloc(desc->size); + if (!payload_buf) + return AVERROR(ENOMEM); + + desc->fill(payload_buf->data, data); + + // All the metadata SEI messages must be unique in an access unit, + // so delete any existing ones of the given type. + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + + err = ff_cbs_sei_add_message(ctx, au, desc->prefix, desc->type, + payload_buf->data, payload_buf); + av_buffer_unref(&payload_buf); + return err; +} + +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type) +{ + const SEIMessageTypeDescriptor *desc; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc) + return AVERROR(EINVAL); + + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + return 0; +} + +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + void *data) +{ + const SEIMessageTypeDescriptor *desc; + SEIRawMessage *message; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->extract) + return AVERROR(EINVAL); + + message = NULL; + err = ff_cbs_sei_find_message(ctx, au, desc->type, &message); + if (err < 0) { + // No message of the given type found. + return err; + } + + desc->extract(data, message->payload); + return 0; +} diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index 5ce4ad3ccd..16fd29265e 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -226,6 +226,8 @@ typedef int (*SEIMessageWriteFunction)(CodedBitstreamContext *ctx, void *current, SEIMessageState *sei); +typedef void (*SEIConvertFunction)(void *dst, const void *src); + typedef struct SEIMessageTypeDescriptor { // Payload type for the message. (-1 in this field ends a list.) int type; @@ -239,6 +241,10 @@ typedef struct SEIMessageTypeDescriptor { SEIMessageReadFunction read; // Write bitstream from SEI message. SEIMessageWriteFunction write; + // Fill SEI structure from AV metadata. + SEIConvertFunction fill; + // Extract SEI metadata to AV structure. + SEIConvertFunction extract; } SEIMessageTypeDescriptor; // Macro for the read/write pair. The clumsy cast is needed because the @@ -248,6 +254,10 @@ typedef struct SEIMessageTypeDescriptor { .read = (SEIMessageReadFunction) cbs_ ## codec ## _read_ ## name, \ .write = (SEIMessageWriteFunction)cbs_ ## codec ## _write_ ## name +#define SEI_MESSAGE_FE(codec, name) \ + .fill = (SEIConvertFunction)cbs_ ## codec ## _fill_ ## name, \ + .extract = (SEIConvertFunction)cbs_ ## codec ## _extract_ ## name + // End-of-list sentinel element. #define SEI_MESSAGE_TYPE_END { .type = -1 } @@ -305,6 +315,8 @@ int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, uint32_t payload_type, SEIRawMessage **message); +enum CBSMetadataType; + /** * Delete all messages with the given payload type from an access unit. */ @@ -312,4 +324,27 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, CodedBitstreamFragment *au, uint32_t payload_type); +/** + * Insert metadata into an access unit. + */ +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + const void *data); + +/** + * Remove metadata from an access unit. + */ +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type); + +/** + * Extract metadata from an access unit. + */ +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + void *data); + #endif /* AVCODEC_CBS_SEI_H */ From patchwork Fri Jan 1 21:35:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24735 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4C2C244BCD1 for ; Fri, 1 Jan 2021 23:38:08 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2EE7B68A878; Fri, 1 Jan 2021 23:38:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 91D0468A86D for ; Fri, 1 Jan 2021 23:38:06 +0200 (EET) Received: by mail-wm1-f45.google.com with SMTP id a6so9864373wmc.2 for ; Fri, 01 Jan 2021 13:38:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fxhiN29K2YVnDleYCGDtqcLccKs9B41X/oWpT7AWuNo=; b=Xe7dNvIoTyatkJ+ykK4yvxIJXZ/1iOq1fBxHado4tfxPm2wE6FwJsJ/5cxcp6RWB7x CVxNzbvanrXGXdL6wrFLFH/ZUPMCYW3K6frkNKTLyzHzamCcZwuao3UHyJwhmuApVFBD apKnuGryc2hmgKCDDKTMc2Sk9LNGT9VWxDqsOD61R7biLG836C/rcO4yaNlXMIb2RCjh UDY8HItRPRF2JPwvg1gPtpy10Q4RCYQ6/8g/BsudFBXSwm9TjRsJoc60oUGZTE90MPTm 0mpRsaSRC8JnopwJ6jvpmuwKunBYeYGM/BR9ciL2LQjr/SS67FcSpu5s3oF3U0nnfmGz XHZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fxhiN29K2YVnDleYCGDtqcLccKs9B41X/oWpT7AWuNo=; b=E4hJO/pxThaegW2Ujr2yEvIupNi03zkPj/VT4xjn41oJWfoSD/QKeX8IwUIshGrq/O 6SU3YobS74/o8v8wd3xWL+8jQ9g4CeUFPTEiGOaJmIsJ1NMbtYBQ0424OdJPYifIviIg cDOmasyFeA1QbqbwuLn4qTs4ayEqiPAsTeodferlDfmsAn5IeuDwpbzm5TuwQbZGG7gJ BB0YfUoNmkwrad+LhMp2UrJeDdr/ePTcbFKwQMht+tQNoWtxpN2ZEz7cd6HoPo7XX/iI 7XqkU2pBHxFHJqMOiWim97iXVgE2VgwaFg1FheX+BDiT+Ebufz39agtX2RDAwo35rv4l ozDA== X-Gm-Message-State: AOAM531MdSNbGv/81lUpT764vU1ab06lIbumeA9vwHRs+We+F6fYFPlf kudnv70HsbngMbBkDVS66/wbXNysbgagow== X-Google-Smtp-Source: ABdhPJw9NKsq7PJH8VgjvtcKS4yHwVFAVLadi7sPLGJRdSO7ywX3RCAj+LM1OLwljrkNGcLRIvvgxg== X-Received: by 2002:a1c:e083:: with SMTP id x125mr17583501wmg.0.1609537085863; Fri, 01 Jan 2021 13:38:05 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:05 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:26 +0000 Message-Id: <20210101213537.169546-17-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 16/27] cbs_sei: Implement fill and extract for HDR SEI messages X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fill and extract both mastering display colour volume and content light level info messages. --- libavcodec/cbs_h2645.c | 130 +++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.c | 4 ++ 2 files changed, 134 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 42b614034e..53d5b50b42 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -18,6 +18,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" +#include "libavutil/mastering_display_metadata.h" #include "bytestream.h" #include "cbs.h" @@ -1504,6 +1505,133 @@ const CodedBitstreamType ff_cbs_type_h265 = { .extract_metadata = &ff_cbs_sei_extract_metadata, }; +static uint32_t rescale_clip(AVRational value, uint32_t scale, + uint32_t min, uint32_t max) +{ + int64_t scaled = av_rescale(scale, value.num, value.den); + return av_clip64(scaled, min, max); +} + +static void cbs_sei_fill_mastering_display_colour_volume + (SEIRawMasteringDisplayColourVolume *mdcv, + const AVMasteringDisplayMetadata *mdm) +{ + memset(mdcv, 0, sizeof(*mdcv)); + + if (mdm->has_primaries) { + // The values in the metadata structure are fractions between 0 and 1, + // while the SEI message contains fixed-point values with an increment + // of 0.00002. So, scale up by 50000 to convert between them and clip + // to the allowed range ([5, 37000] for x, [5, 42000] for y). + + for (int a = 0; a < 3; a++) { + // The metadata structure stores this in RGB order, but the SEI + // wants it in GBR order. + static const uint8_t mapping[] = { 1, 2, 0 }; + int b = mapping[a]; + mdcv->display_primaries_x[a] = + rescale_clip(mdm->display_primaries[b][0], 50000, 5, 37000); + mdcv->display_primaries_y[a] = + rescale_clip(mdm->display_primaries[b][1], 50000, 5, 42000); + } + + mdcv->white_point_x = + rescale_clip(mdm->white_point[0], 50000, 5, 37000); + mdcv->white_point_y = + rescale_clip(mdm->white_point[1], 50000, 5, 42000); + } + + if (mdm->has_luminance) { + // Metadata are rational values in candelas per square metre, SEI + // contains fixed point in units of 0.0001 candelas per square + // metre. So scale up by 10000 to convert between them, and clip to + // the allowed ranges. + + mdcv->max_display_mastering_luminance = + rescale_clip(mdm->max_luminance, 10000, 50000, 100000000); + mdcv->min_display_mastering_luminance = + rescale_clip(mdm->min_luminance, 10000, 1, 50000); + + // The spec requires that they are not equal when in the normal + // range. + if (mdcv->min_display_mastering_luminance >= + mdcv->max_display_mastering_luminance) { + mdcv->min_display_mastering_luminance = + mdcv->max_display_mastering_luminance - 1; + } + } else { + mdcv->max_display_mastering_luminance = 0; + mdcv->min_display_mastering_luminance = 0; + } +} + +static void cbs_sei_extract_mastering_display_colour_volume + (AVMasteringDisplayMetadata *mdm, + const SEIRawMasteringDisplayColourVolume *mdcv) +{ +#define IN_RANGE(v, min, max) ((v) >= (min) && (v) <= (max)) +#define IS_VALID_COORD(x, y) (IN_RANGE(x, 5, 37000) && IN_RANGE(y, 5, 42000)) + int valid_chromaticity = 1; + for (int a = 0; a < 3; a++) { + if (!IS_VALID_COORD(mdcv->display_primaries_x[a], + mdcv->display_primaries_y[a])) + valid_chromaticity = 0; + } + if (!IS_VALID_COORD(mdcv->white_point_x, mdcv->white_point_y)) + valid_chromaticity = 0; + + memset(mdm, 0, sizeof(*mdm)); + + if (valid_chromaticity) { + for (int a = 0; a < 3; a++) { + // SEI message in GBR order, but metadata structure in RGB order. + static const uint8_t mapping[] = { 2, 0, 1 }; + int b = mapping[a]; + + mdm->display_primaries[a][0] = + av_make_q(mdcv->display_primaries_x[b], 50000); + mdm->display_primaries[a][1] = + av_make_q(mdcv->display_primaries_y[b], 50000); + } + + mdm->white_point[0] = av_make_q(mdcv->white_point_x, 50000); + mdm->white_point[1] = av_make_q(mdcv->white_point_y, 50000); + + mdm->has_primaries = 1; + } + + if (IN_RANGE(mdcv->min_display_mastering_luminance, 1, 50000) && + IN_RANGE(mdcv->max_display_mastering_luminance, 50000, 100000000)) { + mdm->min_luminance = av_make_q(mdcv->min_display_mastering_luminance, 10000); + mdm->max_luminance = av_make_q(mdcv->max_display_mastering_luminance, 10000); + + mdm->has_luminance = 1; + } +#undef IN_RANGE +#undef IS_VALID_COORD +} + +static void cbs_sei_fill_content_light_level_info + (SEIRawContentLightLevelInfo *cll, const AVContentLightMetadata *clm) +{ + memset(cll, 0, sizeof(*cll)); + + // Both the metadata and the SEI are in units of candelas per square + // metre, so we only need to clip to ensure that they are in the valid + // range. + + cll->max_content_light_level = av_clip_uintp2(clm->MaxCLL, 16); + cll->max_pic_average_light_level = av_clip_uintp2(clm->MaxFALL, 16); +} + + +static void cbs_sei_extract_content_light_level_info + (AVContentLightMetadata *clm, const SEIRawContentLightLevelInfo *cll) +{ + clm->MaxCLL = cll->max_content_light_level; + clm->MaxFALL = cll->max_pic_average_light_level; +} + static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { { SEI_TYPE_FILLER_PAYLOAD, @@ -1528,12 +1656,14 @@ static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { 1, 0, sizeof(SEIRawMasteringDisplayColourVolume), SEI_MESSAGE_RW(sei, mastering_display_colour_volume), + SEI_MESSAGE_FE(sei, mastering_display_colour_volume), }, { SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, 1, 0, sizeof(SEIRawContentLightLevelInfo), SEI_MESSAGE_RW(sei, content_light_level_info), + SEI_MESSAGE_FE(sei, content_light_level_info), }, { SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS, diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index e5f9e3e403..0c05b2bdc9 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -375,6 +375,10 @@ typedef struct SEIMetadata { } SEIMetadata; static const SEIMetadata cbs_sei_metadata[] = { + { CBS_METADATA_MASTERING_DISPLAY, + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME }, + { CBS_METADATA_CONTENT_LIGHT_LEVEL, + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO }, }; static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata From patchwork Fri Jan 1 21:35:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24736 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5DF9E44BCD1 for ; Fri, 1 Jan 2021 23:38:09 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 44FC668A8F3; Fri, 1 Jan 2021 23:38:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8F13668A84E for ; Fri, 1 Jan 2021 23:38:07 +0200 (EET) Received: by mail-wm1-f46.google.com with SMTP id 190so9871821wmz.0 for ; Fri, 01 Jan 2021 13:38:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=OlCfh+ttkuxvG7JkKZ/vpFyeXLMz1OYzlbA8aC1TvL8=; b=bDoY4VUWSeXatG2qRLtOIy9b1BsSLAfflO84tG0Uf5DbR6ZrxlS1aZVfNMCIz/XzBy +muBQOPC938BEJfm3rP+iDVwheEurpsCYbaXL9QSWWulKbKiarn60IHHC/O6GvQQsMc9 8sWZ4gnPL0/vGKULXmmnp67r7RbvmVEntp23m4ZGKH7LxXXuugWHNMquSxtQow39Foeu Y2KAN53FGZsQRIvaeaJAThcgku02WLT/1/bD0rmo82l9F/I+kZNTQX/fWjaMRpKKPNyW aMzbpHmAwZcy9uo9zMJHKZ8eqHEFGHnCAfCrRRNmYp3e26Lscp2dV/McCd6M4f2wDeen GeBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OlCfh+ttkuxvG7JkKZ/vpFyeXLMz1OYzlbA8aC1TvL8=; b=O3Jsx+5qNaw6m4Pr+vuIRYFKJz0OYX/Vmrwpct5j9MFCTaoDiQBvUyWxRixGEG4rgu ZZL7AkV95oxrj7Uc+ik3W81I81n1V5A8LSA2naP4n87ieU+exXsQ5+oI6v9iVNEqHR51 4iRfIDbFggyiI5o6G6Z7n83XYWVikxuP64zyrgzoC2xcD68E2fpHgtlguZPwVK1Ic5sN c6RT6lXCFxCCyhIaEda6ZssI8GFq1qc4k5tRtA2cXTOd82muqafvGWNT8UkobMaSAgFi Tn4DyD2sx3qaLO4ZLTa1Igyd7Vur271HuNXDfoNCL891JNDiOZiJmdXho2LwGylHShjd 6SwQ== X-Gm-Message-State: AOAM533+g8UBGv1k+bA9tv/HqM5lBGwgI65keX/ZoCS9Fz0/2kAYsU13 gFCwGJ3nRck51/yyK6qrJTZcU3Kfn0Gn1Q== X-Google-Smtp-Source: ABdhPJwUeqKxGaIjcp25ceOcJpFCXkxeCTSoc91bITqk4gOhKd16Y5OnhjyM5iWzNS7VWQXbt9s+0A== X-Received: by 2002:a05:600c:2158:: with SMTP id v24mr16685393wml.129.1609537086683; Fri, 01 Jan 2021 13:38:06 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:06 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:27 +0000 Message-Id: <20210101213537.169546-18-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 17/27] cbs_h264: Implement fill and extract for display orientation SEI message X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 64 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.c | 2 ++ 2 files changed, 66 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 53d5b50b42..4144948b7f 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -18,6 +18,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" +#include "libavutil/display.h" #include "libavutil/mastering_display_metadata.h" #include "bytestream.h" @@ -1674,6 +1675,68 @@ static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { SEI_MESSAGE_TYPE_END, }; +static void cbs_h264_fill_sei_display_orientation + (H264RawSEIDisplayOrientation *disp, const int32_t matrix[9]) +{ + double dmatrix[9]; + int hflip, vflip, i; + double scale_x, scale_y, angle; + + if (matrix[2] != 0 || matrix[5] != 0 || + matrix[6] != 0 || matrix[7] != 0) { + // Not representable. + return; + } + + for (i = 0; i < 9; i++) + dmatrix[i] = matrix[i] / 65536.0; + + // Extract scale factors. + scale_x = hypot(dmatrix[0], dmatrix[3]); + scale_y = hypot(dmatrix[1], dmatrix[4]); + + // Select flips to make the main diagonal positive. + hflip = dmatrix[0] < 0.0; + vflip = dmatrix[4] < 0.0; + if (hflip) + scale_x = -scale_x; + if (vflip) + scale_y = -scale_y; + + // Rescale. + for (i = 0; i < 9; i += 3) { + dmatrix[i] /= scale_x; + dmatrix[i + 1] /= scale_y; + } + + // Extract rotation. + angle = atan2(dmatrix[3], dmatrix[0]); + + if (!(angle >= -M_PI && angle <= M_PI)) { + // Not representable. + return; + } + + *disp = (H264RawSEIDisplayOrientation) { + .hor_flip = hflip, + .ver_flip = vflip, + .anticlockwise_rotation = + (uint16_t)rint((angle >= 0.0 ? angle + : angle + 2 * M_PI) * + 32768.0 / M_PI), + .display_orientation_repetition_period = 1, + }; +} + +static void cbs_h264_extract_sei_display_orientation + (int32_t matrix[9], const H264RawSEIDisplayOrientation *disp) +{ + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 180.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); +} + static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { { SEI_TYPE_BUFFERING_PERIOD, @@ -1704,6 +1767,7 @@ static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { 1, 0, sizeof(H264RawSEIDisplayOrientation), SEI_MESSAGE_RW(h264, sei_display_orientation), + SEI_MESSAGE_FE(h264, sei_display_orientation), }, SEI_MESSAGE_TYPE_END }; diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 0c05b2bdc9..9ea00dd724 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -379,6 +379,8 @@ static const SEIMetadata cbs_sei_metadata[] = { SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME }, { CBS_METADATA_CONTENT_LIGHT_LEVEL, SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO }, + { CBS_METADATA_DISPLAY_MATRIX, + SEI_TYPE_DISPLAY_ORIENTATION }, }; static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata From patchwork Fri Jan 1 21:35:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24737 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5C66F44BCD1 for ; Fri, 1 Jan 2021 23:38:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 376EB68A912; Fri, 1 Jan 2021 23:38:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6870668A800 for ; Fri, 1 Jan 2021 23:38:08 +0200 (EET) Received: by mail-wm1-f51.google.com with SMTP id g185so9976287wmf.3 for ; Fri, 01 Jan 2021 13:38:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=pe7tE7vbgDBnxOZYLm0kiQ7jT/U502E5otzyTM6ueBE=; b=XNH91VjlItKO5ATFf/4UJkofv4UrEt0eCvkmk7xrjBOElymuEzYcevzXRRGiVRR1so Y1tSEizG1PmBOJRsUX+Q2kolSDOcz4ecgqms3iC2WvBf2/u/Gw6k8clrm5M5ri4Tndae 4HX50EsF7KLH/c9Bxy82pGruRFS8LDo389nOESeyTVWNbGqXYk+S4jvo/PAWr53Z11Kl MpKeV2X5+TeXP5PT+oVj7rvWbB7ST2L08GWoYsFbN1mxI+UU/FM07R4olsSb8xQTL0OW qiiNUvjNjGJqSdbDLrM0j5X3YEfpzXo7F3fxodCeDXkG64flh9R4bulymFUmB4Rb65z8 Pzfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pe7tE7vbgDBnxOZYLm0kiQ7jT/U502E5otzyTM6ueBE=; b=I1ebrh9nx694XqszG8egiiGYiRcR2hugMy71ThEsIwSXXDovBS5GwV0jAuIMzU9p0x JzH+Z30Gfr58LVwo5EKwCVS1mrwTVI+n3WpLbvwLXPBhui++BH1sRtXzeiyZ7x+RxCqW jTudIHD821mSmODVyfnKmsvoebKxv8BYPdH0HP1Xiy7Yp0B+R7wO1SHOSH+HwhTBe5zV dItnStLms5W8HjNEq6TAbgJiHJmbjEG4zS0S23dCQ43qlqsGktQz1PmV4aazspKpBdNQ QE4b1G9ftKoCFlq3FavOgQR0/cJY2UsCxj9bNwQfRaDfHJLcSOH9bGDbx+hgaM/MU1Aw 5OqA== X-Gm-Message-State: AOAM533g/3exgYilNxfZned5DkJpsKBg7xey35UtHsx/4/N9kJ+E992w e7G+iJAr/LZgRXProOmGyE+t9OtyBtCpIg== X-Google-Smtp-Source: ABdhPJz7wto1hXIx3Cno70+qlxxlQ+iYbouj5Qug/hIxNi0glb0PkPnKDVrU5nqcLhiZJUjMlocphA== X-Received: by 2002:a7b:cb93:: with SMTP id m19mr16903000wmi.128.1609537087555; Fri, 01 Jan 2021 13:38:07 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:06 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:28 +0000 Message-Id: <20210101213537.169546-19-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 18/27] cbs_av1: Implement metadata manipulation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_av1.c | 138 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index a40c91adff..3bca121fba 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -21,6 +21,7 @@ #include "cbs.h" #include "cbs_internal.h" +#include "cbs_metadata.h" #include "cbs_av1.h" #include "internal.h" @@ -1254,6 +1255,139 @@ static void cbs_av1_close(CodedBitstreamContext *ctx) av_buffer_unref(&priv->frame_header_ref); } +static int cbs_av1_find_metadata_unit(CodedBitstreamFragment *tu, + uint64_t metadata_type, + CodedBitstreamUnit **iter) +{ + int i, found; + + found = 0; + for (i = 0; i < tu->nb_units; i++) { + CodedBitstreamUnit *unit = &tu->units[i]; + AV1RawOBU *obu = unit->content; + + if (obu->header.obu_type == AV1_OBU_METADATA) { + if (!*iter || found) { + *iter = unit; + return 0; + } + if (unit == *iter) + found = 1; + } + } + + return AVERROR(ENOENT); +} + +static uint64_t cbs_av1_find_metadata_type(enum CBSMetadataType type) +{ + struct { + enum CBSMetadataType cbs_type; + uint64_t av1_type; + } map[] = { + }; + + for (int i = 0; i < FF_ARRAY_ELEMS(map); i++) { + if (map[i].cbs_type == type) + return map[i].av1_type; + } + return 0; +} + +static void cbs_av1_delete_metadata_type(CodedBitstreamFragment *tu, + uint64_t metadata_type) +{ + int err; + while (1) { + CodedBitstreamUnit *unit = NULL; + err = cbs_av1_find_metadata_unit(tu, metadata_type, &unit); + if (err < 0) + break; + ff_cbs_delete_unit(tu, unit - tu->units); + } +} + +static int cbs_av1_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *tu, + enum CBSMetadataType type) +{ + uint64_t metadata_type; + + metadata_type = cbs_av1_find_metadata_type(type); + if (metadata_type == 0) + return AVERROR(EINVAL); + + cbs_av1_delete_metadata_type(tu, metadata_type); + return 0; +} + +static int cbs_av1_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *tu, + enum CBSMetadataType type, + const void *data) +{ + AVBufferRef *md_buf; + AV1RawOBU *obu; + uint64_t metadata_type; + int err; + + metadata_type = cbs_av1_find_metadata_type(type); + if (metadata_type == 0) + return AVERROR(EINVAL); + + md_buf = av_buffer_allocz(sizeof(AV1RawOBU)); + if (!md_buf) + return AVERROR(ENOMEM); + + obu = (AV1RawOBU*)md_buf->data; + + obu->header = (AV1RawOBUHeader) { + .obu_type = AV1_OBU_METADATA, + }; + obu->obu.metadata.metadata_type = metadata_type; + + switch (metadata_type) { + default: + av_assert0(0); + } + + cbs_av1_delete_metadata_type(tu, metadata_type); + + err = ff_cbs_insert_unit_content(tu, -1, AV1_OBU_METADATA, + obu, md_buf); + av_buffer_unref(&md_buf); + return err; +} + +static int cbs_av1_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *tu, + enum CBSMetadataType type, + void *data) +{ + CodedBitstreamUnit *unit; + AV1RawOBU *obu; + uint64_t metadata_type; + int err; + + metadata_type = cbs_av1_find_metadata_type(type); + if (metadata_type == 0) + return AVERROR(EINVAL); + + unit = NULL; + err = cbs_av1_find_metadata_unit(tu, metadata_type, &unit); + if (err < 0) { + // No metadata with that type found. + return err; + } + + switch (metadata_type) { + default: + av_assert0(0); + } + + return 0; +} + static void cbs_av1_free_metadata(void *unit, uint8_t *content) { AV1RawOBU *obu = (AV1RawOBU*)content; @@ -1305,4 +1439,8 @@ const CodedBitstreamType ff_cbs_type_av1 = { .flush = &cbs_av1_flush, .close = &cbs_av1_close, + + .insert_metadata = &cbs_av1_insert_metadata, + .remove_metadata = &cbs_av1_remove_metadata, + .extract_metadata = &cbs_av1_extract_metadata, }; From patchwork Fri Jan 1 21:35:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24738 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 57FB944BCD1 for ; Fri, 1 Jan 2021 23:38:12 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 434D368A8FD; Fri, 1 Jan 2021 23:38:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1828068A83C for ; Fri, 1 Jan 2021 23:38:09 +0200 (EET) Received: by mail-wm1-f46.google.com with SMTP id 190so9871956wmz.0 for ; Fri, 01 Jan 2021 13:38:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=gJsul0sEGk67GuRR8Ko9qTTaIEbzG3pDyzrRhKy1khU=; b=J2Q3KXVCKZmdGwxyExh8prisXwM5uOU5EnZCJdbo9Elq+LEtFdMyQsFapIFun28J7S FLXhGARmWAJNOocE7LdWX0e2vRtALV7EvldGIjsQjf9I1tnajTXADLC4rxbTc/N+2+9i 9peJw9nZcQWmp3e/vO1FcYqpBxjFZn3oMuBTKzLzJKmttisPBLA2RMDYzO2yRW9ELd/0 vpbL68xn0ObXQSLUPRPLVY5TQ62iWAVwkUf/56lAI2cBGBhEE5WJGU9d9T6go5+CLw0J 3UIfO7uoMK0YHwpq1uqUv2EdJix7bApI8PA+GHS0RzHSoY+91NZjbr41mIhl0ClHoeWH Vt6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gJsul0sEGk67GuRR8Ko9qTTaIEbzG3pDyzrRhKy1khU=; b=nqJoLHdHYdtSnWNW8lqN1kvxIDBqNhYuWXkdCxSzHI0eqOFclvV88Our/BubTiE3z0 4zR7L/Y7SE73lJK+fjrCces96yQ4+Bl2c8G94DzmsWCs33zYk4Yn9CgqgpYA39wO06QT Lv8a12S0iHQD2zrJCY/31d42br3kKebH2gSMEfICl0KkLuR2dW3ienKZsIVnIrx32VWX POhedBGso9SZ6n/4FKfzCpECADkrhqOqEGBUh4krakKEBq4bFGRI62oJhv/B6FoFDnLs eCKxkqfIQhx54nL8vhsCYWIer+J5MBRzFmQL5lekl6S/Yq2WhEz75Jn8KWrLb68xxREM IOwA== X-Gm-Message-State: AOAM5337OCPJuUtghI0wp1b+4DqvZ5/yAjCQVk4AAh/DWekUAoa9rVvc vc5LAasCiGcCsycXpgClAqzA2zOAjj9/bw== X-Google-Smtp-Source: ABdhPJwFSkgRgSLbpr5zCHr8lt28lNkWUoHCLdHxutiuSww0r7R2mT8rQgWaPzBrWNzFwJCm2sqL8g== X-Received: by 2002:a7b:c19a:: with SMTP id y26mr16512067wmi.20.1609537088348; Fri, 01 Jan 2021 13:38:08 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:07 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:29 +0000 Message-Id: <20210101213537.169546-20-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 19/27] cbs_av1: Support manipulating HDR CLL and MDCV metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_av1.c | 95 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 3bca121fba..2b5fad69e9 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -17,6 +17,7 @@ */ #include "libavutil/avassert.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/pixfmt.h" #include "cbs.h" @@ -1285,6 +1286,10 @@ static uint64_t cbs_av1_find_metadata_type(enum CBSMetadataType type) enum CBSMetadataType cbs_type; uint64_t av1_type; } map[] = { + { CBS_METADATA_CONTENT_LIGHT_LEVEL, + AV1_METADATA_TYPE_HDR_CLL }, + { CBS_METADATA_MASTERING_DISPLAY, + AV1_METADATA_TYPE_HDR_MDCV }, }; for (int i = 0; i < FF_ARRAY_ELEMS(map); i++) { @@ -1321,6 +1326,13 @@ static int cbs_av1_remove_metadata(CodedBitstreamContext *ctx, return 0; } +static uint32_t rescale_clip(AVRational value, uint32_t scale, + uint32_t bits) +{ + int64_t scaled = av_rescale(scale, value.num, value.den); + return av_clip64(scaled, 0, INT64_C(1) << bits); +} + static int cbs_av1_insert_metadata(CodedBitstreamContext *ctx, CodedBitstreamFragment *tu, enum CBSMetadataType type, @@ -1347,6 +1359,51 @@ static int cbs_av1_insert_metadata(CodedBitstreamContext *ctx, obu->obu.metadata.metadata_type = metadata_type; switch (metadata_type) { + case AV1_METADATA_TYPE_HDR_CLL: + { + const AVContentLightMetadata *clm = data; + AV1RawMetadataHDRCLL *cll = + &obu->obu.metadata.metadata.hdr_cll; + + cll->max_cll = av_clip_uintp2(clm->MaxCLL, 16); + cll->max_fall = av_clip_uintp2(clm->MaxFALL, 16); + } + break; + case AV1_METADATA_TYPE_HDR_MDCV: + { + const AVMasteringDisplayMetadata *mdm = data; + AV1RawMetadataHDRMDCV *mdcv = + &obu->obu.metadata.metadata.hdr_mdcv; + + if (mdm->has_primaries) { + for (int i = 0; i < 3; i++) { + mdcv->primary_chromaticity_x[i] = + rescale_clip(mdm->display_primaries[i][0], + 1 << 16, 16); + mdcv->primary_chromaticity_y[i] = + rescale_clip(mdm->display_primaries[i][1], + 1 << 16, 16); + } + + mdcv->white_point_chromaticity_x = + rescale_clip(mdm->white_point[0], 1 << 16, 16); + mdcv->white_point_chromaticity_y = + rescale_clip(mdm->white_point[1], 1 << 16, 16); + } + + if (mdm->has_luminance) { + uint32_t value; + value = rescale_clip(mdm->max_luminance, 1 << 8, 32); + mdcv->luminance_max = value; + av_log(ctx->log_ctx, AV_LOG_ERROR, "mdcv->luminance_max = %d, %d\n", + mdcv->luminance_max, value); + mdcv->luminance_min = + rescale_clip(mdm->min_luminance, 1 << 14, 32); + av_log(ctx->log_ctx, AV_LOG_ERROR, "mdcv->luminance_min = %d\n", + mdcv->luminance_min); + } + } + break; default: av_assert0(0); } @@ -1381,6 +1438,44 @@ static int cbs_av1_extract_metadata(CodedBitstreamContext *ctx, } switch (metadata_type) { + case AV1_METADATA_TYPE_HDR_CLL: + { + AVContentLightMetadata *clm = data; + const AV1RawMetadataHDRCLL *cll = + &obu->obu.metadata.metadata.hdr_cll; + + clm->MaxCLL = cll->max_cll; + clm->MaxFALL = cll->max_fall; + } + break; + case AV1_METADATA_TYPE_HDR_MDCV: + { + AVMasteringDisplayMetadata *mdm = data; + const AV1RawMetadataHDRMDCV *mdcv = + &obu->obu.metadata.metadata.hdr_mdcv; + + for (int i = 0; i < 3; i++) { + mdm->display_primaries[i][0] = + av_make_q(mdcv->primary_chromaticity_x[i], 1 << 16); + mdm->display_primaries[i][1] = + av_make_q(mdcv->primary_chromaticity_y[i], 1 << 16); + } + + mdm->white_point[0] = + av_make_q(mdcv->white_point_chromaticity_x, 1 << 16); + mdm->white_point[1] = + av_make_q(mdcv->white_point_chromaticity_y, 1 << 16); + + mdm->max_luminance = + av_make_q(mdcv->luminance_max, 1 << 8); + mdm->min_luminance = + av_make_q(mdcv->luminance_min, 1 << 14); + + // The standard doesn't allow partially-filled MDCV blocks, so + // if the metadata appears it must be complete. + mdm->has_primaries = mdm->has_luminance = 1; + } + break; default: av_assert0(0); } From patchwork Fri Jan 1 21:35:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24739 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6FE3B44BCD1 for ; Fri, 1 Jan 2021 23:38:13 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 513F168A956; Fri, 1 Jan 2021 23:38:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E0A8F68A8A6 for ; Fri, 1 Jan 2021 23:38:09 +0200 (EET) Received: by mail-wm1-f53.google.com with SMTP id c124so9848010wma.5 for ; Fri, 01 Jan 2021 13:38:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HTTjs6eatYu+1ErDkw+foWwnVx/AXP8L3NH63iKv0DQ=; b=FnsZRgppq3hcTDOXgP7vGiZekBz+r3LraAXtsvYE+RxWZESsJkhcSsLaXVlkoQ2V/u QGTrhNjRFk1R+xy+4KVbapngJ01KpI35agkvq6xaLTzf8ACbT/gOo7r9qlcX2JwHRdG7 Kjvh0XzysxWuK2xHlQ5N3vRR5NRuY1WUcTkw4clLYOWDqkbovVlwJwwX4g6q26Ags54D 5+k+N9DmodyM0cFNvn79QSjV18n+R44824GX1pMXDCGRMRr+3EX6KkQNCanKL0toroz7 5hzIRrxL0S/Sz0WsmG8qiC8KT5WX/MS4FlCATfIflukpDKkymKMZvZM/KdtvbGPHZEdc MTYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HTTjs6eatYu+1ErDkw+foWwnVx/AXP8L3NH63iKv0DQ=; b=oTtOiaE959rtjuzXjW61VqrFNzdoLImAs4mobLKZr88IY5rthgQGKTiMVVpii0wSzR t0Xfbzq8PpieAVfRuvo+oWns9SFZBEEKF4WqZDFW7PWzQbPtUMD0xIoQvU51tJ6siO8/ qqJdOT75MC1yiWv3T4TriLNrVD1cWmGFelrwCDvLuKILz5RfCnm+ver9Lccvkv6HTcii AUV1p0R0HfMFbh4JN3dhiOCa5FHlg3iODMMglAEazCSb3cFkjkS88Pk7EVIrsTDxJzce Aw9lS+Gez3kBDxp9XKGCgQLNyKOdXj2qapIdcLaTY95CnKJAM3ZViXdBqVK0Pgriq4vc ONCw== X-Gm-Message-State: AOAM532xNXjo9/nlV//hcLBvPaVbBRi57NKkgqspy+wZXe6TD2AFgaea 0/lcGtyr7EEAVq3pUP1wiTMD98Ukf3E/4A== X-Google-Smtp-Source: ABdhPJwaOyTh48dXfi2XgNfwpXvuUYYlxDWTlRq1ocXJS+WQqVeqAMbKK5kkJMXrGWx0kEwXE9n2dQ== X-Received: by 2002:a7b:c7d3:: with SMTP id z19mr17298291wmk.31.1609537089197; Fri, 01 Jan 2021 13:38:09 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:08 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:30 +0000 Message-Id: <20210101213537.169546-21-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 20/27] cbs: Add interface to allow BSFs easy access to metadata manipulation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_bsf.c | 73 ++++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_bsf.h | 6 ++++ 2 files changed, 79 insertions(+) diff --git a/libavcodec/cbs_bsf.c b/libavcodec/cbs_bsf.c index 429f360014..fe6304a0d4 100644 --- a/libavcodec/cbs_bsf.c +++ b/libavcodec/cbs_bsf.c @@ -18,6 +18,7 @@ #include "bsf_internal.h" #include "cbs_bsf.h" +#include "cbs_metadata.h" static int ff_cbs_bsf_update_side_data(AVBSFContext *bsf, AVPacket *pkt) { @@ -159,3 +160,75 @@ void ff_cbs_bsf_close(AVBSFContext *bsf) ff_cbs_close(&ctx->input); ff_cbs_close(&ctx->output); } + +int ff_cbs_bsf_apply_metadata(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, int action) +{ + CBSBSFContext *ctx = bsf->priv_data; + const CBSMetadataTypeDescriptor *desc; + uint8_t *side_data; + size_t size; + int err, ignored; + + desc = ff_cbs_metadata_find_type(type); + if (!desc) + return AVERROR(EINVAL); + if (!desc->has_packet_side_data) + return AVERROR(EINVAL); + + switch (action) { + case BSF_ELEMENT_PASS: + // Nothing to do. + return 0; + + case BSF_ELEMENT_REMOVE: + // Remove all existing instances of the metadata. + err = ff_cbs_remove_metadata(ctx->output, frag, type); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to remove %s metadata " + "from %s.\n", desc->name, ctx->type->fragment_name); + } + return err; + + case BSF_ELEMENT_INSERT: + // Insert matching side-data to replace existing metadata. + side_data = av_packet_get_side_data(pkt, + desc->packet_side_data_type, + &ignored); + if (!side_data) { + // No side data of this type. + return 0; + } + err = ff_cbs_insert_metadata(ctx->output, frag, type, side_data); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert %s metadata " + "into %s.\n", desc->name, ctx->type->fragment_name); + } + + case BSF_ELEMENT_EXTRACT: + // Extract the metadata and attach it to the packet as side-data. + side_data = ff_cbs_alloc_metadata(type, &size); + if (!side_data) + return AVERROR(ENOMEM); + err = ff_cbs_extract_metadata(ctx->output, frag, type, side_data); + if (err == AVERROR(ENOENT)) { + // No metadata of that type. + return 0; + } + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to extract %s metadata " + "from %s.\n", desc->name, ctx->type->fragment_name); + return err; + } + err = av_packet_add_side_data(pkt, desc->packet_side_data_type, + side_data, size); + if (err < 0) + av_free(side_data); + return err; + + default: + return AVERROR(EINVAL); + } +} diff --git a/libavcodec/cbs_bsf.h b/libavcodec/cbs_bsf.h index 8cab3f144c..95cae316b2 100644 --- a/libavcodec/cbs_bsf.h +++ b/libavcodec/cbs_bsf.h @@ -55,6 +55,12 @@ int ff_cbs_bsf_filter(AVBSFContext *bsf, AVPacket *pkt); int ff_cbs_bsf_init(AVBSFContext *bsf, const CBSBSFType *type); void ff_cbs_bsf_close(AVBSFContext *bsf); +enum CBSMetadataType; + +int ff_cbs_bsf_apply_metadata(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, int action); // Options for element manipulation. enum { From patchwork Fri Jan 1 21:35:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24740 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 88FE644BCD1 for ; Fri, 1 Jan 2021 23:38:14 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 703D868A9CE; Fri, 1 Jan 2021 23:38:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BE47668A8A6 for ; Fri, 1 Jan 2021 23:38:10 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id w5so22894602wrm.11 for ; Fri, 01 Jan 2021 13:38:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=na4uHB3dcRvELcgEPmQdCUb5FXZJkuITuP8jUbhq4vQ=; b=vom+SWRhO5sfcIjX83lKqr3O8Lg/LKq/0iB3EBLUmGDnNERT6SOaJ4+0ICWL5i1rcV iGETOHwj31j2fQmOlg2veVWc+cOlb+DG7Wm2b2CAxgH4Qb9Y1U1NBk/XWGHPw5VBvLiK DhdeRVpe2wEPJurmogag33Qmv/uxNIC/LpJbM/CIFZUZqoKNddRCwjF342Sdj/u+eiLQ RIKDmrVUsMJIxR7C/zzkZqQsx4UTInJ1mZPB80cMyq9ZIAxtio+xXobO8vSzSvhESYPP q/O6mMLLiJvCe/+ydWCLl7HLuWS4NOL2lr/2xQFe9Ci9M31G4y2mVE670axIib8jBils QtqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=na4uHB3dcRvELcgEPmQdCUb5FXZJkuITuP8jUbhq4vQ=; b=mdsk5885l4iENyXKaKE6Rw1D9uhp6kWjAtURlhORTLV/uHnuuCA59LxPPihW4Mivnk GG2fVw4Tnf5eJLvU0D35VtaDJRpKnI6G2gf7r3gs5mKc5ZhGDkQPaDoG93noZ3q37zQB 44PW4o14XrGWBLKnAwZA7/rexaQqLuh9Ftm7RgWPHlqnC70raprjJTjD66eYp7SNZO2u b1L8+JXI0g9dta/mhmGarsiK/h+vRdclfKC76l3bY45Tow/hoj5NAeDtO1IZ6WW7xw3d ZhBw28yNPQgm3p7mAVKnszfCx94PohlrUQHzTtZTJpEYKc4RJNIhoEu2HY5TUCPa5aSG m2+A== X-Gm-Message-State: AOAM5315iBbtA367ULMNqy9JsUrB61CU1QKhCHOyeOLPyO6xYTz+elSj M8XRxe9fv39QYnE4belRSsbq+tIXgPItYA== X-Google-Smtp-Source: ABdhPJzPJl8gx8A4yvWQEHppysP2jOCeFpaR4jrB/dMbVfd+QkpRH7pd11udyQ7mEMBt125Dx+4sMA== X-Received: by 2002:a5d:604a:: with SMTP id j10mr70779241wrt.290.1609537089991; Fri, 01 Jan 2021 13:38:09 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:09 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:31 +0000 Message-Id: <20210101213537.169546-22-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 21/27] h264_metadata_bsf: Simplify display orientation handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The case where the user supplies new value directly as options still requires a bit of special handling. --- libavcodec/h264_metadata_bsf.c | 118 ++++++--------------------------- 1 file changed, 21 insertions(+), 97 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index f5f439018d..aae91a6e23 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -25,6 +25,7 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h264.h" +#include "cbs_metadata.h" #include "h264.h" #include "h264_levels.h" #include "h264_sei.h" @@ -75,7 +76,6 @@ typedef struct H264MetadataContext { int display_orientation; double rotate; int flip; - H264RawSEIDisplayOrientation display_orientation_payload; int level; } H264MetadataContext; @@ -328,122 +328,46 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, int seek_point) { H264MetadataContext *ctx = bsf->priv_data; - SEIRawMessage *message; int err; - message = NULL; - while (ff_cbs_sei_find_message(ctx->common.output, au, - H264_SEI_TYPE_DISPLAY_ORIENTATION, - &message) == 0) { - H264RawSEIDisplayOrientation *disp = message->payload; - int32_t *matrix; - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) - return AVERROR(ENOMEM); - - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_free(matrix); - return AVERROR(ENOMEM); - } - } - - if (ctx->display_orientation == BSF_ELEMENT_REMOVE || - ctx->display_orientation == BSF_ELEMENT_INSERT) { - ff_cbs_sei_delete_message_type(ctx->common.output, au, - H264_SEI_TYPE_DISPLAY_ORIENTATION); - } - - if (ctx->display_orientation == BSF_ELEMENT_INSERT) { - H264RawSEIDisplayOrientation *disp = - &ctx->display_orientation_payload; + if (ctx->display_orientation != BSF_ELEMENT_INSERT) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_DISPLAY_MATRIX, + ctx->display_orientation); + if (err < 0) + return err; + } else { + int32_t matrix[9]; uint8_t *data; int size; int write = 0; + ff_cbs_sei_delete_message_type(ctx->common.output, au, + H264_SEI_TYPE_DISPLAY_ORIENTATION); + data = av_packet_get_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, &size); if (data && size >= 9 * sizeof(int32_t)) { - int32_t matrix[9]; - double dmatrix[9]; - int hflip, vflip, i; - double scale_x, scale_y, angle; - - memcpy(matrix, data, sizeof(matrix)); - - for (i = 0; i < 9; i++) - dmatrix[i] = matrix[i] / 65536.0; - - // Extract scale factors. - scale_x = hypot(dmatrix[0], dmatrix[3]); - scale_y = hypot(dmatrix[1], dmatrix[4]); - - // Select flips to make the main diagonal positive. - hflip = dmatrix[0] < 0.0; - vflip = dmatrix[4] < 0.0; - if (hflip) - scale_x = -scale_x; - if (vflip) - scale_y = -scale_y; - - // Rescale. - for (i = 0; i < 9; i += 3) { - dmatrix[i] /= scale_x; - dmatrix[i + 1] /= scale_y; - } - - // Extract rotation. - angle = atan2(dmatrix[3], dmatrix[0]); - - if (!(angle >= -M_PI && angle <= M_PI) || - matrix[2] != 0.0 || matrix[5] != 0.0 || - matrix[6] != 0.0 || matrix[7] != 0.0) { - av_log(bsf, AV_LOG_WARNING, "Input display matrix is not " - "representable in H.264 parameters.\n"); - } else { - disp->hor_flip = hflip; - disp->ver_flip = vflip; - disp->anticlockwise_rotation = - (uint16_t)rint((angle >= 0.0 ? angle - : angle + 2 * M_PI) * - 32768.0 / M_PI); - write = 1; - } + memcpy(matrix, data, 9 * sizeof(int32_t)); + write = 1; } if (seek_point) { if (!isnan(ctx->rotate)) { - disp->anticlockwise_rotation = - (uint16_t)rint((ctx->rotate >= 0.0 ? ctx->rotate - : ctx->rotate + 360.0) * - 65536.0 / 360.0); + av_display_rotation_set(matrix, ctx->rotate); write = 1; } if (ctx->flip) { - disp->hor_flip = !!(ctx->flip & FLIP_HORIZONTAL); - disp->ver_flip = !!(ctx->flip & FLIP_VERTICAL); + av_display_matrix_flip(matrix, + !!(ctx->flip & FLIP_HORIZONTAL), + !!(ctx->flip & FLIP_VERTICAL)); write = 1; } } if (write) { - disp->display_orientation_repetition_period = 1; - - err = ff_cbs_sei_add_message(ctx->common.output, au, 1, - H264_SEI_TYPE_DISPLAY_ORIENTATION, - disp, NULL); + err = ff_cbs_sei_insert_metadata(ctx->common.output, au, + CBS_METADATA_DISPLAY_MATRIX, + matrix); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); From patchwork Fri Jan 1 21:35:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24742 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A702E44BCD1 for ; Fri, 1 Jan 2021 23:38:16 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 85CC268A9AA; Fri, 1 Jan 2021 23:38:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 611F268A957 for ; Fri, 1 Jan 2021 23:38:11 +0200 (EET) Received: by mail-wr1-f54.google.com with SMTP id y17so22909161wrr.10 for ; Fri, 01 Jan 2021 13:38:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=agKenjm4YrSwcM6NxpkJ91495JnOXw4KCIb2XONUXi4=; b=OxI9LXRiR4uJn6Ov7jI8fOAPjQJr44Tlsizp2SpjGayM8FRPYZryQH5/GrExbJj3NJ WZ8KVwUxlGCL4Ht4cle5aixF5+p9njTf0HoUtXs4tN4Bh8KjVnJr4Z2RT55LsPsMUG1/ XQODhY/nPpu6bsc2L7nHA6p1ItIiFloR76lhLnsqx+/vpIhWjNwHv+8ilHnOGAtbXh4T 8ZgBpLTcyxW/gJxBUCzwettdLT6b6pu9o5G2U+mY7FhhNJ6odx9/9SvEAbvL55z+vdic i7is65zikkjuKGPN+AHAQK6cTzzHKckmYmgVKziH66r9eHdyz3d2TuwFUUVaMO8Tq26h Mdfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=agKenjm4YrSwcM6NxpkJ91495JnOXw4KCIb2XONUXi4=; b=ioRbEM1CfG93yyQUJLRceaDdJS71wNkixy2efeGGPtPb60s9HWVeZrJMSJbqZQx8oe vmij3DETx5NJyGiJS4sKNOZslBHJj2NnA/WFmFMr/8wUR3efx3uZ8bkGQ44W9M27MmmT f850uNueukCzYHSpSo/tZEBTeqO5rV1CMtBOLFTTsPp/VPcM9w642IG7F4liU4FD0nZn E4gRAPSfjosITA98oHddRFAORD2bk+UTXnNEzbcUkDnkbSX0Zra7mnVE5e11Nd9FmemD 8NGyJAKgH+GAaitOC201NvogglqGDQt7h25+5rB6lIYjbmjvjHJLCmsPlCbsBkh8SaV7 O+4g== X-Gm-Message-State: AOAM533qDU+j1funMAUngUJT6jziA4AfqPazAjJaddEdUacWND81OeJ/ xefOz+gRNAT5vGXbiARh8+DKrLA7ipDmog== X-Google-Smtp-Source: ABdhPJyk9cIT3oP5LS+zI4U5M8yqHG+KBoVshzivQVXLPxq2E6CBnwjYqQoxPjT5Oyb1+7b2y7Qvhg== X-Received: by 2002:adf:84c1:: with SMTP id 59mr72423677wrg.409.1609537090840; Fri, 01 Jan 2021 13:38:10 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:10 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:32 +0000 Message-Id: <20210101213537.169546-23-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 22/27] h264_metadata_bsf: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/bitstream_filters.texi | 17 +++++++++++++++++ libavcodec/h264_metadata_bsf.c | 25 +++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 8a2f55cc41..695825d33f 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -285,6 +285,23 @@ level_idc value (for example, @samp{42}), or the special name @samp{auto} indicating that the filter should attempt to guess the level from the input stream properties. +@item mastering_display +@item content_light_level +Manipulate mastering display colour volume (see H.264 section D.2.29) +and content light level info (see H.264 section D.2.31) SEI messages in +the stream. + +Possible actions: +@table @samp +@item insert +Insert this type of message, taking the values from packet side-data. +@item remove +Remove all instances of this message. +@item extract +Extract the content of this type of message, attaching it to the packets +as side-data. +@end table + @end table @section h264_mp4toannexb diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index aae91a6e23..bdaee78d1e 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -78,6 +78,9 @@ typedef struct H264MetadataContext { int flip; int level; + + int mastering_display; + int content_light_level; } H264MetadataContext; @@ -449,6 +452,20 @@ static int h264_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, return err; } + if (pkt) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_MASTERING_DISPLAY, + ctx->mastering_display); + if (err < 0) + return err; + + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + ctx->content_light_level); + if (err < 0) + return err; + } + if (pkt) ctx->done_first_au = 1; @@ -609,6 +626,14 @@ static const AVOption h264_metadata_options[] = { { LEVEL("6.2", 62) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("mastering_display", + "Mastering display colour volume SEI", + mastering_display, "mdcv"), + + BSF_ELEMENT_OPTIONS_PIRE("content_light_level", + "Content light level information SEI", + content_light_level, "cll"), + { NULL } }; From patchwork Fri Jan 1 21:35:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24741 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6E75244BCD1 for ; Fri, 1 Jan 2021 23:38:15 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 603AE68A9E8; Fri, 1 Jan 2021 23:38:15 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4EEB468A92E for ; Fri, 1 Jan 2021 23:38:12 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id t16so22930633wra.3 for ; Fri, 01 Jan 2021 13:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MZk9uqPRZ8wbEgqaLtbjNrzzymaSMNkB21mH5cJR6ys=; b=laAQwXF5pYE6GswNLGPx2Swq6xfj+NroZBvdbscMuOST64YLuzrpgcILLFfzq84+Za q9mDVCw8coASKx8shohLwFu05rgpAlROyiTP5eaa8phMXnMIin4x/n1pMDP1ZaqWT947 vqmjtv7sut9HJx6El7bbnJQGl/h/X2KVMCjT/5fcd0Cw9ycRicBKpKPrneH/t4qLZPo9 r9fiKu+olBtPOU9YWbFDSMPbAsRxda6XZ7XzbtCxXU75bKCy3EIOAe35K1ukTsfSdgh9 1inovh1/yLsoIAG/bBXLZLxncLBbZR+rLFuXYPaWpOvcXtnAaWMsJ5YJ8lDfKb+2PqCa s5yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MZk9uqPRZ8wbEgqaLtbjNrzzymaSMNkB21mH5cJR6ys=; b=tdMgwNEbzF4AHHhAYIu6jvOZu6tl7SHAPtLlcViJR0aWCQt5mKNT8lTMMuArR0qDJk 7n0JBFRxA3B8eVmLx0IKV/dcF+wy8iLAgzGnuxdGnh+AT8hY2ylEyzMAmXtZsDkxqr5O fzRy9MtKKka4DwZcJQY8tp0MKKUAZmVyD2XGrkJT/p+P60UKbkHsIc9RxSO9noJ6Yg5o XeUsI/mxnOQ1LdUHsEeDBo/AGdlUj7A/9ZUtT3xRSeVnkBYaTgz5w956Oy6As8j72VME Peezyxf1HNdqpSq4cUJg7r7HVuz6a58nfRlnBEcAjtw5jbGuEgdSt/CnDPv89xwKDm3I kUNw== X-Gm-Message-State: AOAM530bV5QTARbmsRU1RueWh7UL8fqZXA5NtE471SHAXAvuwYm4ccnq 6w4YaeplgTg85zSSsuYSKFrJjP+Vyo8Dog== X-Google-Smtp-Source: ABdhPJyZutiPxq+MrP14exnLi8Iqom5UPF57M0peDzhVuF+HDS2GWmvMPqQvjAfJQF8agUdSnyhZkA== X-Received: by 2002:adf:fa86:: with SMTP id h6mr69062265wrr.103.1609537091638; Fri, 01 Jan 2021 13:38:11 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:11 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:33 +0000 Message-Id: <20210101213537.169546-24-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 23/27] h265_metadata_bsf: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/bitstream_filters.texi | 17 +++++++++++++++++ libavcodec/h265_metadata_bsf.c | 26 ++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 695825d33f..4c9a42c24a 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -393,6 +393,23 @@ The argument must be the name of a level (for example, @samp{5.1}), a or the special name @samp{auto} indicating that the filter should attempt to guess the level from the input stream properties. +@item mastering_display +@item content_light_level +Manipulate mastering display colour volume (see H.265 section D.3.28) +and content light level info (see H.265 section D.3.35) SEI messages in +the stream. + +Possible actions: +@table @samp +@item insert +Insert this type of message, taking the values from packet side-data. +@item remove +Remove all instances of this message. +@item extract +Extract the content of this type of message, attaching it to the packets +as side-data. +@end table + @end table @section hevc_mp4toannexb diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index c3eadee92b..4c84d4f5a8 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -23,6 +23,7 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h265.h" +#include "cbs_metadata.h" #include "hevc.h" #include "h265_profile_level.h" @@ -60,6 +61,9 @@ typedef struct H265MetadataContext { int level; int level_guess; int level_warned; + + int mastering_display; + int content_light_level; } H265MetadataContext; @@ -393,6 +397,20 @@ static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, } } + if (pkt) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_MASTERING_DISPLAY, + ctx->mastering_display); + if (err < 0) + return err; + + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + ctx->content_light_level); + if (err < 0) + return err; + } + return 0; } @@ -484,6 +502,14 @@ static const AVOption h265_metadata_options[] = { { LEVEL("8.5", 255) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("mastering_display", + "Mastering display colour volume SEI", + mastering_display, "mdcv"), + + BSF_ELEMENT_OPTIONS_PIRE("content_light_level", + "Content light level information SEI", + content_light_level, "cll"), + { NULL } }; From patchwork Fri Jan 1 21:35:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24743 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A0F2944BCD1 for ; Fri, 1 Jan 2021 23:38:17 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8CB4D68AA00; Fri, 1 Jan 2021 23:38:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 541AD68A995 for ; Fri, 1 Jan 2021 23:38:13 +0200 (EET) Received: by mail-wr1-f41.google.com with SMTP id a12so22897660wrv.8 for ; Fri, 01 Jan 2021 13:38:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=e+A69b92CyJXk51Pidpe14TCeWBn9FZ9FSz92TV0wJY=; b=rhnqPDAkYzzbk1Lbij5Ps0bvL/ZA8kQmmOKiGSUDkMXwXXJH2eypLvvC/xcbdJMzQj UPYyey8aPtqhHiUfkN4f1xmjFMooc+jrAXASVS7s5jTu6j4w1am4yjHc1TjAN2O2x0Qi GODEWeUpeHtmZQdwAFUvLv/wuEYnE0chBZYWTwS5FvpedaWqImOAdKAQVVP/DC+1VULF 76wJ16jxDnGo7w36wmbteke6ZRPMrpoLUbBEALviWvzZipIMjE4aRiUkPUFizKraCd2T LdFwXdsQ1thneWg+xqKCnXQAqWwYinY1oH4DRa/2jO6wWDq0AvSCytoySxz8imS1WvDG oYWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e+A69b92CyJXk51Pidpe14TCeWBn9FZ9FSz92TV0wJY=; b=CAvWBky26fF/KD0qK0/cOwz9iqp5S/B1AFdYMzR/pZr1bAWiNDQDFd8Qlv+u+Y06hY zUePveG99xB8T61WwnsIjOtI3pMSWGBZH7YkawvqZwH99qef2LqCRxkwm25AyQJ9qFrY xdPz768FbtF1aMT4LTDX63mP1OEPSE4VDHNdpVZLZuQ1N93zt2SusMcvTLfIQTB7ejM+ 28yVcpVQ12bg0BnvHL//wCAdJLx+IPZ1Ku82BdHgM+uqECcHz0mnm8NmoLcY5O9Xmohg vfzC7G+d6gERXWDICqQO8TL7pmqsJmMrFqG7Sz4fX0stosnQkzQ0/TxMkXo2v4kHi/+i LcTg== X-Gm-Message-State: AOAM5322QSucgZk/N8HnTQs2T6KCziOLsp6jYA9p/EfZIfI1vlRF+CW+ ysYgLg7y8ZM922G4lPtFrC/mu24FXZH94Q== X-Google-Smtp-Source: ABdhPJw24hKbBuoejAfKBYVhZeVuv9Zxm1zLTWP8xQclcuWkXgoa3FWvjOR7Hs91pByI0WkwiFwpzQ== X-Received: by 2002:adf:dc8b:: with SMTP id r11mr71192270wrj.131.1609537092395; Fri, 01 Jan 2021 13:38:12 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:11 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:34 +0000 Message-Id: <20210101213537.169546-25-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 24/27] av1_metadata_bsf: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/bitstream_filters.texi | 17 +++++++++++++++++ libavcodec/av1_metadata_bsf.c | 26 ++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 4c9a42c24a..3fea06fca7 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -90,6 +90,23 @@ has a fixed framerate. Ignored if @option{tick_rate} is not also set. @item delete_padding Deletes Padding OBUs. +@item hdr_cll +@item hdr_mdcv +Manipulate content light level (see AV1 section 6.7.3) and mastering +display colour volume (see AV1 section 6.7.4) metadata messages in the +stream. + +Possible actions: +@table @samp +@item insert +Insert this type of message, taking the values from packet side-data. +@item remove +Remove all instances of this message. +@item extract +Extract the content of this type of message, attaching it to the packets +as side-data. +@end table + @end table @section chomp diff --git a/libavcodec/av1_metadata_bsf.c b/libavcodec/av1_metadata_bsf.c index 31fd4f0f41..8ac6e32b59 100644 --- a/libavcodec/av1_metadata_bsf.c +++ b/libavcodec/av1_metadata_bsf.c @@ -23,6 +23,7 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_av1.h" +#include "cbs_metadata.h" typedef struct AV1MetadataContext { CBSBSFContext common; @@ -40,6 +41,9 @@ typedef struct AV1MetadataContext { int num_ticks_per_picture; int delete_padding; + + int hdr_cll; + int hdr_mdcv; } AV1MetadataContext; @@ -143,6 +147,20 @@ static int av1_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, } } + if (pkt) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, frag, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + ctx->hdr_mdcv); + if (err < 0) + return err; + + err = ff_cbs_bsf_apply_metadata(bsf, pkt, frag, + CBS_METADATA_MASTERING_DISPLAY, + ctx->hdr_mdcv); + if (err < 0) + return err; + } + return 0; } @@ -203,6 +221,14 @@ static const AVOption av1_metadata_options[] = { OFFSET(delete_padding), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS}, + BSF_ELEMENT_OPTIONS_PIRE("hdr_cll", + "HDR content light level metadata", + hdr_cll, "cll"), + + BSF_ELEMENT_OPTIONS_PIRE("hdr_mdcv", + "HDR mastering display colour volume metadata", + hdr_mdcv, "mdcv"), + { NULL } }; From patchwork Fri Jan 1 21:35:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24744 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8F49544BCD1 for ; Fri, 1 Jan 2021 23:38:18 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7C46468A990; Fri, 1 Jan 2021 23:38:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1460068A990 for ; Fri, 1 Jan 2021 23:38:13 +0200 (EET) Received: by mail-wr1-f47.google.com with SMTP id a12so22897715wrv.8 for ; Fri, 01 Jan 2021 13:38:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hdqnWpJPD/CgEQNBmf9Tzdq96C2f+fcAIRAR8Et1odo=; b=e7mr3hRCdsGlOZhtkXDoOS2E54aztCp3yq9fYOqS7hX0hbGDlLKKqAQR0vqE2VgX3I drPopXuohN+3JcegoMQMQQ5R+ZWBL112Goy84VK03SARifSReGsS15bBFAez3dFbUtcO Wb509cr0Zt1x3KZJo3WCmfnl2M+NnLT6ho/dYCrGFM/KvqXR1KFjStBUxibYFH288XQB 09qhCPx8XLVosZwMNFiuL2oicmP7XdotMybyaxIXe0pLTTK6iSPorTl4F9uK+6xXLdXB CEMWNrBscIWE0rhMivnNPpllz0OcjnMb/HA6RO21Cxd1KQOVnqZXqyV23D/ZXQ/ayT/7 FT4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hdqnWpJPD/CgEQNBmf9Tzdq96C2f+fcAIRAR8Et1odo=; b=a1+TrNQWEoKUhHus9Y12lysSr9PTR0RWsM5F7BTZpdcM9GxjjfioGTQsAt+rXajuyM kv9WTqM9zCKTU1cctCLMdcWESzT46c10EWIkVDkWXipvkSZzPidh+m1ojecjM+qzU3l/ aHQE5ORIcdeoT6SUZfQVBi7lOFJjmumhGPSyDwVlP6NhbR7ssnTGVBgX1Y1K3pz1lewY LMl7BNLsgYFARkIQiyaefac75D0MhXOwFMQ/FHrQRinfZK5qIYEiQCaseKTuk/ivchEm NuuLe3jITwRYEA7cgshtHLjwQt4YCAyd2ef7PMmOvACXZQ2xcfD0WWiRPn4GmKZNIhhV vt6g== X-Gm-Message-State: AOAM531kNEbeMwOLmezawSReZlD4iJW5+1KRaJOqxPz5/f8vEPn8Bmud dJbiY9m8FSF0N4vPww7K7KsT1eS9jz8ytg== X-Google-Smtp-Source: ABdhPJwGbJl/DULUEk2pVU7jjZgitwoojaTObZbiSVdW4eo2jPf4kyMW5jgkue5/UkTIghEKb/CX4g== X-Received: by 2002:adf:c109:: with SMTP id r9mr58186728wre.261.1609537093300; Fri, 01 Jan 2021 13:38:13 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:12 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:35 +0000 Message-Id: <20210101213537.169546-26-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 25/27] vaapi_encode: Unify SEI option mask enum X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This was in two disjoint parts in the H.264 and H.265 files. Unify them in the header to avoid any confusion, because they are really the same enum. --- libavcodec/vaapi_encode.h | 8 ++++++++ libavcodec/vaapi_encode_h264.c | 6 ------ libavcodec/vaapi_encode_h265.c | 5 ----- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index b41604a883..980bbb1b7a 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -474,5 +474,13 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); VAAPI_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate"), \ VAAPI_ENCODE_RC_MODE(AVBR, "Average variable-bitrate") +// SEI type mask used in options for H.264 and H.265. +enum { + SEI_TIMING = 0x01, + SEI_IDENTIFIER = 0x02, + SEI_RECOVERY_POINT = 0x04, + SEI_MASTERING_DISPLAY = 0x08, + SEI_CONTENT_LIGHT_LEVEL = 0x10, +}; #endif /* AVCODEC_VAAPI_ENCODE_H */ diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index d24462414c..7c352e982b 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -35,12 +35,6 @@ #include "internal.h" #include "vaapi_encode.h" -enum { - SEI_TIMING = 0x01, - SEI_IDENTIFIER = 0x02, - SEI_RECOVERY_POINT = 0x04, -}; - // Random (version 4) ISO 11578 UUID. static const uint8_t vaapi_encode_h264_sei_identifier_uuid[16] = { 0x59, 0x94, 0x8b, 0x28, 0x11, 0xec, 0x45, 0xaf, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 2e8e772008..09d5a8b5cb 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -37,11 +37,6 @@ #include "put_bits.h" #include "vaapi_encode.h" -enum { - SEI_MASTERING_DISPLAY = 0x08, - SEI_CONTENT_LIGHT_LEVEL = 0x10, -}; - typedef struct VAAPIEncodeH265Picture { int pic_order_cnt; From patchwork Fri Jan 1 21:35:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24745 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EAACC44AE1C for ; Fri, 1 Jan 2021 23:40:22 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C1DC68AA35; Fri, 1 Jan 2021 23:38:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E698368A995 for ; Fri, 1 Jan 2021 23:38:14 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id c5so22915389wrp.6 for ; Fri, 01 Jan 2021 13:38:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8tMn8FLzLIfoLCdUD+tONTzl97sZtu7BjyIOxs4KRkM=; b=HJERVPLqubgdwzXFa+jKRD2DEc42ay3gtBWaJM4jCQMjC1hR+0SeiHmv6BZv8zfecs TbQ3i/zDDVD6wZLQ5CYn6ymoAO9KOOxmYy7gOZ32B4ffpLbbErS3nriQn5L8Ds/CitLI yQVm7YIunyBokErDpNCq5FoKhmvVAdzg9EbyOxd+bsrzWGDR07yfAEuy74qCFqzZLOU2 J5RhTtBnxUEzwl44fw7aN+Wm6EOswbIr9PqrxToPOPWYfvyaSCIlr2hXEItrKcTAgIVR YAR30xz6MiGncDFrYdEnS5M8fys5akbtpLc1EIviQ3gVEFvMveikbiJtFkPbt6uZ3Cka cz6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8tMn8FLzLIfoLCdUD+tONTzl97sZtu7BjyIOxs4KRkM=; b=WJeuOzZEaEUb20f/K3EmrYY2Ar5d6Yd+P1AfNo0VK7+bS5ByvcdJNfcvntTQo2+zEG 9wo5Xyxm13LTEBoTyWdvAF7NyhzPF/z1MhcxzSRz6WBwMobN8C5PhtTGdhMyBBH2y46q wcQyG8SRgvQF3Geq4ZNOc6zr8kJuuDU2Aqnla0FEf8pA3XHw5sWBjeQOvYxhjDvDWKbY ZlVEpgPyKcyAqYsSoRjiLXq2RTX2v28oErxgD881fCN3y59EAFi4T/wVED+TABDF0CnH wUB4xjPyOHFdEptG20Mw3qGnXXub4zN7mmfyMAWZdHJmKRKABX01cxA1miBPhX1ZVvA1 vgLw== X-Gm-Message-State: AOAM532oMylXUGeFmFTVMIAx5nwaESYdd8Vt3RqiIPhclhBxb3iveAi/ URNZNeFmGIm6JLvqDV/Cyz5OPrCISy8YMw== X-Google-Smtp-Source: ABdhPJw7S+AZZxmZcRKjPLGsS04rFBX3n3jNK6oq50tRxgiIyyjryAkzjFVxK3+mllXs66yJDFvBpQ== X-Received: by 2002:a5d:5146:: with SMTP id u6mr69720924wrt.46.1609537094237; Fri, 01 Jan 2021 13:38:14 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:13 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:36 +0000 Message-Id: <20210101213537.169546-27-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 26/27] vaapi_encode_h265: Ensure that HDR metadata is included with seek points X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Save the most recent HDR side data structures and use them when generating an IDR or CRA frame. --- libavcodec/vaapi_encode_h265.c | 93 ++++++++++++---------------------- 1 file changed, 32 insertions(+), 61 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 09d5a8b5cb..75ebb63e74 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -30,6 +30,7 @@ #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" +#include "cbs_metadata.h" #include "h265_profile_level.h" #include "hevc.h" #include "hevc_sei.h" @@ -70,13 +71,15 @@ typedef struct VAAPIEncodeH265Context { H265RawPPS raw_pps; H265RawSlice raw_slice; - SEIRawMasteringDisplayColourVolume sei_mastering_display; - SEIRawContentLightLevelInfo sei_content_light_level; - CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; int aud_needed; int sei_needed; + + int have_mastering_display; + AVMasteringDisplayMetadata mastering_display; + int have_content_light_level; + AVContentLightMetadata content_light_level; } VAAPIEncodeH265Context; @@ -200,17 +203,17 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, } if (priv->sei_needed & SEI_MASTERING_DISPLAY) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO, - &priv->sei_mastering_display, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_MASTERING_DISPLAY, + &priv->mastering_display); if (err < 0) goto fail; } if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, - &priv->sei_content_light_level, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &priv->content_light_level); if (err < 0) goto fail; } @@ -746,72 +749,40 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, priv->sei_needed = 0; - // Only look for the metadata on I/IDR frame on the output. We - // may force an IDR frame on the output where the medadata gets - // changed on the input frame. - if ((priv->sei & SEI_MASTERING_DISPLAY) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + // We check for new metadata on every frame, but only include the + // SEI with IRAP frames. + if (priv->sei & SEI_MASTERING_DISPLAY) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); if (sd) { - AVMasteringDisplayMetadata *mdm = - (AVMasteringDisplayMetadata *)sd->data; - - // SEI is needed when both the primaries and luminance are set - if (mdm->has_primaries && mdm->has_luminance) { - SEIRawMasteringDisplayColourVolume *mdcv = - &priv->sei_mastering_display; - const int mapping[3] = {1, 2, 0}; - const int chroma_den = 50000; - const int luma_den = 10000; - - for (i = 0; i < 3; i++) { - const int j = mapping[i]; - mdcv->display_primaries_x[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][0])), - chroma_den); - mdcv->display_primaries_y[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][1])), - chroma_den); - } - - mdcv->white_point_x = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), - chroma_den); - mdcv->white_point_y = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), - chroma_den); - - mdcv->max_display_mastering_luminance = - lrint(luma_den * av_q2d(mdm->max_luminance)); - mdcv->min_display_mastering_luminance = - FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), - mdcv->max_display_mastering_luminance); - - priv->sei_needed |= SEI_MASTERING_DISPLAY; - } + av_assert0(sd->size >= sizeof(priv->mastering_display)); + priv->have_mastering_display = 1; + memcpy(&priv->mastering_display, sd->data, + sizeof(priv->mastering_display)); + } + + if (priv->have_mastering_display && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_MASTERING_DISPLAY; } } - if ((priv->sei & SEI_CONTENT_LIGHT_LEVEL) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + if (priv->sei & SEI_CONTENT_LIGHT_LEVEL) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); if (sd) { - AVContentLightMetadata *clm = - (AVContentLightMetadata *)sd->data; - SEIRawContentLightLevelInfo *clli = - &priv->sei_content_light_level; - - clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); - clli->max_pic_average_light_level = FFMIN(clm->MaxFALL, 65535); + av_assert0(sd->size >= sizeof(priv->content_light_level)); + priv->have_mastering_display = 1; + memcpy(&priv->content_light_level, sd->data, + sizeof(priv->content_light_level)); + } + if (priv->have_content_light_level && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; } } From patchwork Fri Jan 1 21:35:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 24746 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3CF4B44AE1C for ; Fri, 1 Jan 2021 23:40:34 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6AC6F68AA4A; Fri, 1 Jan 2021 23:38:20 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A988568A7C9 for ; Fri, 1 Jan 2021 23:38:15 +0200 (EET) Received: by mail-wr1-f54.google.com with SMTP id w5so22895048wrm.11 for ; Fri, 01 Jan 2021 13:38:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hBf8DbD1+Bgi/564oCdVgNTuR0dzvlQ4LCJgxLpOljg=; b=kR0JK7nfs27kkgZa0BmSjA9f2vHxSIKM+3iU2SxgWj8jrvUqObAu0NxyHrRAVMXqpZ FTN2pn+B9jmeKK8QPJt3mcK58JZpPcOvmk3vMF9YRZKGU5lKJDVi614wQsKuDwM0e22K VGiV0fv6RPxVcCvFg/Zg0+Noi/s1emAR+YuIdZ309QK0RoRt7bgwDhFuJtMAZez+DIPd 4RSVdcw3d1+EwHPS+0qTJ4dFeXRCj/cc6XNEGA40jzYMluKs4cj5+apl+9BYmcNUzwLq vGq45C72xEsV6a88kB17IXqw3i4cOFtf7FQRm4yCSt3IABkkqbEcikderPytUuCPjS4I lAOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hBf8DbD1+Bgi/564oCdVgNTuR0dzvlQ4LCJgxLpOljg=; b=pi/O7PaYH/iU61GodmJx2p4UJpGSP/cB8s5r/mytLZhtaF8F0LcJsW4TApsUhD3hKf tAGZxGl/09mSm4O6rkBPVcO6ogj6hxRnnKR6I9JrfZSKTi06+0dMlrttR8VMH0J3Q+iB YSBAL/9n2sw5MHVobkc4nU5Pu25S3FMVKPJRaWbl20apST/Ps0WrB+PIuWsRjad6y4+E Z937b6R8WrpTdedh/ezqaRi/hRNic6r3oEJV9O86jxx2+qoB7Eg8YzVhuHhiq4+FFs2M TeZwlspIqPRdOx+K+IMvEGeYjJdjfbnb1n5bYErmsW8xF15p7mvzUwgbKXam9LNM1CA5 L0OQ== X-Gm-Message-State: AOAM532v1qxVNPZmSgjEDCVDEFhhJ9nK99085/iGJAU4jZ9OaiR0PO5P FIv2Y+pc+UxCVVGni/OPK9n/5+kI63DHaQ== X-Google-Smtp-Source: ABdhPJz4mC0sj1i7HarZz9sNh50rpLZuPfkNMYHRc5lWiUwkHQMHAnQrQu/szFThfDU4ZCKO5bvbww== X-Received: by 2002:a5d:43cc:: with SMTP id v12mr43489506wrr.319.1609537095044; Fri, 01 Jan 2021 13:38:15 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id v20sm18784595wml.34.2021.01.01.13.38.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jan 2021 13:38:14 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 1 Jan 2021 21:35:37 +0000 Message-Id: <20210101213537.169546-28-sw@jkqxz.net> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210101213537.169546-1-sw@jkqxz.net> References: <20210101213537.169546-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 27/27] vaapi_encode_h264: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This can now be done in exactly the same way as it is for H.265. --- libavcodec/vaapi_encode_h264.c | 66 +++++++++++++++++++++++++++++++++- 1 file changed, 65 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index 7c352e982b..ab1a09f674 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -24,11 +24,13 @@ #include "libavutil/avassert.h" #include "libavutil/common.h" #include "libavutil/internal.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/opt.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h264.h" +#include "cbs_metadata.h" #include "h264.h" #include "h264_levels.h" #include "h264_sei.h" @@ -95,6 +97,11 @@ typedef struct VAAPIEncodeH264Context { int aud_needed; int sei_needed; int sei_cbr_workaround_needed; + + int have_mastering_display; + AVMasteringDisplayMetadata mastering_display; + int have_content_light_level; + AVContentLightMetadata content_light_level; } VAAPIEncodeH264Context; @@ -241,6 +248,20 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, if (err < 0) goto fail; } + if (priv->sei_needed & SEI_MASTERING_DISPLAY) { + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_MASTERING_DISPLAY, + &priv->mastering_display); + if (err < 0) + goto fail; + } + if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &priv->content_light_level); + if (err < 0) + goto fail; + } priv->sei_needed = 0; @@ -674,6 +695,42 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_needed |= SEI_RECOVERY_POINT; } + if (priv->sei & SEI_MASTERING_DISPLAY) { + AVFrameSideData *sd = + av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + + if (sd) { + av_assert0(sd->size >= sizeof(priv->mastering_display)); + priv->have_mastering_display = 1; + memcpy(&priv->mastering_display, sd->data, + sizeof(priv->mastering_display)); + } + + if (priv->have_mastering_display && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_MASTERING_DISPLAY; + } + } + + if (priv->sei & SEI_CONTENT_LIGHT_LEVEL) { + AVFrameSideData *sd = + av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + + if (sd) { + av_assert0(sd->size >= sizeof(priv->content_light_level)); + priv->have_mastering_display = 1; + memcpy(&priv->content_light_level, sd->data, + sizeof(priv->content_light_level)); + } + + if (priv->have_content_light_level && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; + } + } + vpic->CurrPic = (VAPictureH264) { .picture_id = pic->recon_surface, .frame_idx = hpic->frame_num, @@ -1245,7 +1302,8 @@ static const AVOption vaapi_encode_h264_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT }, + { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT | + SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, 0, INT_MAX, FLAGS, "sei" }, { "identifier", "Include encoder version identifier", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_IDENTIFIER }, @@ -1256,6 +1314,12 @@ static const AVOption vaapi_encode_h264_options[] = { { "recovery_point", "Include recovery points where appropriate", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_RECOVERY_POINT }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "hdr", + "Include HDR metadata (mastering display colour volume " + "and content light level information)", + 0, AV_OPT_TYPE_CONST, + { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "profile", "Set profile (profile_idc and constraint_set*_flag)", OFFSET(profile), AV_OPT_TYPE_INT,