From patchwork Wed Aug 12 23:20:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 21619 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8DFD3448961 for ; Thu, 13 Aug 2020 02:25:14 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6339868B4B2; Thu, 13 Aug 2020 02:25:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3C7CD680A1A for ; Thu, 13 Aug 2020 02:25:08 +0300 (EEST) Received: by mail-wr1-f66.google.com with SMTP id z18so3535285wrm.12 for ; Wed, 12 Aug 2020 16:25:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bG1lp0YFOFp7sgNPSUHrvgy6PU0kFJWELLZG1Cs+6Bo=; b=AvmwAjraEE9roKGAVgZwlBztA1D41BNFUjPemPFCM5an/QboAuHIEM4UPnkFcCUivL 96tfPLckFPSMfpzPoNgxbpYyONzTKnHQiJyEOUywj81hqlLbDPMR9mDXHT1yMgfOMLgt v5q9aPC4XHIVjQd0ActOCADHN0zbRjKd2jOAXp8b3jC8FEcqy/XILsIIVzlUh81eo0aq P8mGQq8619fUhpDwhxBC/CwBK7b0Sx6GaRfnRBhb+/BOTNSzcp6akrAFxC/WTlOZw9ii PhuHtQFQRFucYsFr4eI95kj4qOg4GsrAeVdfSkP+QNN5qCGrSoMqyE+TIthieGATB5Xr MCFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bG1lp0YFOFp7sgNPSUHrvgy6PU0kFJWELLZG1Cs+6Bo=; b=AE4nn+0sv6493vnD5M2BLvnwm7YXeczLnxa/Vx6W3dWJOBG32UGLEUGUQiQQ9N+Fjw 90gQbrVFE+y9d354bb3mpq8sKSspmUdyfezC8JB5kA6puVn7SlB6+YPkPlLr2oxW8oyt XIlNml5e8YsY7KQGVer+zACpwNvGST5SJnoz5mECxw9u9U4W4fLFtU5gSgf113LcvSAd PHvDw4pFrzSAPIsWe+p97Cz2jiA78ki4O7aHjYCogUB5bdiOXdVK2+OEKFNGQchQbr02 ZhsEVT4gTAVhRF+HXsAAmfA2Ehv514C2Mnye/Cv26bKhZw+LD0B5GQ1ZTaHkKCuvEuWQ F+uQ== X-Gm-Message-State: AOAM531EWL/vpAkuCOplxLRqGfxjuYJwTIlOw6ymj6U1Hvt8gKTLbIeJ OHa50vplKh2MnHUp0b3O7VDwOYuZPc8= X-Google-Smtp-Source: ABdhPJxTh8rTVNwDCMNMUKuBI5xB2EJaka8Ocgvjx20nUevQRPwLR/kR9MGQaRZTXmdfITcxAaq9Pg== X-Received: by 2002:adf:a1c6:: with SMTP id v6mr1227526wrv.197.1597274707369; Wed, 12 Aug 2020 16:25:07 -0700 (PDT) Received: from localhost.localdomain (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id p25sm6019534wma.39.2020.08.12.16.25.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Aug 2020 16:25:06 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Thu, 13 Aug 2020 00:20:53 +0100 Message-Id: <20200812232054.451992-1-sw@jkqxz.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <38a52214-4fd6-b465-4029-0eba332ba8c5@jkqxz.net> References: <38a52214-4fd6-b465-4029-0eba332ba8c5@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 21.1/23] h264_metadata: Move SEI user data parsing to init time X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 91 ++++++++++++++++------------------ 1 file changed, 43 insertions(+), 48 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index eb1503159b..e59b5b8369 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -78,6 +78,7 @@ typedef struct H264MetadataContext { int crop_bottom; const char *sei_user_data; + H264RawSEIPayload sei_user_data_payload; int delete_filler; @@ -413,54 +414,10 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) // Only insert the SEI in access units containing SPSs, and also // unconditionally in the first access unit we ever see. if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - H264RawSEIPayload payload = { - .payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED, - }; - H264RawSEIUserDataUnregistered *udu = - &payload.payload.user_data_unregistered; - - for (i = j = 0; j < 32 && ctx->sei_user_data[i]; i++) { - int c, v; - c = ctx->sei_user_data[i]; - if (c == '-') { - continue; - } else if (av_isxdigit(c)) { - c = av_tolower(c); - v = (c <= '9' ? c - '0' : c - 'a' + 10); - } else { - goto invalid_user_data; - } - if (j & 1) - udu->uuid_iso_iec_11578[j / 2] |= v; - else - udu->uuid_iso_iec_11578[j / 2] = v << 4; - ++j; - } - if (j == 32 && ctx->sei_user_data[i] == '+') { - size_t len = strlen(ctx->sei_user_data + i + 1); - - udu->data_ref = av_buffer_alloc(len + 1); - if (!udu->data_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - - udu->data = udu->data_ref->data; - udu->data_length = len + 1; - memcpy(udu->data, ctx->sei_user_data + i + 1, len + 1); - - err = ff_cbs_h264_add_sei_message(au, &payload); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " - "message to access unit.\n"); - goto fail; - } - - } else { - invalid_user_data: - av_log(bsf, AV_LOG_ERROR, "Invalid user data: " - "must be \"UUID+string\".\n"); - err = AVERROR(EINVAL); + err = ff_cbs_h264_add_sei_message(au, &ctx->sei_user_data_payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " + "message to access unit.\n"); goto fail; } } @@ -627,6 +584,44 @@ static int h264_metadata_init(AVBSFContext *bsf) CodedBitstreamFragment *au = &ctx->access_unit; int err, i; + if (ctx->sei_user_data) { + H264RawSEIUserDataUnregistered *udu = + &ctx->sei_user_data_payload.payload.user_data_unregistered; + int j; + + ctx->sei_user_data_payload.payload_type = + H264_SEI_TYPE_USER_DATA_UNREGISTERED; + + // Parse UUID. It must be a hex string of length 32, possibly + // containing '-'s between hex digits (which we ignore). + for (i = j = 0; j < 32 && i < 64 && ctx->sei_user_data[i]; i++) { + int c, v; + c = ctx->sei_user_data[i]; + if (c == '-') { + continue; + } else if (av_isxdigit(c)) { + c = av_tolower(c); + v = (c <= '9' ? c - '0' : c - 'a' + 10); + } else { + break; + } + if (j & 1) + udu->uuid_iso_iec_11578[j / 2] |= v; + else + udu->uuid_iso_iec_11578[j / 2] = v << 4; + ++j; + } + if (j == 32 && ctx->sei_user_data[i] == '+') { + udu->data = (uint8_t*)ctx->sei_user_data + i + 1; + udu->data_length = strlen(udu->data) + 1; + } else { + av_log(bsf, AV_LOG_ERROR, "Invalid user data: " + "must be \"UUID+string\".\n"); + err = AVERROR(EINVAL); + goto fail; + } + } + err = ff_cbs_init(&ctx->input, AV_CODEC_ID_H264, bsf); if (err < 0) return err; From patchwork Wed Aug 12 23:20:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 21620 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 73BE7448961 for ; Thu, 13 Aug 2020 02:25:16 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BA5D68B20E; Thu, 13 Aug 2020 02:25:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8086668B4E8 for ; Thu, 13 Aug 2020 02:25:09 +0300 (EEST) Received: by mail-wr1-f65.google.com with SMTP id 88so3572704wrh.3 for ; Wed, 12 Aug 2020 16:25:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=C3IjQ2mBuiJuxkcERBJR6C/3NC15XGPqENzgzFZCXbM=; b=oJUWFLy+EfpWQpD8lsGew14BwCBcX814KlSfRvEuuN6d1lfqJsFhusyh4pY15AqUsj iI7AZkQ+4zPoqOL+e32Krwg3sZ2a2Me7DI+6NV2FFllfjiGlxFcFgMuO+f3rkMbvhmXS s4jgGRLH/5jMUyFTkQN91YK2IRYZnDfmjc9HOyPL3ndhi8pBljAsTQcUlw/z+KH0UnMD jrblQSBNRCSxUi2jhl7Lj1ByQot4RuAv+uKuEG+lo0XtAGAlEGoMrwULVJPBYQnr14TE 4PBARax5rhqrD8UajpNNGjqJ8zJCa5ZrKdRQ8FWpUnxjw0o5rnBLOyaSpXP6j4QmR7O/ HLRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C3IjQ2mBuiJuxkcERBJR6C/3NC15XGPqENzgzFZCXbM=; b=A1WJvxZiWahm1fGiR9lPPqeO6zbgV+Q78qUSEKb8SabmABmNfSotjzXq+ZF6ns+hKb WrUkxybA+40qSOEC1FTJrtySNKww+lSe/NohKQ8qnVI99x29Hr4uH/j5JKUFn7o6ezWM mEMAy7MbF6dd93fXHs5kKj7CmfMJ//+NuCwsay8OMgNUbP99T6Y9jWTxLdmsOrxWkcFN 3WY4CBoZwrnXofJPleiFIecBIzLRkbl8++dqPUrT4VVwf/QZ9Sa/bDW6Da03f/L+aPTd NW4uR2JThlL3pf0U0FWz3Z8b3+n1hpiwWqUdMXGu247HLpY9TRDzHHpgVggrZiAUUQBO xntQ== X-Gm-Message-State: AOAM533WAJE7faP/Z0qsqaFjzxyqd7u/Net+SBmmCoxvWX4ZL8MWp2s+ 9lwf6c0XCDi7otEGDphyAb74JOX2QbU= X-Google-Smtp-Source: ABdhPJxzIXFcxOmY4ydjEtynef3KyGxCdxaFViyCNTGy+xEBtJ/xJCp/op661ccZqvpSN6ZXHANS2A== X-Received: by 2002:a5d:6541:: with SMTP id z1mr1211540wrv.320.1597274708308; Wed, 12 Aug 2020 16:25:08 -0700 (PDT) Received: from localhost.localdomain (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id p25sm6019534wma.39.2020.08.12.16.25.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Aug 2020 16:25:07 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Thu, 13 Aug 2020 00:20:54 +0100 Message-Id: <20200812232054.451992-2-sw@jkqxz.net> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200812232054.451992-1-sw@jkqxz.net> References: <38a52214-4fd6-b465-4029-0eba332ba8c5@jkqxz.net> <20200812232054.451992-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 21.2/23] h264_metadata_bsf: Refactor the filter function into smaller parts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 348 ++++++++++++++++++--------------- 1 file changed, 191 insertions(+), 157 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index e59b5b8369..d69713cbab 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -56,6 +56,7 @@ typedef struct H264MetadataContext { int done_first_au; int aud; + H264RawAUD aud_nal; AVRational sample_aspect_ratio; @@ -90,6 +91,59 @@ typedef struct H264MetadataContext { } H264MetadataContext; +static int h264_metadata_insert_aud(AVBSFContext *bsf, + CodedBitstreamFragment *au) +{ + H264MetadataContext *ctx = bsf->priv_data; + int primary_pic_type_mask = 0xff; + int err, i, j; + + static const int primary_pic_type_table[] = { + 0x084, // 2, 7 + 0x0a5, // 0, 2, 5, 7 + 0x0e7, // 0, 1, 2, 5, 6, 7 + 0x210, // 4, 9 + 0x318, // 3, 4, 8, 9 + 0x294, // 2, 4, 7, 9 + 0x3bd, // 0, 2, 3, 4, 5, 7, 8, 9 + 0x3ff, // 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 + }; + + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SLICE || + au->units[i].type == H264_NAL_IDR_SLICE) { + H264RawSlice *slice = au->units[i].content; + for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) { + if (!(primary_pic_type_table[j] & + (1 << slice->header.slice_type))) + primary_pic_type_mask &= ~(1 << j); + } + } + } + for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) + if (primary_pic_type_mask & (1 << j)) + break; + if (j >= FF_ARRAY_ELEMS(primary_pic_type_table)) { + av_log(bsf, AV_LOG_ERROR, "No usable primary_pic_type: " + "invalid slice types?\n"); + return AVERROR_INVALIDDATA; + } + + ctx->aud_nal = (H264RawAUD) { + .nal_unit_header.nal_unit_type = H264_NAL_AUD, + .primary_pic_type = j, + }; + + err = ff_cbs_insert_unit_content(au, 0, H264_NAL_AUD, + &ctx->aud_nal, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + return err; + } + + return 0; +} + static int h264_metadata_update_sps(AVBSFContext *bsf, H264RawSPS *sps) { @@ -321,175 +375,59 @@ static int h264_metadata_update_side_data(AVBSFContext *bsf, AVPacket *pkt) return 0; } -static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *au, + int seek_point) { H264MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - int err, i, j, has_sps; - H264RawAUD aud; - - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; - - err = h264_metadata_update_side_data(bsf, pkt); - if (err < 0) - goto fail; - - err = ff_cbs_read_packet(ctx->input, au, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } - - if (au->nb_units == 0) { - av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - - // If an AUD is present, it must be the first NAL unit. - if (au->units[0].type == H264_NAL_AUD) { - if (ctx->aud == REMOVE) - ff_cbs_delete_unit(au, 0); - } else { - if (ctx->aud == INSERT) { - static const int primary_pic_type_table[] = { - 0x084, // 2, 7 - 0x0a5, // 0, 2, 5, 7 - 0x0e7, // 0, 1, 2, 5, 6, 7 - 0x210, // 4, 9 - 0x318, // 3, 4, 8, 9 - 0x294, // 2, 4, 7, 9 - 0x3bd, // 0, 2, 3, 4, 5, 7, 8, 9 - 0x3ff, // 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 - }; - int primary_pic_type_mask = 0xff; - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SLICE || - au->units[i].type == H264_NAL_IDR_SLICE) { - H264RawSlice *slice = au->units[i].content; - for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) { - if (!(primary_pic_type_table[j] & - (1 << slice->header.slice_type))) - primary_pic_type_mask &= ~(1 << j); - } - } - } - for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) - if (primary_pic_type_mask & (1 << j)) - break; - if (j >= FF_ARRAY_ELEMS(primary_pic_type_table)) { - av_log(bsf, AV_LOG_ERROR, "No usable primary_pic_type: " - "invalid slice types?\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } + int err, i, j; - aud = (H264RawAUD) { - .nal_unit_header.nal_unit_type = H264_NAL_AUD, - .primary_pic_type = j, - }; + for (i = au->nb_units - 1; i >= 0; i--) { + H264RawSEI *sei; + if (au->units[i].type != H264_NAL_SEI) + continue; + sei = au->units[i].content; - err = ff_cbs_insert_unit_content(au, - 0, H264_NAL_AUD, &aud, NULL); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); - goto fail; - } - } - } + for (j = sei->payload_count - 1; j >= 0; j--) { + H264RawSEIDisplayOrientation *disp; + int32_t *matrix; - has_sps = 0; - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SPS) { - err = h264_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - goto fail; - has_sps = 1; - } - } - - // Only insert the SEI in access units containing SPSs, and also - // unconditionally in the first access unit we ever see. - if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - err = ff_cbs_h264_add_sei_message(au, &ctx->sei_user_data_payload); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " - "message to access unit.\n"); - goto fail; - } - } + if (sei->payload[j].payload_type != + H264_SEI_TYPE_DISPLAY_ORIENTATION) + continue; + disp = &sei->payload[j].payload.display_orientation; - if (ctx->delete_filler) { - for (i = au->nb_units - 1; i >= 0; i--) { - if (au->units[i].type == H264_NAL_FILLER_DATA) { - ff_cbs_delete_unit(au, i); + if (ctx->display_orientation == REMOVE || + ctx->display_orientation == INSERT) { + ff_cbs_h264_delete_sei_message(au, &au->units[i], j); continue; } - if (au->units[i].type == H264_NAL_SEI) { - // Filler SEI messages. - H264RawSEI *sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - if (sei->payload[j].payload_type == - H264_SEI_TYPE_FILLER_PAYLOAD) - ff_cbs_h264_delete_sei_message(au, &au->units[i], j); - } + matrix = av_malloc(9 * sizeof(int32_t)); + if (!matrix) + return AVERROR(ENOMEM); + + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 180.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); + + // If there are multiple display orientation messages in an + // access unit, then the last one added to the packet (i.e. + // the first one in the access unit) will prevail. + err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, + (uint8_t*)matrix, + 9 * sizeof(int32_t)); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " + "displaymatrix side data to packet.\n"); + av_free(matrix); + return AVERROR(ENOMEM); } } } - if (ctx->display_orientation != PASS) { - for (i = au->nb_units - 1; i >= 0; i--) { - H264RawSEI *sei; - if (au->units[i].type != H264_NAL_SEI) - continue; - sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - H264RawSEIDisplayOrientation *disp; - int32_t *matrix; - - if (sei->payload[j].payload_type != - H264_SEI_TYPE_DISPLAY_ORIENTATION) - continue; - disp = &sei->payload[j].payload.display_orientation; - - if (ctx->display_orientation == REMOVE || - ctx->display_orientation == INSERT) { - ff_cbs_h264_delete_sei_message(au, &au->units[i], j); - continue; - } - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) { - err = AVERROR(ENOMEM); - goto fail; - } - - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_free(matrix); - goto fail; - } - } - } - } if (ctx->display_orientation == INSERT) { H264RawSEIPayload payload = { .payload_type = H264_SEI_TYPE_DISPLAY_ORIENTATION, @@ -533,7 +471,7 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) } } - if (has_sps || !ctx->done_first_au) { + if (seek_point) { if (!isnan(ctx->rotate)) { disp->anticlockwise_rotation = (uint16_t)rint((ctx->rotate >= 0.0 ? ctx->rotate @@ -555,11 +493,107 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); + return err; + } + } + } + + return 0; +} + +static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +{ + H264MetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err, i, j, has_sps, seek_point; + + err = ff_bsf_get_packet_ref(bsf, pkt); + if (err < 0) + return err; + + err = h264_metadata_update_side_data(bsf, pkt); + if (err < 0) + goto fail; + + err = ff_cbs_read_packet(ctx->input, au, pkt); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (au->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + // If an AUD is present, it must be the first NAL unit. + if (au->units[0].type == H264_NAL_AUD) { + if (ctx->aud == REMOVE) + ff_cbs_delete_unit(au, 0); + } else { + if (ctx->aud == INSERT) { + err = h264_metadata_insert_aud(bsf, au); + if (err < 0) goto fail; + } + } + + has_sps = 0; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SPS) { + err = h264_metadata_update_sps(bsf, au->units[i].content); + if (err < 0) + goto fail; + has_sps = 1; + } + } + + // The current packet should be treated as a seek point for metadata + // insertion if any of: + // - It is the first packet in the stream. + // - It contains an SPS, indicating that a sequence might start here. + // - It is marked as containing a key frame. + seek_point = !ctx->done_first_au || has_sps || + (pkt->flags & AV_PKT_FLAG_KEY); + + if (ctx->sei_user_data && seek_point) { + err = ff_cbs_h264_add_sei_message(au, &ctx->sei_user_data_payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " + "message to access unit.\n"); + goto fail; + } + } + + if (ctx->delete_filler) { + for (i = au->nb_units - 1; i >= 0; i--) { + if (au->units[i].type == H264_NAL_FILLER_DATA) { + ff_cbs_delete_unit(au, i); + continue; + } + + if (au->units[i].type == H264_NAL_SEI) { + // Filler SEI messages. + H264RawSEI *sei = au->units[i].content; + + for (j = sei->payload_count - 1; j >= 0; j--) { + if (sei->payload[j].payload_type == + H264_SEI_TYPE_FILLER_PAYLOAD) + ff_cbs_h264_delete_sei_message(au, + &au->units[i], j); + } } } } + if (ctx->display_orientation != PASS) { + err = h264_metadata_handle_display_orientation(bsf, pkt, au, + seek_point); + if (err < 0) + goto fail; + } + err = ff_cbs_write_packet(ctx->output, pkt, au); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n");