From patchwork Sat Feb 4 00:41:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 40243 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1617218pzk; Fri, 3 Feb 2023 16:42:58 -0800 (PST) X-Google-Smtp-Source: AK7set8KxlVhZ263Oz6Pphl/X4YKtSOlNSjWOTOoWnIl6KItzHGAcnj8/vFl+T9M7n7ystzvLsQR X-Received: by 2002:a17:907:1687:b0:872:84dd:8903 with SMTP id hc7-20020a170907168700b0087284dd8903mr15146484ejc.59.1675471378670; Fri, 03 Feb 2023 16:42:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675471378; cv=none; d=google.com; s=arc-20160816; b=cYcAdnuQBrpmH5/KhAscNhJjqvmpShm8KigR4vzy/b25KMZM91gNAUtc3mETf+EEbg teEX0YdnQGmb4u8KOfSTiwkHmiVacf33Zs5NCjPwjgYyMfxeV3/CSbBg3VtUlmGsGKQb d8S5l9J6HyfDVpi6NwFtl36vTc8Ux8+gYIbq5+mxKs0ELFaNnd5hzPZO8I6RlpUQVuEv zuIvqP2zrIRczZ9fhFT+o7aNEU7xz5OiLjk0DKNZL9v94OrVJBlMcFuf/5gsydN40I9e GVnibUIxKNvllPU86SIeReravYBpYiwYkknTvEH74zqf8wec57FE4p91/sLG4+stBpuK +DTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=I50dbtcGsMS31EDkaE20Gzm+O7FYvRwwj9tSARMr3Cg=; b=cTTUTcyKXXK6midjIbLsfllJJ4xWrA9Dz9REHoGL2poejgiFxy4zk6TrSmu6E0N6OH /a6OBlsN5NDl8yomH4blSaWrBrwVMMy7SWT2wC+e7oj7O0cuwMF79r/7/cySnZ9SdHdU BZ6DdaL5e++KA8oYhSCW0qOosZiYMReunGSKucQK2+QR2he3vJQ5M+AqmuyZSGX6EKRI RPizoPm6SFrFbhClCxJ1rjNnluayeY9Wf9lfs1+6IHNgN+x7/Krqt7r9QOT40C7SClbp G5+lnsWM2YhXGbrB9hDG6Jlozo6ny7WJF1hvvBLXsDqkgIZnrGK7n1oy/YgCmWAojGt+ oHDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PpPxVozp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id wj17-20020a170907051100b0087777bb4127si4831286ejb.172.2023.02.03.16.42.58; Fri, 03 Feb 2023 16:42:58 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PpPxVozp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F1F6F68BEC8; Sat, 4 Feb 2023 02:42:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0140868BE90 for ; Sat, 4 Feb 2023 02:42:01 +0200 (EET) Received: by mail-pj1-f52.google.com with SMTP id l4-20020a17090a850400b0023013402671so10332834pjn.5 for ; Fri, 03 Feb 2023 16:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=Wq+5njkHHHPr7Y9utdj6he1G+n/NFwNTuHdN1g7puGk=; b=PpPxVozpZ+kKiqz+/M9uS9I9YBHjZHPTbZUEOO5DdpOEUU+viYNWSGUyGW3f91gJfM uR9fDTGC8VaLS/XYpw5h3F7qVo5J+4t2r+yHmhAP3nxVYUrvZoelt4OimqUcNaX9icY8 dta2mtAGTATH2GMwLT2ZmOMBGD9IaGljnuBRX0weMINipzbYbJpcol8WLmbrtORQeFCW UB4SXauo1LNAMi1VVLGSj8cWMPwYkn0qjYsHXD45F6Lrr+8mbXQg2xxYKZ65ShHsIMij QwAgjUECKxy5yCISqAm2S89TqEEjke54I67x3ewu02ffbUvoD9UFAlmYVbSNH7S1b02m XPMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Wq+5njkHHHPr7Y9utdj6he1G+n/NFwNTuHdN1g7puGk=; b=Ksi1wJufA80O8HIdj2pkI22+IiMv2cGU6aG6WENX/Qz88QPWpKlgpzUz5nHt9vfxKB Jrcp8of3+HumZoxa/M2EZ+b92/Ybg0fKbRLGwZ0s2jWDwmiLMHBXTRmnpR9f0GY1Ttbc ro897jXMUKkgiGOeM3YRsacVEK35Zwi48JpKuszIvBmoRTuUPo5/lcp5RE8hdoT0ReS2 ZGZm6vjVShhSZUcz7BxUasHVYE+/H9he0BiqEFXFVcSnN7bZYaBTpkwxfzUdCmd7m9Ts 6Ga2gmt2X6HPzwpF8GYGmcuGwFXylKZZFwRdpzf78D59w32ab+MED7s0MvCTIEh5uab0 6BSw== X-Gm-Message-State: AO0yUKXhoVHnnnAkC7USbQI24IPrFJ7FZJeNmysWVkhCifFNJcWkOroW 7wZpvXE2ww9GBl/9UIm+F9RFEt9cMmM= X-Received: by 2002:a05:6a21:8695:b0:bf:6cd3:9546 with SMTP id ox21-20020a056a21869500b000bf6cd39546mr8452677pzb.30.1675471320302; Fri, 03 Feb 2023 16:42:00 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id v30-20020a637a1e000000b0047917991e83sm2053687pgc.48.2023.02.03.16.41.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Feb 2023 16:41:59 -0800 (PST) From: Aman Karmani X-Google-Original-From: Aman Karmani Message-Id: <8aa6c32db86a00c50708736d86861e21a66697a5.1675471315.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Sat, 04 Feb 2023 00:41:55 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 5/5] avcodec/h265_metadata_bsf: add support for a/53 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Karmani Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xBqL7xan2MZb From: Aman Karmani Signed-off-by: Aman Karmani --- doc/bitstream_filters.texi | 11 +++ libavcodec/Makefile | 2 +- libavcodec/h265_metadata_bsf.c | 144 +++++++++++++++++++++++++++++++++ 3 files changed, 156 insertions(+), 1 deletion(-) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 3d97f66315..57f8682d9f 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -398,6 +398,17 @@ confuse other transformations which require correct extradata. Modify metadata embedded in an HEVC stream. @table @option +@item a53_cc +Insert or remove registered userdata SEI NAL units containing A/53 closed captions. + +@table @samp +@item pass +@item insert +@item remove +@end table + +Default is pass. + @item aud Insert or remove AUD NAL units in all access units of the stream. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 9f97c9b5e5..156fd6215a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1217,7 +1217,7 @@ OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF) += h264_mp4toannexb_bsf.o OBJS-$(CONFIG_H264_REDUNDANT_PPS_BSF) += h264_redundant_pps_bsf.o OBJS-$(CONFIG_HAPQA_EXTRACT_BSF) += hapqa_extract_bsf.o hap.o OBJS-$(CONFIG_HEVC_METADATA_BSF) += h265_metadata_bsf.o h265_profile_level.o \ - h2645data.o + h2645data.o cbs_misc.o OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF) += hevc_mp4toannexb_bsf.o OBJS-$(CONFIG_IMX_DUMP_HEADER_BSF) += imx_dump_header_bsf.o OBJS-$(CONFIG_MEDIA100_TO_MJPEGB_BSF) += media100_to_mjpegb_bsf.o diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index 6787bd14a1..c3acbfb167 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -24,6 +24,8 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h265.h" +#include "cbs_misc.h" +#include "cbs_sei.h" #include "h2645data.h" #include "hevc.h" #include "h265_profile_level.h" @@ -62,6 +64,7 @@ typedef struct H265MetadataContext { int level; int level_guess; int level_warned; + int a53_cc; } H265MetadataContext; @@ -322,6 +325,137 @@ static int h265_metadata_update_sps(AVBSFContext *bsf, return 0; } +static int h265_metadata_extract_a53_cc(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au) +{ + H265MetadataContext *ctx = bsf->priv_data; + SEIRawMessage *message; + int err; + + message = NULL; + while (ff_cbs_sei_find_message(ctx->common.output, au, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &message) == 0) { + SEIRawUserDataRegistered *udr = message->payload; + A53UserData a53_ud; + uint8_t *a53_side_data = NULL; + size_t a53_side_data_size = 0; + + if (udr->data_length < 2) { + // Too short to contain a provider code. + continue; + } + + if (AV_RB16(udr->data) != 0x31) { // provider_code as atsc_provider_code + // Not ATSC. + continue; + } + + // The first two bytes of the message is provider_code + err = ff_cbs_read_a53_user_data(ctx->common.output, + &a53_ud, + udr->data + 2, + udr->data_length - 2); + if (err < 0) { + // Invalid or something completely different. + continue; + } + if (a53_ud.user_identifier != A53_USER_IDENTIFIER_ATSC || + a53_ud.atsc.user_data_type_code != + A53_USER_DATA_TYPE_CODE_CC_DATA) { + // Valid but something else (e.g. AFD). + continue; + } + + err = ff_cbs_write_a53_cc_side_data(ctx->common.output, + &a53_side_data, + &a53_side_data_size, + &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data for packet side data.\n"); + return err; + } + + if (a53_side_data) { + err = av_packet_add_side_data(pkt, AV_PKT_DATA_A53_CC, + a53_side_data, a53_side_data_size); + if (err) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted A/53 " + "side data to packet.\n"); + av_freep(&a53_side_data); + return err; + } + + if (ctx->a53_cc == BSF_ELEMENT_REMOVE || + ctx->a53_cc == BSF_ELEMENT_INSERT) { + ff_cbs_sei_delete_message(ctx->common.output, au, message); + + // Reset iteration + message = NULL; + } + } + } + + if (ctx->a53_cc == BSF_ELEMENT_INSERT) { + uint8_t *data; + size_t size; + int offset = 0; + + data = av_packet_get_side_data(pkt, AV_PKT_DATA_A53_CC, &size); + while (size > 0) { + A53UserData a53_ud; + int rsize = FFMIN(93, size); + + err = ff_cbs_read_a53_cc_side_data(ctx->common.output, &a53_ud, + data + offset, rsize); + offset += rsize; + size -= rsize; + if (err < 0) { + av_log(bsf, AV_LOG_WARNING, "Invalid A/53 closed captions " + "in packet side data dropped.\n"); + } else { + AVBufferRef *udr_buf = av_buffer_allocz(sizeof(SEIRawUserDataRegistered)); + SEIRawUserDataRegistered *udr = (SEIRawUserDataRegistered*)udr_buf->data; + size_t size = 9 + 3 * a53_ud.atsc.cc_data.cc_count; + + udr->data_ref = av_buffer_allocz(2 + size); + if (!udr->data_ref) { + return AVERROR(ENOMEM); + } + udr->data = udr->data_ref->data; + udr->data_length = udr->data_ref->size; + + udr->itu_t_t35_country_code = 0xB5; // usa_country_code + AV_WB16(udr->data, 0x31); // provider_code as atsc_provider_code + + err = ff_cbs_write_a53_user_data(ctx->common.output, udr->data + 2, + &size, &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data.\n"); + av_buffer_unref(&udr->data_ref); + av_buffer_unref(&udr_buf); + return err; + } + + err = ff_cbs_sei_add_message(ctx->common.output, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + udr, udr_buf); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add A/53 user data " + "SEI message to access unit.\n"); + av_buffer_unref(&udr->data_ref); + av_buffer_unref(&udr_buf); + return err; + } + } + } + } + + return 0; +} + static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, CodedBitstreamFragment *au) { @@ -387,6 +521,12 @@ static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, } } + if (pkt && ctx->a53_cc != BSF_ELEMENT_PASS) { + err = h265_metadata_extract_a53_cc(bsf, pkt, au); + if (err < 0) + return err; + } + return 0; } @@ -478,6 +618,10 @@ static const AVOption h265_metadata_options[] = { { LEVEL("8.5", 255) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("a53_cc", + "A/53 Closed Captions in SEI NAL units", + a53_cc, FLAGS), + { NULL } };