From patchwork Sun Mar 11 18:30:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 7915 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp827050jad; Sun, 11 Mar 2018 11:37:07 -0700 (PDT) X-Google-Smtp-Source: AG47ELs+EYyjadyFvIfMAr87LcnmmvTVRlXfkUsU1L9Jvvs0QFIeO8ELXqkHXM16kJnR7gf728lK X-Received: by 10.28.183.9 with SMTP id h9mr3292285wmf.99.1520793427125; Sun, 11 Mar 2018 11:37:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1520793427; cv=none; d=google.com; s=arc-20160816; b=M5NbtRDCsMS225OV6Rnp2yzRdDWOBAmbg4o+dlVBopWzXlREFji292pM+Tx33c/aI1 YsztkPHsMQwMzT61/E8kJkEmkKnj5i9RxGTOIDu+cS5xb21M98uls5Ae2bo3UEvMguGO d0fkYB9FZ9bL20UK3QJ9OXEU4ohBZOL/moy+SsjFTrTX1lFG2Vy2Rkf+IfAXpSZcFGBM MszjqCdEf7Rkm2q1X+adOh7lgt0qj+yMBpg73+8k7v09kAnvPCnDw7hgQHRZ7ziPw4DX Kg4Nng0GpHb139kG1kuAP+fScJkDzUgf8OwA06mMeVup8kranA+vpUhrPcrS44T2E4HU euzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=vaPPd7F08LlERf+JaTV3i99GHpSN+QKL1Mfjo+aA6oI=; b=TmrmZmgCkrm3IjxtWwVd2dpPMm6Xqog08cgInFFRAy+nIbFgpSWDb+8b7+vkOPmwU0 Omye1LdgaxcGhFzWrGcbybqCLBhpCTpHlSK1efBmGeRi6jl0k8rtp1xYCIQZ7RaogiP5 Sg7/NQ4o24lGsbYlaA9ZDNRWUNzGY8XjpO+gaZjgA2kUNNejICRlG8eiEx9QmJ+XYpmj L29oWkTBLKxc1hS1OIc4s+EgRIUQyi7wz0pmWIGQ10EBVTJ9luUx3FLj1BySW2YxAoqK Ag/x7MYMqPxQ2x+/g+Qp1Ql5H2moZI9kD0Vtc265hmob7lvpnEDDNfkbQXM2cEg/z9bO w29w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=HrOFElMy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 131si2210879wmt.225.2018.03.11.11.37.06; Sun, 11 Mar 2018 11:37:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=HrOFElMy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B1E1E68A295; Sun, 11 Mar 2018 20:36:53 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f53.google.com (mail-wm0-f53.google.com [74.125.82.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 349C368A270 for ; Sun, 11 Mar 2018 20:36:52 +0200 (EET) Received: by mail-wm0-f53.google.com with SMTP id e194so12343713wmd.3 for ; Sun, 11 Mar 2018 11:37:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ERW77M0h/A9OK7P5LszqdVgRJRgBCNq3O90vhkzUiv0=; b=HrOFElMy3TvvAa5hMFHelp5XnP7+FN/hFiL+ZbGqraNJu9NHvk02i/B87RVVyVWs61 cQS3sC2etGxJkCS2ErCE5MXQFXUyJQnajmDub8gECuZ6ZI2j1FqNAoOh8jdeB+YwKgCB se9TR5Rbn9BejZnhoegKfxXyXmwWt2PchWBuEtrIXODQQ5hGMTxpYDl/MF6v5896fBoi x4C4Tk2r07rn2QZGjXldaJ78Nof0ucrTsxlqfEwOKc3eGBSLv+9xm3Uo8kRxmwlBfhDN 79fohnf2pMboaY8V3fvKhlZrO56kBtfDsj6xkOzzjnBiuppQUsDqPjzvq2nEvSw0KW/a OWNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ERW77M0h/A9OK7P5LszqdVgRJRgBCNq3O90vhkzUiv0=; b=VOzNkMYOE5Xlf2eAEIiOCb6zv/3t06e2Rd6crfqn9PwMSdM4llcDbmHVwNobcw0c1s xcLEG2I3X7qA6unvFCpk35fKQTXhtupZAhstpXjb2t0Yej7J/Ej8W3XZAdI73lk9D7t7 4IyoDSwv6N4Jfwy0+9u9Eah4oibVY9rVZADT9HgtyxzVFsW11le2J2C9a2FTjP04YjSa Q9QkswEgelDJ8a8X/D4Ma/me6WdNvtehyUWl7voaTYiWuZOFDH9FpUBjQLnnjcAmHaD4 YknD0uzYPBrIcCbmipF5UHmOZLi10ELinKMvLkrV3avSXjryKCUxpuJxtgapMVnc1DVu mjkw== X-Gm-Message-State: AElRT7HoHlCS6gryhBDtI5XjFQMKN8tnu3ut2Ebjpz7u9t5EqMIET2qD MjVUdnSRgTjdj4krkVYZN1wti53d X-Received: by 10.28.9.81 with SMTP id 78mr3531385wmj.61.1520793028331; Sun, 11 Mar 2018 11:30:28 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id 31sm5064514wrr.59.2018.03.11.11.30.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Mar 2018 11:30:27 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 11 Mar 2018 18:30:17 +0000 Message-Id: <20180311183021.25556-4-sw@jkqxz.net> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180311183021.25556-1-sw@jkqxz.net> References: <20180311183021.25556-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 4/8] h264_metadata: Add support for A/53 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 121 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 36047887ca..d340c55990 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -77,6 +77,8 @@ typedef struct H264MetadataContext { int display_orientation; double rotate; int flip; + + int a53_cc; } H264MetadataContext; @@ -225,6 +227,8 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) int err, i, j, has_sps; uint8_t *displaymatrix_side_data = NULL; size_t displaymatrix_side_data_size = 0; + uint8_t *a53_side_data = NULL; + size_t a53_side_data_size = 0; err = ff_bsf_get_packet(bsf, &in); if (err < 0) @@ -514,6 +518,104 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) } } + if (ctx->a53_cc == INSERT) { + uint8_t *data; + int size; + + data = av_packet_get_side_data(in, AV_PKT_DATA_A53_CC, &size); + if (data) { + H264RawSEIPayload payload = { + .payload_type = H264_SEI_TYPE_USER_DATA_REGISTERED, + }; + H264RawSEIUserDataRegistered *udr = + &payload.payload.user_data_registered; + + av_log(bsf, AV_LOG_WARNING, "A53 CC insert: %d bytes.\n", size); + + udr->data_length = size + 10; + udr->data_ref = av_buffer_alloc(udr->data_length); + if (!udr->data_ref) { + err = AVERROR(ENOMEM); + goto fail; + } + udr->data = udr->data_ref->data; + + udr->itu_t_t35_country_code = 181; + udr->data[0] = 0; + udr->data[1] = 49; + AV_WB32(udr->data + 2, MKBETAG('G', 'A', '9', '4')); + udr->data[6] = 3; + udr->data[7] = ((size / 3) & 0x1f) | 0x40; + udr->data[8] = 0; + memcpy(udr->data + 9, data, size); + udr->data[size + 9] = 0xff; + + err = ff_cbs_h264_add_sei_message(ctx->cbc, au, &payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " + "message to access unit.\n"); + av_buffer_unref(&udr->data_ref); + goto fail; + } + } + + } else if (ctx->a53_cc == REMOVE || ctx->a53_cc == EXTRACT) { + for (i = 0; i < au->nb_units; i++) { + H264RawSEI *sei; + if (au->units[i].type != H264_NAL_SEI) + continue; + sei = au->units[i].content; + + for (j = 0; j < sei->payload_count; j++) { + H264RawSEIUserDataRegistered *udr; + uint32_t tag; + uint8_t type_code, count; + + if (sei->payload[j].payload_type != + H264_SEI_TYPE_USER_DATA_REGISTERED) + continue; + udr = &sei->payload[j].payload.user_data_registered; + tag = AV_RB32(udr->data + 2); + type_code = udr->data[6]; + if (tag != MKBETAG('G', 'A', '9', '4') || type_code != 3) + continue; + + if (ctx->a53_cc == REMOVE) { + err = ff_cbs_h264_delete_sei_message(ctx->cbc, au, + &au->units[i], j); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to delete " + "A53 CC SEI message.\n"); + goto fail; + } + av_log(bsf, AV_LOG_WARNING, "A53 CC remove!.\n"); + + --i; + break; + } + + // Extract. + count = udr->data[7] & 0x1f; + if (3 * count + 10 > udr->data_length) { + av_log(bsf, AV_LOG_ERROR, "Invalid A/53 closed caption " + "data: count %d overflows length %zu.\n", + count, udr->data_length); + continue; + } + + av_log(bsf, AV_LOG_WARNING, "A53 CC extract: %zu bytes.\n", udr->data_length); + + err = av_reallocp(&a53_side_data, + a53_side_data_size + 3 * count); + if (err) + goto fail; + memcpy(a53_side_data + a53_side_data_size, + udr->data + 9, 3 * count); + a53_side_data_size += 3 * count; + } + } + } + err = ff_cbs_write_packet(ctx->cbc, out, au); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); @@ -535,6 +637,16 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) } displaymatrix_side_data = NULL; } + if (a53_side_data) { + err = av_packet_add_side_data(out, AV_PKT_DATA_A53_CC, + a53_side_data, a53_side_data_size); + if (err) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted A/53 " + "side data to packet.\n"); + goto fail; + } + a53_side_data = NULL; + } ctx->done_first_au = 1; @@ -542,6 +654,7 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) fail: ff_cbs_fragment_uninit(ctx->cbc, au); av_freep(&displaymatrix_side_data); + av_freep(&a53_side_data); av_packet_free(&in); @@ -670,6 +783,14 @@ static const AVOption h264_metadata_options[] = { { "vertical", "Set ver_flip", 0, AV_OPT_TYPE_CONST, { .i64 = FLIP_VERTICAL }, .unit ="flip" }, + { "a53_cc", "A/53 Closed Captions in SEI NAL units", + OFFSET(a53_cc), AV_OPT_TYPE_INT, + { .i64 = PASS }, PASS, EXTRACT, 0, "a53_cc" }, + { "pass", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = PASS }, .unit = "a53_cc" }, + { "insert", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = INSERT }, .unit = "a53_cc" }, + { "remove", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = REMOVE }, .unit = "a53_cc" }, + { "extract", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = EXTRACT }, .unit = "a53_cc" }, + { NULL } };