From patchwork Sun Mar 25 17:41:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 8161 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp2707146jad; Sun, 25 Mar 2018 10:41:59 -0700 (PDT) X-Google-Smtp-Source: AIpwx4/4NAs/zLMAxPIfzEn7FiRj849AF0DPGXVur4aXQvUbOE9W+4zqvxo+nv69UyjWlaMm/8iN X-Received: by 10.223.164.28 with SMTP id d28mr1769362wra.113.1521999719281; Sun, 25 Mar 2018 10:41:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521999719; cv=none; d=google.com; s=arc-20160816; b=XNwUN+57bOvSXlIDz2Iflfg+fN5bFIq8ICVyNDIJZlqZzNk+DJvijeaB1BoKbTAvmO SUihL/JsNoL0sZ4m1TO7ySJsJxF8VYDB95Y/JRL4pWakwDHF7/FChFT3rGwOnST8rzs4 HTbc1qcFofnLCqEimNnyNvB1Z3tPo0L+AK83djqDlA5xacjyMP1xxWPUE9NKLYWfNb46 NOps+F7kJoS2tIMDCeG8R7zr9vS/t6k6L7DZK4pWgU+x5tBMtosSuDDMzk5qh5nkMV0G dimsKnCvjck/dQKCL1bABFK+EDauHiRqIrQrQ9UIwyK/XYZtUbm6W+P3mp6LLPZ80aqc YwuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=nwJLR18b//ySo6rKUsVCuqstmfJOwITIc21pmf2pqoE=; b=B9D1GHJMAEZ6upfNeSQ5+yvC5Cg3koZit14dlB/yKuBv/6nzjp1IKK76IZzg++TFTF Sh3uv25iBvByaI61/7DFIqBlzrs2aBC318gHhbjUqzPssaAzPlVpqumVR0WPkccSz/g0 HwLQRyJT2pPj41doJeDzZkqVRm9TNhFsWtl8MVjtvHk+/CBzf7KbZvPb1e6D9nf9cRtY 9xsatkURal0kfW/UJVJ7Nf3kHjlQMtTPEL8fCfp7EVummXEj/Q6HMCNfcKzfiV4BYcyI HkS4c7QbDs/qbXvbehHB6vygfgthbiR/mzHeTOYfhz9g+Pn0qdJtx/0F1u83LHno6lqF pVvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=npMn3Y8D; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j44si10190580wre.473.2018.03.25.10.41.58; Sun, 25 Mar 2018 10:41:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=npMn3Y8D; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 84BE3680C89; Sun, 25 Mar 2018 20:41:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f45.google.com (mail-wm0-f45.google.com [74.125.82.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1E8B7680375 for ; Sun, 25 Mar 2018 20:41:26 +0300 (EEST) Received: by mail-wm0-f45.google.com with SMTP id t6so11458414wmt.5 for ; Sun, 25 Mar 2018 10:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=gWq3hxuk5WavS9u1T3OmccvFkNaiw/BrPmtGJo1VElw=; b=npMn3Y8DlBxj1BeS0FKg3PZhY2tM1bmbsAUqMP2jlEG5lDx87AdldfXwb245QMsglQ 2MHJicUeNPYwzG3tdR6dXsxS253H7E1grpbV21P1j8wJciXkvXrj9v4lZoL9MZm8YW+G EcUNN18kCNE8HVWCMfnq1VkLBhUeYV172oP39hgJDVmxjpnNo04YkufPUhzz1N1J3vXU KYM2C2oT7S//50c1O94oOpHmS+zc7ahdhOW5/NXKSG68+fQwnYvLJMj+bCzgu0vLyPTA PaCuK+DF5rf5yEi1qn/6t2lmVqJTJ+jLknVl5YStafiv3BxYaJBhiDmOiBhxBbSYatet E6EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=gWq3hxuk5WavS9u1T3OmccvFkNaiw/BrPmtGJo1VElw=; b=RItE3n0cNKD/tMqtsSO4zWDpMdmKi//d04MFgl49A8GQ8oBXNdXuns4Zyoc0FNJdYj DqlUz9pUyeAwnBXrqrf79X+OBPPjgvY5SbanS0yyTk0uaL8vicHMaFWr6FTjF7JYqV+x ZHPaivMAiZk3KexNbh8TewekkobrIdPC+AXAaFU/LnQ3mdgsrTHq4bgTnnQ2vBn6zA0S C1Zhp+jY4qoTvuuYmNZK26KstxdCsL1iA0uzJuOPCHT+6yDSwz/1xXPSp5k8TX/M0XAo lbuLIuVh70ToiSOYlbX8RDNyZD5iwNE4RBWESNnwSRca5Wqt242Q9eBW5qDFe1jV8brx EMqg== X-Gm-Message-State: AElRT7FS69bOlgqQW+xV5go0GxctiYqK150hGNe3m4kwi+T5X2jt/a1T +tMZ4U+PTSvi8zpGhmMu8bJ3KzfZ X-Received: by 10.28.245.9 with SMTP id t9mr14127099wmh.33.1521999702192; Sun, 25 Mar 2018 10:41:42 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id x78sm23123579wmd.2.2018.03.25.10.41.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Mar 2018 10:41:41 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 25 Mar 2018 18:41:34 +0100 Message-Id: <20180325174137.14749-2-sw@jkqxz.net> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180325174137.14749-1-sw@jkqxz.net> References: <20180325174137.14749-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 2/5] h264_metadata: Add support for A/53 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Allows insertion (from side data), extraction (to side data), and removal of closed captions in SEI messages. --- libavcodec/Makefile | 2 +- libavcodec/h264_metadata_bsf.c | 138 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+), 1 deletion(-) diff --git a/libavcodec/Makefile b/libavcodec/Makefile index aaef6c3ab8..cfde104055 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1044,7 +1044,7 @@ OBJS-$(CONFIG_DCA_CORE_BSF) += dca_core_bsf.o OBJS-$(CONFIG_EXTRACT_EXTRADATA_BSF) += extract_extradata_bsf.o \ h2645_parse.o OBJS-$(CONFIG_FILTER_UNITS_BSF) += filter_units_bsf.o -OBJS-$(CONFIG_H264_METADATA_BSF) += h264_metadata_bsf.o +OBJS-$(CONFIG_H264_METADATA_BSF) += h264_metadata_bsf.o cbs_misc.o OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF) += h264_mp4toannexb_bsf.o OBJS-$(CONFIG_H264_REDUNDANT_PPS_BSF) += h264_redundant_pps_bsf.o OBJS-$(CONFIG_HAPQA_EXTRACT_BSF) += hapqa_extract_bsf.o hap.o diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 27053dbdcf..d79e6c0c87 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -24,6 +24,7 @@ #include "bsf.h" #include "cbs.h" #include "cbs_h264.h" +#include "cbs_misc.h" #include "h264.h" #include "h264_sei.h" @@ -74,6 +75,8 @@ typedef struct H264MetadataContext { int display_orientation; double rotate; int flip; + + int a53_cc; } H264MetadataContext; @@ -222,6 +225,8 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) int err, i, j, has_sps; uint8_t *displaymatrix_side_data = NULL; size_t displaymatrix_side_data_size = 0; + uint8_t *a53_side_data = NULL; + size_t a53_side_data_size = 0; err = ff_bsf_get_packet(bsf, &in); if (err < 0) @@ -516,6 +521,116 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) } } + if (ctx->a53_cc == INSERT) { + uint8_t *data; + int size; + + data = av_packet_get_side_data(in, AV_PKT_DATA_A53_CC, &size); + if (data) { + A53UserData a53_ud; + + err = ff_cbs_read_a53_cc_side_data(ctx->cbc, &a53_ud, + data, size); + if (err < 0) { + av_log(bsf, AV_LOG_WARNING, "Invalid A/53 closed captions " + "in packet side data dropped.\n"); + } else { + H264RawSEIPayload payload = { + .payload_type = H264_SEI_TYPE_USER_DATA_REGISTERED, + }; + H264RawSEIUserDataRegistered *udr = + &payload.payload.user_data_registered; + size_t size = 9 + 3 * a53_ud.atsc.cc_data.cc_count; + + udr->data_ref = av_buffer_alloc(2 + size); + if (!udr->data_ref) { + err = AVERROR(ENOMEM); + goto fail; + } + udr->data = udr->data_ref->data; + + udr->itu_t_t35_country_code = 181; + AV_WB16(udr->data, 49); // provider_code + + err = ff_cbs_write_a53_user_data(ctx->cbc, udr->data + 2, + &size, &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data.\n"); + av_buffer_unref(&udr->data_ref); + goto fail; + } + udr->data_length = size + 2; + + err = ff_cbs_h264_add_sei_message(ctx->cbc, au, &payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add A/53 user data " + "SEI message to access unit.\n"); + av_buffer_unref(&udr->data_ref); + goto fail; + } + } + } + + } else if (ctx->a53_cc == REMOVE || ctx->a53_cc == EXTRACT) { + for (i = 0; i < au->nb_units; i++) { + H264RawSEI *sei; + if (au->units[i].type != H264_NAL_SEI) + continue; + sei = au->units[i].content; + + for (j = 0; j < sei->payload_count; j++) { + H264RawSEIUserDataRegistered *udr; + A53UserData a53_ud; + + if (sei->payload[j].payload_type != + H264_SEI_TYPE_USER_DATA_REGISTERED) + continue; + udr = &sei->payload[j].payload.user_data_registered; + if (udr->data_length < 6) { + // Can't be relevant. + continue; + } + + err = ff_cbs_read_a53_user_data(ctx->cbc, &a53_ud, + udr->data + 2, + udr->data_length - 2); + if (err < 0) { + // Invalid or something completely different. + continue; + } + if (a53_ud.user_identifier != A53_USER_IDENTIFIER_ATSC || + a53_ud.atsc.user_data_type_code != + A53_USER_DATA_TYPE_CODE_CC_DATA) { + // Valid but something else (e.g. AFD). + continue; + } + + if (ctx->a53_cc == REMOVE) { + err = ff_cbs_h264_delete_sei_message(ctx->cbc, au, + &au->units[i], j); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to delete " + "A/53 CC SEI message.\n"); + goto fail; + } + --i; + break; + } else if(ctx->a53_cc == EXTRACT) { + err = ff_cbs_write_a53_cc_side_data(ctx->cbc, + &a53_side_data, + &a53_side_data_size, + &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data for packet side data.\n"); + goto fail; + } + } + } + } + } + err = ff_cbs_write_packet(ctx->cbc, out, au); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); @@ -537,6 +652,16 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) } displaymatrix_side_data = NULL; } + if (a53_side_data) { + err = av_packet_add_side_data(out, AV_PKT_DATA_A53_CC, + a53_side_data, a53_side_data_size); + if (err) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted A/53 " + "side data to packet.\n"); + goto fail; + } + a53_side_data = NULL; + } ctx->done_first_au = 1; @@ -544,6 +669,7 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *out) fail: ff_cbs_fragment_uninit(ctx->cbc, au); av_freep(&displaymatrix_side_data); + av_freep(&a53_side_data); if (err < 0) av_packet_unref(out); @@ -684,6 +810,18 @@ static const AVOption h264_metadata_options[] = { 0, AV_OPT_TYPE_CONST, { .i64 = FLIP_VERTICAL }, .flags = FLAGS, .unit = "flip" }, + { "a53_cc", "A/53 Closed Captions in SEI NAL units", + OFFSET(a53_cc), AV_OPT_TYPE_INT, + { .i64 = PASS }, PASS, EXTRACT, FLAGS, "a53_cc" }, + { "pass", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = PASS }, .flags = FLAGS, .unit = "a53_cc" }, + { "insert", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = INSERT }, .flags = FLAGS, .unit = "a53_cc" }, + { "remove", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = REMOVE }, .flags = FLAGS, .unit = "a53_cc" }, + { "extract", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = EXTRACT }, .flags = FLAGS, .unit = "a53_cc" }, + { NULL } };