From patchwork Sat Feb 4 00:41:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 40239 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1616987pzk; Fri, 3 Feb 2023 16:42:21 -0800 (PST) X-Google-Smtp-Source: AK7set+hl5B7/xV9cglEw6s59gCQxCQyVUys818dcP/v3bOdS0Io2RMhmhZ5d12gC76VY9bjTyXm X-Received: by 2002:a17:906:9c8d:b0:878:78bc:975c with SMTP id fj13-20020a1709069c8d00b0087878bc975cmr10733367ejc.36.1675471341370; Fri, 03 Feb 2023 16:42:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675471341; cv=none; d=google.com; s=arc-20160816; b=w83GWsZvdSenCLxfto5ZhaNdLVxWPU1khZmsUPxR6OCkOMKB5l18z/0V8sObTYqVlh 98iPqWy1HsiC2+C07ukyX4p9GXU29/xRoZ1SDz82fqj5rc9XsnC+M+1bEaESy0hLKye0 wOb/EF8ypnTE1U/cIPZ4FhIj/SyyRfDiyFrveVnlV3sTDGlIseFaFxhMf2A6VcHCvqdU MTwA4ftVCWjqR2t2j7KfQNCSQCLfoWE5LuUZIbZqGWB+0bc/qaIyOp9hZAqFqz3GwDBH oXucxDKqwIvmS7Av/sqxuZW+tp3VvXUPCquYs0g9SfjciisO7Drz/zLSCUeaY+eBbRqe rTfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=mZ0yQHNJOawrgLLYMsBkB4OHIrB+//6S+xz6Mzu8Wqg=; b=LjdIQbOvHGADg9FllfKXlScOtFDZGF69x2JljTBZRwBRwxLJoFyPjnAnN+W3qgyjWZ 1O0xfIjPsWO/CZ69IjmropqKNP1p0bKg10Go9G2lT2zYEBa+iwfb6ha/nALzXiRiJxAw ezYaDzlAjs2wlJ2pz2gZQWF2Opm5+QArpqKffdCRpXIs0dZh5yXSwANyZjtHzxC2RA4h JDOw2d10H6wVteG8u/2p9MRXfRjRaGsYKlR5k+D+GO43+HGAHWbMIRnmDqQsJL5z1Pcf ugTq9gc5QGRjUMKkAHOhh3c9/3szUGDmXDnjMiuGqv6D6TwFxc2+Jxmf/OnJCjJi4Cd1 soTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=ZRdF24wy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j9-20020a170906104900b0088a161c2342si5343459ejj.192.2023.02.03.16.42.21; Fri, 03 Feb 2023 16:42:21 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=ZRdF24wy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E791768BE88; Sat, 4 Feb 2023 02:42:06 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCEAE68BC53 for ; Sat, 4 Feb 2023 02:41:59 +0200 (EET) Received: by mail-pj1-f48.google.com with SMTP id nm12-20020a17090b19cc00b0022c2155cc0bso6452653pjb.4 for ; Fri, 03 Feb 2023 16:41:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=3R+UpWpyPt3/kwa9pCa5gPZIcV7gSjTdcwTllCOEiyo=; b=ZRdF24wyk4FDvJyPVqxWLTVu0qRtRsuJuU3WGuU3EyLCLPtk3QySOcXKb04Pqmu9uV BhpTTUDvSk0dIJJ7BPZ3prj3mq9NKN1cWJyvYcpSzD+mz7ohERyrZBiGrGki3A4eL4PE iysgZGnV3QF/oatSwfOq/JAlpRTQmfTJ1TMUV+Hxb/R1tTm4Eys8DrL2HOyJm0FrhKFX U6JCAziNgA3ZuzwD0ggpK2ekTgugtSjXote3UBuDxBekeqrqc5niIeP/X21c5qo8Xiax ItKcU16VJDIdNlOlvLzPLhjhgKXPBVS8vkCLBMkAP/xfAOiBzIPsrUEo/wOVC8i8uRy4 vdnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=3R+UpWpyPt3/kwa9pCa5gPZIcV7gSjTdcwTllCOEiyo=; b=JucOP8wOvmrZbKqsLniN0akeX5V53dqhvlP1JJ2l8VLDl/9cVhzr6bjhbT+LpEXlto qY7SARjebwmCZg59XhwX6XBEUiXW+JKc/3Qgu5PoY2v59Udu775fots37XeB/nMSITGF B3j8Eag6devr+xFxu1ch+JOioP1IHDM982pAezKaUPSklKpLSbaUACfsiOPmLuan02tO F6hsRHuzk4KnRsscTAIJ3RBiQu46c3ULsynYhFVc1xYYCsiY7vUmSe3CpD0usGqznC6j THVzr8XSn8Dy+XFHGemA1DrnDy/h6hs1B1oIx3XyCh4vKjRzso8y3NBmBezyCItR9fpo uDyQ== X-Gm-Message-State: AO0yUKXz5S63OfnLuyhQXklQfkdZ6695z3SqbTwukQsrG6M2FfWMnhJD 7ymHzQ0rubzCNQDXdXF9EXrQfyLm6PM= X-Received: by 2002:a17:902:db12:b0:198:f13d:1208 with SMTP id m18-20020a170902db1200b00198f13d1208mr688114plx.61.1675471317662; Fri, 03 Feb 2023 16:41:57 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id u13-20020aa7848d000000b00593ce42db6bsm2434757pfn.59.2023.02.03.16.41.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Feb 2023 16:41:57 -0800 (PST) From: Mark Thompson X-Google-Original-From: Mark Thompson Message-Id: <9a539e4caab098bc69af9ec30d19608372e3ddd6.1675471315.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Sat, 04 Feb 2023 00:41:51 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/5] cbs: Add some common code for read/write of miscellaneous user data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson , Aman Gupta Karmani Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5wK+oc8B7dPB From: Mark Thompson Supports closed captions, active format and bar data as defined by SCTE 128 part 1 or A/53 part 4, suitable for use with both MPEG-2 and H.264. Signed-off-by: Aman Karmani --- libavcodec/cbs_misc.c | 217 ++++++++++++++++++++++++++ libavcodec/cbs_misc.h | 109 +++++++++++++ libavcodec/cbs_misc_syntax_template.c | 150 ++++++++++++++++++ 3 files changed, 476 insertions(+) create mode 100644 libavcodec/cbs_misc.c create mode 100644 libavcodec/cbs_misc.h create mode 100644 libavcodec/cbs_misc_syntax_template.c diff --git a/libavcodec/cbs_misc.c b/libavcodec/cbs_misc.c new file mode 100644 index 0000000000..d0ced562f5 --- /dev/null +++ b/libavcodec/cbs_misc.c @@ -0,0 +1,217 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/attributes.h" +#include "libavutil/avassert.h" + +#include "cbs.h" +#include "cbs_internal.h" +#include "cbs_misc.h" + +#define CHECK(call) do { \ + err = (call); \ + if (err < 0) \ + return err; \ + } while (0) + +#define FUNC_NAME(rw, codec, name) cbs_ ## codec ## _ ## rw ## _ ## name +#define FUNC_MISC(rw, name) FUNC_NAME(rw, misc, name) +#define FUNC(name) FUNC_MISC(READWRITE, name) + + +#define READWRITE read +#define RWContext GetBitContext + +#define xui(width, name, var) do { \ + uint32_t value = 0; \ + CHECK(ff_cbs_read_unsigned(ctx, rw, width, #name, NULL, \ + &value, 0, MAX_UINT_BITS(width))); \ + var = value; \ + } while (0) + +#define ui(width, name) \ + xui(width, name, current->name) + +#define fixed(width, name, expected) do { \ + av_unused uint32_t value; \ + CHECK(ff_cbs_read_unsigned(ctx, rw, width, #name, NULL, \ + &value, expected, expected)); \ + } while (0) + +#include "cbs_misc_syntax_template.c" + +#undef READWRITE +#undef RWContext +#undef xui +#undef ui +#undef fixed + + +#define READWRITE write +#define RWContext PutBitContext + +#define xui(width, name, var) do { \ + CHECK(ff_cbs_write_unsigned(ctx, rw, width, #name, NULL, \ + var, 0, MAX_UINT_BITS(width))); \ + } while (0) + +#define ui(width, name) \ + xui(width, name, current->name) + +#define fixed(width, name, value) do { \ + CHECK(ff_cbs_write_unsigned(ctx, rw, width, #name, NULL, \ + value, value, value)); \ + } while (0) + +#include "cbs_misc_syntax_template.c" + +#undef READWRITE +#undef RWContext +#undef xui +#undef ui +#undef fixed + + +int ff_cbs_read_a53_user_data(CodedBitstreamContext *ctx, + A53UserData *data, + const uint8_t *read_buffer, size_t length) +{ + GetBitContext gbc; + int err; + + err = init_get_bits(&gbc, read_buffer, 8 * length); + if (err < 0) + return err; + + return cbs_misc_read_a53_user_data(ctx, &gbc, data); +} + +int ff_cbs_write_a53_user_data(CodedBitstreamContext *ctx, + uint8_t *write_buffer, size_t *length, + A53UserData *data) +{ + PutBitContext pbc; + int err; + + init_put_bits(&pbc, write_buffer, *length); + + err = cbs_misc_write_a53_user_data(ctx, &pbc, data); + if (err < 0) { + // Includes AVERROR(ENOSPC). + return err; + } + + // That output must be aligned. + av_assert0(put_bits_count(&pbc) % 8 == 0); + + *length = put_bits_count(&pbc) / 8; + + flush_put_bits(&pbc); + + return 0; +} + +int ff_cbs_read_a53_cc_side_data(CodedBitstreamContext *ctx, + A53UserData *data, + const uint8_t *side_data, + size_t side_data_size) +{ + GetBitContext gbc; + CEA708CCData *cc; + int err, i, cc_count; + + if (side_data_size % 3) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "A53 CC side data length must " + "be a multiple of 3 (got %zu).\n", side_data_size); + return AVERROR(EINVAL); + } + cc_count = side_data_size / 3; + if (cc_count > 31) { + av_log(ctx->log_ctx, AV_LOG_ERROR, "A53 CC can only fit 31 packets " + "in a single user data block (got %d).\n", cc_count); + return AVERROR(EINVAL); + } + + *data = (A53UserData) { + .user_identifier = A53_USER_IDENTIFIER_ATSC, + + .atsc = { + .user_data_type_code = A53_USER_DATA_TYPE_CODE_CC_DATA, + + .cc_data = { + .process_em_data_flag = 0, + .process_cc_data_flag = 1, + .additional_data_flag = 0, + + .em_data = 0, + + .cc_count = cc_count, + }, + }, + }; + cc = &data->atsc.cc_data; + + err = init_get_bits(&gbc, side_data, 8 * side_data_size); + if (err < 0) + return err; + + for (i = 0; i < cc->cc_count; i++) { + err = cbs_misc_read_cea708_cc_data_packet(ctx, &gbc, + &cc->cc_data_pkts[i]); + if (err < 0) + return err; + } + + return 0; +} + +int ff_cbs_write_a53_cc_side_data(CodedBitstreamContext *ctx, + uint8_t **side_data, + size_t *side_data_size, + A53UserData *data) +{ + PutBitContext pbc; + CEA708CCData *cc; + int err, i; + + if (data->user_identifier != A53_USER_IDENTIFIER_ATSC || + data->atsc.user_data_type_code != A53_USER_DATA_TYPE_CODE_CC_DATA) + return AVERROR(EINVAL); + + cc = &data->atsc.cc_data; + + err = av_reallocp(side_data, *side_data_size + 3 * cc->cc_count); + if (err < 0) + return err; + + init_put_bits(&pbc, *side_data + *side_data_size, 3 * cc->cc_count); + + for (i = 0; i < cc->cc_count; i++) { + err = cbs_misc_write_cea708_cc_data_packet(ctx, &pbc, + &cc->cc_data_pkts[i]); + if (err < 0) { + av_freep(side_data); + return err; + } + } + + flush_put_bits(&pbc); + *side_data_size += 3 * cc->cc_count; + + return 0; +} diff --git a/libavcodec/cbs_misc.h b/libavcodec/cbs_misc.h new file mode 100644 index 0000000000..0d7ab2c8e7 --- /dev/null +++ b/libavcodec/cbs_misc.h @@ -0,0 +1,109 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_MISC_H +#define AVCODEC_CBS_MISC_H + +#include +#include + +#include "libavutil/common.h" + + +enum { + A53_USER_IDENTIFIER_ATSC = MKBETAG('G', 'A', '9', '4'), + A53_USER_IDENTIFIER_AFD = MKBETAG('D', 'T', 'G', '1'), +}; + +enum { + A53_USER_DATA_TYPE_CODE_CC_DATA = 0x03, + A53_USER_DATA_TYPE_CODE_BAR_DATA = 0x06, +}; + +typedef struct A53BarData { + uint8_t top_bar_flag; + uint8_t bottom_bar_flag; + uint8_t left_bar_flag; + uint8_t right_bar_flag; + + uint16_t line_number_end_of_top_bar; + uint16_t line_number_end_of_bottom_bar; + uint16_t line_number_end_of_left_bar; + uint16_t line_number_end_of_right_bar; +} A53BarData; + +typedef struct CEA708CCDataPacket { + uint8_t cc_valid; + uint8_t cc_type; + uint8_t cc_data_1; + uint8_t cc_data_2; +} CEA708CCDataPacket; + +typedef struct CEA708CCData { + uint8_t process_em_data_flag; + uint8_t process_cc_data_flag; + uint8_t additional_data_flag; + + uint8_t em_data; + + uint8_t cc_count; + CEA708CCDataPacket cc_data_pkts[31]; +} CEA708CCData; + +typedef struct A53ATSCUserData { + uint8_t user_data_type_code; + union { + CEA708CCData cc_data; + A53BarData bar_data; + }; +} A53ATSCUserData; + +typedef struct A53AFDData { + uint8_t active_format_flag; + uint8_t active_format; +} A53AFDData; + +typedef struct A53UserData { + uint32_t user_identifier; + union { + A53ATSCUserData atsc; + A53AFDData afd; + }; +} A53UserData; + + +int ff_cbs_read_a53_user_data(CodedBitstreamContext *ctx, + A53UserData *data, + const uint8_t *read_buffer, size_t length); + +int ff_cbs_write_a53_user_data(CodedBitstreamContext *ctx, + uint8_t *write_buffer, size_t *length, + A53UserData *data); + +int ff_cbs_read_a53_cc_side_data(CodedBitstreamContext *ctx, + A53UserData *data, + const uint8_t *side_data, + size_t side_data_size); + +int ff_cbs_write_a53_cc_side_data(CodedBitstreamContext *ctx, + uint8_t **side_data, + size_t *side_data_length, + A53UserData *data); + + +#endif /* AVCODEC_CBS_MISC_H */ diff --git a/libavcodec/cbs_misc_syntax_template.c b/libavcodec/cbs_misc_syntax_template.c new file mode 100644 index 0000000000..7b98c7cc85 --- /dev/null +++ b/libavcodec/cbs_misc_syntax_template.c @@ -0,0 +1,150 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +static int FUNC(a53_bar_data)(CodedBitstreamContext *ctx, RWContext *rw, + A53BarData *current) +{ + int err; + + ui(1, top_bar_flag); + ui(1, bottom_bar_flag); + ui(1, left_bar_flag); + ui(1, right_bar_flag); + fixed(4, reserved, 0xf); + + if (current->top_bar_flag) { + fixed(2, one_bits, 3); + ui(14, line_number_end_of_top_bar); + } + if (current->bottom_bar_flag) { + fixed(2, one_bits, 3); + ui(14, line_number_end_of_bottom_bar); + } + if (current->left_bar_flag) { + fixed(2, one_bits, 3); + ui(14, line_number_end_of_left_bar); + } + if (current->right_bar_flag) { + fixed(2, one_bits, 3); + ui(14, line_number_end_of_right_bar); + } + + return 0; +} + +static int FUNC(cea708_cc_data_packet)(CodedBitstreamContext *ctx, + RWContext *rw, + CEA708CCDataPacket *current) +{ + int err; + + fixed(5, marker_bits, 0x1f); + ui(1, cc_valid); + ui(2, cc_type); + + ui(8, cc_data_1); + ui(8, cc_data_2); + + return 0; +} + +static int FUNC(cea708_cc_data)(CodedBitstreamContext *ctx, RWContext *rw, + CEA708CCData *current) +{ + int err, i; + + ui(1, process_em_data_flag); + ui(1, process_cc_data_flag); + ui(1, additional_data_flag); + + ui(5, cc_count); + + ui(8, em_data); + + for (i = 0; i < current->cc_count; i++) { + CHECK(FUNC(cea708_cc_data_packet)(ctx, rw, + ¤t->cc_data_pkts[i])); + } + + fixed(8, marker_bits, 0xff); + + if (current->additional_data_flag) { + // Ignored. + } + + return 0; +} + +static int FUNC(a53_atsc_user_data)(CodedBitstreamContext *ctx, RWContext *rw, + A53ATSCUserData *current) +{ + int err; + + ui(8, user_data_type_code); + + switch (current->user_data_type_code) { + case A53_USER_DATA_TYPE_CODE_CC_DATA: + return FUNC(cea708_cc_data)(ctx, rw, ¤t->cc_data); + case A53_USER_DATA_TYPE_CODE_BAR_DATA: + return FUNC(a53_bar_data)(ctx, rw, ¤t->bar_data); + default: + av_log(ctx->log_ctx, AV_LOG_WARNING, + "Unknown ATSC user data found: type code %#02x.\n", + current->user_data_type_code); + } + + return 0; +} + +static int FUNC(a53_afd_data)(CodedBitstreamContext *ctx, RWContext *rw, + A53AFDData *current) +{ + int err; + + fixed(1, zero_bit, 0); + ui(1, active_format_flag); + fixed(6, alignment_bits, 1); + + if (current->active_format_flag) { + fixed(4, reserved, 0xf); + ui(4, active_format); + } + + return 0; +} + +static int FUNC(a53_user_data)(CodedBitstreamContext *ctx, RWContext *rw, + A53UserData *current) +{ + int err; + + ui(32, user_identifier); + + switch (current->user_identifier) { + case A53_USER_IDENTIFIER_ATSC: + return FUNC(a53_atsc_user_data)(ctx, rw, ¤t->atsc); + case A53_USER_IDENTIFIER_AFD: + return FUNC(a53_afd_data)(ctx, rw, ¤t->afd); + default: + av_log(ctx->log_ctx, AV_LOG_WARNING, + "Unknown registered user data found: identifier %#08x.\n", + current->user_identifier); + } + + return 0; +} From patchwork Sat Feb 4 00:41:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 40240 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1617041pzk; Fri, 3 Feb 2023 16:42:30 -0800 (PST) X-Google-Smtp-Source: AK7set/XhvKW0wzzJ4H1vmqqiPc+9WiSfHP1RqDwzaV51P/esTjCkuyp4v53P8HNAPPGQ5JmZcMI X-Received: by 2002:a17:906:8a5b:b0:7c0:efb9:bc0e with SMTP id gx27-20020a1709068a5b00b007c0efb9bc0emr12309583ejc.62.1675471350504; Fri, 03 Feb 2023 16:42:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675471350; cv=none; d=google.com; s=arc-20160816; b=I/Mr11cfxvAaJfuSmYRS7LUfy2wRWZU+qadfuX5aWz2Cfu4XHxQQG1FPphyvmsD1lJ N4cjM6ZDq/RTjRgkNp6pyq1/Fq+2ALiBZpdxX+j8f2KZE4uF47MSdw6+l8yMlEtKK7fj 5hoDC7V62Z9DKyNrLaS+rohV8lGhHN5QcNI/P8pzzIa256ptW0VYaGTBfH9+6/lW0gAY LeBidW4mojS8pZaZc7rkrW+660aX0+X1oQXZpUS4szdiRA+7oUfiIfscBtpNDdzDIiyC wB3ieZBgMfgIiz55dQH9TBL+g8kKkKse2IupFXbl7f7FxnCrlmBNdEP3T37T39mSuarZ GLTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=Wos3Xse0rL0yn2IO6DRhn4ieRTu41k9Hy0KBQE93W5c=; b=BDST27G1Uo+oyjnDRAkQ6zlWXhefYEUP9IXs+qZpO4HgqJ/N7qBQKb0v6H4zXkEqmu BqPBXJ03aPM1gFlLypzsBV9yeNthGdTEh3BbUn4mMoGrlbWin6xpeMFcSmC91cWiDydi sXsNtWPA6p6Ndk7abcvyaKs1GnswM8UpcfU03G4GAcLz+NSnuDEU+IKJ2JDuCtFhswlU Il3+jVa8+OjF8OCOmcM6TeZ0gegnAPK8Z1jubIpzuiBulWYwZ4KgS+IemPj99BY5Wi/m z1Dt9nzxvn+NDKZ3QAVlkw9lzYTWTZ1Qg5I09abntNHAEdYxVS3bAtP0AkHJMdoo1RhX 7R/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=fIKHg9s1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id su21-20020a17090703d500b0088458045d8fsi4000546ejb.639.2023.02.03.16.42.30; Fri, 03 Feb 2023 16:42:30 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=fIKHg9s1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 13A4F68BEAA; Sat, 4 Feb 2023 02:42:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3CA4368B6C4 for ; Sat, 4 Feb 2023 02:42:00 +0200 (EET) Received: by mail-pg1-f176.google.com with SMTP id 78so4759286pgb.8 for ; Fri, 03 Feb 2023 16:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=kYWLa0Lt1EGmmEwqLeYWzotFAEn+MZ5WCTz/uSjUrSY=; b=fIKHg9s1LteZsL75wMP9WJG3BgEGOUg9GUn4VyxLuoaA4YYzUH2W1ZErX9N3lotVbe 3LkCtrYqCl0IOLrFwOZVyT6bUXzljcVHLZoeUiXARRFqoyB1k6viA6p7W2vu37ugxQ0X ujsQsu+og+SS4K5FknwVTN37blZHSezeQ2TxF8dDx5JELjU6iv4lAxvVKIl/bM0SiLKb mrrUyVzvl1BUkvLiMn5CXN37R+jvI2bl9i4Ihz8dBJfr+Ke+dR3VeFRSI+UeliEs27+p 7j4d0+rOIUVltkVkk96Z2O+M8LPVrG1bYaXGxDzYejNjWQ8oe4/5l99wxZEzOeahLLiQ cJqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=kYWLa0Lt1EGmmEwqLeYWzotFAEn+MZ5WCTz/uSjUrSY=; b=62acrfl3lmhqssEgV1SRPZagDMNzpsDyFb/3dJIuGyrGseyEKXl4hzUVTV7z5WZ7ca M+39mRcndXZa5iJwiPy1llVygOUe9/3cI6f6OSiTdk4D6O9TM3WwZuAIuuss300OU3JS 21LUX9I1/Y9kmrBcT20H0u7Y9xhH0rF1ikK0aiOx1Aja7kA1B/vtw57WywDbr9pfndHO 2/qHCLe3Rle4bBSIOSgCgFQpCycW6fgDWE3QSMYrfPtSP5z3CX4LLMji6/L9HwvKlHCD Rv6JCPgHuhZQkP0mwhPKmluzFoEOoF+JFCkq39JPVOBnr/Ug2M+wvsh5cimTGj3AX09z 4kTA== X-Gm-Message-State: AO0yUKWvDLhgudipL5Caic/OCiOWpY+ha81UPYMbFZAow2pq6bSjh6Se TD1sfDAZU4XAdx4CjFQjQHxbEY3bwQk= X-Received: by 2002:a05:6a00:4acc:b0:58b:b29c:50c2 with SMTP id ds12-20020a056a004acc00b0058bb29c50c2mr11657299pfb.26.1675471318318; Fri, 03 Feb 2023 16:41:58 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id z15-20020aa791cf000000b00590163e1762sm2400003pfa.200.2023.02.03.16.41.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Feb 2023 16:41:57 -0800 (PST) From: Eric Lindvall X-Google-Original-From: Eric Lindvall Message-Id: <8164f307e2a4c93dc7ad0dbcfb33d493bbede6b3.1675471315.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Sat, 04 Feb 2023 00:41:52 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 2/5] cbs_sei: add ff_cbs_sei_delete_message helper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Eric Lindvall , Aman Gupta Karmani Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: D1aW0RG0Qq2L From: Eric Lindvall Signed-off-by: Aman Karmani --- libavcodec/cbs_sei.c | 25 +++++++++++++++++++++++++ libavcodec/cbs_sei.h | 7 +++++++ 2 files changed, 32 insertions(+) diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 50a513f592..1ea2bbc109 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -347,6 +347,31 @@ static void cbs_sei_delete_message(SEIRawMessageList *list, } } +int ff_cbs_sei_delete_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + SEIRawMessage *message) +{ + int err, i, j; + + for (i = 0; i < au->nb_units; i++) { + CodedBitstreamUnit *unit = &au->units[i]; + SEIRawMessageList *list; + + err = cbs_sei_get_message_list(ctx, unit, &list); + if (err < 0) + continue; + + for (j = 0; j < list->nb_messages; j++) { + if (message == &list->messages[j]) { + cbs_sei_delete_message(list, j); + return 0; + } + } + } + + return AVERROR(ENOENT); +} + void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, CodedBitstreamFragment *au, uint32_t payload_type) diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index 1c327a4689..96bd116a53 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -202,4 +202,11 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, CodedBitstreamFragment *au, uint32_t payload_type); +/** + * Delete a message from the access unit. + */ +int ff_cbs_sei_delete_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + SEIRawMessage *message); + #endif /* AVCODEC_CBS_SEI_H */ From patchwork Sat Feb 4 00:41:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 40241 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1617107pzk; Fri, 3 Feb 2023 16:42:39 -0800 (PST) X-Google-Smtp-Source: AK7set/JhBqjfnrxTltTJLSLpCdNWzA2zr9PSzBA86DkWF6X/nqikfhfMOEu6V1KYgXI+BkZ+PFi X-Received: by 2002:a17:906:7141:b0:877:7157:9358 with SMTP id z1-20020a170906714100b0087771579358mr3860304ejj.10.1675471359374; Fri, 03 Feb 2023 16:42:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675471359; cv=none; d=google.com; s=arc-20160816; b=w71wXiBLNQZ2yGaD/uEBL/DNQEuSV0g8tnW1YKEhvqOL6QI1W4Yet2t8EN+wr8Byfj SsiGw9hQZjSi2CqXmFLy/QqHlD4yUQaRkWjtmUV953R6r3lVWAVPOj06eNg9AVuNI3Rx EVcivbqAO25+fhK1Mcd7FtsOoDjjAzgVGc1dDkNU2kcjHh6PYHZ2s+dTHooSf9fzkkcn XDJZ09zmcEtq+4VJyZ+4rKmbZNM774MvWAxa2rrl1ovLBbsGdbetuwU5V6cHVoFD97V2 tGwHuC5UOL1Dh8ohpFmJAG89M8OvurKxBtdBZ0/oj3lAgXBUTx4Lbz6P2aVVZ1OBrL4d LvzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=PY6MA6QCX+m/KhUPTwYcCfhTcsSNS/33hSxlItDBTL4=; b=v5RoJf+J0Kw8Zv52q1jMJX5G/K2ToBV35y5t8OKtI6Ndp++9djDQOlAopwvc5Cff3a k8v+sWidgWrndh6VTAukKJkvzyJ7t7CVUIL5HUmU/mFDjjQ4cewHPjGiK0nlSZTGpYNs vd2nIJQtyWw3rbF0wwf15AIT2u0Vg6nbrCqwiWedEBSuQY41OpeLAOUA/MxIzASE7xjo oeUHi2U8OPgtL7fM2WXkF+6v+ATHe5EMKQo0oHPCIVQeTNn5TEV0u6JYhfc405ODx1lH 7UOY0iuCpW6AAYFr53A+2t1eWN217KGpVGnwrh+ZqtqHkAlcKrYcxHQ54k4g0oB9N/m5 y6Cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=O73g61Iq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 30-20020a170906209e00b0087a56fcad93si4911430ejq.262.2023.02.03.16.42.38; Fri, 03 Feb 2023 16:42:39 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=O73g61Iq; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1ABD268BEB6; Sat, 4 Feb 2023 02:42:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B6E3168BE80 for ; Sat, 4 Feb 2023 02:42:00 +0200 (EET) Received: by mail-pl1-f182.google.com with SMTP id n13so6927650plf.11 for ; Fri, 03 Feb 2023 16:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=0C/IOruxjajzb2eOlVuufvt3+1AGg1WTFQdZpHki1FI=; b=O73g61Iqqdzyazpph7HF40T0UmhB0EZPojzNCXUSB7g1WshqT1lF8ues+jwlwj6FNb Wy/gVw9czl5/55BWnND8dQ5tGVneeuTLOUgAcxFplGXokctGVDHKBtcbpf3uNaNRBfCd 59azczrkFcBsszSKNhAZ3vzcHcKTjuAUALfOFo6NlQpkecTq+rEPWAfzOSjewAkxFkWu K+5Qyyfiy6TEVNLCmCq3hM9CvsWhqEVYyVxDbaIQ6lLSaDEAIXncJ3IzZfMLieEOYCMV 2vBApqx9e/ywmlGcC5Hzz7LYqdIP17yMg/CnlSTW7SWfItuMW+KmfO7nWgBhanLftlCq DBnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=0C/IOruxjajzb2eOlVuufvt3+1AGg1WTFQdZpHki1FI=; b=UKu9fn4v1kGfg/v/S0FV+9nbelkdUhOyPNz4RPt5gins3ukE81u48DHBbqybNYH0aT TOy6MiebnwpIoJuwDtjPz1MMaE2tTCQxrfAZkiWVexYo0HhhK12a8hL2yIUM2q2zOAxM 7P0o2FNsTIybUeYPD3h9zGK6JQ+32FyXE+wYzJRmgTr1R/BzFtfLINjH8JUebRAABjui RFXgBieeiEZxFAmahmnrRVss/umEo3CH/09XiQM5+DIimqKa4fZ9iO6cvu6YPfnQefyb 6K4K3e4jcK6w7LCSoNF8Ujy5XWoXrGabbXnJRZt2Fc/hS0CYtCykMVYAdT0mjs2OFVet 7L2Q== X-Gm-Message-State: AO0yUKWBwASzrzTBoNecVxTvWen4PCLmfLocLSUOI3lx7s4r/kgFd315 Sbl0DlY6q8osEsTumyg+AM411ldwWZM= X-Received: by 2002:a05:6a21:100d:b0:b9:4afb:1472 with SMTP id nk13-20020a056a21100d00b000b94afb1472mr10841402pzb.13.1675471319048; Fri, 03 Feb 2023 16:41:59 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id p18-20020a170902b09200b0019101215f63sm1328246plr.93.2023.02.03.16.41.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Feb 2023 16:41:58 -0800 (PST) From: Aman Karmani X-Google-Original-From: Aman Karmani Message-Id: In-Reply-To: References: Date: Sat, 04 Feb 2023 00:41:53 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 3/5] avcodec/mpeg2_metadata_bsf: add support for a/53 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Karmani Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NviC3yPQCWBO From: Aman Karmani Signed-off-by: Aman Karmani --- doc/bitstream_filters.texi | 11 +++++ libavcodec/Makefile | 2 +- libavcodec/mpeg2_metadata_bsf.c | 80 +++++++++++++++++++++++++++++++++ 3 files changed, 92 insertions(+), 1 deletion(-) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index c63c20370f..c036d37403 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -537,6 +537,17 @@ Decompress non-standard compressed MP3 audio headers. Modify metadata embedded in an MPEG-2 stream. @table @option +@item a53_cc +Insert or remove user data containing A/53 closed captions. + +@table @samp +@item pass +@item insert +@item remove +@end table + +Default is pass. + @item display_aspect_ratio Set the display aspect ratio in the stream. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 1fb963f820..2a36a31ff2 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1227,7 +1227,7 @@ OBJS-$(CONFIG_MPEG4_UNPACK_BFRAMES_BSF) += mpeg4_unpack_bframes_bsf.o OBJS-$(CONFIG_MOV2TEXTSUB_BSF) += movsub_bsf.o OBJS-$(CONFIG_MP3_HEADER_DECOMPRESS_BSF) += mp3_header_decompress_bsf.o \ mpegaudiotabs.o -OBJS-$(CONFIG_MPEG2_METADATA_BSF) += mpeg2_metadata_bsf.o +OBJS-$(CONFIG_MPEG2_METADATA_BSF) += mpeg2_metadata_bsf.o cbs_misc.o OBJS-$(CONFIG_NOISE_BSF) += noise_bsf.o OBJS-$(CONFIG_NULL_BSF) += null_bsf.o OBJS-$(CONFIG_OPUS_METADATA_BSF) += opus_metadata_bsf.o diff --git a/libavcodec/mpeg2_metadata_bsf.c b/libavcodec/mpeg2_metadata_bsf.c index 1aa6e02791..888bace6b2 100644 --- a/libavcodec/mpeg2_metadata_bsf.c +++ b/libavcodec/mpeg2_metadata_bsf.c @@ -23,6 +23,7 @@ #include "bsf_internal.h" #include "cbs.h" #include "cbs_bsf.h" +#include "cbs_misc.h" #include "cbs_mpeg2.h" #include "mpeg12.h" @@ -39,10 +40,79 @@ typedef struct MPEG2MetadataContext { int colour_primaries; int transfer_characteristics; int matrix_coefficients; + int a53_cc; int mpeg1_warned; } MPEG2MetadataContext; +static int mpeg2_metadata_handle_a53_cc(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *frag) +{ + MPEG2MetadataContext *ctx = bsf->priv_data; + int err, i; + uint8_t *a53_side_data = NULL; + size_t a53_side_data_size = 0; + + if (ctx->a53_cc == BSF_ELEMENT_REMOVE || ctx->a53_cc == BSF_ELEMENT_EXTRACT) { + for (i = 0; i < frag->nb_units; i++) { + MPEG2RawUserData *ud; + A53UserData a53_ud; + + if (frag->units[i].type != MPEG2_START_USER_DATA) + continue; + ud = frag->units[i].content; + + err = ff_cbs_read_a53_user_data(ctx->common.output, &a53_ud, ud->user_data, + ud->user_data_length); + if (err < 0) { + // Invalid or something completely different. + continue; + } + if (a53_ud.user_identifier != A53_USER_IDENTIFIER_ATSC || + a53_ud.atsc.user_data_type_code != + A53_USER_DATA_TYPE_CODE_CC_DATA) { + // Valid but something else (e.g. AFD). + continue; + } + + if (ctx->a53_cc == BSF_ELEMENT_REMOVE) { + ff_cbs_delete_unit(frag, i); + --i; + break; + } else if(ctx->a53_cc == BSF_ELEMENT_EXTRACT) { + err = ff_cbs_write_a53_cc_side_data(ctx->common.output, + &a53_side_data, + &a53_side_data_size, + &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data for packet side data.\n"); + goto fail; + } + + if (a53_side_data) { + err = av_packet_add_side_data(pkt, AV_PKT_DATA_A53_CC, + a53_side_data, a53_side_data_size); + if (err) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted A/53 " + "side data to packet.\n"); + goto fail; + } + a53_side_data = NULL; + } + } + } + } + + err = 0; + +fail: + av_freep(&a53_side_data); + + return err; +} + static int mpeg2_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, @@ -54,6 +124,13 @@ static int mpeg2_metadata_update_fragment(AVBSFContext *bsf, MPEG2RawSequenceDisplayExtension *sde = NULL; int i, se_pos; + if (pkt && ctx->a53_cc != BSF_ELEMENT_PASS) { + int err; + err = mpeg2_metadata_handle_a53_cc(bsf, pkt, frag); + if (err < 0) + return err; + } + for (i = 0; i < frag->nb_units; i++) { if (frag->units[i].type == MPEG2_START_SEQUENCE_HEADER) { sh = frag->units[i].content; @@ -218,6 +295,9 @@ static const AVOption mpeg2_metadata_options[] = { OFFSET(matrix_coefficients), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 255, FLAGS }, + BSF_ELEMENT_OPTIONS_PIRE("a53_cc", + "A/53 Closed Captions in user data", + a53_cc, FLAGS), { NULL } }; From patchwork Sat Feb 4 00:41:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 40242 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1617172pzk; Fri, 3 Feb 2023 16:42:49 -0800 (PST) X-Google-Smtp-Source: AK7set8x3rxpg7wUUVBPzKOo3HIrSBPgo6zUyI3vNcvsp3CIpfSEvCGx7sHmG7lTOE7AYG//dWWw X-Received: by 2002:a17:906:8586:b0:889:1eb1:7517 with SMTP id v6-20020a170906858600b008891eb17517mr12173107ejx.30.1675471369204; Fri, 03 Feb 2023 16:42:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675471369; cv=none; d=google.com; s=arc-20160816; b=R29KQYKzwhsZLaxBd1+lYMANTTvXbx29xkt/r6bDthqs88Ey/4J3gRsRtLgfr0DDcI Syvpl+Ls+wwiHtvvFIT0xvi6lQsWFkoKJDsfWdPLjyQFoCxBfuPxzMuva9GgfB3WhHsk S0rWm+qs7GxayVvTo/6uRgEFFBE7CMmTtp6ADSZcq2KdYzhG+y56Cjivhzpp/gYOQl3Z ZrBpEbuVgDEqDcvZL87aArsZD0zAgAYNOIEgXdmLYzA4FuCZOVC6mIrW10QNzjf8eHae ULKSO0rSM9Q+K489aQ57xqxt3KGFK3cfrV6XSpTvyzsA7elNUaStWRYa5jVDLhx/Cx+6 uZHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=Q2zp/KBDW8PkRmMFoNZRdR538tGeudXpw7zDNZcv3JA=; b=CZbCuAutxVbJOVD0CXMcZM3DF1HNm/o/u4wx/CJYKfrHwDR3ruFUYsxjnanEzOZdVp ghoOIPk2XDm92BEzwPGgmmUZTlVEjAQ8yDq4ualp38huoyVX9oDTiq1VUq1WfMZ2Zr12 NfVEOZw5LBXpS3l3pOvQQf6apdju1YtmVIpWWiYfEDav6NPZkiSqt7yEGIncWPioF7/6 VQednV4cMFMQRlntlQKBl2x+frpGy51P1r+3mHto/uX9H9NK2sK4ERj8H/5ISCRk2IPc Mx4CSVKd5HlguddXvaRDsFCnQbI+PNv5PQxnqHqC7JdrkYriRNxLW/Vmwxm+pJ1thr0d 3kNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=eB5ELDnj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 14-20020a170906024e00b0087879f8c291si5272078ejl.703.2023.02.03.16.42.48; Fri, 03 Feb 2023 16:42:49 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=eB5ELDnj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 12D3A68BEBF; Sat, 4 Feb 2023 02:42:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4C98868BE88 for ; Sat, 4 Feb 2023 02:42:01 +0200 (EET) Received: by mail-pg1-f169.google.com with SMTP id 7so4763923pgh.7 for ; Fri, 03 Feb 2023 16:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=aNEtYWwNlxyEBV/qVhmAs8GEVMAR28imFQr1nK2kqvY=; b=eB5ELDnjLFABGL/kJVC8ZH1B2bwMv2QqTBjjtLjfNdL5DgbGxioMjAdkl4MPUz+Rp3 g+9PVzNN5bN4XHKSgb40D6Wd4tXIde7bKHRcx6V41ITnKWMkI7w8Zy5GDh/CRZfTDwEZ yehSiuR81UJCw9kLQQzRzvUeWSzkjC5WWKS/4fLwfPJejrh/xUEQeXLdE/Fa9pBuIzUm 4tn5zqDO3lGl1Hbu2a5jA9z50CZJKgXVFszTJ1bhs0lnHSmExTqJQs7iAeYdtyJiqJI1 JfNEs2w8QCuVF6E3Mf7bvzWLx3wJJSW6sg1Ei5L285ZEqlnDY8gaEUa+9OsmScM6dJnU tTSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=aNEtYWwNlxyEBV/qVhmAs8GEVMAR28imFQr1nK2kqvY=; b=171Po4iQ4ANvosz+zaKJ58gF4oCI89GRF+NmSSL6ZBm/iNuLA+edVuTBhZo9SsxN8L Ue7Lbc+ZREUM+sTVlr22k1tMOsuY6NEhNMb44hfjeIDvPfbsqeaukvXYqbK2fEzCG+Op l5JhjfRXk7zZNQP7qGVHtL8GaNag/o/McYc7BwG9TqY8rJ9G+b0CHHcLlbgFKF+3SZLX oJ5uTKz2JF3c7JX69RQ5EGlxztpKm3+DvEDVCwzckHGs5lBrJSkjJGva9H6JMsYFI7gb +1rgHYxAKTNtGFPyQR9nL/qERbD5sbS/gSLEOsUod0b6xIF1w5njp9UpKz3cq4adKbZr SHvg== X-Gm-Message-State: AO0yUKXqAJ1RbVeV/zywUgpZlyAISYpHpVwjJas/Va3NQn3Npw0riFi4 ziibb76RCVZ/vwWQMaP8wDRBbtQgY2U= X-Received: by 2002:a05:6a00:2911:b0:58b:c1a1:4006 with SMTP id cg17-20020a056a00291100b0058bc1a14006mr10497969pfb.18.1675471319666; Fri, 03 Feb 2023 16:41:59 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id x11-20020aa793ab000000b0058bba6f06c5sm2432872pff.8.2023.02.03.16.41.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Feb 2023 16:41:59 -0800 (PST) From: Aman Karmani X-Google-Original-From: Aman Karmani Message-Id: In-Reply-To: References: Date: Sat, 04 Feb 2023 00:41:54 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 4/5] avcodec/h264_metadata_bsf: add support for a/53 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Karmani Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6fTsINEvwxZw From: Aman Karmani Signed-off-by: Aman Karmani --- doc/bitstream_filters.texi | 11 +++ libavcodec/Makefile | 2 +- libavcodec/h264_metadata_bsf.c | 144 +++++++++++++++++++++++++++++++++ 3 files changed, 156 insertions(+), 1 deletion(-) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index c036d37403..3d97f66315 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -243,6 +243,17 @@ ffmpeg -i hapqa_inputfile.mov -c copy -bsf:v hapqa_extract=texture=alpha -tag:v Modify metadata embedded in an H.264 stream. @table @option +@item a53_cc +Insert or remove registered userdata SEI NAL units containing A/53 closed captions. + +@table @samp +@item pass +@item insert +@item remove +@end table + +Default is pass. + @item aud Insert or remove AUD NAL units in all access units of the stream. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 2a36a31ff2..9f97c9b5e5 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1212,7 +1212,7 @@ OBJS-$(CONFIG_EXTRACT_EXTRADATA_BSF) += extract_extradata_bsf.o \ av1_parse.o h2645_parse.o OBJS-$(CONFIG_FILTER_UNITS_BSF) += filter_units_bsf.o OBJS-$(CONFIG_H264_METADATA_BSF) += h264_metadata_bsf.o h264_levels.o \ - h2645data.o + h2645data.o cbs_misc.o OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF) += h264_mp4toannexb_bsf.o OBJS-$(CONFIG_H264_REDUNDANT_PPS_BSF) += h264_redundant_pps_bsf.o OBJS-$(CONFIG_HAPQA_EXTRACT_BSF) += hapqa_extract_bsf.o hap.o diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index d318bf0cee..bbe74c3cee 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -26,6 +26,8 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h264.h" +#include "cbs_misc.h" +#include "cbs_sei.h" #include "h264.h" #include "h264_levels.h" #include "h264_sei.h" @@ -81,6 +83,7 @@ typedef struct H264MetadataContext { H264RawSEIDisplayOrientation display_orientation_payload; int level; + int a53_cc; } H264MetadataContext; @@ -463,6 +466,137 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, return 0; } +static int h264_metadata_handle_a53_cc(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au) +{ + H264MetadataContext *ctx = bsf->priv_data; + SEIRawMessage *message; + int err; + + message = NULL; + while (ff_cbs_sei_find_message(ctx->common.output, au, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &message) == 0) { + SEIRawUserDataRegistered *udr = message->payload; + A53UserData a53_ud; + uint8_t *a53_side_data = NULL; + size_t a53_side_data_size = 0; + + if (udr->data_length < 2) { + // Too short to contain a provider code. + continue; + } + + if (AV_RB16(udr->data) != 0x31) { // provider_code as atsc_provider_code + // Not ATSC. + continue; + } + + // The first two bytes of the message is provider_code so we offset the data by that + err = ff_cbs_read_a53_user_data(ctx->common.output, + &a53_ud, + udr->data + 2, + udr->data_length - 2); + if (err < 0) { + // Invalid or something completely different. + continue; + } + if (a53_ud.user_identifier != A53_USER_IDENTIFIER_ATSC || + a53_ud.atsc.user_data_type_code != + A53_USER_DATA_TYPE_CODE_CC_DATA) { + // Valid but something else (e.g. AFD). + continue; + } + + err = ff_cbs_write_a53_cc_side_data(ctx->common.output, + &a53_side_data, + &a53_side_data_size, + &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data for packet side data.\n"); + return err; + } + + if (a53_side_data) { + err = av_packet_add_side_data(pkt, AV_PKT_DATA_A53_CC, + a53_side_data, a53_side_data_size); + if (err) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted A/53 " + "side data to packet.\n"); + av_freep(&a53_side_data); + return err; + } + + if (ctx->a53_cc == BSF_ELEMENT_REMOVE || + ctx->a53_cc == BSF_ELEMENT_INSERT) { + ff_cbs_sei_delete_message(ctx->common.output, au, message); + + // Reset iteration + message = NULL; + } + } + } + + if (ctx->a53_cc == BSF_ELEMENT_INSERT) { + uint8_t *data; + size_t size; + int offset = 0; + + data = av_packet_get_side_data(pkt, AV_PKT_DATA_A53_CC, &size); + while (size > 0) { + A53UserData a53_ud; + int rsize = FFMIN(93, size); + + err = ff_cbs_read_a53_cc_side_data(ctx->common.output, &a53_ud, + data + offset, rsize); + offset += rsize; + size -= rsize; + if (err < 0) { + av_log(bsf, AV_LOG_WARNING, "Invalid A/53 closed captions " + "in packet side data dropped.\n"); + } else { + AVBufferRef *udr_buf = av_buffer_allocz(sizeof(SEIRawUserDataRegistered)); + SEIRawUserDataRegistered *udr = (SEIRawUserDataRegistered*)udr_buf->data; + size_t size = 9 + 3 * a53_ud.atsc.cc_data.cc_count; + + udr->data_ref = av_buffer_allocz(2 + size); + if (!udr->data_ref) { + return AVERROR(ENOMEM); + } + udr->data = udr->data_ref->data; + udr->data_length = udr->data_ref->size; + + udr->itu_t_t35_country_code = 0xB5; // usa_country_code + AV_WB16(udr->data, 0x31); // provider_code as atsc_provider_code + + err = ff_cbs_write_a53_user_data(ctx->common.output, udr->data + 2, + &size, &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data.\n"); + av_buffer_unref(&udr->data_ref); + av_buffer_unref(&udr_buf); + return err; + } + + err = ff_cbs_sei_add_message(ctx->common.output, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + udr, udr_buf); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add A/53 user data " + "SEI message to access unit.\n"); + av_buffer_unref(&udr->data_ref); + av_buffer_unref(&udr_buf); + return err; + } + } + } + } + + return 0; +} + static int h264_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, CodedBitstreamFragment *au) { @@ -533,6 +667,12 @@ static int h264_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, return err; } + if (pkt && ctx->a53_cc != BSF_ELEMENT_PASS) { + err = h264_metadata_handle_a53_cc(bsf, pkt, au); + if (err < 0) + return err; + } + if (pkt) ctx->done_first_au = 1; @@ -696,6 +836,10 @@ static const AVOption h264_metadata_options[] = { { LEVEL("6.2", 62) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("a53_cc", + "A/53 Closed Captions in SEI NAL units", + a53_cc, FLAGS), + { NULL } }; From patchwork Sat Feb 4 00:41:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 40243 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp1617218pzk; Fri, 3 Feb 2023 16:42:58 -0800 (PST) X-Google-Smtp-Source: AK7set8KxlVhZ263Oz6Pphl/X4YKtSOlNSjWOTOoWnIl6KItzHGAcnj8/vFl+T9M7n7ystzvLsQR X-Received: by 2002:a17:907:1687:b0:872:84dd:8903 with SMTP id hc7-20020a170907168700b0087284dd8903mr15146484ejc.59.1675471378670; Fri, 03 Feb 2023 16:42:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675471378; cv=none; d=google.com; s=arc-20160816; b=cYcAdnuQBrpmH5/KhAscNhJjqvmpShm8KigR4vzy/b25KMZM91gNAUtc3mETf+EEbg teEX0YdnQGmb4u8KOfSTiwkHmiVacf33Zs5NCjPwjgYyMfxeV3/CSbBg3VtUlmGsGKQb d8S5l9J6HyfDVpi6NwFtl36vTc8Ux8+gYIbq5+mxKs0ELFaNnd5hzPZO8I6RlpUQVuEv zuIvqP2zrIRczZ9fhFT+o7aNEU7xz5OiLjk0DKNZL9v94OrVJBlMcFuf/5gsydN40I9e GVnibUIxKNvllPU86SIeReravYBpYiwYkknTvEH74zqf8wec57FE4p91/sLG4+stBpuK +DTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=I50dbtcGsMS31EDkaE20Gzm+O7FYvRwwj9tSARMr3Cg=; b=cTTUTcyKXXK6midjIbLsfllJJ4xWrA9Dz9REHoGL2poejgiFxy4zk6TrSmu6E0N6OH /a6OBlsN5NDl8yomH4blSaWrBrwVMMy7SWT2wC+e7oj7O0cuwMF79r/7/cySnZ9SdHdU BZ6DdaL5e++KA8oYhSCW0qOosZiYMReunGSKucQK2+QR2he3vJQ5M+AqmuyZSGX6EKRI RPizoPm6SFrFbhClCxJ1rjNnluayeY9Wf9lfs1+6IHNgN+x7/Krqt7r9QOT40C7SClbp G5+lnsWM2YhXGbrB9hDG6Jlozo6ny7WJF1hvvBLXsDqkgIZnrGK7n1oy/YgCmWAojGt+ oHDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PpPxVozp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id wj17-20020a170907051100b0087777bb4127si4831286ejb.172.2023.02.03.16.42.58; Fri, 03 Feb 2023 16:42:58 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=PpPxVozp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F1F6F68BEC8; Sat, 4 Feb 2023 02:42:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0140868BE90 for ; Sat, 4 Feb 2023 02:42:01 +0200 (EET) Received: by mail-pj1-f52.google.com with SMTP id l4-20020a17090a850400b0023013402671so10332834pjn.5 for ; Fri, 03 Feb 2023 16:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=Wq+5njkHHHPr7Y9utdj6he1G+n/NFwNTuHdN1g7puGk=; b=PpPxVozpZ+kKiqz+/M9uS9I9YBHjZHPTbZUEOO5DdpOEUU+viYNWSGUyGW3f91gJfM uR9fDTGC8VaLS/XYpw5h3F7qVo5J+4t2r+yHmhAP3nxVYUrvZoelt4OimqUcNaX9icY8 dta2mtAGTATH2GMwLT2ZmOMBGD9IaGljnuBRX0weMINipzbYbJpcol8WLmbrtORQeFCW UB4SXauo1LNAMi1VVLGSj8cWMPwYkn0qjYsHXD45F6Lrr+8mbXQg2xxYKZ65ShHsIMij QwAgjUECKxy5yCISqAm2S89TqEEjke54I67x3ewu02ffbUvoD9UFAlmYVbSNH7S1b02m XPMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Wq+5njkHHHPr7Y9utdj6he1G+n/NFwNTuHdN1g7puGk=; b=Ksi1wJufA80O8HIdj2pkI22+IiMv2cGU6aG6WENX/Qz88QPWpKlgpzUz5nHt9vfxKB Jrcp8of3+HumZoxa/M2EZ+b92/Ybg0fKbRLGwZ0s2jWDwmiLMHBXTRmnpR9f0GY1Ttbc ro897jXMUKkgiGOeM3YRsacVEK35Zwi48JpKuszIvBmoRTuUPo5/lcp5RE8hdoT0ReS2 ZGZm6vjVShhSZUcz7BxUasHVYE+/H9he0BiqEFXFVcSnN7bZYaBTpkwxfzUdCmd7m9Ts 6Ga2gmt2X6HPzwpF8GYGmcuGwFXylKZZFwRdpzf78D59w32ab+MED7s0MvCTIEh5uab0 6BSw== X-Gm-Message-State: AO0yUKXhoVHnnnAkC7USbQI24IPrFJ7FZJeNmysWVkhCifFNJcWkOroW 7wZpvXE2ww9GBl/9UIm+F9RFEt9cMmM= X-Received: by 2002:a05:6a21:8695:b0:bf:6cd3:9546 with SMTP id ox21-20020a056a21869500b000bf6cd39546mr8452677pzb.30.1675471320302; Fri, 03 Feb 2023 16:42:00 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id v30-20020a637a1e000000b0047917991e83sm2053687pgc.48.2023.02.03.16.41.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Feb 2023 16:41:59 -0800 (PST) From: Aman Karmani X-Google-Original-From: Aman Karmani Message-Id: <8aa6c32db86a00c50708736d86861e21a66697a5.1675471315.git.ffmpegagent@gmail.com> In-Reply-To: References: Date: Sat, 04 Feb 2023 00:41:55 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 5/5] avcodec/h265_metadata_bsf: add support for a/53 closed captions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Karmani Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xBqL7xan2MZb From: Aman Karmani Signed-off-by: Aman Karmani --- doc/bitstream_filters.texi | 11 +++ libavcodec/Makefile | 2 +- libavcodec/h265_metadata_bsf.c | 144 +++++++++++++++++++++++++++++++++ 3 files changed, 156 insertions(+), 1 deletion(-) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 3d97f66315..57f8682d9f 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -398,6 +398,17 @@ confuse other transformations which require correct extradata. Modify metadata embedded in an HEVC stream. @table @option +@item a53_cc +Insert or remove registered userdata SEI NAL units containing A/53 closed captions. + +@table @samp +@item pass +@item insert +@item remove +@end table + +Default is pass. + @item aud Insert or remove AUD NAL units in all access units of the stream. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 9f97c9b5e5..156fd6215a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1217,7 +1217,7 @@ OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF) += h264_mp4toannexb_bsf.o OBJS-$(CONFIG_H264_REDUNDANT_PPS_BSF) += h264_redundant_pps_bsf.o OBJS-$(CONFIG_HAPQA_EXTRACT_BSF) += hapqa_extract_bsf.o hap.o OBJS-$(CONFIG_HEVC_METADATA_BSF) += h265_metadata_bsf.o h265_profile_level.o \ - h2645data.o + h2645data.o cbs_misc.o OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF) += hevc_mp4toannexb_bsf.o OBJS-$(CONFIG_IMX_DUMP_HEADER_BSF) += imx_dump_header_bsf.o OBJS-$(CONFIG_MEDIA100_TO_MJPEGB_BSF) += media100_to_mjpegb_bsf.o diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index 6787bd14a1..c3acbfb167 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -24,6 +24,8 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h265.h" +#include "cbs_misc.h" +#include "cbs_sei.h" #include "h2645data.h" #include "hevc.h" #include "h265_profile_level.h" @@ -62,6 +64,7 @@ typedef struct H265MetadataContext { int level; int level_guess; int level_warned; + int a53_cc; } H265MetadataContext; @@ -322,6 +325,137 @@ static int h265_metadata_update_sps(AVBSFContext *bsf, return 0; } +static int h265_metadata_extract_a53_cc(AVBSFContext *bsf, AVPacket *pkt, + CodedBitstreamFragment *au) +{ + H265MetadataContext *ctx = bsf->priv_data; + SEIRawMessage *message; + int err; + + message = NULL; + while (ff_cbs_sei_find_message(ctx->common.output, au, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &message) == 0) { + SEIRawUserDataRegistered *udr = message->payload; + A53UserData a53_ud; + uint8_t *a53_side_data = NULL; + size_t a53_side_data_size = 0; + + if (udr->data_length < 2) { + // Too short to contain a provider code. + continue; + } + + if (AV_RB16(udr->data) != 0x31) { // provider_code as atsc_provider_code + // Not ATSC. + continue; + } + + // The first two bytes of the message is provider_code + err = ff_cbs_read_a53_user_data(ctx->common.output, + &a53_ud, + udr->data + 2, + udr->data_length - 2); + if (err < 0) { + // Invalid or something completely different. + continue; + } + if (a53_ud.user_identifier != A53_USER_IDENTIFIER_ATSC || + a53_ud.atsc.user_data_type_code != + A53_USER_DATA_TYPE_CODE_CC_DATA) { + // Valid but something else (e.g. AFD). + continue; + } + + err = ff_cbs_write_a53_cc_side_data(ctx->common.output, + &a53_side_data, + &a53_side_data_size, + &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data for packet side data.\n"); + return err; + } + + if (a53_side_data) { + err = av_packet_add_side_data(pkt, AV_PKT_DATA_A53_CC, + a53_side_data, a53_side_data_size); + if (err) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted A/53 " + "side data to packet.\n"); + av_freep(&a53_side_data); + return err; + } + + if (ctx->a53_cc == BSF_ELEMENT_REMOVE || + ctx->a53_cc == BSF_ELEMENT_INSERT) { + ff_cbs_sei_delete_message(ctx->common.output, au, message); + + // Reset iteration + message = NULL; + } + } + } + + if (ctx->a53_cc == BSF_ELEMENT_INSERT) { + uint8_t *data; + size_t size; + int offset = 0; + + data = av_packet_get_side_data(pkt, AV_PKT_DATA_A53_CC, &size); + while (size > 0) { + A53UserData a53_ud; + int rsize = FFMIN(93, size); + + err = ff_cbs_read_a53_cc_side_data(ctx->common.output, &a53_ud, + data + offset, rsize); + offset += rsize; + size -= rsize; + if (err < 0) { + av_log(bsf, AV_LOG_WARNING, "Invalid A/53 closed captions " + "in packet side data dropped.\n"); + } else { + AVBufferRef *udr_buf = av_buffer_allocz(sizeof(SEIRawUserDataRegistered)); + SEIRawUserDataRegistered *udr = (SEIRawUserDataRegistered*)udr_buf->data; + size_t size = 9 + 3 * a53_ud.atsc.cc_data.cc_count; + + udr->data_ref = av_buffer_allocz(2 + size); + if (!udr->data_ref) { + return AVERROR(ENOMEM); + } + udr->data = udr->data_ref->data; + udr->data_length = udr->data_ref->size; + + udr->itu_t_t35_country_code = 0xB5; // usa_country_code + AV_WB16(udr->data, 0x31); // provider_code as atsc_provider_code + + err = ff_cbs_write_a53_user_data(ctx->common.output, udr->data + 2, + &size, &a53_ud); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to write " + "A/53 user data.\n"); + av_buffer_unref(&udr->data_ref); + av_buffer_unref(&udr_buf); + return err; + } + + err = ff_cbs_sei_add_message(ctx->common.output, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + udr, udr_buf); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add A/53 user data " + "SEI message to access unit.\n"); + av_buffer_unref(&udr->data_ref); + av_buffer_unref(&udr_buf); + return err; + } + } + } + } + + return 0; +} + static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, CodedBitstreamFragment *au) { @@ -387,6 +521,12 @@ static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, } } + if (pkt && ctx->a53_cc != BSF_ELEMENT_PASS) { + err = h265_metadata_extract_a53_cc(bsf, pkt, au); + if (err < 0) + return err; + } + return 0; } @@ -478,6 +618,10 @@ static const AVOption h265_metadata_options[] = { { LEVEL("8.5", 255) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("a53_cc", + "A/53 Closed Captions in SEI NAL units", + a53_cc, FLAGS), + { NULL } };