From patchwork Fri Jul 21 20:37:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 42894 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be1d:b0:130:ccc6:6c4b with SMTP id ge29csp479208pzb; Fri, 21 Jul 2023 13:38:52 -0700 (PDT) X-Google-Smtp-Source: APBJJlFlU2TqBDdtUhLfRccb/anMJPFP3WMEqF4MEn2H3xWdHu5o50MaASQDVZVmybarz0t9Uy+y X-Received: by 2002:a50:fc16:0:b0:51e:293b:e1ce with SMTP id i22-20020a50fc16000000b0051e293be1cemr2574131edr.31.1689971932263; Fri, 21 Jul 2023 13:38:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689971932; cv=none; d=google.com; s=arc-20160816; b=idUg+DkZr61XDQPA0atQO6n1ALBKQcPwq0MeNLxdhnD0oO9iAL2jpCl232HwYtds26 hPEgOBDRqqfsCu0tV2MyFhQAQ4ZisN4x3x5YZF0AENxyWvk32GMyqpYsEUi6pn/OtTHz KQmXXCV51EK0SUBbaXCNTviMqyFJCvb5duHw/QOMtjvb5fCmuWB0KiPCWtwBwI2NWhjy XYKu9a88EJAjBtUspAw4SizYJw9fok+HvAPuEJerb4sWO81wci/EXIzus0duvZMsx47j CbLemKEzqwS2cy94VEe1btgahEnIbAyM4//vgrhk5DpXDEo/z4d4orkrp/md3CMXusMJ JnYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=R19BIZqh6ktKC1syY3jq7DfwIgfeqGldy+5aN89kAw0=; fh=DKzdJpyueDOox6ShoNYUi750QZs6R2naOS9+hcsrM90=; b=u3aAn1s+WGM2oVQ+rI8c3BonuF1z6PoUh8X8WMovOFaCPEzXe49O7+gA6Bv3xkM9Sn T0rP7+d1UdtrzztkHcSX2SeWmtCOvrwUcq8C8IsM+gUuBbDc4YQ5LoruO7bJF1K7prxu Yq1LKBrVYrJhsC51chD0ygZI08OE1E6h8TSCZZQ6xPUkP7RMT/PnOTviAqZF/p3DqPEu uIZmzKWTWjCySUwHSy3w1op6p5lRW7LI/6DxYfdfxn9zDEyt0TRGBKPZn3BRpIQgZpvp vYIhKthE0hzZyLTNqKjwhA50TuIka/JM11gGCU1NvYbX7uOVa3urVkg6Xk19fzJN8tjz 04vQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=hOY0ZSmb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d10-20020aa7c1ca000000b0051e5cabfebdsi2811012edp.373.2023.07.21.13.38.50; Fri, 21 Jul 2023 13:38:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=hOY0ZSmb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AFB2068C646; Fri, 21 Jul 2023 23:38:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vs1-f47.google.com (mail-vs1-f47.google.com [209.85.217.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D18CF68C5E8 for ; Fri, 21 Jul 2023 23:38:05 +0300 (EEST) Received: by mail-vs1-f47.google.com with SMTP id ada2fe7eead31-440db8e60c8so1003101137.0 for ; Fri, 21 Jul 2023 13:38:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1689971884; x=1690576684; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=/4KI71djtdlc3uKHLZgVuizAPZmaWbdtBbAJ/xcyZco=; b=hOY0ZSmbhNiGN04L89dW+wbPqU+N8FeDW8wEKRWEfV/9rtn4uH2agjtLeVnZ1vlmhM 7D6OhsTLVXv0q8wmRH7lJNWoIOQl4nckFEnm/KrX96H0YkV1EYYm/LQJHh7VdzW5mnnM hpU6yjX2JL7fO0exDTGpDT2w690kbITHrJyouX20srRjHZDMbJw6W5ZNVlKii5S36AY2 WH9YNhGQX3PcL+SBFhU2NMn1qfo3+AJSDrygLnLksBvt+p8XtGIm7JSlOi/mtbRIS0a5 Ya/0pAkkBDKoRxeB/753ShiinzsHAqSeTzcxmET55M0LNbtwldNBO3AtnJzwDsztaKQV JxEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689971884; x=1690576684; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/4KI71djtdlc3uKHLZgVuizAPZmaWbdtBbAJ/xcyZco=; b=gv7oyDlpoAUJ+b9v5ZmHK2sR7P0JtvQ0gaaiefoHi1k45bI31IG5F1kFOenmYTbWMQ jLEKt+Oqsk8a51VIpRXVW6egCEOY0uLbdLOwLLgOZ/K0sO8qBJuS9YVAiw700kseLCXA pWagO3Aqc5g3yH7YpuxmtE9srD8hRqw3ftaVXmhY+62/ni6IcriG+Q8/LijvaORSNBUr +K2o2PjZgWmtP6GhPYVKAQ60G/G9avkA3AIqsRInhnm+nzmpMFDa7X8bh/CLddj+DcT5 2qDQngy0/RpFiwq/67ChrcAreMsHdUVHwWsldNBYTwskrchDsV/4XEorqKJrrco6D9C5 xYIg== X-Gm-Message-State: ABy/qLYMAzia9Meik4n2awILbxi45/2+m3ahR9JbAczVj1e0lGwmeBwC q2WuifeyUzJ1AlFLNGylzMobyURzpI3YXmH5XpY= X-Received: by 2002:a67:edc1:0:b0:443:6457:101 with SMTP id e1-20020a67edc1000000b0044364570101mr867723vsp.7.1689971883494; Fri, 21 Jul 2023 13:38:03 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id i16-20020a0cf390000000b0063646f1147asm1534077qvk.135.2023.07.21.13.38.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jul 2023 13:38:03 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 16:37:58 -0400 Message-Id: <1689971878-26226-5-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1689971878-26226-1-git-send-email-dheitmueller@ltnglobal.com> References: <1689971878-26226-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [PATCH v3 4/4] bsf: Add new bitstream filter to set SCTE-35 pts_adjustment when reclocking X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bYpkfU5pTFV6 Because SCTE-35 messages are represented in TS streams as sections rather than PES packets, we cannot rely on ffmpeg's standard mechanisms to adjust PTS values if reclocking the stream. This filter will leverage the SCTE-35 pts_adjust field to compensate for any change in the PTS values in the stream. See SCTE-35 2019 Sec 9.6 for information about the use of the pts_adjust field. This filter also tweaks the mpegtsenc mux to automatically add it so the user doesn't have to include it manually. Thanks to Andreas Rheinhardt for providing feedback/suggestions on improving the patch. Signed-off-by: Devin Heitmueller --- doc/bitstream_filters.texi | 9 ++++ libavcodec/Makefile | 1 + libavcodec/bitstream_filters.c | 1 + libavcodec/scte35ptsadjust_bsf.c | 103 +++++++++++++++++++++++++++++++++++++++ libavformat/mpegtsenc.c | 2 + 5 files changed, 116 insertions(+) create mode 100644 libavcodec/scte35ptsadjust_bsf.c diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index c63c203..068b0c9 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -797,6 +797,15 @@ Remove extradata from all frames. @end table @end table +@section scte35ptsadjust +This bitstream filter makes use of side data injected by the MPEG-TS demux +in order to rewrite the PTS adjust field within SCTE-35 packets. This +ensures the pts_adjust field contains a valid value if the caller changes +the timebase of the stream. + +The bitstream filter is added automatically by the mpegtsenc mux, and no +action is required on the part of the user. + @section setts Set PTS and DTS in packets. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 1b0226c..4c1b312 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1253,6 +1253,7 @@ OBJS-$(CONFIG_PCM_RECHUNK_BSF) += pcm_rechunk_bsf.o OBJS-$(CONFIG_PGS_FRAME_MERGE_BSF) += pgs_frame_merge_bsf.o OBJS-$(CONFIG_PRORES_METADATA_BSF) += prores_metadata_bsf.o OBJS-$(CONFIG_REMOVE_EXTRADATA_BSF) += remove_extradata_bsf.o av1_parse.o +OBJS-$(CONFIG_SCTE35PTSADJUST_BSF) += scte35ptsadjust_bsf.o OBJS-$(CONFIG_SETTS_BSF) += setts_bsf.o OBJS-$(CONFIG_TEXT2MOVSUB_BSF) += movsub_bsf.o OBJS-$(CONFIG_TRACE_HEADERS_BSF) += trace_headers_bsf.o diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index 1e9a676..60ed164 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -57,6 +57,7 @@ extern const FFBitStreamFilter ff_pcm_rechunk_bsf; extern const FFBitStreamFilter ff_pgs_frame_merge_bsf; extern const FFBitStreamFilter ff_prores_metadata_bsf; extern const FFBitStreamFilter ff_remove_extradata_bsf; +extern const FFBitStreamFilter ff_scte35ptsadjust_bsf; extern const FFBitStreamFilter ff_setts_bsf; extern const FFBitStreamFilter ff_text2movsub_bsf; extern const FFBitStreamFilter ff_trace_headers_bsf; diff --git a/libavcodec/scte35ptsadjust_bsf.c b/libavcodec/scte35ptsadjust_bsf.c new file mode 100644 index 0000000..9870737 --- /dev/null +++ b/libavcodec/scte35ptsadjust_bsf.c @@ -0,0 +1,103 @@ +/* + * SCTE-35 PTS fixup bitstream filter + * Copyright (c) 2023 LTN Global Communications, Inc. + * + * Author: Devin Heitmueller + * + * Because SCTE-35 messages are represented in TS streams as sections + * rather than PES packets, we cannot rely on ffmpeg's standard + * mechanisms to adjust PTS values if reclocking the stream. + * This filter will leverage the SCTE-35 pts_adjust field to + * compensate for any change in the PTS values in the stream. + * + * See SCTE-35 2019 Sec 9.6 for information about the use of + * the pts_adjust field. + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bsf.h" +#include "bsf_internal.h" +#include "defs.h" +#include "libavutil/intreadwrite.h" + +static int scte35ptsadjust_filter(AVBSFContext *ctx, AVPacket *pkt) +{ + const AVTransportTimestamp *transport_ts; + int64_t cur_pts_adjust; + int ret = 0; + + ret = ff_bsf_get_packet_ref(ctx, pkt); + if (ret < 0) + return ret; + + /* Retrieve the original PTS, which will be used to calculate the pts_adjust */ + transport_ts = (AVTransportTimestamp *) av_packet_get_side_data(pkt, AV_PKT_DATA_TRANSPORT_TIMESTAMP, NULL); + if (transport_ts == NULL) { + /* No original PTS specified, so just pass the packet through */ + return 0; + } + + /* The pts_adjust field is logically buf[4]-buf[8] of the payload */ + if (pkt->size < 9) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + /* Extract the current pts_adjust value from the packet */ + cur_pts_adjust = ((int64_t) pkt->data[4] & (int64_t) 0x01 << 32) | + ((int64_t) pkt->data[5] << 24) | + ((int64_t) pkt->data[6] << 16) | + ((int64_t) pkt->data[7] << 8) | + ((int64_t) pkt->data[8] ); + + av_log(ctx, AV_LOG_DEBUG, "pts=%" PRId64 "(%d/%d) orig_pts=%" PRId64 "(%d/%d) pts_adjust=%" PRId64 "\n", + pkt->pts, pkt->time_base.num, pkt->time_base.den, + transport_ts->pts, transport_ts->time_base.num, transport_ts->time_base.den, cur_pts_adjust); + + /* Compute the new PTS adjust value */ + cur_pts_adjust -= av_rescale_q(transport_ts->pts, transport_ts->time_base, (AVRational){1, 90000}); + cur_pts_adjust += av_rescale_q(pkt->pts, pkt->time_base, (AVRational){1, 90000}); + cur_pts_adjust &= (int64_t) 0x1ffffffff; + + av_log(ctx, AV_LOG_DEBUG, "new pts_adjust=%" PRId64 "\n", cur_pts_adjust); + + ret = av_packet_make_writable(pkt); + if (ret < 0) + goto fail; + + /* Insert the updated pts_adjust value */ + pkt->data[4] &= 0xfe; /* Preserve top 7 unrelated bits */ + pkt->data[4] |= cur_pts_adjust >> 32; + AV_WB32(pkt->data + 5, cur_pts_adjust); + +fail: + if (ret < 0) + av_packet_unref(pkt); + + return ret; +} + +static const enum AVCodecID codec_ids[] = { + AV_CODEC_ID_SCTE_35, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_scte35ptsadjust_bsf = { + .p.name = "scte35ptsadjust", + .p.codec_ids = codec_ids, + .filter = scte35ptsadjust_filter, +}; diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c index c6cd1fd..48d7833 100644 --- a/libavformat/mpegtsenc.c +++ b/libavformat/mpegtsenc.c @@ -2337,6 +2337,8 @@ static int mpegts_check_bitstream(AVFormatContext *s, AVStream *st, (st->codecpar->extradata_size > 0 && st->codecpar->extradata[0] == 1))) ret = ff_stream_add_bitstream_filter(st, "hevc_mp4toannexb", NULL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_SCTE_35) { + ret = ff_stream_add_bitstream_filter(st, "scte35ptsadjust", NULL); } return ret;