From patchwork Thu Feb 23 22:25:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40476 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:5494:b0:bf:7b3a:fd32 with SMTP id i20csp756701pzk; Thu, 23 Feb 2023 14:25:38 -0800 (PST) X-Google-Smtp-Source: AK7set+d8Wt1D8u/V0VZ9sfqhmTvlAkoTWzQH1VnR/w2yS9fgt4XVvVaOpfU7i2VMY4u2CXOqHWE X-Received: by 2002:a17:906:9f29:b0:8b1:3821:1406 with SMTP id fy41-20020a1709069f2900b008b138211406mr29209381ejc.45.1677191138544; Thu, 23 Feb 2023 14:25:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677191138; cv=none; d=google.com; s=arc-20160816; b=Uz1vGa2L6XOvrKQcV0+dVQHOHbIEjFlkPC92yq8iV9PfN8HSa7U8LScU0YMW0sPCwp VuR5Rj/YRRNPptekTZI5SUJcdJT3rRLwmkG434oIgRjkt8+j5F6jlszQBkqDl2vcDLFd rS8DZGAzYFeRmP2zKyjoYDaSwjTcpxioLSlni5QW1d77d5rhawF90VfxJlfm9gdZFO/2 L2P4Nrj6eX8Dpm9Vqv7+6Sy9d5qAlLCdmGAJpV3ZF/uhBVB03sZm5xJeLcELxOii2byh ZFyFh6NGZQ95I1X2fiw/0eUQ2F/iLtBx4FShvqACUouPLlGnVsHB0szMaX6TGzzuQN0c Re+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=4BgoWQ4acugBaoMapZ4ZYpNjgeDhjGogAjuEwV/Gijc=; b=T9zB9alpd4N6zGaBEsWga8VwEQvtStDKdUipzCm46+O+z4c3Pk0aLXCbrjg7WWNu/T bcavTiGKGx9tB7fwiOD8w02U+FE8vtj3o/r6HUzAjU3Ek/JOG3lmGfRsjGDohVPrOWIt bXK6WiQiR0FCJaScZwIxe2O9F1CqsUx8/7dgP77QJjYwLhIjLJ4tgJTDPcDfbT1MW6rk ivPQv8qhhiwVhtta3YQjMyLpQ3p4KnF0k/HzynoC5ptRk87vGevUQstMkCqCtO2/7/Ps UIc9Kaha8z2ZyzLhq+zvwNsj86p4TNsJBrUZZR2vYlk9z7SqahjVzupUhL9Ats47cfz3 Mxcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=uH1cqBLP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id wi3-20020a170906fd4300b008d030b1b8f2si12632318ejb.409.2023.02.23.14.25.38; Thu, 23 Feb 2023 14:25:38 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=uH1cqBLP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 14D8568C04E; Fri, 24 Feb 2023 00:25:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DABA268C03C for ; Fri, 24 Feb 2023 00:25:19 +0200 (EET) Received: by mail-qv1-f45.google.com with SMTP id y3so12272342qvn.4 for ; Thu, 23 Feb 2023 14:25:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1677191118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=R9r+myOsaLfr5V6tJ0iXzoDctzruxcUf2o7AhE7a32U=; b=uH1cqBLPoW9e/8z8oB+KNFucPcCPY3eacaGsIbL1WNMi/12EhABh7EiQkknZEVHEfn 0LLMNfSn525duxEL/OIDxxqt7EAnyrg6YA9hFUzurkUVfMF0RQ2ioRFcXGRoXEB7Q70b 7eWGC0cSISY+K6u2g4xSdNbBiDK+i1ogBNT9gzTjph248v+L0Ms1shHSAyHhaMieBJAy iX9YDFRPi9b6hAvixRd5JQTVlTyc7Eu6Gnsq2ave68d2PVzlOcJJ1WYwrEkqqa4l3wuy OCbSDe7FG3giucDNhXXb1BDrKtVyOb5XG/zDKex2jHwAOD/7xXHnvbyOMcaoHV+LBrn1 x3xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677191118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=R9r+myOsaLfr5V6tJ0iXzoDctzruxcUf2o7AhE7a32U=; b=0LC5HVjyxKnueG+jGOWKy5dixM0C8RmE+wotiZUIDw8ZnKd4QU0lZ2DbWPMA41X+qQ S7sHxcByT4CIipO0qyneXfgXR0z25pg1L+8l2dcdIFp8mqa29FdVmGgpbAEpaJ1L33TH CiqsmXFMxGTwPcrYpKK+x2JNyl/69EQk8LN9q2vHBLBGKqgaFenRSxdqxS6DjHzZmqE0 qZwa4kY7UX8BpSnOmdHuGe1Tnp7N9feanBcCiertsi+cA1nln6sY/WkgVfZRBmfwscxP pVk/usZ97DqdVsh/QWZEELfdpWlHRe6xLnRYhOOS3WGe4V6Xi6nQAzkH1yRAW/Kt/CNx zg9g== X-Gm-Message-State: AO0yUKWlo7fbtxRcFEgQKTmn9fqIACm1T+V+xwmDq4aUvlxZpuj6ROau eRJZJCndvWQQ4y8306nPR7wcMfA7kAbNoIOD X-Received: by 2002:a05:6214:e4f:b0:571:cb52:7ee2 with SMTP id o15-20020a0562140e4f00b00571cb527ee2mr13458844qvc.52.1677191117926; Thu, 23 Feb 2023 14:25:17 -0800 (PST) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id s75-20020a37a94e000000b0073d873df3fesm9537062qke.30.2023.02.23.14.25.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Feb 2023 14:25:17 -0800 (PST) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Thu, 23 Feb 2023 17:25:11 -0500 Message-Id: <20230223222513.3759-2-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 In-Reply-To: <20230223222513.3759-1-dheitmueller@ltnglobal.com> References: <20230223222513.3759-1-dheitmueller@ltnglobal.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] decklink: Don't take for granted that first frame to decklink output will be PTS 0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: tAhZgEyPnxkw The existing code assumed that the first frame received by the decklink output would always be PTS zero. However if running in other timing modes than the default of CBR, items such as frame dropping at the beginning may result in starting at a non-zero PTS. For example, in our setup because we discard probing data and run with "-vsync 2" the first video frame scheduled to the decklink output will have a PTS around 170. Scheduling frames too far into the future will either fail or cause a backlog of frames scheduled far enough into the future that the entire pipeline will stall. Issue can be reproduced with the following command-line: ./ffmpeg -copyts -i foo.ts -f decklink -vcodec v210 -ac 2 'DeckLink Duo (4)' Keep track of the PTS of the first frame received, so that when we enable start playback we can provide that value to the decklink driver. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_common.h | 1 + libavdevice/decklink_enc.cpp | 7 +++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 79d6ac5b38..088e165ee7 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -118,6 +118,7 @@ struct decklink_ctx { /* Status */ int playback_started; + int64_t first_pts; int64_t last_pts; unsigned long frameCount; unsigned int dropped; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index fb686b9032..c3dc2c0cac 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -486,6 +486,9 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) ctx->frames_buffer_available_spots--; pthread_mutex_unlock(&ctx->mutex); + if (ctx->first_pts == 0) + ctx->first_pts = pkt->pts; + /* Schedule frame for playback. */ hr = ctx->dlo->ScheduleVideoFrame((class IDeckLinkVideoFrame *) frame, pkt->pts * ctx->bmd_tb_num, @@ -505,14 +508,14 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) " Video may misbehave!\n"); /* Preroll video frames. */ - if (!ctx->playback_started && pkt->pts > ctx->frames_preroll) { + if (!ctx->playback_started && pkt->pts > (ctx->first_pts + ctx->frames_preroll)) { av_log(avctx, AV_LOG_DEBUG, "Ending audio preroll.\n"); if (ctx->audio && ctx->dlo->EndAudioPreroll() != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not end audio preroll!\n"); return AVERROR(EIO); } av_log(avctx, AV_LOG_DEBUG, "Starting scheduled playback.\n"); - if (ctx->dlo->StartScheduledPlayback(0, ctx->bmd_tb_den, 1.0) != S_OK) { + if (ctx->dlo->StartScheduledPlayback(ctx->first_pts * ctx->bmd_tb_num, ctx->bmd_tb_den, 1.0) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not start scheduled playback!\n"); return AVERROR(EIO); }