From patchwork Fri Mar 3 21:08:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 40579 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp1077960pzb; Fri, 3 Mar 2023 13:08:34 -0800 (PST) X-Google-Smtp-Source: AK7set/ibNirwTnFfVxwmtMlhONmdyA91K1KIoro7QZEWYYOXfceUYLyzcDJbAvUZ0GRe1gyAka+ X-Received: by 2002:a17:906:a0d8:b0:8f3:dc49:d8eb with SMTP id bh24-20020a170906a0d800b008f3dc49d8ebmr2676660ejb.71.1677877713936; Fri, 03 Mar 2023 13:08:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677877713; cv=none; d=google.com; s=arc-20160816; b=uV8WGQGUpkY/2jvkQKqU8USQ6ihd7ByC2Us/ZqzcACteaF8NGjcPSi/UoNY+JQXmtr s0MrC61zFfZOFtZn+o4VMX1V2oudxixW79zjXSuMpvV9wvlVdCL7Xwf+MEGy2QiIPk4X O4C5nNEN/4TTSkoeRUtNXgSB+DUPsZnyvJX7oJUSSF1IPaHk2H0lCupAptXmUu1NdPgE jIvm1dn2GBjqyLOfBjMex3Bs9cbCzYM8UlJLfbuLrioh3JLb5VOwPztS4q5wk2l6UoL6 +Q2WZi8UBWjkW4C22C151t4xYhf+9YJnZmUHq4RjTZVAL7JTUqB2o1zxviw15tH1svJd K0GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=oTovg6qzwwBrShwPUNSr08ocpgcpUxLeTjGs20TJ5Ko=; b=VU1jjod6WBkcSkNLd0e84WWpEI88uhzO21THVPZ7QtsPcemzj2xtt81AcP2NR5g7Jl d8jZWmlr3QE4BLt7csTMQ69YEFPvxGqGjvnUwNKMzBeBLtyUmy+zc10XNTHgc5AB+FLt YFTjEUTfK+0rfiVkmWH8ss+U/kilTD195hY9rhSmrva9ouMrSwh2RLyCIPLkSKWqnUho L0o+b4XEHrholR92Po+oa4hfYYhER3ggwg96AX5/4w7yL2uaPxHncpMgLjleXBCwNpdl Jdc0HiLQ8PGV/8RnW7OenJEYnq+JaIJHNyG1LkBd+UZz+aVhRrkfx12Ze7Pd0uKNQ7fR 7GBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=CqcN1R8l; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y18-20020a170906449200b008b97df5bd54si3239537ejo.961.2023.03.03.13.08.33; Fri, 03 Mar 2023 13:08:33 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20210112.gappssmtp.com header.s=20210112 header.b=CqcN1R8l; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6969568B5A9; Fri, 3 Mar 2023 23:08:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F087768B49F for ; Fri, 3 Mar 2023 23:08:21 +0200 (EET) Received: by mail-qv1-f45.google.com with SMTP id m4so2665282qvq.3 for ; Fri, 03 Mar 2023 13:08:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20210112.gappssmtp.com; s=20210112; t=1677877700; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=92Jj6SFG1oJ7czyTFRGcmWKJ3QlYHEFh/5meE1aok+Y=; b=CqcN1R8lmwP0F98tEss+aytfbvb2XF+XkMTYvXF9TUbemdMiK3OWsVOAA8e3Zot3jQ 3h/1vNrNVTTJd1VEvE33X7aKeelqRNQqM7B/nByTaVaRFQc/QbELP6maBMejyx4FNR+1 tZsffxOaOm85VFHfy00iHOBkCNYcjwu6NHYzggf5oDjiVAiCDkhqemX09lToQcwOdvnQ t7KLg4exU2pxJlqh0VwnZlzN6E6/q0xrorrjm88QFfE1orgo5fox+lI5FbY6ouQ7ytWw uS4O2XOfXzjgvNciBZGT5ZIDTBUzLVYZo7XV5hm2qq7dqDvJQc5UJUsDAsQkZQEHNae+ O0Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677877700; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=92Jj6SFG1oJ7czyTFRGcmWKJ3QlYHEFh/5meE1aok+Y=; b=Jek4lZllRn+gPe57O2zWS7j545uesDLvEvndnyGxms8tsgH+M+2qLWTANw27y4oRNQ dceTyNy82Hcm5QEMIrNsFa8T8FVW2XdMLsekjGtctOjjPyetgQJhMFac4gNYEL57FHnL mQ7Alo9WvC+qqwKNVg7w1laQIQMSIEk15pDq467Xq0w225OpSYQKEUN5BOqWkvuI7RV8 dBaHVViR4MfkidUjfzZ7nv9CmSvpM2NTdmyLeBYZorXw/FVVxAVTLAg2DLYXCJ8zH1e8 vK712G4uyZ1tJRjl5+idromkdVdTtU41KqU/nkOtA1lfsL5Pq/pY/2opWkxvziGLbREu Lcjg== X-Gm-Message-State: AO0yUKXlBoUsBWI/6zSk1LvtTcacjmcWFCmIlFBtGcWvHyJ0EZDkch1g Zz17yQtvnjdI08KxOo+B6K/wJLkuNZUjp2eFB9c= X-Received: by 2002:a05:6214:ca2:b0:56e:a620:7b39 with SMTP id s2-20020a0562140ca200b0056ea6207b39mr12423154qvs.4.1677877700195; Fri, 03 Mar 2023 13:08:20 -0800 (PST) Received: from klab-nyc-kernellabs2.localdomain (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id e10-20020a05620a014a00b0071a02d712b0sm2442991qkn.99.2023.03.03.13.08.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Mar 2023 13:08:19 -0800 (PST) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 3 Mar 2023 16:08:13 -0500 Message-Id: <20230303210815.23692-1-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 2.35.1.655.ga68dfadae5 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/3] decklink: Don't take for granted that first frame to decklink output will be PTS 0 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FTIuhTliO0V0 The existing code assumed that the first frame received by the decklink output would always be PTS zero. However if running in other timing modes than the default of CBR, items such as frame dropping at the beginning may result in starting at a non-zero PTS. For example, in our setup because we discard probing data and run with "-vsync 2" the first video frame scheduled to the decklink output will have a PTS around 170. Scheduling frames too far into the future will either fail or cause a backlog of frames scheduled far enough into the future that the entire pipeline will stall. Issue can be reproduced with the following command-line: ./ffmpeg -copyts -i foo.ts -f decklink -vcodec v210 -ac 2 'DeckLink Duo (4)' Keep track of the PTS of the first frame received, so that when we enable start playback we can provide that value to the decklink driver. Thanks to Marton Balint for review and suggestion to use AV_NOPTS_VALUE rather than zero for the initial value. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_common.h | 1 + libavdevice/decklink_enc.cpp | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 79d6ac5b38..088e165ee7 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -118,6 +118,7 @@ struct decklink_ctx { /* Status */ int playback_started; + int64_t first_pts; int64_t last_pts; unsigned long frameCount; unsigned int dropped; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index fb686b9032..5a435ddc78 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -486,6 +486,9 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) ctx->frames_buffer_available_spots--; pthread_mutex_unlock(&ctx->mutex); + if (ctx->first_pts == AV_NOPTS_VALUE) + ctx->first_pts = pkt->pts; + /* Schedule frame for playback. */ hr = ctx->dlo->ScheduleVideoFrame((class IDeckLinkVideoFrame *) frame, pkt->pts * ctx->bmd_tb_num, @@ -505,14 +508,14 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) " Video may misbehave!\n"); /* Preroll video frames. */ - if (!ctx->playback_started && pkt->pts > ctx->frames_preroll) { + if (!ctx->playback_started && pkt->pts > (ctx->first_pts + ctx->frames_preroll)) { av_log(avctx, AV_LOG_DEBUG, "Ending audio preroll.\n"); if (ctx->audio && ctx->dlo->EndAudioPreroll() != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not end audio preroll!\n"); return AVERROR(EIO); } av_log(avctx, AV_LOG_DEBUG, "Starting scheduled playback.\n"); - if (ctx->dlo->StartScheduledPlayback(0, ctx->bmd_tb_den, 1.0) != S_OK) { + if (ctx->dlo->StartScheduledPlayback(ctx->first_pts * ctx->bmd_tb_num, ctx->bmd_tb_den, 1.0) != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not start scheduled playback!\n"); return AVERROR(EIO); } @@ -559,6 +562,7 @@ av_cold int ff_decklink_write_header(AVFormatContext *avctx) ctx->list_formats = cctx->list_formats; ctx->preroll = cctx->preroll; ctx->duplex_mode = cctx->duplex_mode; + ctx->first_pts = AV_NOPTS_VALUE; if (cctx->link > 0 && (unsigned int)cctx->link < FF_ARRAY_ELEMS(decklink_link_conf_map)) ctx->link = decklink_link_conf_map[cctx->link]; cctx->ctx = ctx;