From patchwork Wed Mar 11 23:42:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 18132 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B897944A6E4 for ; Thu, 12 Mar 2020 02:07:43 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 958C168ABAC; Thu, 12 Mar 2020 02:07:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0C532689BC2 for ; Thu, 12 Mar 2020 02:07:37 +0200 (EET) Received: by mail-ed1-f48.google.com with SMTP id m25so5139512edq.8 for ; Wed, 11 Mar 2020 17:07:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=cZjpzZsIJ6LyBfwWACrA8Fg0MonkoB9cbgSX/8pEPlM=; b=rZs6SokyhB8ptzOETXHFBIeo73NfWRU4mJPRgn8dm1tt+Rxdv7Ou5t5jv5COoUWiLr UrEu3d8ykxDVpCmp6MrVkMRXB6t28TsmcW9krnM4ZHxNu1wU8Kp1IljUVfcyDElpxsrR MWiH+yp/8E4I7JK8qIUgdjPoBy91KibydvqXOnZ3T4o6EEFBQzCYpi43Z3zi2pBvUIv9 +OFwWdWRJRa4XnzwL1ou/wjJUWy+jrRnyubTx8SyMK1u1CtkUfqBr9bfDqxTr+QRFGzW lR2xzV/gvZAvn2X4xCfg8hETFiMygmmW2fGkikQXxVELf2KQ4iamc0NTU0icg/gW83i7 MAOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=cZjpzZsIJ6LyBfwWACrA8Fg0MonkoB9cbgSX/8pEPlM=; b=LTpDreygmJuXlgLJJ0lce9Hr45fk85l3r+Txj2HxYeY8stknBOvckXcim21k9ftATb 0aP5ZV25pEOwvzx49EnN/TzgWGe1LoHiL18JG4lTKJ9CUAFVxQ44y7XRMsaumYAJ75VG oA9rO9zxkd65mgSfi/9S9k0OSGj3YAMv7eCEcfQrATKczOroDULR/UFL/6/aWz7Jh1vq QuQivmpEVSGuI1YctMXIQMmNPhct7HXCxLZ1zLtj+5E3tqmTOP0VqKf4Z1jIUMuyKPvJ YSzc1+Pkd1LDrbgY0eN5lxH5m6Yn9V19BtFNtAGbty/vfOtqz7HBK5nGd0ofu0k9S1QN 4KQg== X-Gm-Message-State: ANhLgQ1tIsJ68nULfCwquDdJbb1SLEeEfX8X56JD4+H1wpxYF/a1GWw9 w7lVbC7FLSsyEk2c9EqYR1QT6We1 X-Google-Smtp-Source: ADFU+vuJbV1aYekFWLjEb6evsVhgFuCaywjKrTizbSrIdwH5nXyZV2RQKkL3mwJHrFdpfQ69HHrCEw== X-Received: by 2002:a2e:8112:: with SMTP id d18mr3322238ljg.137.1583970165805; Wed, 11 Mar 2020 16:42:45 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id e9sm16471250ljp.24.2020.03.11.16.42.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Mar 2020 16:42:45 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Thu, 12 Mar 2020 01:42:42 +0200 Message-Id: <20200311234244.28303-1-jeebjp@gmail.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] ffmpeg: explicitly handle sub2video subpicture initialization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Each time the sub2video structure is initialized, the sub2video subpicture is initialized together with the first received heartbeat. The heartbeat's PTS is utilized as the subpicture start time. Additionally, add some documentation on the stages. --- fftools/ffmpeg.c | 22 +++++++++++++++------- fftools/ffmpeg.h | 3 ++- fftools/ffmpeg_filter.c | 8 +++++++- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index e5fbd479a8..aaaf241314 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -237,7 +237,7 @@ static void sub2video_push_ref(InputStream *ist, int64_t pts) } } -void sub2video_update(InputStream *ist, AVSubtitle *sub) +void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) { AVFrame *frame = ist->sub2video.frame; int8_t *dst; @@ -254,7 +254,12 @@ void sub2video_update(InputStream *ist, AVSubtitle *sub) AV_TIME_BASE_Q, ist->st->time_base); num_rects = sub->num_rects; } else { - pts = ist->sub2video.end_pts; + /* If we are initializing the system, utilize current heartbeat + PTS as the start time, and show until the following subpicture + is received. Otherwise, utilize the previous subpicture's end time + as the fall-back value. */ + pts = ist->sub2video.initialize ? + heartbeat_pts : ist->sub2video.end_pts; end_pts = INT64_MAX; num_rects = 0; } @@ -269,6 +274,7 @@ void sub2video_update(InputStream *ist, AVSubtitle *sub) sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height, sub->rects[i]); sub2video_push_ref(ist, pts); ist->sub2video.end_pts = end_pts; + ist->sub2video.initialize = 0; } static void sub2video_heartbeat(InputStream *ist, int64_t pts) @@ -291,9 +297,11 @@ static void sub2video_heartbeat(InputStream *ist, int64_t pts) /* do not send the heartbeat frame if the subtitle is already ahead */ if (pts2 <= ist2->sub2video.last_pts) continue; - if (pts2 >= ist2->sub2video.end_pts || - (!ist2->sub2video.frame->data[0] && ist2->sub2video.end_pts < INT64_MAX)) - sub2video_update(ist2, NULL); + if (pts2 >= ist2->sub2video.end_pts || ist2->sub2video.initialize) + /* if we have hit the end of the current displayed subpicture, + or if we need to initialize the system, update the + overlayed subpicture and its start/end times */ + sub2video_update(ist2, pts2 + 1, NULL); for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); if (nb_reqs) @@ -307,7 +315,7 @@ static void sub2video_flush(InputStream *ist) int ret; if (ist->sub2video.end_pts < INT64_MAX) - sub2video_update(ist, NULL); + sub2video_update(ist, INT64_MAX, NULL); for (i = 0; i < ist->nb_filters; i++) { ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); if (ret != AVERROR_EOF && ret < 0) @@ -2507,7 +2515,7 @@ static int transcode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, return ret; if (ist->sub2video.frame) { - sub2video_update(ist, &subtitle); + sub2video_update(ist, INT64_MIN, &subtitle); } else if (ist->nb_filters) { if (!ist->sub2video.sub_queue) ist->sub2video.sub_queue = av_fifo_alloc(8 * sizeof(AVSubtitle)); diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index c0b8eb599f..fbaae15377 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -348,6 +348,7 @@ typedef struct InputStream { AVFifoBuffer *sub_queue; ///< queue of AVSubtitle* before filter init AVFrame *frame; int w, h; + unsigned int initialize; ///< marks if sub2video_update should force an initialization } sub2video; int dr1; @@ -645,7 +646,7 @@ int filtergraph_is_simple(FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost); int init_complex_filtergraph(FilterGraph *fg); -void sub2video_update(InputStream *ist, AVSubtitle *sub); +void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 40cc4c191c..b66faa50b5 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -740,6 +740,12 @@ static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) return AVERROR(ENOMEM); ist->sub2video.last_pts = INT64_MIN; ist->sub2video.end_pts = INT64_MIN; + + /* sub2video structure has been (re-)initialized. + Mark it as such so that the system will be + initialized with the first received heartbeat. */ + ist->sub2video.initialize = 1; + return 0; } @@ -1168,7 +1174,7 @@ int configure_filtergraph(FilterGraph *fg) while (av_fifo_size(ist->sub2video.sub_queue)) { AVSubtitle tmp; av_fifo_generic_read(ist->sub2video.sub_queue, &tmp, sizeof(tmp), NULL); - sub2video_update(ist, &tmp); + sub2video_update(ist, INT64_MIN, &tmp); avsubtitle_free(&tmp); } }