From patchwork Tue Feb 26 00:12:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 12159 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id F09E5447EFA for ; Tue, 26 Feb 2019 02:17:56 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D01A168082C; Tue, 26 Feb 2019 02:17:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D03A368082B for ; Tue, 26 Feb 2019 02:17:50 +0200 (EET) Received: by mail-lj1-f195.google.com with SMTP id d24so8857998ljc.12 for ; Mon, 25 Feb 2019 16:17:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=YRpLrJgTuSnJixsImwkZFbacCxU7Hu2R93grXXycCqE=; b=pXWL0U16jjcZ1CGeS2Q9Rl9Xf7LyScMfi7qCQSFf8mWtJqw/PvJ+JBv8p0hTo2gdZo SdloCTg7Q2IJzOFf4xy4Fwj7gxpHVVu5NvPZmMjJxFro1aDZ8eXpt/mxdJlrJBReN5kd Cg04G/fqFV/eFSe8mD9r6MEs98Bd2S8jte69CGq/sihy8Xx3B1iJas4O88ashn1ahBt3 z7HhZt/mS+N4Fs1CTJPI+uEZJ3IH5sPv8xySywP+ZQzSai5sY94bHOEJN9UaU7xEAj1+ n9P0mMuayWNpfusD2rNqBxIQ8xAbSIda8jzeZnzy+lIEEGDqYzOL1UIsSNplYZohmNl8 tgrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YRpLrJgTuSnJixsImwkZFbacCxU7Hu2R93grXXycCqE=; b=qaMfPEPtazW85Hgj0ydOvgQaxo8BxaKLShyL3lnypxiLxEIhsLO5S61gRK4F5nVQeg K1xhAs4RQtevMqgmGnElj10zmLc5i74LfLdKuDli4uQRB1uT8NOl48dHQ0tNGAilMOnz mFryw1Aq7QZj89QcmcAKPoZ9J9NncTtrZI0AjBKsd3/K1mK1QLm25ZGg3TWeZTucznlk fnimsRKjQxs1kYZgCYIMhCVKTJPMjYVt8uJqyxFyGHRguHaW9RdLN5BHLnuDGlezYuCI mKygKE5jdXVkBxJBLOyD9Z1Qp215mCHJ3JtRFPsiqLkXuildaRaoEBmHHjvHZYbFPZ+w Yrtw== X-Gm-Message-State: AHQUAuZcJ16zLUnHAMqUoUqWKixPLEZubGcdC4ekRDkGYHNwUxD1CKVM Xm+cMRfIYQhBvP1ExEg+2xG/5Shw X-Google-Smtp-Source: AHgI3Iai8d4GDEP6LoouC7yu0zt6Z4fmEkak7YxUVlHd9pLndIxEOnXL33nHa9fv0n0178tI2NWHng== X-Received: by 2002:a2e:2202:: with SMTP id i2mr12277495lji.170.1551139941140; Mon, 25 Feb 2019 16:12:21 -0800 (PST) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id c2sm2868583ljj.45.2019.02.25.16.12.20 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 25 Feb 2019 16:12:20 -0800 (PST) From: =?UTF-8?q?Jan=20Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Feb 2019 02:12:20 +0200 Message-Id: <20190226001220.27888-1-jeebjp@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190222235722.21864-1-jeebjp@gmail.com> References: <20190222235722.21864-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] ffmpeg: explicitly handle sub2video subpicture initialization X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Each time the sub2video structure is initialized, the sub2video subpicture is initialized together with the first received heartbeat. The heartbeat's PTS is utilized as the subpicture start time. Additionally, add some documentation on the stages. --- fftools/ffmpeg.c | 22 +++++++++++++++------- fftools/ffmpeg.h | 3 ++- fftools/ffmpeg_filter.c | 8 +++++++- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 544f1a1cef..efe903cc41 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -237,7 +237,7 @@ static void sub2video_push_ref(InputStream *ist, int64_t pts) } } -void sub2video_update(InputStream *ist, AVSubtitle *sub) +void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) { AVFrame *frame = ist->sub2video.frame; int8_t *dst; @@ -254,7 +254,12 @@ void sub2video_update(InputStream *ist, AVSubtitle *sub) AV_TIME_BASE_Q, ist->st->time_base); num_rects = sub->num_rects; } else { - pts = ist->sub2video.end_pts; + /* If we are initializing the system, utilize current heartbeat + PTS as the start time, and show until the following subpicture + is received. Otherwise, utilize the previous subpicture's end time + as the fall-back value. */ + pts = ist->sub2video.initialize ? + heartbeat_pts : ist->sub2video.end_pts; end_pts = INT64_MAX; num_rects = 0; } @@ -269,6 +274,7 @@ void sub2video_update(InputStream *ist, AVSubtitle *sub) sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height, sub->rects[i]); sub2video_push_ref(ist, pts); ist->sub2video.end_pts = end_pts; + ist->sub2video.initialize = 0; } static void sub2video_heartbeat(InputStream *ist, int64_t pts) @@ -291,9 +297,11 @@ static void sub2video_heartbeat(InputStream *ist, int64_t pts) /* do not send the heartbeat frame if the subtitle is already ahead */ if (pts2 <= ist2->sub2video.last_pts) continue; - if (pts2 >= ist2->sub2video.end_pts || - (!ist2->sub2video.frame->data[0] && ist2->sub2video.end_pts < INT64_MAX)) - sub2video_update(ist2, NULL); + if (pts2 >= ist2->sub2video.end_pts || ist2->sub2video.initialize) + /* if we have hit the end of the current displayed subpicture, + or if we need to initialize the system, update the + overlayed subpicture and its start/end times */ + sub2video_update(ist2, pts2 + 1, NULL); for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++) nb_reqs += av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter); if (nb_reqs) @@ -307,7 +315,7 @@ static void sub2video_flush(InputStream *ist) int ret; if (ist->sub2video.end_pts < INT64_MAX) - sub2video_update(ist, NULL); + sub2video_update(ist, INT64_MAX, NULL); for (i = 0; i < ist->nb_filters; i++) { ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL); if (ret != AVERROR_EOF && ret < 0) @@ -2514,7 +2522,7 @@ static int transcode_subtitles(InputStream *ist, AVPacket *pkt, int *got_output, return ret; if (ist->sub2video.frame) { - sub2video_update(ist, &subtitle); + sub2video_update(ist, INT64_MIN, &subtitle); } else if (ist->nb_filters) { if (!ist->sub2video.sub_queue) ist->sub2video.sub_queue = av_fifo_alloc(8 * sizeof(AVSubtitle)); diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index eb1eaf6363..b2129a3f28 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -349,6 +349,7 @@ typedef struct InputStream { AVFifoBuffer *sub_queue; ///< queue of AVSubtitle* before filter init AVFrame *frame; int w, h; + unsigned int initialize; ///< marks if sub2video_update should force an initialization } sub2video; int dr1; @@ -646,7 +647,7 @@ int filtergraph_is_simple(FilterGraph *fg); int init_simple_filtergraph(InputStream *ist, OutputStream *ost); int init_complex_filtergraph(FilterGraph *fg); -void sub2video_update(InputStream *ist, AVSubtitle *sub); +void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub); int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 72838de1e2..d81bb9742f 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -740,6 +740,12 @@ static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) return AVERROR(ENOMEM); ist->sub2video.last_pts = INT64_MIN; ist->sub2video.end_pts = INT64_MIN; + + /* sub2video structure has been (re-)initialized. + Mark it as such so that the system will be + initialized with the first received heartbeat. */ + ist->sub2video.initialize = 1; + return 0; } @@ -1169,7 +1175,7 @@ int configure_filtergraph(FilterGraph *fg) while (av_fifo_size(ist->sub2video.sub_queue)) { AVSubtitle tmp; av_fifo_generic_read(ist->sub2video.sub_queue, &tmp, sizeof(tmp), NULL); - sub2video_update(ist, &tmp); + sub2video_update(ist, INT64_MIN, &tmp); avsubtitle_free(&tmp); } }