From patchwork Sat Nov 4 07:56:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 44522 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3aa6:b0:181:818d:5e7f with SMTP id d38csp336964pzh; Sat, 4 Nov 2023 02:25:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEQXAOlzhrQZS053aOdFaZm1wCyYwPIaCmyRZ3KkVaxMD2rRls56U6HGnXX+Xtz2ArNItty X-Received: by 2002:aa7:d78a:0:b0:53d:d4a0:3154 with SMTP id s10-20020aa7d78a000000b0053dd4a03154mr19330743edq.31.1699089905467; Sat, 04 Nov 2023 02:25:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699089905; cv=none; d=google.com; s=arc-20160816; b=qq5P9neu8jtIQQsg1Mj3l8+5dDIuSBU+lTFlmHUtYm8hpm0xMyYqH1/L4RvZzXfFIY UTXNJznCDCf71ew9ifqxWp/ZApiN9ETeo9WDOy0yQWJNdzgTVpyd48U80wANP+dXX6lu bDJYCcbcRzdZ3Pa/eKkv80Oz9v1yFjNchfMM4rH56RQ6R/Jsa0QB6qVaRqxUNPRhbfl1 n2ILK6mSsEHkodZ/H47NH7ZERLP6udclT77XHjI8TzXkCY9xBAq1xCxM7W0/mYFi5oTJ iA9xjffLthV0Qdm+Ni6UzzBKEQKHkrMnQHVpT+OK6D2digSYhUkh9r9g5NpcRmpCl2pm e1oA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=DK7yIayQyT+guTMXMPw1gpnLiF9s9s64Ieui6BFMPdU=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=QSZGHUprSHQL9MMTDobWx1KFdndfTVvaLcOaqOxuc7uGZSGKfOjcOwnGiDjIGsmTcz zU2bAHME7IKOqtHvHsizfNrWtov+KNLHhxFO6PjA8KD+eu7/LCRTHd4fqlOqx1e2j4SO c+lvSfEchbTkf6BT1e6gTc8Z1RpWjbD/otDNm41mXj1MITGgD3ZmbDQ8zABg5AGCkhzR cOTGhdZprgmZPgYfrfccZDvapnleKS7ganpfTi5DqbhbBNocx/d542LIkxYbVgL0wHiN lGzY+5jFlA4xgp4kFiYuKGb13BFCYyKvzw1DDtslIhxOYoiwNErtzjW+3UHcgYwaZbFj KK3A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t2-20020a50d702000000b0053ef9264fbasi1987917edi.138.2023.11.04.02.25.05; Sat, 04 Nov 2023 02:25:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A5F868CEC1; Sat, 4 Nov 2023 11:22:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 69E8368CE57 for ; Sat, 4 Nov 2023 11:21:55 +0200 (EET) Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 4C5D4951 for ; Sat, 4 Nov 2023 10:21:55 +0100 (CET) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id 0XFYCM6CPys9 for ; Sat, 4 Nov 2023 10:21:54 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 15B551546 for ; Sat, 4 Nov 2023 10:21:47 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id C9D043A150D for ; Sat, 4 Nov 2023 10:21:40 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 4 Nov 2023 08:56:25 +0100 Message-ID: <20231104092125.10213-17-anton@khirnov.net> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231104092125.10213-1-anton@khirnov.net> References: <20231104092125.10213-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 16/24] fftools/ffmpeg: disable -fix_sub_duration_heartbeat X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uSDvi2tNIjX4 As it causes subtitle packets processed by encoders/muxers to signal back to decoding, it depends on packets being processed in a specific order and is thus in its current form fundamentally incompatible with threading architecture. --- fftools/ffmpeg.c | 31 ------------------------------- fftools/ffmpeg.h | 10 ---------- fftools/ffmpeg_dec.c | 23 ----------------------- fftools/ffmpeg_enc.c | 7 ------- fftools/ffmpeg_mux.c | 10 ---------- fftools/ffmpeg_mux_init.c | 4 ---- fftools/ffmpeg_opt.c | 9 +++++++-- tests/fate/ffmpeg.mak | 24 ++++++++++++------------ 8 files changed, 19 insertions(+), 99 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 038649d9b5..f2293e0250 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -769,37 +769,6 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle *subtitle, int copy) return 0; } -int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt) -{ - OutputFile *of = output_files[ost->file_index]; - int64_t signal_pts = av_rescale_q(pkt->pts, pkt->time_base, - AV_TIME_BASE_Q); - - if (!ost->fix_sub_duration_heartbeat || !(pkt->flags & AV_PKT_FLAG_KEY)) - // we are only interested in heartbeats on streams configured, and - // only on random access points. - return 0; - - for (int i = 0; i < of->nb_streams; i++) { - OutputStream *iter_ost = of->streams[i]; - InputStream *ist = iter_ost->ist; - int ret = AVERROR_BUG; - - if (iter_ost == ost || !ist || !ist->decoding_needed || - ist->dec_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE) - // We wish to skip the stream that causes the heartbeat, - // output streams without an input stream, streams not decoded - // (as fix_sub_duration is only done for decoded subtitles) as - // well as non-subtitle streams. - continue; - - if ((ret = fix_sub_duration_heartbeat(ist, signal_pts)) < 0) - return ret; - } - - return 0; -} - /* pkt = NULL means EOF (needed to flush decoder buffers) */ static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eof) { diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 8de91ab85a..c954ed5ebf 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -248,8 +248,6 @@ typedef struct OptionsContext { int nb_reinit_filters; SpecifierOpt *fix_sub_duration; int nb_fix_sub_duration; - SpecifierOpt *fix_sub_duration_heartbeat; - int nb_fix_sub_duration_heartbeat; SpecifierOpt *canvas_sizes; int nb_canvas_sizes; SpecifierOpt *pass; @@ -604,12 +602,6 @@ typedef struct OutputStream { EncStats enc_stats_pre; EncStats enc_stats_post; - - /* - * bool on whether this stream should be utilized for splitting - * subtitles utilizing fix_sub_duration at random access points. - */ - unsigned int fix_sub_duration_heartbeat; } OutputStream; typedef struct OutputFile { @@ -875,8 +867,6 @@ InputStream *ist_iter(InputStream *prev); OutputStream *ost_iter(OutputStream *prev); void close_output_stream(OutputStream *ost); -int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt); -int fix_sub_duration_heartbeat(InputStream *ist, int64_t signal_pts); void update_benchmark(const char *fmt, ...); #define SPECIFIER_OPT_FMT_str "%s" diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index b60bad1220..798ddc25b3 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -439,29 +439,6 @@ static int process_subtitle(InputStream *ist, AVFrame *frame) return 0; } -int fix_sub_duration_heartbeat(InputStream *ist, int64_t signal_pts) -{ - Decoder *d = ist->decoder; - int ret = AVERROR_BUG; - AVSubtitle *prev_subtitle = d->sub_prev[0]->buf[0] ? - (AVSubtitle*)d->sub_prev[0]->buf[0]->data : NULL; - AVSubtitle *subtitle; - - if (!ist->fix_sub_duration || !prev_subtitle || - !prev_subtitle->num_rects || signal_pts <= prev_subtitle->pts) - return 0; - - av_frame_unref(d->sub_heartbeat); - ret = subtitle_wrap_frame(d->sub_heartbeat, prev_subtitle, 1); - if (ret < 0) - return ret; - - subtitle = (AVSubtitle*)d->sub_heartbeat->buf[0]->data; - subtitle->pts = signal_pts; - - return process_subtitle(ist, d->sub_heartbeat); -} - static int transcode_subtitles(InputStream *ist, const AVPacket *pkt, AVFrame *frame) { diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index fa4539664f..aae0ba7a73 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -692,13 +692,6 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame) av_ts2str(pkt->duration), av_ts2timestr(pkt->duration, &enc->time_base)); } - if ((ret = trigger_fix_sub_duration_heartbeat(ost, pkt)) < 0) { - av_log(NULL, AV_LOG_ERROR, - "Subtitle heartbeat logic failed in %s! (%s)\n", - __func__, av_err2str(ret)); - return ret; - } - e->data_size += pkt->size; e->packets_encoded++; diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c index 57fb8a8413..bc6ce33483 100644 --- a/fftools/ffmpeg_mux.c +++ b/fftools/ffmpeg_mux.c @@ -502,16 +502,6 @@ int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts) } opkt->dts -= ts_offset; - { - int ret = trigger_fix_sub_duration_heartbeat(ost, pkt); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, - "Subtitle heartbeat logic failed in %s! (%s)\n", - __func__, av_err2str(ret)); - return ret; - } - } - ret = of_output_packet(of, ost, opkt); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 63a25a350f..d5a10e92bd 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -64,7 +64,6 @@ static const char *const opt_name_enc_stats_post_fmt[] = {"enc_stats_post static const char *const opt_name_mux_stats_fmt[] = {"mux_stats_fmt", NULL}; static const char *const opt_name_filters[] = {"filter", "af", "vf", NULL}; static const char *const opt_name_filter_scripts[] = {"filter_script", NULL}; -static const char *const opt_name_fix_sub_duration_heartbeat[] = {"fix_sub_duration_heartbeat", NULL}; static const char *const opt_name_fps_mode[] = {"fps_mode", NULL}; static const char *const opt_name_force_fps[] = {"force_fps", NULL}; static const char *const opt_name_forced_key_frames[] = {"forced_key_frames", NULL}; @@ -1389,9 +1388,6 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, MATCH_PER_STREAM_OPT(bits_per_raw_sample, i, ost->bits_per_raw_sample, oc, st); - MATCH_PER_STREAM_OPT(fix_sub_duration_heartbeat, i, ost->fix_sub_duration_heartbeat, - oc, st); - if (oc->oformat->flags & AVFMT_GLOBALHEADER && ost->enc_ctx) ost->enc_ctx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER; diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 304471dd03..cd1aaabccc 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -970,6 +970,12 @@ static int opt_vstats(void *optctx, const char *opt, const char *arg) return opt_vstats_file(NULL, opt, filename); } +static int opt_fix_sub_duration_heartbeat(void *optctx, const char *opt, const char *arg) +{ + av_log(NULL, AV_LOG_FATAL, "Option '%s' is disabled\n", opt); + return AVERROR(ENOSYS); +} + static int opt_video_frames(void *optctx, const char *opt, const char *arg) { OptionsContext *o = optctx; @@ -1740,8 +1746,7 @@ const OptionDef options[] = { { "autoscale", HAS_ARG | OPT_BOOL | OPT_SPEC | OPT_EXPERT | OPT_OUTPUT, { .off = OFFSET(autoscale) }, "automatically insert a scale filter at the end of the filter graph" }, - { "fix_sub_duration_heartbeat", OPT_VIDEO | OPT_BOOL | OPT_EXPERT | - OPT_SPEC | OPT_OUTPUT, { .off = OFFSET(fix_sub_duration_heartbeat) }, + { "fix_sub_duration_heartbeat", OPT_VIDEO | OPT_EXPERT, { .func_arg = opt_fix_sub_duration_heartbeat }, "set this video output stream to be a heartbeat stream for " "fix_sub_duration, according to which subtitles should be split at " "random access points" }, diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak index 835770a924..ebc1e1f189 100644 --- a/tests/fate/ffmpeg.mak +++ b/tests/fate/ffmpeg.mak @@ -139,18 +139,18 @@ fate-ffmpeg-fix_sub_duration: CMD = fmtstdout srt -fix_sub_duration \ # Basic test for fix_sub_duration_heartbeat, which causes a buffered subtitle # to be pushed out when a video keyframe is received from an encoder. -FATE_SAMPLES_FFMPEG-$(call FILTERDEMDECENCMUX, MOVIE, MPEGVIDEO, \ - MPEG2VIDEO, SUBRIP, SRT, LAVFI_INDEV \ - MPEGVIDEO_PARSER CCAPTION_DECODER \ - MPEG2VIDEO_ENCODER NULL_MUXER PIPE_PROTOCOL) \ - += fate-ffmpeg-fix_sub_duration_heartbeat -fate-ffmpeg-fix_sub_duration_heartbeat: CMD = fmtstdout srt -fix_sub_duration \ - -real_time 1 -f lavfi \ - -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]" \ - -map 0:v -map 0:s -fix_sub_duration_heartbeat:v:0 \ - -c:v mpeg2video -b:v 2M -g 30 -sc_threshold 1000000000 \ - -c:s srt \ - -f null - +#FATE_SAMPLES_FFMPEG-$(call FILTERDEMDECENCMUX, MOVIE, MPEGVIDEO, \ +# MPEG2VIDEO, SUBRIP, SRT, LAVFI_INDEV \ +# MPEGVIDEO_PARSER CCAPTION_DECODER \ +# MPEG2VIDEO_ENCODER NULL_MUXER PIPE_PROTOCOL) \ +# += fate-ffmpeg-fix_sub_duration_heartbeat +#fate-ffmpeg-fix_sub_duration_heartbeat: CMD = fmtstdout srt -fix_sub_duration \ +# -real_time 1 -f lavfi \ +# -i "movie=$(TARGET_SAMPLES)/sub/Closedcaption_rollup.m2v[out0+subcc]" \ +# -map 0:v -map 0:s -fix_sub_duration_heartbeat:v:0 \ +# -c:v mpeg2video -b:v 2M -g 30 -sc_threshold 1000000000 \ +# -c:s srt \ +# -f null - # FIXME: the integer AAC decoder does not produce the same output on all platforms # so until that is fixed we use the volume filter to silence the data