From patchwork Tue Sep 19 19:10:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 43808 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a886:b0:149:dfde:5c0a with SMTP id ca6csp170545pzb; Tue, 19 Sep 2023 12:24:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEvNRp3muQaZo6oWABiuExtpzdNkDr5Ri/5KX0j/oYLVoj5Prxad1qG4DE/JqO4ip0k7117 X-Received: by 2002:a17:907:75c2:b0:9ae:3f69:9b8c with SMTP id jl2-20020a17090775c200b009ae3f699b8cmr293226ejc.2.1695151493678; Tue, 19 Sep 2023 12:24:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695151493; cv=none; d=google.com; s=arc-20160816; b=dprdk74LtBIWQ/LfoROU9g8l0SDSS6rpOiZx/He55LAL2/p2o2No0jBWPlDpbY47Ia aTTmDVZp8fr53w0qDA/1Af8mztuQzMr2WifNXK+Yxjy819Q0SSYlNe32Bk1XJevnId+o W2wL9gI8a25ZJuLlnFSPuh1Tl6hsLH78Nw181EOnvtlegl6l++HOxb2eMzh7PT2yCJrf 6mStl7760SWg5Ji9KRNM9QUklegQg+5JXJwQGzs0ok4jWcM7on1zUxdmZhtNUc83vi5e HD/QFnp8l5hDcEEFnH/9OTyzFL8S7JXkO3TRUFSMB9QHtBAAHw880znxlCBHjO7QR/FH SFNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=UkBqrTI1+82KgzbGRhpVaKAdVZEpM41w6HogD5ndTQk=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=YMAWpBcF7PUaHUf3YIf3tDbtynG5z4DMhgKh7RFY9rhSg19ViIquM2A7nHNwPypFS2 W861rDW6UTJVJAfiIERd52Q1YlnO0lmenv55D1zeG+DAW0Dc9imFrl/8WmVc2G5D1SYj 2wdT14usMXELI8U9JyJ8RTZ+g3mB1KoBT6sn0oHHZN8mw8CiFAlVAQxBWCZbZJwMPIMe 6prKx+uX8rOuK5+O+xsEYki0NDDGqTVfzZPDvpLDMphMmSfElox3sQOT3iR8KheghKaV iqCYv3hG94bck6WzYPHBbW6uj4UOEfr21JWk3iVt2UWT2rmBjQKCbPBOjZXINuP9yNuk OU7g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jj12-20020a170907984c00b0098874379199si10470961ejc.163.2023.09.19.12.24.53; Tue, 19 Sep 2023 12:24:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AE55068C9C5; Tue, 19 Sep 2023 22:21:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C3CD868C9B3 for ; Tue, 19 Sep 2023 22:21:50 +0300 (EEST) Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 6C2C110 for ; Tue, 19 Sep 2023 21:21:50 +0200 (CEST) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id x6oMwU6bj9ui for ; Tue, 19 Sep 2023 21:21:45 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 45B3C522E for ; Tue, 19 Sep 2023 21:20:43 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 3919E3A0212 for ; Tue, 19 Sep 2023 21:20:43 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Sep 2023 21:10:52 +0200 Message-Id: <20230919191044.18873-26-anton@khirnov.net> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230919191044.18873-1-anton@khirnov.net> References: <20230919191044.18873-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 25/27] WIP fftools/ffmpeg_enc: convert to the scheduler X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PLVJ8vqZP8mI --- fftools/ffmpeg.c | 1 - fftools/ffmpeg.h | 4 +- fftools/ffmpeg_dec.c | 1 + fftools/ffmpeg_enc.c | 231 ++++++-------------------------------- fftools/ffmpeg_mux_init.c | 43 ++----- 5 files changed, 45 insertions(+), 235 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index a09a9e1200..69e73b84ed 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -1228,7 +1228,6 @@ static int transcode(Scheduler *sch, int *err_rate_exceeded) } else if (err_rate) av_log(ist, AV_LOG_VERBOSE, "Decode error rate %g\n", err_rate); } - ret = err_merge(ret, enc_flush()); term_exit(); diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index eb4e8e27a9..66b5e398bc 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -797,10 +797,8 @@ int enc_alloc(Encoder **penc, const AVCodec *codec, Scheduler *sch, unsigned sch_idx); void enc_free(Encoder **penc); -int enc_open(OutputStream *ost, const AVFrame *frame); +int enc_open(void *opaque, const AVFrame *frame); int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub); -int enc_frame(OutputStream *ost, AVFrame *frame); -int enc_flush(void); /* * Initialize muxing state for the given stream, should be called diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 400fa666b9..d2dc27303d 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -377,6 +377,7 @@ static int process_subtitle(InputStream *ist, AVFrame *frame) if (!ost->enc || ost->type != AVMEDIA_TYPE_SUBTITLE) continue; + // XXX ret = enc_subtitle(output_files[ost->file_index], ost, subtitle); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c index 9bede78a1e..0944cddf1f 100644 --- a/fftools/ffmpeg_enc.c +++ b/fftools/ffmpeg_enc.c @@ -57,21 +57,6 @@ struct Encoder { Scheduler *sch; unsigned sch_idx; - - pthread_t thread; - /** - * Queue for sending frames from the main thread to - * the encoder thread. - */ - ThreadQueue *queue_in; - /** - * Queue for sending encoded packets from the encoder thread - * to the main thread. - * - * An empty packet is sent to signal that a previously sent - * frame has been fully processed. - */ - ThreadQueue *queue_out; }; // data that is local to the decoder thread and not visible outside of it @@ -80,24 +65,6 @@ typedef struct EncoderThread { AVPacket *pkt; } EncoderThread; -static int enc_thread_stop(Encoder *e) -{ - void *ret; - - if (!e->queue_in) - return 0; - - tq_send_finish(e->queue_in, 0); - tq_receive_finish(e->queue_out, 0); - - pthread_join(e->thread, &ret); - - tq_free(&e->queue_in); - tq_free(&e->queue_out); - - return (int)(intptr_t)ret; -} - void enc_free(Encoder **penc) { Encoder *enc = *penc; @@ -105,8 +72,6 @@ void enc_free(Encoder **penc) if (!enc) return; - enc_thread_stop(enc); - av_frame_free(&enc->sq_frame); av_frame_free(&enc->sub_frame); @@ -223,52 +188,9 @@ static int set_encoder_id(OutputFile *of, OutputStream *ost) return 0; } -static int enc_thread_start(OutputStream *ost) -{ - Encoder *e = ost->enc; - ObjPool *op; - int ret = 0; - - op = objpool_alloc_frames(); - if (!op) - return AVERROR(ENOMEM); - - e->queue_in = tq_alloc(1, 1, op, frame_move); - if (!e->queue_in) { - objpool_free(&op); - return AVERROR(ENOMEM); - } - - op = objpool_alloc_packets(); - if (!op) - goto fail; - - e->queue_out = tq_alloc(1, 4, op, pkt_move); - if (!e->queue_out) { - objpool_free(&op); - goto fail; - } - - ret = pthread_create(&e->thread, NULL, encoder_thread, ost); - if (ret) { - ret = AVERROR(ret); - av_log(ost, AV_LOG_ERROR, "pthread_create() failed: %s\n", - av_err2str(ret)); - goto fail; - } - - return 0; -fail: - if (ret >= 0) - ret = AVERROR(ENOMEM); - - tq_free(&e->queue_in); - tq_free(&e->queue_out); - return ret; -} - -int enc_open(OutputStream *ost, const AVFrame *frame) +int enc_open(void *opaque, const AVFrame *frame) { + OutputStream *ost = opaque; InputStream *ist = ost->ist; Encoder *e = ost->enc; AVCodecContext *enc_ctx = ost->enc_ctx; @@ -276,6 +198,7 @@ int enc_open(OutputStream *ost, const AVFrame *frame) const AVCodec *enc = enc_ctx->codec; OutputFile *of = output_files[ost->file_index]; FrameData *fd; + int frame_samples = 0; int ret; if (e->opened) @@ -425,11 +348,8 @@ int enc_open(OutputStream *ost, const AVFrame *frame) return AVERROR(ENOMEM); } - if (ost->enc_ctx->frame_size) { - av_assert0(ost->sq_idx_encode >= 0); - sq_frame_samples(output_files[ost->file_index]->sq_encode, - ost->sq_idx_encode, ost->enc_ctx->frame_size); - } + if (ost->enc_ctx->frame_size) + frame_samples = ost->enc_ctx->frame_size; ret = check_avoptions(ost->encoder_opts); if (ret < 0) @@ -489,18 +409,11 @@ int enc_open(OutputStream *ost, const AVFrame *frame) if (ost->st->time_base.num <= 0 || ost->st->time_base.den <= 0) ost->st->time_base = av_add_q(ost->enc_ctx->time_base, (AVRational){0, 1}); - ret = enc_thread_start(ost); - if (ret < 0) { - av_log(ost, AV_LOG_ERROR, "Error starting encoder thread: %s\n", - av_err2str(ret)); - return ret; - } - ret = of_stream_init(of, ost); if (ret < 0) return ret; - return 0; + return frame_samples; } static int check_recording_time(OutputStream *ost, int64_t ts, AVRational tb) @@ -593,7 +506,7 @@ static int do_subtitle_out(OutputFile *of, OutputStream *ost, const AVSubtitle * } pkt->dts = pkt->pts; - ret = tq_send(e->queue_out, 0, pkt); + ret = sch_enc_send(e->sch, e->sch_idx, pkt); if (ret < 0) { av_packet_unref(pkt); return ret; @@ -827,7 +740,7 @@ static int encode_frame(OutputFile *of, OutputStream *ost, AVFrame *frame, e->packets_encoded++; - ret = tq_send(e->queue_out, 0, pkt); + ret = sch_enc_send(e->sch, e->sch_idx, pkt); if (ret < 0) { av_packet_unref(pkt); return ret; @@ -1047,13 +960,32 @@ void *encoder_thread(void *arg) enc_thread_set_name(ost); - while (!input_status) { - int dummy; + /* Open the subtitle encoders immediately. AVFrame-based encoders + * are opened through a callback from the scheduler once they get + * their first frame */ + if (ost->type != AVMEDIA_TYPE_VIDEO && ost->type != AVMEDIA_TYPE_AUDIO) { + ret = enc_open(ost, NULL); + if (ret < 0) + goto finish; + } - input_status = tq_receive(e->queue_in, &dummy, et.frame); - if (input_status < 0) + while (!input_status) { + input_status = sch_enc_receive(e->sch, e->sch_idx, et.frame); + if (input_status == AVERROR_EOF) { av_log(ost, AV_LOG_VERBOSE, "Encoder thread received EOF\n"); + if (!e->opened) { + av_log(ost, AV_LOG_ERROR, "Could not open encoder before EOF\n"); + ret = AVERROR(EINVAL); + goto finish; + } + } else if (input_status < 0) { + ret = input_status; + av_log(ost, AV_LOG_ERROR, "Error receiving a frame for encoding: %s\n", + av_err2str(ret)); + goto finish; + } + ret = frame_encode(ost, input_status >= 0 ? et.frame : NULL, et.pkt); av_packet_unref(et.pkt); @@ -1067,15 +999,6 @@ void *encoder_thread(void *arg) av_err2str(ret)); break; } - - // signal to the consumer thread that the frame was encoded - ret = tq_send(e->queue_out, 0, et.pkt); - if (ret < 0) { - if (ret != AVERROR_EOF) - av_log(ost, AV_LOG_ERROR, - "Error communicating with the main thread\n"); - break; - } } // EOF is normal thread termination @@ -1086,8 +1009,7 @@ finish: if (ost->sq_idx_encode >= 0) sq_send(of->sq_encode, ost->sq_idx_encode, SQFRAME(NULL)); - tq_receive_finish(e->queue_in, 0); - tq_send_finish (e->queue_out, 0); + sch_enc_send(e->sch, e->sch_idx, NULL); enc_thread_uninit(&et); @@ -1096,62 +1018,6 @@ finish: return (void*)(intptr_t)ret; } -int enc_frame(OutputStream *ost, AVFrame *frame) -{ - OutputFile *of = output_files[ost->file_index]; - Encoder *e = ost->enc; - int ret, thread_ret; - - ret = enc_open(ost, frame); - if (ret < 0) - return ret; - - // thread already joined - // XXX check EOF handling - if (!e->queue_in) - return AVERROR_EOF; - - // send the frame/EOF to the encoder thread - if (frame) { - ret = tq_send(e->queue_in, 0, frame); - if (ret < 0) - goto finish; - } else - tq_send_finish(e->queue_in, 0); - - // retrieve all encoded data for the frame - while (1) { - int dummy; - - ret = tq_receive(e->queue_out, &dummy, e->pkt); - if (ret < 0) - break; - - // frame fully encoded - if (!e->pkt->data && !e->pkt->side_data_elems) - return 0; - - // process the encoded packet - ret = of_output_packet(of, ost, e->pkt); - if (ret < 0) - goto finish; - } - -finish: - thread_ret = enc_thread_stop(e); - if (thread_ret < 0) { - av_log(ost, AV_LOG_ERROR, "Encoder thread returned error: %s\n", - av_err2str(thread_ret)); - ret = err_merge(ret, thread_ret); - } - - if (ret < 0 && ret != AVERROR_EOF) - return ret; - - // signal EOF to the muxer - return of_output_packet(of, ost, NULL); -} - int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) { Encoder *e = ost->enc; @@ -1166,38 +1032,11 @@ int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub) if (ret < 0) return ret; + // XXX +#if 0 ret = enc_frame(ost, f); +#endif av_frame_unref(f); return ret; } - -int enc_flush(void) -{ - int ret = 0; - - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { - OutputFile *of = output_files[ost->file_index]; - if (ost->sq_idx_encode >= 0) - sq_send(of->sq_encode, ost->sq_idx_encode, SQFRAME(NULL)); - } - - for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) { - Encoder *e = ost->enc; - AVCodecContext *enc = ost->enc_ctx; - int err; - - if (!enc || !e->opened || - (enc->codec_type != AVMEDIA_TYPE_VIDEO && enc->codec_type != AVMEDIA_TYPE_AUDIO)) - continue; - - err = enc_frame(ost, NULL); - // XXX check EOF handling - if (err != AVERROR_EOF && ret < 0) - ret = err_merge(ret, err); - - av_assert0(!e->queue_in); - } - - return ret; -} diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 3380cbeb5c..9fb6a74393 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1186,7 +1186,8 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, if (!ost->enc_ctx) return AVERROR(ENOMEM); - ret = sch_add_enc(mux->sch, encoder_thread, ost, NULL); + ret = sch_add_enc(mux->sch, encoder_thread, ost, + ost->type == AVMEDIA_TYPE_SUBTITLE ? NULL : enc_open); if (ret < 0) return ret; ms->sched_idx_enc = ret; @@ -2621,23 +2622,6 @@ static int validate_enc_avopt(Muxer *mux, const AVDictionary *codec_avopt) return 0; } -static int init_output_stream_nofilter(OutputStream *ost) -{ - int ret = 0; - - if (ost->enc_ctx) { - ret = enc_open(ost, NULL); - if (ret < 0) - return ret; - } else { - ret = of_stream_init(output_files[ost->file_index], ost); - if (ret < 0) - return ret; - } - - return ret; -} - static const char *output_file_item_name(void *obj) { const Muxer *mux = obj; @@ -2822,26 +2806,15 @@ int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) of->url = filename; - /* initialize stream copy and subtitle/data streams. - * Encoded AVFrame based streams will get initialized when the first AVFrame - * is received in do_video_out - */ + /* initialize streamcopy streams. */ for (int i = 0; i < of->nb_streams; i++) { OutputStream *ost = of->streams[i]; - if (ost->filter) - continue; - - err = init_output_stream_nofilter(ost); - if (err < 0) - return err; - } - - /* write the header for files with no streams */ - if (of->format->flags & AVFMT_NOSTREAMS && oc->nb_streams == 0) { - int ret = mux_check_init(mux); - if (ret < 0) - return ret; + if (!ost->enc) { + err = of_stream_init(of, ost); + if (err < 0) + return err; + } } return 0;