From patchwork Wed Jan 24 19:29:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 45796 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:120f:b0:199:de12:6fa6 with SMTP id v15csp1493595pzf; Wed, 24 Jan 2024 11:29:31 -0800 (PST) X-Google-Smtp-Source: AGHT+IHjpDZsk5Ey278WhvFHjkYdUcHajz1pfezjez3pZHFTfiZJgeaIH3ubR3rTw2sl6crsP9KL X-Received: by 2002:a05:600c:548e:b0:40e:b95e:6ef2 with SMTP id iv14-20020a05600c548e00b0040eb95e6ef2mr1545205wmb.22.1706124571107; Wed, 24 Jan 2024 11:29:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1706124571; cv=none; d=google.com; s=arc-20160816; b=kBspz9vCULH5neo4rzJjAYlBbGWqF4wfdbhXvNipjjRT4Zl7P70GTvpBffUn5o0PUx Nq+bCWf9PZVzAlK39HKGL5/+y5D9uULXbz6b27o1WAuCvfUj54dcvyreQ7XF1DevHgOt oqeYgrYkdkIcQ5PZr7S7dwzoT/1kIOLTSyNQ/V2TzDRonNu/NR9fL2REsFbNX2B1sA6E u39ATFsZ1aBo0Oyg1l5v994kRgxl0hbATJvEVrQuZt6lJ/X2x1LcWjCRcaXbjKZhkpk+ PxId92xkhqy8bkIq5SVtrF0mTeu4Wj/DwqPLDN8TgbRLfAQfcvMN0wb6rvbeKn4ttq0Q nvJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=jP848/6NqmfR/p1nssaZkAJSYiU/yf9uIpLckRCiCfA=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=bcqTvyAIv1YKaKXStLzsGbEpkCYwgneS6+si7kpkNMm/5oTajyb2+SKhSZmZ9oThOo FGBK1ghQKriGsaxLPa0rlutB3vt86TiW27X8plT184qub/lpFYMOLve97xnuRDunwUUg 0NaylEQbjpcqvbnA1RdrWI8p8ggy1svZ8fbje8l75EoIHwX/C4icMjOIndE+xFUqAT28 6kM0G/SkesZjZ0MkeCujQM4pXJZefl3Ilz6D9zIfH4PxZmYMwTP+XLxXg+gkGQ/eD/VL n6EGaHTA8gn1CzNn3eAe6ORWiIZHEJhLi5+g4rHnZeaXUDAQ1w3RvpzAHWBuc2vMoO5e 2h6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=Lkaa8UCT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ce25-20020a170906b25900b00a30136c1e5csi175683ejb.212.2024.01.24.11.29.30; Wed, 24 Jan 2024 11:29:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@khirnov.net header.s=mail header.b=Lkaa8UCT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 10B3C68D0D9; Wed, 24 Jan 2024 21:29:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail1.khirnov.net (quelana.khirnov.net [94.230.150.81]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 15C2468CD89 for ; Wed, 24 Jan 2024 21:29:21 +0200 (EET) Authentication-Results: mail1.khirnov.net; dkim=pass (2048-bit key; unprotected) header.d=khirnov.net header.i=@khirnov.net header.a=rsa-sha256 header.s=mail header.b=Lkaa8UCT; dkim-atps=neutral Received: from localhost (mail1.khirnov.net [IPv6:::1]) by mail1.khirnov.net (Postfix) with ESMTP id 3F1E219C0 for ; Wed, 24 Jan 2024 20:29:20 +0100 (CET) Received: from mail1.khirnov.net ([IPv6:::1]) by localhost (mail1.khirnov.net [IPv6:::1]) (amavis, port 10024) with ESMTP id sr5RVjGtLsZf for ; Wed, 24 Jan 2024 20:29:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=khirnov.net; s=mail; t=1706124559; bh=pmlzyu9BrF1QyNR5meGRWG8yFKDEbHtaTrFPdQSyK4M=; h=From:To:Subject:Date:From; b=Lkaa8UCTNT8qtlaaqDIeNhrpcXLNED5vbQE71DQpJyFhQoffbk0GN0Losa3jV/DzW 0UDolTElcfQlnG6qkNmo/84U6gVszT9fSzP5TNW4FsYIuj7hpf6FfChkJ+42emZYwz r0wRYL1a7RNEWHKv2DhUaoZVuKF3Pw8LsDZkvlGXEG906MVpKfwZnzXtY+OXjQumqL DjbWqafi5N0uNJVoIicLayIBiSqFFG0ybd0RjRO2Egf+4jvDfdFEfAxYk9pghaX/+E oTiyWKcLqOSu2FKxdSXRbsuYVziMPEIPYFz3WMPsDnVt2Cx6K4FNgTNLttklrgN/5z fl0y3/BdZVjyw== Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail1.khirnov.net (Postfix) with ESMTPS id 338B3EE9 for ; Wed, 24 Jan 2024 20:29:19 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 0AC113A0534 for ; Wed, 24 Jan 2024 20:29:19 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Wed, 24 Jan 2024 20:29:16 +0100 Message-ID: <20240124192916.22857-1-anton@khirnov.net> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] fftools/ffmpeg: optimize inter-thread queue sizes X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: LTLkPOAQweQA Use 8 packets/frames by default rather than 1, which seems to provide better throughput. Allow -thread_queue_size to set the muxer queue size manually again. --- fftools/ffmpeg_mux.h | 2 -- fftools/ffmpeg_mux_init.c | 3 +-- fftools/ffmpeg_opt.c | 2 +- fftools/ffmpeg_sched.c | 15 ++++++++++----- fftools/ffmpeg_sched.h | 4 +++- tests/ref/fate/ffmpeg-fix_sub_duration_heartbeat | 5 ----- 6 files changed, 15 insertions(+), 16 deletions(-) diff --git a/fftools/ffmpeg_mux.h b/fftools/ffmpeg_mux.h index d0be8a51ea..e1b44142cf 100644 --- a/fftools/ffmpeg_mux.h +++ b/fftools/ffmpeg_mux.h @@ -94,8 +94,6 @@ typedef struct Muxer { AVDictionary *opts; - int thread_queue_size; - /* filesize limit expressed in bytes */ int64_t limit_filesize; atomic_int_least64_t last_filesize; diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 6b5e4f8b3c..8ada837555 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -3047,7 +3047,6 @@ int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) of->start_time = o->start_time; of->shortest = o->shortest; - mux->thread_queue_size = o->thread_queue_size > 0 ? o->thread_queue_size : 8; mux->limit_filesize = o->limit_filesize; av_dict_copy(&mux->opts, o->g->format_opts, 0); @@ -3081,7 +3080,7 @@ int of_open(const OptionsContext *o, const char *filename, Scheduler *sch) } err = sch_add_mux(sch, muxer_thread, mux_check_init, mux, - !strcmp(oc->oformat->name, "rtp")); + !strcmp(oc->oformat->name, "rtp"), o->thread_queue_size); if (err < 0) return err; mux->sch = sch; diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 304c493dcf..7505b0cf90 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -144,7 +144,7 @@ static void init_options(OptionsContext *o) o->limit_filesize = INT64_MAX; o->chapters_input_file = INT_MAX; o->accurate_seek = 1; - o->thread_queue_size = -1; + o->thread_queue_size = 0; o->input_sync_ref = -1; o->find_stream_info = 1; o->shortest_buf_duration = 10.f; diff --git a/fftools/ffmpeg_sched.c b/fftools/ffmpeg_sched.c index 4fc5a33941..62a40c6057 100644 --- a/fftools/ffmpeg_sched.c +++ b/fftools/ffmpeg_sched.c @@ -218,6 +218,7 @@ typedef struct SchMux { */ atomic_int mux_started; ThreadQueue *queue; + unsigned queue_size; AVPacket *sub_heartbeat_pkt; } SchMux; @@ -358,6 +359,8 @@ static int queue_alloc(ThreadQueue **ptq, unsigned nb_streams, unsigned queue_si ThreadQueue *tq; ObjPool *op; + queue_size = queue_size > 0 ? queue_size : 8; + op = (type == QUEUE_PACKETS) ? objpool_alloc_packets() : objpool_alloc_frames(); if (!op) @@ -653,7 +656,7 @@ static const AVClass sch_mux_class = { }; int sch_add_mux(Scheduler *sch, SchThreadFunc func, int (*init)(void *), - void *arg, int sdp_auto) + void *arg, int sdp_auto, unsigned thread_queue_size) { const unsigned idx = sch->nb_mux; @@ -667,6 +670,7 @@ int sch_add_mux(Scheduler *sch, SchThreadFunc func, int (*init)(void *), mux = &sch->mux[idx]; mux->class = &sch_mux_class; mux->init = init; + mux->queue_size = thread_queue_size; task_init(sch, &mux->task, SCH_NODE_TYPE_MUX, idx, func, arg); @@ -773,7 +777,7 @@ int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *ctx, if (!dec->send_frame) return AVERROR(ENOMEM); - ret = queue_alloc(&dec->queue, 1, 1, QUEUE_PACKETS); + ret = queue_alloc(&dec->queue, 1, 0, QUEUE_PACKETS); if (ret < 0) return ret; @@ -813,7 +817,7 @@ int sch_add_enc(Scheduler *sch, SchThreadFunc func, void *ctx, task_init(sch, &enc->task, SCH_NODE_TYPE_ENC, idx, func, ctx); - ret = queue_alloc(&enc->queue, 1, 1, QUEUE_FRAMES); + ret = queue_alloc(&enc->queue, 1, 0, QUEUE_FRAMES); if (ret < 0) return ret; @@ -861,7 +865,7 @@ int sch_add_filtergraph(Scheduler *sch, unsigned nb_inputs, unsigned nb_outputs, if (ret < 0) return ret; - ret = queue_alloc(&fg->queue, fg->nb_inputs + 1, 1, QUEUE_FRAMES); + ret = queue_alloc(&fg->queue, fg->nb_inputs + 1, 0, QUEUE_FRAMES); if (ret < 0) return ret; @@ -1313,7 +1317,8 @@ int sch_start(Scheduler *sch) } } - ret = queue_alloc(&mux->queue, mux->nb_streams, 1, QUEUE_PACKETS); + ret = queue_alloc(&mux->queue, mux->nb_streams, mux->queue_size, + QUEUE_PACKETS); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_sched.h b/fftools/ffmpeg_sched.h index b167d8d158..d12affa69d 100644 --- a/fftools/ffmpeg_sched.h +++ b/fftools/ffmpeg_sched.h @@ -225,12 +225,14 @@ int sch_add_filtergraph(Scheduler *sch, unsigned nb_inputs, unsigned nb_outputs, * streams in the muxer. * @param ctx Muxer state; will be passed to func/init and used for logging. * @param sdp_auto Determines automatic SDP writing - see sch_sdp_filename(). + * @param thread_queue_size number of packets that can be buffered before + * sending to the muxer blocks * * @retval ">=0" Index of the newly-created muxer. * @retval "<0" Error code. */ int sch_add_mux(Scheduler *sch, SchThreadFunc func, int (*init)(void *), - void *ctx, int sdp_auto); + void *ctx, int sdp_auto, unsigned thread_queue_size); /** * Add a muxed stream for a previously added muxer. * diff --git a/tests/ref/fate/ffmpeg-fix_sub_duration_heartbeat b/tests/ref/fate/ffmpeg-fix_sub_duration_heartbeat index bc9b833799..3a3ec96637 100644 --- a/tests/ref/fate/ffmpeg-fix_sub_duration_heartbeat +++ b/tests/ref/fate/ffmpeg-fix_sub_duration_heartbeat @@ -33,8 +33,3 @@ {\an7}( inaudible radio chatter ) >> Safety remains our numb -9 -00:00:03,704 --> 00:00:04,004 -{\an7}( inaudible radio chatter ) ->> Safety remains our number one -