From patchwork Sun Feb 25 11:18:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 46516 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51b:b0:19e:cdac:8cce with SMTP id gm27csp828166pzb; Sun, 25 Feb 2024 03:18:25 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCXpaF0WVMvqW0XZ24Arb9/G2EtxDmWfXgPqpJkE0gvE45oTdR537KzgbgfNS62Mkm2ggrXt3EV9wzWaMuAb7MQd9IW1nmkZ94p1kw== X-Google-Smtp-Source: AGHT+IHlcedpYeZww6XKweCrCw2abl4ORk87gNYEINusXobZ3HG6oKXg/2udu5Kh9VNyHKNTW4o1 X-Received: by 2002:a05:6402:2c7:b0:565:6ddf:32cf with SMTP id b7-20020a05640202c700b005656ddf32cfmr2630692edx.26.1708859905044; Sun, 25 Feb 2024 03:18:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708859905; cv=none; d=google.com; s=arc-20160816; b=SN08uwfuOelGTkKh+myu/14vH29BmE68GOnMEjuhzkdQodb6L409i4gqoZLcA3z/Qv 2tL3j+gg2HPVrDhfChSTLrkj0n9a7EhcKBi+uHCYI3JwwHeVRxn5s1XlvOFCE/Y3jZw/ AA54Yj1VevqkfkCFqVmGl0FecniT5PNpITs0J8KvuRhXoa/KulmloSeAiNszBMiE/RBo GFz+Z0w5otq8yh3YIdwjIqgZNDLkDgNQHJl7HaX4YhlKQI5Wv5M+QNwwpgglsLHG6EAU Vne1rvjPLaGRyhJ/JO18lVXvrUFUPcnt9SgdlGX+fAZB7IMImWW/L23Kc8qyKkDz/0J4 jq9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:from:to:content-language:user-agent:mime-version :date:message-id:dkim-signature:delivered-to; bh=poPLMxWx+j2BvGM2gMgUl1r867ZXiviT1+XESmvGk50=; fh=e5zN9xSzcxLA6bGo3lF+CqTbY/oLwzApV03EO/RBfgQ=; b=tetFLcaL5DVC/CpZlZGNiYv5P0STO5x0Af5zL6nwHD+IOybBhxqPdBpvXq5wZ0WN7w vvI5AJeJ9B8BqzXRXT4nXkTqFNH0QrA5NUgLbDtd/vIcc9bdemtg9+jiAB/KGXZ20tdq ZmP/PSB7XozBS6EUwiUBCgWtGRdlj4kXWbxjffPe2ZsM7kKOC55JGA8IeS10x78jSivD uclfVG/YZt8vbGFGi5woMh8sRs/tHr7DchrXPgeFzZblDgYDyMBQ4CrAAvV12RCNQz2U a98TKFj+9KSNsVJ6sjUBGjCf3PYqbIJbPnOxB0XVnem+uwnlIUfi8nfnyFi5hIyRGI49 bcEw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20230601.gappssmtp.com header.s=20230601 header.b=gMLQ7GKY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h9-20020a056402280900b00565bc93ae7esi770485ede.295.2024.02.25.03.18.24; Sun, 25 Feb 2024 03:18:25 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20230601.gappssmtp.com header.s=20230601 header.b=gMLQ7GKY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6E62968C74E; Sun, 25 Feb 2024 13:18:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B888268C490 for ; Sun, 25 Feb 2024 13:18:14 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-33d6bd39470so1154641f8f.3 for ; Sun, 25 Feb 2024 03:18:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20230601.gappssmtp.com; s=20230601; t=1708859894; x=1709464694; darn=ffmpeg.org; h=content-transfer-encoding:subject:from:to:content-language :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=fN5OOCV+IhRF9BmumYm5RwCxGJ7Jcz5CJXqNn3i1p9Q=; b=gMLQ7GKYz0nRpCyjP273PBPAxvFgI++ArorlH937FcaY64dux9vICeV7ovU2+zoTDR VNoVTadZDFeY1wUhCEC/ZtwXvTsUCBMtsWmqgZ3ho9rpTJp+YslRAJn6jXAveBgqPVj1 e5hIj4iqbwozAEqfIXMmWSwX4P1PRDNjDvwNUSAJX4nqjpJMawQeigWpqUQJq5BDt2eu Gsd6WDoFqaaugPcTvw2mx/xLhPTrsqPSBFEJqm0gCPC4F8haTqbp+QEwn06PUT84KDH7 PFNO4EEPGwgloZX0e258Ecy5gcVpFHfu4XeZNCLaS9f8jgt3u5Qzq91BwrLsN7yKeCsE zdfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708859894; x=1709464694; h=content-transfer-encoding:subject:from:to:content-language :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fN5OOCV+IhRF9BmumYm5RwCxGJ7Jcz5CJXqNn3i1p9Q=; b=JDwsxRWECdpVbuX2GTKw3S1rheossyKVL5BcrWs94bBgi3Jy542MBfHNQwxbJtyiXC aPKOSBmT6NYwSj7wFh7rMlHcqmYQ1sL4wJYp9do71lTYBf9kUmtkSTWeDkzlYvXp9tW5 LSka4r/u5h8RzrlN4AUd/w5kGMUaQmKMEaqJJAZjjjEMsdomu6zZ4HlsgLx65yPZlTQc oplvIJsEANvkQeezOPMbsK/SrHuBWHymNhhB6fMfMTEDNwfAAiQdBXpCRHJapLQLKQw8 l6bcMWzA3FI6T3bqAxd7ztCTF94Shs/ZusZQ2x4WmkMmpzvwxyhP+kvN/OyjBhtOps82 YYJQ== X-Gm-Message-State: AOJu0YyblFmKg8ERxoolR3qs77mCrh8ZBZAeG9gqmsfWS7fpf2vgcOWH Ns8irjrWGAg9JzmlEh5gLlSsw3ssFYi2bENB9R5s7ZR60VmT6sEoToLj+9b0r2TcKOCT2pnFQVN L X-Received: by 2002:adf:db4b:0:b0:33d:76a9:89ae with SMTP id f11-20020adfdb4b000000b0033d76a989aemr2836703wrj.12.1708859893793; Sun, 25 Feb 2024 03:18:13 -0800 (PST) Received: from [192.168.0.15] (cpc92302-cmbg19-2-0-cust1183.5-4.cable.virginm.net. [82.1.212.160]) by smtp.gmail.com with ESMTPSA id g9-20020a5d5549000000b0033b1b1da2adsm4735104wrw.99.2024.02.25.03.18.13 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 25 Feb 2024 03:18:13 -0800 (PST) Message-ID: <3deacbbf-0087-477c-83b9-910bdba15b7d@jkqxz.net> Date: Sun, 25 Feb 2024 11:18:42 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: FFmpeg development discussions and patches From: Mark Thompson Subject: [FFmpeg-devel] [PATCH] ffmpeg: set extra_hw_frames to account for frames held in queues X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: YHi4Rwz2+7KG Since e0da916b8f5b079a4865eef7f64863f50785463d the ffmpeg utility has held multiple frames output by the decoder in internal queues without telling the decoder that it is going to do so. When the decoder has a fixed-size pool of frames (common in some hardware APIs where the output frames must be stored as an array texture) this could lead to the pool being exhausted and the decoder getting stuck. Fix this by telling the decoder to allocate additional frames according to the queue size. --- fftools/ffmpeg_dec.c | 13 +++++++++++++ fftools/ffmpeg_sched.c | 11 ++++++++++- fftools/ffmpeg_sched.h | 7 +++++++ 3 files changed, 30 insertions(+), 1 deletion(-) diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 8c92b27cc1..0dab989cdd 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -1146,6 +1146,19 @@ int dec_open(Decoder **pdec, Scheduler *sch, goto fail; } + if (dp->dec_ctx->hw_device_ctx) { + // Update decoder extra_hw_frames option to account for the + // frames held in queues inside the ffmpeg utility. This is + // called after avcodec_open2() because the user-set value of + // extra_hw_frames becomes valid in there, and we need to add + // this on top of it. + int extra_frames = DEFAULT_THREAD_QUEUE_SIZE; + if (dp->dec_ctx->extra_hw_frames >= 0) + dp->dec_ctx->extra_hw_frames += extra_frames; + else + dp->dec_ctx->extra_hw_frames = extra_frames; + } + ret = check_avoptions(*dec_opts); if (ret < 0) goto fail; diff --git a/fftools/ffmpeg_sched.c b/fftools/ffmpeg_sched.c index 1144fce958..9ff9431270 100644 --- a/fftools/ffmpeg_sched.c +++ b/fftools/ffmpeg_sched.c @@ -361,7 +361,16 @@ static int queue_alloc(ThreadQueue **ptq, unsigned nb_streams, unsigned queue_si ThreadQueue *tq; ObjPool *op; - queue_size = queue_size > 0 ? queue_size : 8; + queue_size = queue_size > 0 ? queue_size : DEFAULT_THREAD_QUEUE_SIZE; + + if (type == QUEUE_FRAMES) { + // This queue length is used in the decoder code to ensure that + // there are enough entries in fixed-size frame pools to account + // for frames held in queues inside the ffmpeg utility. If this + // can ever dynamically change then the corresponding decode + // code needs to be updated as well. + av_assert0(queue_size == DEFAULT_THREAD_QUEUE_SIZE); + } op = (type == QUEUE_PACKETS) ? objpool_alloc_packets() : objpool_alloc_frames(); diff --git a/fftools/ffmpeg_sched.h b/fftools/ffmpeg_sched.h index 95f9c1d4db..315053ae42 100644 --- a/fftools/ffmpeg_sched.h +++ b/fftools/ffmpeg_sched.h @@ -233,6 +233,13 @@ int sch_add_filtergraph(Scheduler *sch, unsigned nb_inputs, unsigned nb_outputs, */ int sch_add_mux(Scheduler *sch, SchThreadFunc func, int (*init)(void *), void *ctx, int sdp_auto, unsigned thread_queue_size); + +/** + * Default size of a thread queue, used if thread_queue_size is not set on a + * call to sch_add_mux(). + */ +#define DEFAULT_THREAD_QUEUE_SIZE 8 + /** * Add a muxed stream for a previously added muxer. *