From patchwork Wed Apr 10 03:01:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 47992 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c90a:b0:1a7:a0dc:8de5 with SMTP id gx10csp316754pzb; Tue, 9 Apr 2024 20:01:27 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUODTyh0rWFKPN7EHco4B3VexhcfjQFr+Yh06zCFG5V+K1n/PXkJfFVk9riwvcua3mPhvPi+AQoLJfHCMj/pPJ7mgCOU9SENtwviQ== X-Google-Smtp-Source: AGHT+IHCO9FMzcbtwYfsbAgWgK5UPkpX5pVqzzd6tToJlGK+AFE9xg98hfalc2KHFx2qCVZFANjp X-Received: by 2002:a17:907:25cd:b0:a51:db9e:4cce with SMTP id ae13-20020a17090725cd00b00a51db9e4ccemr827628ejc.3.1712718087335; Tue, 09 Apr 2024 20:01:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712718087; cv=none; d=google.com; s=arc-20160816; b=T5tXFobyMIw4uNNe/liOgxcyie7MZDK5K9WdbiaheqceYbt1WfMoFTp0GpgzF5j1pf nrwx2Kh7OPe18otPHfHE8GqAsS0GS0/phjm7iM07z0cURYF5UDibFf83RYeD9SqC3kb9 lASFMGGhOuXWvngflCV5iGmSyFI0E9JMm569G3cbi8djd7tbWOghkZSKAwdohQlSqF0P pIM+MvKkxFq8K/zUk1N/8kKNrW5BWsnpgDlFz7urMthdhbCcloC17il2NUpHmHkfMID4 JoCZ5YWppPLsamntJtZC8MHjut3A410V4bWja7UuuLnwKXtsUxnUqPtEE9+uL09Sv9Z9 iG3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=h7TjVQZqdyz0nv74VHMTBstufgPZjnWKYICKEtaBK0Y=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=q/uoCVQR8vZCEGtZ9qC6ZkBzMzbd+HhjXtHOhRxdnnnx6svzX2VK2o18C7z7dXmY7Y f9YI3+BT8t6hnsAMTBqnseRxZCYB97s5hEVxzxOkBICM/ek2w4xl8lFhsenDwJbOfNIt 5sOA0QQjQiH9PoQFfzTB07pg2TjwCGx2+3Pshsx+ahttM/vdyGvnpbzuGK9pRKacQkLK d7SD4iidWtkzwU4qp2NiHr+v0g2OW6s7zAZdV1IlNSly3+UC4oRtlnVqRGcEdJsl4MSe 3O69sDYXccIB6GGgoUVqlEnpqHBhoGjdAmIsRbdaWnfut2889vnZ8QiWDaq+jxwgCcwJ o59g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=D1q5N63J; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lv23-20020a170906bc9700b00a4d4254c397si5212325ejb.323.2024.04.09.20.01.26; Tue, 09 Apr 2024 20:01:27 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=D1q5N63J; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8A4C568D109; Wed, 10 Apr 2024 06:01:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 18EF368C8F2 for ; Wed, 10 Apr 2024 06:01:15 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1712718081; x=1744254081; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=F4ha7n2u3ZSIJvVTGGzfSGiN5hkXl3D1XTJHE4Z5OHM=; b=D1q5N63JPHv9xWwHAk2ZGxo4V0+ZojIjYwFKweBzWFM1X95TJEoSO4KP 6ew7ehZXfJLPPidZuVPQURVLjdg4n17c176y6IF16VCDQnGYRic9FH6SV uRs3P5kOZz4s7QGqr+GMIMurG3+lI4ydPNOA7hkFrpjF3+QVPnVZD+guc S6HrLVZkUn4+nEQDlHsnTpZ2XYh61W+GR/A5VL8nQHFlU+tqt6aLzxvF/ mApBvWKm1ZrUJQ9VNCbk0rvMq/f7hGaR85vg+MVNKyc0KFC5pDLclrzZT cfJJ9kJ6TjnBQIknaTCP7aDMr96spvK/lv903K4mw+oOZGK8c2Ks/0srK w==; X-CSE-ConnectionGUID: /OjrDvR1QrqyEAi8JMrATg== X-CSE-MsgGUID: GBg2Q/8DQgWBw2DfKFHM2A== X-IronPort-AV: E=McAfee;i="6600,9927,11039"; a="11901362" X-IronPort-AV: E=Sophos;i="6.07,190,1708416000"; d="scan'208";a="11901362" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Apr 2024 20:01:13 -0700 X-CSE-ConnectionGUID: Q1Kah7LJQIOziRhUqu7A9w== X-CSE-MsgGUID: LDWk1mmyRuKVMKO4T/yiHg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,190,1708416000"; d="scan'208";a="24897110" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Apr 2024 20:01:13 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Apr 2024 11:01:02 +0800 Message-Id: <20240410030103.520402-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] lavc/vaapi_decode: Use dynamic frame pool if possible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bcw2uO4hpuD1 From: Haihao Xiang libva2 doesn't require a fixed surface-array any more, so we may use dynamic frame pool for decoding when libva2 is available, which allows a downstream element stores more frames from VAAPI decoders and fixes the error below: $ ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi \ -i input.mp4 -c:v hevc_vaapi -f null - ... [h264 @ 0x557a075a1400] get_buffer() failed [h264 @ 0x557a075a1400] thread_get_buffer() failed [h264 @ 0x557a075a1400] decode_slice_header error [h264 @ 0x557a075a1400] no frame! Signed-off-by: Haihao Xiang --- libavcodec/vaapi_decode.c | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index 5665639dd7..21b273cd0f 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -599,22 +599,26 @@ static int vaapi_decode_make_config(AVCodecContext *avctx, if (err < 0) goto fail; - frames->initial_pool_size = 1; - // Add per-codec number of surfaces used for storing reference frames. - switch (avctx->codec_id) { - case AV_CODEC_ID_H264: - case AV_CODEC_ID_HEVC: - case AV_CODEC_ID_AV1: - frames->initial_pool_size += 16; - break; - case AV_CODEC_ID_VP9: - frames->initial_pool_size += 8; - break; - case AV_CODEC_ID_VP8: - frames->initial_pool_size += 3; - break; - default: - frames->initial_pool_size += 2; + if (CONFIG_VAAPI_1) + frames->initial_pool_size = 0; + else { + frames->initial_pool_size = 1; + // Add per-codec number of surfaces used for storing reference frames. + switch (avctx->codec_id) { + case AV_CODEC_ID_H264: + case AV_CODEC_ID_HEVC: + case AV_CODEC_ID_AV1: + frames->initial_pool_size += 16; + break; + case AV_CODEC_ID_VP9: + frames->initial_pool_size += 8; + break; + case AV_CODEC_ID_VP8: + frames->initial_pool_size += 3; + break; + default: + frames->initial_pool_size += 2; + } } } From patchwork Wed Apr 10 03:01:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 47993 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c90a:b0:1a7:a0dc:8de5 with SMTP id gx10csp316826pzb; Tue, 9 Apr 2024 20:01:36 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWIGIT+asVk6RFBmUeJ3OZ09R1zKMHz81udXGUjXrDQlf52XSjfekz8G8SD/NPSaDrOtX79PeqeP3CMroNAFz1tyyW4KqxA7h3jaQ== X-Google-Smtp-Source: AGHT+IEUIsJY5H2Vun4bpkrsTbm2pfSQrLI5F2q0VHBbSZ+CHhSFRkQ6expU4H7A16ceRmv6S9WT X-Received: by 2002:a50:d5dd:0:b0:56e:2c34:cff2 with SMTP id g29-20020a50d5dd000000b0056e2c34cff2mr1308050edj.19.1712718096261; Tue, 09 Apr 2024 20:01:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1712718096; cv=none; d=google.com; s=arc-20160816; b=uf8U1RFAqeiLPdjE9YsNfzCKu1i6iMt94VAVU08mYRkl6+3nL5v7rDRAqtmWSJ/4Wl 2nl+vMGIinFomzAI9bW38JXdAYwtuJPCXlbbawpmP6PAcXbK0bfTvIw/qQU24n8Rltt1 ssYZ3BnsHTPjqNkFX4Hvt6rOnUGzQPWSs3ct0OBMNCC9waZBqlBi+AGK2IxdA2KjuXoe oB+9ePPLMFV5viFj8YAWZNvjOpR/iBJEMQkWYksonXaAt/DQH5F2gI0LQeMtkDLZgV0l xI3R4pYyfrELAU8N6RkviN7jv+wnByNuNjoAec1QG22LeKgWfl3D4enYuOxFP00I5QHK 3lrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=a7riJ6UFN+no7QQ50bvCTRBa4mT/smBS+97KUB3VGyU=; fh=1kpmTRI1sW42eNCnNMby7VLIISQBX6KJuVYxOEG0t2A=; b=UxGKYbBpjDdeqOcGNumcfZVKzIp8d2aSiJTxG4Cw436NdrVR/4IbFwCXMvP4IMlknu AbQB8QNqLMuMVqtcttYudPl2p/kGSx+2cZAUo/UphLFCgJBVMURezDC82PAiBOMW++zU Q2RTCZiPyi5LjuK5mv2IIwg8mWgm6kSJu4dleLQHi0fQoNnRWVDi6wBOvrwlO/T5Dvav /kDn3lYHSjAOpvkqEufpsuTrzry6GmI2pzBIAsMGgbaiVNkqZbMT0fJ9VFaChqNif7Na oMqw0w2XEFdL5yWyamBGLI1+ZQON4FLUhccPQMcF5APNvfq5ts+yKtQ+bDjzFDS0kOKQ SvrA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=dPHi4cUj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b15-20020a50b40f000000b0056dfb2fe573si5538786edh.394.2024.04.09.20.01.35; Tue, 09 Apr 2024 20:01:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=dPHi4cUj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A11BF68D128; Wed, 10 Apr 2024 06:01:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0DEBB68C7F8 for ; Wed, 10 Apr 2024 06:01:21 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1712718087; x=1744254087; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4uupQrwimWKZ5re7HbjXE/KB0Cak7AnVZIBnB4SzoU0=; b=dPHi4cUj3l3gajCmPRpJ6jLiy+qVhGgFZMgMoGmxSotF6QqdLQrDDVVW wUl2FDCVoUcw8P/bV9BCu6GJHiRA6Uf0pB94nsziLqt7h4gdXpxNdgfVv DS8K6Pla+R3Y72eQbVx/YCSxp8GZbZ8jV2dGy29ixrALyjsOVoTCUQ9Iz PGLupEmqzymPdtA05NctBNT61B7JTKVZwiVv9jURSdBi5C9SpX/T8ltXp +93iVIeHrIeiKrn3hhhfqtPtEfsiELRdz0Xc21kAWrhWpmI8WbkfC1Zyw webSlLu/bkFhxFrWy4bRyWhvGjBCi3HN7Lc2ehEbB1ZfNaPKSqQMJTe8h A==; X-CSE-ConnectionGUID: lAjpa4JSS5S+aN3uh0YxTQ== X-CSE-MsgGUID: um5uMLf3RViVzoTYQh6THw== X-IronPort-AV: E=McAfee;i="6600,9927,11039"; a="11901364" X-IronPort-AV: E=Sophos;i="6.07,190,1708416000"; d="scan'208";a="11901364" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Apr 2024 20:01:14 -0700 X-CSE-ConnectionGUID: jfP970VdSAGPA6NI1JTzkQ== X-CSE-MsgGUID: YYmaFAd5Srm3BpfsN06YDQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,190,1708416000"; d="scan'208";a="24897115" Received: from unknown (HELO xhh-dg264.sh.intel.com) ([10.238.2.76]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Apr 2024 20:01:14 -0700 From: "Xiang, Haihao" To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Apr 2024 11:01:03 +0800 Message-Id: <20240410030103.520402-2-haihao.xiang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240410030103.520402-1-haihao.xiang@intel.com> References: <20240410030103.520402-1-haihao.xiang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/2] lavfi/vaapi_vpp: Use dynamic frame pool in outlink if possible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5ys21/tKLUN8 From: Haihao Xiang This can avoid to exhaust the buffers within outlink when libva2 is available. For example: $ ffmpeg -hwaccel_output_format vaapi -hwaccel vaapi -i input.mp4 \ -vf 'scale_vaapi=w=720:h=480' -c:v hevc_vaapi -f null - ... [vf#0:0 @ 0x55acad91f400] Error while filtering: Cannot allocate memory [vf#0:0 @ 0x55acad91f400] Task finished with error code: -12 (Cannot allocate memory) [vf#0:0 @ 0x55acad91f400] Terminating thread with return code -12 (Cannot allocate memory) Signed-off-by: Haihao Xiang --- libavfilter/vaapi_vpp.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c index ace1153a23..9ef7a289fb 100644 --- a/libavfilter/vaapi_vpp.c +++ b/libavfilter/vaapi_vpp.c @@ -204,7 +204,10 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink) output_frames->width = ctx->output_width; output_frames->height = ctx->output_height; - output_frames->initial_pool_size = 4; + if (CONFIG_VAAPI_1) + output_frames->initial_pool_size = 0; + else + output_frames->initial_pool_size = 4; err = ff_filter_init_hw_frames(avctx, outlink, 10); if (err < 0) @@ -220,6 +223,8 @@ int ff_vaapi_vpp_config_output(AVFilterLink *outlink) va_frames = output_frames->hwctx; av_assert0(ctx->va_context == VA_INVALID_ID); + av_assert0(output_frames->initial_pool_size || + (va_frames->surface_ids == NULL && va_frames->nb_surfaces == 0)); vas = vaCreateContext(ctx->hwctx->display, ctx->va_config, ctx->output_width, ctx->output_height, VA_PROGRESSIVE,