From patchwork Tue Jan 11 06:55:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 33201 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp3432743iog; Mon, 10 Jan 2022 22:57:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJyOnvT/lSrnPYahGbWgDWhQpiNzrniWzsDqPY0rsQ402CuKGytLHRj2lc602i9ZyH3XWgt3 X-Received: by 2002:a17:906:540f:: with SMTP id q15mr2550161ejo.641.1641884226974; Mon, 10 Jan 2022 22:57:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1641884226; cv=none; d=google.com; s=arc-20160816; b=G+ptnV/74ufAiDVY3niOMail6o2CY0PH2gKhvPTYikbrejFLcg/NuCiRvkeflohrME RiCm8Q1HuLXaeCkjMnA53hwfHBFAmRJVqp3bVsHMLnclQmYCR8KavQRbZGlwBnsz8q0Z oyJqctG5Ik+PsEptgIoj+WU8YZr2R7RGS5SYH0JUjvB7hbHvfDg/29gxovKj6ZvKyYHA BVP9fC+F8sYsqFiPBDCBV0Z8qI5Ml5VjvfOpwwIJo46nyoTap3l26d6wKyXC+wuoullQ RRYTMDFKVZRMxkUGVrlpBT9WcJQaqGYAMv/QIv8uoSPDqFJHU1GO1F3BiBzqV2efvS00 ufKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=Bg2o7pCCTtYm3WYaKb2IoCXTAFdbdRv90XQI2Jj7jfU=; b=TYW0ztTGQx2cFf8BLf1gOvVptnlvxDa0GxqVf8epJizAXp6hL1KGGhP9REGdv+KbVb nWXfBm/crvE51l48pBnmd5a7yX21IKxQafG2DVTBctbtpVpVbe9cjriTUhCBrxEWCJyz YwtHLUCvy0x83mcAWYO1IRc7vn5kDaAgQ2KxFLUTa9yte8jdhl1SKcwYebNJ3qjNebgE 1EvXY+900UPakjzhisdChbNROJr9aqi2/TDJ/5tJUKOGCa3jV0EOsysP5Ial3khvQqok NaBla1Jgk//yLSocrG5GEiuOHaMdlwwb5lD5ND/6hrUGtSx3Xxetc1z81fqI+Pqes9ND 4WNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=RgUGo0xA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s1si5871175ejn.485.2022.01.10.22.57.06; Mon, 10 Jan 2022 22:57:06 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=RgUGo0xA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 714BD68AECA; Tue, 11 Jan 2022 08:57:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 57EBF68A8E6 for ; Tue, 11 Jan 2022 08:56:54 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641884220; x=1673420220; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=RzNg75RP5h4qCVvbQA5kGCoXWuoRaE+gBXWNlPmNQQU=; b=RgUGo0xA1eOy1++ASnFN2u4f8n2WIPRklCyHYcnCBMXsz4zxGGOIPRtk ToMdFU/6uWj61G/kBQ//Ffic5LIBRnTE5amWtE7JuyVhY5pMP7EtpAYM/ Z5t6AtPlniAbgHjMdmufob76OP08KGg1SzGVwWewtNSTpTnlO7FuoQmdm MpBNM9rQTl7h4+OgKBb5uw3IMY3zcSn/gbxW9PZkmK72uq14Sq6R/aF7k Fw8Gyng87cJIeQPlvCmjWr1QTaSEmumt+A/ibQbUEqFpFLJnTjkIS2jdn ZIsA3UdA6uJmcONLAqBjalqnciXr0ZOmijVNMIc+ihWqVPco6x/XGnyD2 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10223"; a="243214618" X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="243214618" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jan 2022 22:56:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="474427063" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga006.jf.intel.com with ESMTP; 10 Jan 2022 22:56:52 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 11 Jan 2022 14:55:37 +0800 Message-Id: <20220111065538.750653-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V5 1/2] libavcodec/vaapi_decode: fix the problem that init_pool_size < nb_surface X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: F8iLmB8Sn2rU For vaapi if the init_pool_size is not zero, the pool size is fixed. This means max surfaces is init_pool_size, but when mapping vaapi frame to qsv frame, the init_pool_size < nb_surface. The cause is that vaapi_decode_make_config() config the init_pool_size and it is called twice. The first time is to init frame_context and the second time is to init codec. On the second time the init_pool_size is changed to original value so the init_pool_size is lower than the reall size because pool_size used to initialize frame_context need to plus thread_count and 3 (guarantee 4 base work surfaces). Now add code to make sure init_pool_size is only set once. Now the following commandline works: ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 \ -hwaccel_output_format vaapi -i input.264 \ -vf "hwmap=derive_device=qsv,format=qsv" \ -c:v h264_qsv output.264 Signed-off-by: Wenbin Chen --- libavcodec/vaapi_decode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index 665af370ed..da0e72b10b 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -650,7 +650,7 @@ int ff_vaapi_decode_init(AVCodecContext *avctx) ctx->hwctx = ctx->device->hwctx; err = vaapi_decode_make_config(avctx, ctx->frames->device_ref, - &ctx->va_config, avctx->hw_frames_ctx); + &ctx->va_config, NULL); if (err) goto fail; From patchwork Tue Jan 11 06:55:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 33202 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp3432873iog; Mon, 10 Jan 2022 22:57:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJxBfqFuDrsrbNMGrQ1bY/N3Sks+ysAPJOWPiGww8zPHgvnvPhcgQ4ppCORyo4dtnFGwKQCZ X-Received: by 2002:a17:907:7e97:: with SMTP id qb23mr2575134ejc.363.1641884237486; Mon, 10 Jan 2022 22:57:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1641884237; cv=none; d=google.com; s=arc-20160816; b=CrJuNiSOLZExhHnWsCMio9EsHPyxdfWMo5DoqdYbW/eo4qtsVx8idzOHDtRA4/yZBJ 8cvoyppGwXIwwju4xVdmKUMCyLlSl+3PwQYhlwXxH+2w60Od2GrhfUUSQknC04ksVLtL MrRoKljJ5RYiFnvPbk/63XX94fGx8YjbUaBfDJNulrDE5HTM//2JmDB+9JGv1Z2hDoQz 9tGIrU4JCdLMfG5X4SNMOV5wZnXj0NWnRF0+P+p3JRZcpUSt89GqQSQiIBrsNfpVrc60 RuVlySIiJlA0St5T99K/MC7riMoAabKlH/t8Xs6qv1DERoK6QMEv0MlBMNw2Fofv7wgN EdPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=SkGxLtyLxJDmP5FOPmQnBQ/S3RUhsSw3N2dLUt3iw8U=; b=eXxIVEOnXAiiqCro7JG1kn9Qv18uiop37BMw+OsWbMRDU8cGudPLjvAoiIF9sp1SRr bFFT/f3YsS0+ZBnitzLWcC3Hfs0CYPKRc0/xqentzZyf+dgt4sRaeUhY2YyC7G0uWs9S X/VV9EQk7BG0IwyyCw97m3k0psqhgSnVEMWj9QBcFSt2D2MLGrvp38AaBBzDoyXmdpFS h6w+NAkFG2MPw2KDuK2iyRpIyLvtp6BGN/kEAv1obsQdylrtYORhflBn3XeRrMxxbKuV JJGMVg/28xW6MAAT1SkowDWGbMjR5IXl2LrCxd4ergbQVMNLBnMpdyBlnHHHuaEJIeDD yspQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Z9P1Oi+K; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gl1si5463783ejc.903.2022.01.10.22.57.16; Mon, 10 Jan 2022 22:57:17 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Z9P1Oi+K; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8F66568AEE2; Tue, 11 Jan 2022 08:57:03 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9545568ACA5 for ; Tue, 11 Jan 2022 08:56:55 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1641884220; x=1673420220; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=bzf8aR9MXsJKSoR89E8dAEJGZBAnBzlw5iYIstI5djg=; b=Z9P1Oi+KLM8LLUgk1KGitPRjgw0c6Onk1/ZxFpAWqLjMQuGjFRCwy+jA +3ibyrkGk5A/oWd8ESW8pajolQiD29kUDjRnYV4YSd6N9oaeDWhFR0DiG l5rB4esIrqz/Dnco1qOnxAj6Yz54opyDvomWqZBSuHC95VV8k8R3c0QxM Vtpl5sUht7+kmudOCnuS5p80lFYEukzSqxoRcPIHN+dqWs2HXyh289v2b qPKUfkRtzRBJ1DHs3GpYn04XHCnRFHZNJkKyMp2guIyb7wpg+SQxLjpsM 2Ics70SmQBgRyWUiIBmWD3b26nWt85n5e2hNdnte44C1UerZW2kUuTJpk w==; X-IronPort-AV: E=McAfee;i="6200,9189,10223"; a="243214623" X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="243214623" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jan 2022 22:56:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,279,1635231600"; d="scan'208";a="474427064" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by orsmga006.jf.intel.com with ESMTP; 10 Jan 2022 22:56:52 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Tue, 11 Jan 2022 14:55:38 +0800 Message-Id: <20220111065538.750653-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220111065538.750653-1-wenbin.chen@intel.com> References: <20220111065538.750653-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V5 2/2] libavutil/hwcontext_opencl: fix a bug for mapping qsv frame to opencl X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Uoj/8GA4An3/ From: nyanmisaka mfxHDLPair was added to qsv, so modify qsv->opencl map function as well. Now the following commandline works: ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 \ -init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va -filter_hw_device ocl \ -hwaccel qsv -hwaccel_output_format qsv -hwaccel_device qs -c:v h264_qsv \ -i input.264 -vf "hwmap=derive_device=opencl,format=opencl,avgblur_opencl, \ hwmap=derive_device=qsv:reverse=1:extra_hw_frames=32,format=qsv" \ -c:v h264_qsv output.264 Signed-off-by: nyanmisaka Signed-off-by: Wenbin Chen --- libavutil/hwcontext_opencl.c | 14 +++++++++++++- libavutil/hwcontext_qsv.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 26a3a24593..4e2ab18ede 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -72,6 +72,12 @@ #include "hwcontext_drm.h" #endif +#if HAVE_OPENCL_VAAPI_INTEL_MEDIA && CONFIG_LIBMFX +extern int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, + enum AVHWDeviceType base_dev_typ, + void **base_handle); +#endif + typedef struct OpenCLDeviceContext { // Default command queue to use for transfer/mapping operations on @@ -2248,8 +2254,14 @@ static int opencl_map_from_qsv(AVHWFramesContext *dst_fc, AVFrame *dst, #if CONFIG_LIBMFX if (src->format == AV_PIX_FMT_QSV) { + void *base_handle; mfxFrameSurface1 *mfx_surface = (mfxFrameSurface1*)src->data[3]; - va_surface = *(VASurfaceID*)mfx_surface->Data.MemId; + err = ff_qsv_get_surface_base_handle(mfx_surface, + AV_HWDEVICE_TYPE_VAAPI, + &base_handle); + if (err < 0) + return err; + va_surface = *(VASurfaceID *)base_handle; } else #endif if (src->format == AV_PIX_FMT_VAAPI) { diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 853fb7f60d..6d9b8324c2 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -112,6 +112,40 @@ static const struct { #endif }; +extern int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, + enum AVHWDeviceType base_dev_type, + void **base_handle); + +/** + * Caller needs to allocate enough space for base_handle pointer. + **/ +int ff_qsv_get_surface_base_handle(mfxFrameSurface1 *surf, + enum AVHWDeviceType base_dev_type, + void **base_handle) +{ + mfxHDLPair *handle_pair; + handle_pair = surf->Data.MemId; + switch (base_dev_type) { +#if CONFIG_VAAPI + case AV_HWDEVICE_TYPE_VAAPI: + base_handle[0] = handle_pair->first; + return 0; +#endif +#if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + base_handle[0] = handle_pair->first; + base_handle[1] = handle_pair->secode; + return 0; +#endif +#if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + base_handle[0] = handle_pair->first; + return 0; +#endif + } + return AVERROR(EINVAL); +} + static uint32_t qsv_fourcc_from_pix_fmt(enum AVPixelFormat pix_fmt) { int i;