From patchwork Thu Nov 4 05:44:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31292 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:a610:0:0:0:0:0 with SMTP id q16csp146084ioi; Wed, 3 Nov 2021 22:48:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyZ9WO1C5gw0nVwbMlO0vOucvLaaoBwF02bvfWDGo0NBP4zbga+c88lmpEUoM+yE5FuyzkC X-Received: by 2002:a17:906:edce:: with SMTP id sb14mr44586194ejb.25.1636004914176; Wed, 03 Nov 2021 22:48:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1636004914; cv=none; d=google.com; s=arc-20160816; b=AuRNwRDXueyA5V0aq9p3Rrhkr/mgLmwMzeVl8XQQv16jun3WhNbziwANrAxKEOSiCo //JoxsJTHwcMmKgStOmgrZL+fS+J8DdkZKaTsRPEUwSL0Ycu3EtGLy9AxZJquZehnWv8 8fdM6YawqMX0A6p7tBut+mmxauO4QoXavBmEhudbjY6WrT5qgK88GFY5lvokG/IMRNdh SLiGJJhdj0fqpjihxDLyCEaaux7UsnJCBGXeWi6AHcN3B0tDkBzsdI38erhMlvTj3ILG N4QCKc/8IsKxxG31gRJfy86MNkxwG0skIrRZB8Ftlkom7dQBifsoHBZkCiWjcDWxZqEF 7JjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=3Na9adASDRsn9gr8dEYOs/TT6fRIQmTPO3tf9PjkndQ=; b=sQ/quBLAAK1V/Gst0qJbHhbEUa1oMR2mfvpGLBZLMxwXAV2Px2jZVuY5QNMNjr7gw2 EhUL8r4YP02dg1UUrYeVhm975ljupZYGL56Z1FuQwAqeO5GIp8fY0XgLrxNMTXbsSet6 pMJa1orK5eL0lCiZM5viATR1Anh4UrKqDzeAID/J8P20oytgXCByfwnrIo59IVSoaOPp aYpZ/tQNx0CE4qHiBT87QFYWF0sBiKLyhwuGGjfzh40ZPqoTVDo0URiRPGbp0l1KbTyi tm6sHG9ReDV75MSqKXyY6Hee0fQmviYqIp6W6z6Um/HXfr4YFO2S/v2dV3c/Xe4l65Ss j4Rw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y5si7817587edp.88.2021.11.03.22.48.32; Wed, 03 Nov 2021 22:48:34 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E322B68ACA5; Thu, 4 Nov 2021 07:48:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6020968A993 for ; Thu, 4 Nov 2021 07:48:20 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10157"; a="212398282" X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="212398282" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Nov 2021 22:48:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="639158169" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by fmsmga001.fm.intel.com with ESMTP; 03 Nov 2021 22:48:15 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 Nov 2021 13:44:00 +0800 Message-Id: <20211104054403.1251208-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] libavutil/hwcontext_d3d11va: Add nb_surfaces to AVD3D11VAFramesContext X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hm+mFxK8kQE/ Adding nb_surfaces in AVD3D11VAFramesContext in the end of the structure to support flexible size of this arrays and align to AVDXVA2FramesContext and AVVAAPIFramesContext. Signed-off-by Wenbin Chen --- libavutil/hwcontext_d3d11va.c | 3 +-- libavutil/hwcontext_d3d11va.h | 2 ++ 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index 8ab96bad25..086e7b9daa 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -72,7 +72,6 @@ static av_cold void load_functions(void) } typedef struct D3D11VAFramesContext { - int nb_surfaces; int nb_surfaces_used; DXGI_FORMAT format; @@ -287,7 +286,7 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) hwctx->texture_infos = av_calloc(ctx->initial_pool_size, sizeof(*hwctx->texture_infos)); if (!hwctx->texture_infos) return AVERROR(ENOMEM); - s->nb_surfaces = ctx->initial_pool_size; + hwctx->nb_surfaces = ctx->initial_pool_size; ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(AVD3D11FrameDescriptor), ctx, d3d11va_pool_alloc, NULL); diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h index 77d2d72f1b..b0df470190 100644 --- a/libavutil/hwcontext_d3d11va.h +++ b/libavutil/hwcontext_d3d11va.h @@ -173,6 +173,8 @@ typedef struct AVD3D11VAFramesContext { * This field is ignored/invalid if a user-allocated texture is provided. */ AVD3D11FrameDescriptor *texture_infos; + + int nb_surfaces; } AVD3D11VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D11VA_H */ From patchwork Thu Nov 4 05:44:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31291 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:a610:0:0:0:0:0 with SMTP id q16csp146181ioi; Wed, 3 Nov 2021 22:48:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwviwCiFthoJfbBfygcwPJRNEJQFLWYYc9inUG0W5cosrQZY4RUSxUzm0T2ADxb8zCQrCR0 X-Received: by 2002:a05:6402:2550:: with SMTP id l16mr27385003edb.142.1636004924089; Wed, 03 Nov 2021 22:48:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1636004924; cv=none; d=google.com; s=arc-20160816; b=Okel8WNSoGMx9Z/a8c7l8kQ/oBRYimaBugLO0OPWQcZsOlB32F6p1L1lO9DVcX8Ve+ JDV9fUWszeHQPC6bVwLDHBgygklWEt4PcGmBB1JwdPOZ2dHs1Ii+6k9hteia3Zw9bJCx n6Giv+/tLaEcWe+OHHIzKyxeLeeMy0IeRInIoI2ftj4R5MfZatcYkETTaQGwxRakkYJQ 9uOlsJw2GNt7OdZVyMaA24TVi3ibWbzEfg3/BnNF5RbNrnklzAuJy6tFhITqN/CoUsDp VdVo6exdwOtHW89CkoXGxz5ach9xnRce2Q5k/Or1Sv6OC4NhglqFnvjnMSiu/Ue8nybr Jz7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=dBfB43HfCLYydx6wdS6bNbRyR98nPdmm0NZ7Li0vOYw=; b=wiMooKDKxCjDErPI5P76cMSl7QXYnU31uxXFaRaSX3ZxVw2ZBWXRQ8dg1hDwv4RzdT GbiyDSphFza3by45ObVxbShWbC6vILJFVFd/YALNy56vCCBajj9Zrb36ZvhTK12qMnbH TsSpTkHvlSyKVph2Zv9wVPEP9qoBryN1+HA6HJtAP8AdsinSdtP7iZg5aXjazA/jWVDk E5j7gjQpUvGtHdKxBLzaKhYCgV8+aaIIAzGfj/VK9l4h0k6dN5HHyooWT8/rwSLXEzU/ 39oMmddXKVWYrHMeir4e2LfZ6/8MqN7eseCiu3R1+juhY8kF6ZpO9L+qT6eiJ9BmBBVY 6wzw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h26si5694077edw.33.2021.11.03.22.48.43; Wed, 03 Nov 2021 22:48:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 20D0468AD12; Thu, 4 Nov 2021 07:48:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F158968996A for ; Thu, 4 Nov 2021 07:48:21 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10157"; a="212398284" X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="212398284" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Nov 2021 22:48:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="639158174" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by fmsmga001.fm.intel.com with ESMTP; 03 Nov 2021 22:48:17 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 Nov 2021 13:44:01 +0800 Message-Id: <20211104054403.1251208-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211104054403.1251208-1-wenbin.chen@intel.com> References: <20211104054403.1251208-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] libavutil/hwcontext_qsv: fix a bug when malloc handle_pairs_internal X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oUBF15o8iuwc This commandline cause core dumped: ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 \ -hwaccel_output_format vaapi -i input.264 \ -vf "hwmap=derive_device=qsv,format=qsv" \ -c:v h264_qsv output.264 reason: We use nb_surfaces to assign surface to handle_pairs_internal but handle_pairs_internal is alloced with the size of init_pool_size. This lead to access to illegal address. Now change it to use nb_surfaces to allocate handle_pairs_internal and the core dumped error is unseen. Also change D3D11VA to use nb_surfaces to align to VAAPI and DXVA2. Signed-off-by: Wenbin Chen --- libavutil/hwcontext_qsv.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index c18747f7eb..5a285fd25b 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -1123,8 +1123,7 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, case AV_HWDEVICE_TYPE_VAAPI: { AVVAAPIFramesContext *src_hwctx = src_ctx->hwctx; - s->handle_pairs_internal = av_calloc(src_ctx->initial_pool_size, - sizeof(*s->handle_pairs_internal)); + s->handle_pairs_internal = av_calloc(src_hwctx->nb_surfaces, sizeof(*s->handle_pairs_internal)); if (!s->handle_pairs_internal) return AVERROR(ENOMEM); s->surfaces_internal = av_calloc(src_hwctx->nb_surfaces, @@ -1146,15 +1145,15 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, case AV_HWDEVICE_TYPE_D3D11VA: { AVD3D11VAFramesContext *src_hwctx = src_ctx->hwctx; - s->handle_pairs_internal = av_calloc(src_ctx->initial_pool_size, + s->handle_pairs_internal = av_calloc(src_ctx->nb_surfaces, sizeof(*s->handle_pairs_internal)); if (!s->handle_pairs_internal) return AVERROR(ENOMEM); - s->surfaces_internal = av_calloc(src_ctx->initial_pool_size, + s->surfaces_internal = av_calloc(src_ctx->nb_surfaces, sizeof(*s->surfaces_internal)); if (!s->surfaces_internal) return AVERROR(ENOMEM); - for (i = 0; i < src_ctx->initial_pool_size; i++) { + for (i = 0; i < src_ctx->nb_surfaces; i++) { qsv_init_surface(dst_ctx, &s->surfaces_internal[i]); s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->texture_infos[i].texture; if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { @@ -1164,7 +1163,7 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, } s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; } - dst_hwctx->nb_surfaces = src_ctx->initial_pool_size; + dst_hwctx->nb_surfaces = src_ctx->nb_surfaces; if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { dst_hwctx->frame_type |= MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; } else { @@ -1177,7 +1176,7 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, case AV_HWDEVICE_TYPE_DXVA2: { AVDXVA2FramesContext *src_hwctx = src_ctx->hwctx; - s->handle_pairs_internal = av_calloc(src_ctx->initial_pool_size, + s->handle_pairs_internal = av_calloc(src_ctx->nb_surfaces, sizeof(*s->handle_pairs_internal)); if (!s->handle_pairs_internal) return AVERROR(ENOMEM); From patchwork Thu Nov 4 05:44:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31290 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:a610:0:0:0:0:0 with SMTP id q16csp146295ioi; Wed, 3 Nov 2021 22:48:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwQrc0JxGhHHYAVsWDSBP5aafqN7iZ3XIK5AWhTqDtKQHi7+x1DWbuxyoF0vSB+mQQxLbxF X-Received: by 2002:aa7:db93:: with SMTP id u19mr66555380edt.179.1636004934688; Wed, 03 Nov 2021 22:48:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1636004934; cv=none; d=google.com; s=arc-20160816; b=Vkn4Ti1lF7D63gm8FRNSJc190orEQXz4bcmHZYYDyzL/9aYRyLxv/pcsRktsAMFdxV 4jz2OSifgOjXYMmd8dDRjAGLj8kK68QKMP9Ok7+Qk2HlFgBLqhCdAAfR9BBaZcs5G9pL n3L7u7+ZAm23wZdajT21z/R102xcls5kzythOUEoNZaYkF8MQR3/H3MT4h0/G6UDFJt9 mQhGWQTyIYtBa8hb53vSH9cYMd9ZYZqbAuUFV1XlxGpvmRE2Bm8DAAqZO8t9w0mOT6tx iN56jS7yRiMFg8K5JK4lc+ul6OoVO+EmVmzEMnFt973PXbsMDP0ajCzWpfPo0GiDN6Ke Yf6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=jvwaXTdqQ9YhP1SPMHuF8JWPAnX5kGS26pZiTMDI/t8=; b=jOkpwB66AutFw0OqAvW+yLLwmdLceaxOjsGZHlaKxi7U40d1Z3fZmZo1QguTdB9bUN hT0/U3GDAoErRaBUe/eJe4jfzmHeEZ6a0d6aY/fFsWZ2PyTAz8VzLC8U0/2UPFA3uuGe OSF8zl3n5Hpx55JOfqI1xlx1QTq5LnLanFsXyfo2Ql4P0R5qh7uLUhQKkFLITbc7nMWZ cRHnfnSrTgNVUvy3PTyUy01i/4bA2LrLSyy+cfJeWtRa/749Vy/2PiJyx4Oed7Bkz0vR SPU1MKWxgRm4zExGvKmWjKdQkpc9HS0+2BYIz5sStZZKviIINMYCf0uCJBI+3sySyR/t KhBQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dr15si9116436ejc.170.2021.11.03.22.48.54; Wed, 03 Nov 2021 22:48:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 50F1668AD5F; Thu, 4 Nov 2021 07:48:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E872A68AD0A for ; Thu, 4 Nov 2021 07:48:23 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10157"; a="212398287" X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="212398287" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Nov 2021 22:48:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="639158181" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by fmsmga001.fm.intel.com with ESMTP; 03 Nov 2021 22:48:19 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 Nov 2021 13:44:02 +0800 Message-Id: <20211104054403.1251208-3-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211104054403.1251208-1-wenbin.chen@intel.com> References: <20211104054403.1251208-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] libavutil/hwcontext_qsv: fix a bug for mapping vaapi frame to qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen , nyanmisaka Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6fxSmjLgF/RZ From: nyanmisaka The data stored in data[3] in VAAPI AVFrame is VASurfaceID while the data stored in pair->first is the pointer of VASurfaceID, so we need to do cast to make following commandline works: ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 \ -hwaccel_output_format vaapi -i input.264 \ -vf "hwmap=derive_device=qsv,format=qsv" -c:v h264_qsv output.264 Signed-off-by: nyanmisaka Signed-off-by: Wenbin Chen --- libavutil/hwcontext_qsv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 5a285fd25b..8075c27862 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -1219,7 +1219,7 @@ static int qsv_map_to(AVHWFramesContext *dst_ctx, case AV_PIX_FMT_VAAPI: { mfxHDLPair *pair = (mfxHDLPair*)hwctx->surfaces[i].Data.MemId; - if (pair->first == src->data[3]) { + if (*(VASurfaceID*)pair->first == (VASurfaceID)src->data[3]) { index = i; break; } From patchwork Thu Nov 4 05:44:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 31293 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:a610:0:0:0:0:0 with SMTP id q16csp146567ioi; Wed, 3 Nov 2021 22:49:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyW5BLTNBCITVU22cwuePFd18QDcs9Q4I1UWQiRA4BUIHs26B97OYNHZytl5qV45V8ifTk9 X-Received: by 2002:a17:906:942:: with SMTP id j2mr61721990ejd.303.1636004957494; Wed, 03 Nov 2021 22:49:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1636004957; cv=none; d=google.com; s=arc-20160816; b=DTT7eUFIjyR4Op17QYUsQXSB1CKy24ugpB3nz/VPCJIX3fkUqNUviRRuYU3GE7vScV IRdO0NI7Ts0g8xhZkzppSWh5xNOdZFb2t+eqouHlvnIhTogKHcQtDhtcVyuRknNzVgl/ JPep3fBwVaDmKgTpVDRf/ubhIJjvEnSJ1okbFpOYAz0UMp4BA4D3i4ZpMHQEnwHqOUME KXFt+oZ6x4ctmATVC6R06Cmzz1JBZ/SozIqEwn+60rt1aRHqRzrNWtBsNfk8pPFllk8R 0DfrXlHpx6UyT+KchuCRyAoxGRbe12LDA8uHD1jQvT0rcbzNg/rp1cdsK4I/zoDlv9Xw 5hUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:delivered-to; bh=kPZPPdpH5v6lW50leFe99JgWwLkHcvxUKfLcFnVaw3o=; b=s0lTiOZsC2DB7TM0AQ1dH1ZJ0QV5Qqg+L5Ju0aG06RzHpTK3NTNn7Hqo1DnLPYdkbb rz+HqKig6msrP0Yj9fc/3SHXE5rDa03lWsB1JVBOb+FG5SKK6KFuyaaqc/XcLnyfdQ4D gZu3jD6Q0v0Uc2UQ5l7p4EXwGaFoYpPCiQczdizUbaOfjN6yeWJwZDMaM0mm6Ip8xHZ0 Xpf/ieT1brkE8AGD4cKcmZM9/T7RIZ2QDQEk7mvpbTgH2BoRzYdCmEf4gLf44J4Iceve CZeN7djgUVputV+VHd+BJUo9Sw1AmtoaXE8JtBzS2EJv1/rFpiuVuoWn0FgSl2vsYvEQ of0A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x19si6863594edl.627.2021.11.03.22.49.05; Wed, 03 Nov 2021 22:49:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 837DB68ACE7; Thu, 4 Nov 2021 07:48:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 676D968AD2A for ; Thu, 4 Nov 2021 07:48:26 +0200 (EET) X-IronPort-AV: E=McAfee;i="6200,9189,10157"; a="212398289" X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="212398289" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Nov 2021 22:48:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,208,1631602800"; d="scan'208";a="639158187" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by fmsmga001.fm.intel.com with ESMTP; 03 Nov 2021 22:48:21 -0700 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Thu, 4 Nov 2021 13:44:03 +0800 Message-Id: <20211104054403.1251208-4-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211104054403.1251208-1-wenbin.chen@intel.com> References: <20211104054403.1251208-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4] libavutil/hwcontext_opencl: fix a bug for mapping qsv frame to opencl X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbin Chen , nyanmisaka Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KoE+AM8fM8eP From: nyanmisaka mfxHDLPair was added to qsv, so modify qsv->opencl map function as well. Now the following commandline works: ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 \ -init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va -filter_hw_device ocl \ -hwaccel qsv -hwaccel_output_format qsv -hwaccel_device qs -c:v h264_qsv \ -i input.264 -vf "hwmap=derive_device=opencl,format=opencl,avgblur_opencl, \ hwmap=derive_device=qsv:reverse=1:extra_hw_frames=32,format=qsv" \ -c:v h264_qsv output.264 Signed-off-by: nyanmisaka Signed-off-by: Wenbin Chen --- libavutil/hwcontext_opencl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 26a3a24593..4b6e74ff6f 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -2249,7 +2249,8 @@ static int opencl_map_from_qsv(AVHWFramesContext *dst_fc, AVFrame *dst, #if CONFIG_LIBMFX if (src->format == AV_PIX_FMT_QSV) { mfxFrameSurface1 *mfx_surface = (mfxFrameSurface1*)src->data[3]; - va_surface = *(VASurfaceID*)mfx_surface->Data.MemId; + mfxHDLPair *pair = (mfxHDLPair*)mfx_surface->Data.MemId; + va_surface = *(VASurfaceID*)pair->first; } else #endif if (src->format == AV_PIX_FMT_VAAPI) {