From patchwork Thu Jul 22 10:56:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 29011 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp411676ios; Thu, 22 Jul 2021 04:03:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzEnyykgUmnvxGJpVRkTsnuWwjWc2gwOfwZAiSxd1KU7rXJ9+lQjPfVW2RwdI4GnNWkmM8X X-Received: by 2002:a17:906:6814:: with SMTP id k20mr42440311ejr.381.1626951831855; Thu, 22 Jul 2021 04:03:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1626951831; cv=none; d=google.com; s=arc-20160816; b=XHf2gXzIikcfL2EjfAcPiqiIV07zHNAa/lCy3kz5y3wfkptxv21JE4FEXAbC4tb0f2 TAnAFDksnoZm4WK6ABVSa2E5CwLii2cHTwe75LidS8AbbgfOm9ZEDqdsFmU3uF3rK4M2 efAihICbRhxeNA7ZIfp4rBHceeZimvBlK7nO13wobgxPckqM73UWzD+F4cx1yqATFCYG mwcvi6cNjXuai7Jlvl2I0uTlQOA0fMmnPvtzEmVh5HW/zeLZG48OiHuunJomrikOgb/g vu8Z5+8Xl4rR7vbie4RWypKpj4THIfNIEy/5+kgvS+G2sWamxZFAAEDLWWajBS8nI4RQ ZsyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=kx4cflEi4zqhXzmFQRCirOhwyRLt8htc23qi5yXj8rI=; b=B9HESbPPqjcvZp4Rm+0FeuX0tjtNDrB+D8gmDvzbFX2nJcMHt7kT6m453FgDcDx367 PpriIQuV5HMj3SbjBKIKBnivCmRn4BeNmomSsf2ABxFCvpbRPm1O4BlJnKJXOTYwtJpK UuHmon+t7CyFtpVyTD+BLCe3cSEcsTpnjdkPz6QrPNtpO8Miz76H9ZR3M3TcycsjAy2l Zc/dbo+iop3dKWaFZNmVuIMKQnTI41GQd+BxeZx/nYEplIa3TJVqGLcHjbM+38uvrbTG tVv8RGcPvH9IExGp/zhECoVJ7IfSNH4n4e5L0fHemdmFVlndqoXyW0WiT+rhDQc5GDFz b5ww== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="Zu9/Yogp"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lr6si9594747ejb.129.2021.07.22.04.03.51; Thu, 22 Jul 2021 04:03:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="Zu9/Yogp"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B4D4568A883; Thu, 22 Jul 2021 14:03:48 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 37AF168A82C for ; Thu, 22 Jul 2021 14:03:40 +0300 (EEST) Received: by mail-wr1-f51.google.com with SMTP id t5so5455513wrw.12; Thu, 22 Jul 2021 04:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OLEao4lJNi/60jLkTLc7O6XjU9kS69Wc8hDJarJPWuw=; b=Zu9/YogpYzr+aYQIG8MUobzPh00uYXZV5COHGnEJpzbxfsPBQPqUAwdarXhVN6m1Hl ZqP7tuaSES+3i3w1aYhSs7SiLSMXx3unwnkp0RIKFct7m/a4LcjORpKepbUaKh3eTjpn h+oPzPnP898oVimEGXeq3I0PUs20EiZP7WMKGBf56zUk0DaDFdfJYAeNTcijJZcjphDG Snn8OaJcJBAFbf6jgjoLf9vMmgyuodfnd8h2R1Jx9WpatKZV+TmvRDL5K4tbi2pCIF4B fZpqZ8OY7mL1xQGhQFvwbT//hJeOM81xPsBq/MgorChZaFuYrbhrnmq4p9vEz/s5Nynb KL8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OLEao4lJNi/60jLkTLc7O6XjU9kS69Wc8hDJarJPWuw=; b=K/no48RU7HTNbjf/zIBOdl0ZztvFj0NSXchNvDgYirvZsHAmgh7/rrDBRzJvd2AzQK OwOL9EyMt4cQdMacuT3zKl0wYIJLMI3aO0FTqw8tyFSl8Y2vMaXjdmSgUaomEm2bZ5ZT sfOO0RkyYDBbCdgN9fANV4Z3fGMTaQkb+6pYxSbclDSYyh2aP4I9QWKwUIXJQcJAk5wV icbljUcOcsBZqsxE4VbMvMmyvmGUOwzXBpzmdDy/1nSdzkkHO7kEKFgENBSRLLHLMEKG IPqPSENZXhOqw5nir55qjkofDwFFY8pA8BtXYvZv95tHUuL09l+VfNWyaN6rjnqzc5kI 260w== X-Gm-Message-State: AOAM530jo9Ayk20E3bumzIKLq24xSkJKc/nUuMgTJ9Dxvk3BMVETkbz0 jRrNnStRqvDEbph2gKYmsufnaL6KlhDxtiAp X-Received: by 2002:a2e:a5c5:: with SMTP id n5mr32364275ljp.197.1626951411388; Thu, 22 Jul 2021 03:56:51 -0700 (PDT) Received: from localhost.localdomain (37-144-244-78.broadband.corbina.ru. [37.144.244.78]) by smtp.googlemail.com with ESMTPSA id z4sm1951236lfh.2.2021.07.22.03.56.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jul 2021 03:56:51 -0700 (PDT) From: Artem Galin To: ffmpeg-devel@ffmpeg.org, ffmpeg-devep@ffmpeg.org Date: Thu, 22 Jul 2021 11:56:31 +0100 Message-Id: <20210722105635.48075-4-artem.galin@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210722105635.48075-1-artem.galin@gmail.com> References: <20210722105635.48075-1-artem.galin@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/8] libavutil/hwcontext_d3d11va: adding more texture information to the D3D11 hwcontext API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Itq8ntTeNHZ6 From: Artem Galin Microsoft VideoProcessor requires texture with D3DUSAGE_RENDERTARGET flag as output. There is no way to allocate array of textures with D3D11_BIND_RENDER_TARGET flag and .ArraySize > 2 by ID3D11Device_CreateTexture2D due to the Microsoft limitation. Adding AVD3D11FrameDescriptors array to store array of single textures instead of texture with multiple slices resolves this. Signed-off-by: Artem Galin --- libavutil/hwcontext_d3d11va.c | 24 ++++++++++++++++++----- libavutil/hwcontext_d3d11va.h | 9 +++++++++ libavutil/hwcontext_qsv.c | 37 +++++++++++++++++++++++------------ 3 files changed, 53 insertions(+), 17 deletions(-) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index 27274ee3fa..272a19da47 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -72,6 +72,7 @@ static av_cold void load_functions(void) } typedef struct D3D11VAFramesContext { + int nb_surfaces; int nb_surfaces_used; DXGI_FORMAT format; @@ -112,6 +113,8 @@ static void d3d11va_frames_uninit(AVHWFramesContext *ctx) if (s->staging_texture) ID3D11Texture2D_Release(s->staging_texture); s->staging_texture = NULL; + + av_freep(&frames_hwctx->texture_infos); } static int d3d11va_frames_get_constraints(AVHWDeviceContext *ctx, @@ -152,15 +155,21 @@ static void free_texture(void *opaque, uint8_t *data) av_free(data); } -static AVBufferRef *wrap_texture_buf(ID3D11Texture2D *tex, int index) +static AVBufferRef *wrap_texture_buf(AVHWFramesContext *ctx, ID3D11Texture2D *tex, int index) { AVBufferRef *buf; - AVD3D11FrameDescriptor *desc = av_mallocz(sizeof(*desc)); + AVD3D11FrameDescriptor *desc = av_mallocz(sizeof(*desc)); + D3D11VAFramesContext *s = ctx->internal->priv; + AVD3D11VAFramesContext *frames_hwctx = ctx->hwctx; if (!desc) { ID3D11Texture2D_Release(tex); return NULL; } + frames_hwctx->texture_infos[s->nb_surfaces_used].texture = tex; + frames_hwctx->texture_infos[s->nb_surfaces_used].index = index; + s->nb_surfaces_used++; + desc->texture = tex; desc->index = index; @@ -199,7 +208,7 @@ static AVBufferRef *d3d11va_alloc_single(AVHWFramesContext *ctx) return NULL; } - return wrap_texture_buf(tex, 0); + return wrap_texture_buf(ctx, tex, 0); } static AVBufferRef *d3d11va_pool_alloc(void *opaque, size_t size) @@ -220,7 +229,7 @@ static AVBufferRef *d3d11va_pool_alloc(void *opaque, size_t size) } ID3D11Texture2D_AddRef(hwctx->texture); - return wrap_texture_buf(hwctx->texture, s->nb_surfaces_used++); + return wrap_texture_buf(ctx, hwctx->texture, s->nb_surfaces_used); } static int d3d11va_frames_init(AVHWFramesContext *ctx) @@ -267,7 +276,7 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) av_log(ctx, AV_LOG_ERROR, "User-provided texture has mismatching parameters\n"); return AVERROR(EINVAL); } - } else if (texDesc.ArraySize > 0) { + } else if (!(texDesc.BindFlags & D3D11_BIND_RENDER_TARGET) && texDesc.ArraySize > 0) { hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, NULL, &hwctx->texture); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Could not create the texture (%lx)\n", (long)hr); @@ -275,6 +284,11 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) } } + hwctx->texture_infos = av_mallocz_array(ctx->initial_pool_size, sizeof(*hwctx->texture_infos)); + if (!hwctx->texture_infos) + return AVERROR(ENOMEM); + s->nb_surfaces = ctx->initial_pool_size; + ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(AVD3D11FrameDescriptor), ctx, d3d11va_pool_alloc, NULL); if (!ctx->internal->pool_internal) diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h index 9f91e9b1b6..77d2d72f1b 100644 --- a/libavutil/hwcontext_d3d11va.h +++ b/libavutil/hwcontext_d3d11va.h @@ -164,6 +164,15 @@ typedef struct AVD3D11VAFramesContext { * This field is ignored/invalid if a user-allocated texture is provided. */ UINT MiscFlags; + + /** + * In case if texture structure member above is not NULL contains the same texture + * pointer for all elements and different indexes into the array texture. + * In case if texture structure member above is NULL, all elements contains + * pointers to separate non-array textures and 0 indexes. + * This field is ignored/invalid if a user-allocated texture is provided. + */ + AVD3D11FrameDescriptor *texture_infos; } AVD3D11VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D11VA_H */ diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index f7cd13cf1b..b8294f2511 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -120,6 +120,23 @@ static uint32_t qsv_fourcc_from_pix_fmt(enum AVPixelFormat pix_fmt) return 0; } +#if CONFIG_D3D11VA +static uint32_t qsv_get_d3d11va_bind_flags(int mem_type) +{ + uint32_t bind_flags = 0; + + if ((mem_type & MFX_MEMTYPE_VIDEO_MEMORY_ENCODER_TARGET) && (mem_type & MFX_MEMTYPE_INTERNAL_FRAME)) + bind_flags = D3D11_BIND_DECODER | D3D11_BIND_VIDEO_ENCODER; + else + bind_flags = D3D11_BIND_DECODER; + + if ((MFX_MEMTYPE_FROM_VPPOUT & mem_type) || (MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET & mem_type)) + bind_flags = D3D11_BIND_RENDER_TARGET; + + return bind_flags; +} +#endif + static int qsv_device_init(AVHWDeviceContext *ctx) { AVQSVDeviceContext *hwctx = ctx->hwctx; @@ -295,12 +312,11 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) #if CONFIG_D3D11VA if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; - if (hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) - child_frames_hwctx->BindFlags = D3D11_BIND_RENDER_TARGET; - else - child_frames_hwctx->BindFlags = D3D11_BIND_DECODER; + if (hwctx->frame_type == 0) + hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; if (hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) child_frames_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; + child_frames_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(hwctx->frame_type); } #endif #if CONFIG_DXVA2 @@ -334,11 +350,11 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; for (i = 0; i < ctx->initial_pool_size; i++) { - s->handle_pairs_internal[i].first = (mfxMemId)child_frames_hwctx->texture; + s->handle_pairs_internal[i].first = (mfxMemId)child_frames_hwctx->texture_infos[i].texture; if(child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; } else { - s->handle_pairs_internal[i].second = (mfxMemId)i; + s->handle_pairs_internal[i].second = (mfxMemId)child_frames_hwctx->texture_infos[i].index; } s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; } @@ -714,10 +730,7 @@ static int qsv_frames_derive_from(AVHWFramesContext *dst_ctx, dst_hwctx->texture = (ID3D11Texture2D*)pair->first; if (src_hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; - if (src_hwctx->frame_type == MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) - dst_hwctx->BindFlags = D3D11_BIND_DECODER; - else - dst_hwctx->BindFlags = D3D11_BIND_RENDER_TARGET; + dst_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(src_hwctx->frame_type); } break; #endif @@ -1137,11 +1150,11 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, return AVERROR(ENOMEM); for (i = 0; i < src_ctx->initial_pool_size; i++) { qsv_init_surface(dst_ctx, &s->surfaces_internal[i]); - s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->texture; + s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->texture_infos[i].texture; if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; } else { - s->handle_pairs_internal[i].second = (mfxMemId)i; + s->handle_pairs_internal[i].second = (mfxMemId)src_hwctx->texture_infos[i].index; } s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; }