From patchwork Tue Nov 3 18:45:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: galinart X-Patchwork-Id: 23373 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1605C449A23 for ; Tue, 3 Nov 2020 21:13:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E9D8468B6CB; Tue, 3 Nov 2020 21:13:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f66.google.com (mail-ej1-f66.google.com [209.85.218.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1AD1168AC9B for ; Tue, 3 Nov 2020 21:13:05 +0200 (EET) Received: by mail-ej1-f66.google.com with SMTP id cw8so12286519ejb.8 for ; Tue, 03 Nov 2020 11:13:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pKrnCKWAkAKyvgsAX4sdedDTJsaoCyAIeyYtfw7KZaw=; b=LYXkr9XTJJGFBjt9ntonZrpci+6O6hZoF95aoBeJn0AUdAaHWYpXm5Q+QkIGGKTIch Q1wIFoCBE3c7KBz+4mHgfK1o+6fOIx2oHu6Aw7KVtbzYCgq98qnwY1e8S2rqiDAnL0D1 kmnkUCwHsgDAiiG0KFKNAvuxH2ORn4u1kLNJ2mxWUi/l3S4wKkTBNpFGSIbKigsP7TvJ whEskobAh+PE/+2jnMezKu3W0YXewRKpE0+OaAGtupasSZ1AR07JxdpstQDcOWAQGHpR wO0W/kAxeFaPXd4FYY9C9dnRTdNi5frJM6QWneMhuB2U5p+5Vq4uIxzDiunGl2ze1had aedg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pKrnCKWAkAKyvgsAX4sdedDTJsaoCyAIeyYtfw7KZaw=; b=Gm1109YwqIfBUv3e0eG5mIJVxboUHtZSZcsUkaf0f4US8IGMwRlRNfmpp8xg9zuuM3 eLiGZtM8veRGt9ifSBq2ap8z22CiYhQGKL17K3xX+wFD1VbgshLU0Bhm1W/d44xWeA27 RX6CJCT1w+WWyIdfKdXP+CcDgAa/KANonSaKlduJPRuWiuiiN59ZwxZvqKMbUsCzsd4k G+VOF09cCj0UXmOpFuCV6f7B42iSEMoKs+F5C5cUJahExjgi1gxX1WX9DnBoYggBN204 0c/aS/re6P8HZ9zI5f21C6nQQ6nfz4ra4914GSt1OlMjHFeDIvKGyNCt0aMd1g7LhPC+ udlg== X-Gm-Message-State: AOAM5325G3pPuIpWCRbEiQNrcHVyoMQqM+BX4L2h9lY9Fq3uPF7xA9dk Kg+IDJ6AytDBMJLXdeSzd5i7YFjIxOxB1A== X-Google-Smtp-Source: ABdhPJy62p35OjfWU24y8SuLh7mFkAu049A8dlpIjEVJElhT/POyXi2Dj8vc4YGYto7GOtGKF/m21g== X-Received: by 2002:a05:6000:107:: with SMTP id o7mr27420047wrx.354.1604429274269; Tue, 03 Nov 2020 10:47:54 -0800 (PST) Received: from localhost.localdomain ([2a02:c7f:3015:4e00:e4c4:5a5e:dd23:ce22]) by smtp.googlemail.com with ESMTPSA id z2sm3905249wmf.45.2020.11.03.10.47.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Nov 2020 10:47:53 -0800 (PST) From: Artem Galin X-Google-Original-From: Artem Galin To: ffmpeg-devel@ffmpeg.org Date: Tue, 3 Nov 2020 18:45:53 +0000 Message-Id: <20201103184557.18444-4-artem.galin@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201103184557.18444-1-artem.galin@intel.com> References: <20201103184557.18444-1-artem.galin@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v7 4/8] libavutil/hwcontext_d3d11va: adding more texture information to the D3D11 hwcontext API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Artem Galin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Microsoft VideoProcessor requires texture with D3DUSAGE_RENDERTARGET flag as output. There is no way to allocate array of textures with D3D11_BIND_RENDER_TARGET flag and .ArraySize > 2 by ID3D11Device_CreateTexture2D due to the Microsoft limitation. Adding AVD3D11FrameDescriptors array to store array of single textures instead of texture with multiple slices resolves this. Signed-off-by: Artem Galin --- libavutil/hwcontext_d3d11va.c | 24 ++++++++++++++++++----- libavutil/hwcontext_d3d11va.h | 9 +++++++++ libavutil/hwcontext_qsv.c | 37 +++++++++++++++++++++++------------ 3 files changed, 53 insertions(+), 17 deletions(-) diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index c8ae58f908..1f95eae24a 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -72,6 +72,7 @@ static av_cold void load_functions(void) } typedef struct D3D11VAFramesContext { + int nb_surfaces; int nb_surfaces_used; DXGI_FORMAT format; @@ -112,6 +113,8 @@ static void d3d11va_frames_uninit(AVHWFramesContext *ctx) if (s->staging_texture) ID3D11Texture2D_Release(s->staging_texture); s->staging_texture = NULL; + + av_freep(&frames_hwctx->texture_infos); } static int d3d11va_frames_get_constraints(AVHWDeviceContext *ctx, @@ -152,15 +155,21 @@ static void free_texture(void *opaque, uint8_t *data) av_free(data); } -static AVBufferRef *wrap_texture_buf(ID3D11Texture2D *tex, int index) +static AVBufferRef *wrap_texture_buf(AVHWFramesContext *ctx, ID3D11Texture2D *tex, int index) { AVBufferRef *buf; - AVD3D11FrameDescriptor *desc = av_mallocz(sizeof(*desc)); + AVD3D11FrameDescriptor *desc = av_mallocz(sizeof(*desc)); + D3D11VAFramesContext *s = ctx->internal->priv; + AVD3D11VAFramesContext *frames_hwctx = ctx->hwctx; if (!desc) { ID3D11Texture2D_Release(tex); return NULL; } + frames_hwctx->texture_infos[s->nb_surfaces_used].texture = tex; + frames_hwctx->texture_infos[s->nb_surfaces_used].index = index; + s->nb_surfaces_used++; + desc->texture = tex; desc->index = index; @@ -199,7 +208,7 @@ static AVBufferRef *d3d11va_alloc_single(AVHWFramesContext *ctx) return NULL; } - return wrap_texture_buf(tex, 0); + return wrap_texture_buf(ctx, tex, 0); } static AVBufferRef *d3d11va_pool_alloc(void *opaque, int size) @@ -220,7 +229,7 @@ static AVBufferRef *d3d11va_pool_alloc(void *opaque, int size) } ID3D11Texture2D_AddRef(hwctx->texture); - return wrap_texture_buf(hwctx->texture, s->nb_surfaces_used++); + return wrap_texture_buf(ctx, hwctx->texture, s->nb_surfaces_used); } static int d3d11va_frames_init(AVHWFramesContext *ctx) @@ -267,7 +276,7 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) av_log(ctx, AV_LOG_ERROR, "User-provided texture has mismatching parameters\n"); return AVERROR(EINVAL); } - } else if (texDesc.ArraySize > 0) { + } else if (!(texDesc.BindFlags & D3D11_BIND_RENDER_TARGET) && texDesc.ArraySize > 0) { hr = ID3D11Device_CreateTexture2D(device_hwctx->device, &texDesc, NULL, &hwctx->texture); if (FAILED(hr)) { av_log(ctx, AV_LOG_ERROR, "Could not create the texture (%lx)\n", (long)hr); @@ -275,6 +284,11 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) } } + hwctx->texture_infos = av_mallocz_array(ctx->initial_pool_size, sizeof(*hwctx->texture_infos)); + if (!hwctx->texture_infos) + return AVERROR(ENOMEM); + s->nb_surfaces = ctx->initial_pool_size; + ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(AVD3D11FrameDescriptor), ctx, d3d11va_pool_alloc, NULL); if (!ctx->internal->pool_internal) diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h index 9f91e9b1b6..77d2d72f1b 100644 --- a/libavutil/hwcontext_d3d11va.h +++ b/libavutil/hwcontext_d3d11va.h @@ -164,6 +164,15 @@ typedef struct AVD3D11VAFramesContext { * This field is ignored/invalid if a user-allocated texture is provided. */ UINT MiscFlags; + + /** + * In case if texture structure member above is not NULL contains the same texture + * pointer for all elements and different indexes into the array texture. + * In case if texture structure member above is NULL, all elements contains + * pointers to separate non-array textures and 0 indexes. + * This field is ignored/invalid if a user-allocated texture is provided. + */ + AVD3D11FrameDescriptor *texture_infos; } AVD3D11VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D11VA_H */ diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index c0a71c9822..b4fa1d845e 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -120,6 +120,23 @@ static uint32_t qsv_fourcc_from_pix_fmt(enum AVPixelFormat pix_fmt) return 0; } +#if CONFIG_D3D11VA +static uint32_t qsv_get_d3d11va_bind_flags(int mem_type) +{ + uint32_t bind_flags = 0; + + if ((mem_type & MFX_MEMTYPE_VIDEO_MEMORY_ENCODER_TARGET) && (mem_type & MFX_MEMTYPE_INTERNAL_FRAME)) + bind_flags = D3D11_BIND_DECODER | D3D11_BIND_VIDEO_ENCODER; + else + bind_flags = D3D11_BIND_DECODER; + + if ((MFX_MEMTYPE_FROM_VPPOUT & mem_type) || (MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET & mem_type)) + bind_flags = D3D11_BIND_RENDER_TARGET; + + return bind_flags; +} +#endif + static int qsv_device_init(AVHWDeviceContext *ctx) { AVQSVDeviceContext *hwctx = ctx->hwctx; @@ -295,12 +312,11 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) #if CONFIG_D3D11VA if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; - if (hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET) - child_frames_hwctx->BindFlags = D3D11_BIND_RENDER_TARGET; - else - child_frames_hwctx->BindFlags = D3D11_BIND_DECODER; + if (hwctx->frame_type == 0) + hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET; if (hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) child_frames_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; + child_frames_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(hwctx->frame_type); } #endif #if CONFIG_DXVA2 @@ -334,11 +350,11 @@ static int qsv_init_child_ctx(AVHWFramesContext *ctx) if (child_device_ctx->type == AV_HWDEVICE_TYPE_D3D11VA) { AVD3D11VAFramesContext *child_frames_hwctx = child_frames_ctx->hwctx; for (i = 0; i < ctx->initial_pool_size; i++) { - s->handle_pairs_internal[i].first = (mfxMemId)child_frames_hwctx->texture; + s->handle_pairs_internal[i].first = (mfxMemId)child_frames_hwctx->texture_infos[i].texture; if(child_frames_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; } else { - s->handle_pairs_internal[i].second = (mfxMemId)i; + s->handle_pairs_internal[i].second = (mfxMemId)child_frames_hwctx->texture_infos[i].index; } s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; } @@ -714,10 +730,7 @@ static int qsv_frames_derive_from(AVHWFramesContext *dst_ctx, dst_hwctx->texture = (ID3D11Texture2D*)pair->first; if (src_hwctx->frame_type & MFX_MEMTYPE_SHARED_RESOURCE) dst_hwctx->MiscFlags = D3D11_RESOURCE_MISC_SHARED; - if (src_hwctx->frame_type == MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) - dst_hwctx->BindFlags = D3D11_BIND_DECODER; - else - dst_hwctx->BindFlags = D3D11_BIND_RENDER_TARGET; + dst_hwctx->BindFlags = qsv_get_d3d11va_bind_flags(src_hwctx->frame_type); } break; #endif @@ -1137,11 +1150,11 @@ static int qsv_frames_derive_to(AVHWFramesContext *dst_ctx, return AVERROR(ENOMEM); for (i = 0; i < src_ctx->initial_pool_size; i++) { qsv_init_surface(dst_ctx, &s->surfaces_internal[i]); - s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->texture; + s->handle_pairs_internal[i].first = (mfxMemId)src_hwctx->texture_infos[i].texture; if (src_hwctx->BindFlags & D3D11_BIND_RENDER_TARGET) { s->handle_pairs_internal[i].second = (mfxMemId)MFX_INFINITE; } else { - s->handle_pairs_internal[i].second = (mfxMemId)i; + s->handle_pairs_internal[i].second = (mfxMemId)src_hwctx->texture_infos[i].index; } s->surfaces_internal[i].Data.MemId = (mfxMemId)&s->handle_pairs_internal[i]; }