From patchwork Mon Apr 9 16:48:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kravchenko X-Patchwork-Id: 8365 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp91129jad; Mon, 9 Apr 2018 09:55:27 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+qpESF39pL90Xcte3jkrQPmhopi9EdgbnvFO4QzkZCFCu73TzWgdB+JilFkhOQiOdHtIWp X-Received: by 10.223.196.132 with SMTP id m4mr29389669wrf.9.1523292926982; Mon, 09 Apr 2018 09:55:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523292926; cv=none; d=google.com; s=arc-20160816; b=hsp/nGJtZIayCZ4H1TSV+RRqJFYiBiCDDLluuqyNJA0xXUY6PKfWS0bgj3zsUAVGPy QhnKUwV9h4bzl19GZPT7khAhD3hNkRBSDCKqNWT95xClBABYmnlq6KybubLRKUsilTgR 7QGxenygbm3VuIWiyPLvKX1gUZ7ZbM1tJkoY+VA5uh6+2VmvIwg7KHRKKC1aIBOAPJiX SSxYPfI6htZ+jBaliurCXL/6XxzG5I/nwiOiQgBqYLUOlmDlDh/wrzJDMjN7ynze15wd yELkJdBvE2gY08gKYbnqh29hKW0se0/4VtPYmnYgXqmNn7Hkhwb+l2jT6b350RDI2sjG Gl+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:thread-index:content-language:mime-version :message-id:date:in-reply-to:references:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=ufdolTgCJRoZg/SLMhe2vcbHPxSRs5sEZdYyrySALpI=; b=jqyct50ikGhccFsfMFpNNm/+mtjEIdrGs363NtBstZYHMerSmcIJcGYOzu6CsG9wMb R7NDq+2AMKMzdUJKthF1Un2gYCz6TfaukNrktmVykuZAF9CXOqyYdHEPPiyWL+FULcbk YiWJV9eo8AubYAuf+hheaWDVyOvUSbPjC5S8m/zmwjixJVNv0uJbNey6J142tin5W2Bl HVD/D/NYjC5LeA/VqdY5d+ppwzmDOaMx+b8MS7cCqQbQoyifsHQxaYFW+QjnA/WqgC+9 Q+0iUHYxD2pYZoiSy7VtS2sjF71Ns01/fvGGypkQ4f364spZjpQ5G3s8g0vcYMxjzT8o UxOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=BdqQJ5VS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x25si660257wmh.120.2018.04.09.09.55.25; Mon, 09 Apr 2018 09:55:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=BdqQJ5VS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 064FD689F22; Mon, 9 Apr 2018 19:55:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 38DF6689E45 for ; Mon, 9 Apr 2018 19:54:55 +0300 (EEST) Received: by mail-lf0-f53.google.com with SMTP id e5-v6so7570348lfb.7 for ; Mon, 09 Apr 2018 09:55:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-transfer-encoding:content-language:thread-index; bh=qpYhrW79KKhhtFz7tPEPiGVCApdX+03a1aO4vMPHbv8=; b=BdqQJ5VSyev+PXSHLwHz9+lClvCfY26kZEMirVC+HAf6P+gtmktygRbmPT6Oxvz+rU wWW5vo2u+Bem9cQsAbMSkVZ54UYlo8Ad4QrO+5PNO7B6nvWDIcbwrpCJeZAwHsnxLyaj rHGTUUbvcDXSSnxDXj1IJ671ivqC8UOzwzzbkfv8REhSHhQPw1ePHNb9k9/+Zl7bYAxx BoeiFYnKGZY/33f8fFsRIY8kp0Z80984WIpYUeIDAAfD5bey5MS7Iewz3npKEHkotXcM 1MZSgkE/B9Ghcm7DawRSeAVDXAU/I1KhBbLyC8VMIWGcjZSDtVFoBvbqyOkIoPNmMmlC eQXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:content-language :thread-index; bh=qpYhrW79KKhhtFz7tPEPiGVCApdX+03a1aO4vMPHbv8=; b=BEc/EIpHqGE8SsI7z2yJ/0IL88HQUVkifVrSVB08I60dnMZzZjiYk68bm6Av/knD7M /dz6RLLvl+fSZl0EUpXN5F8liiSgHgVKC/Z5okr2YhStPsC0WEtcWuyNwqEKgCaYpyYf aa1Md2IcnyQLn+SMsj3eGVsxTlcIoPn8r9Fg99XRUZlHTS7msnVHw8JPKjlsVGMlxIWk VI0LpUC7pMKiw6xMhsCCFGFYs+VRSzInSXEBdIf7Fn2W7TNL1m1LkG8hB+UU5STlQrlc luywrQDDZz813gwmpVTXYWBg9uhRaXVKj+vnkA8UKaUy9JCZBEdep9ocF3TAzFbc6WQe 5dNA== X-Gm-Message-State: ALQs6tCZVIHspioD4Fa6NUpMVuul5fAR6/Z4q+xocVi1f+1zu+hQcPbf Ixvmqlw5tea1etnH8bVH9Qqi3W26 X-Received: by 10.46.58.16 with SMTP id h16mr13345340lja.146.1523292515251; Mon, 09 Apr 2018 09:48:35 -0700 (PDT) Received: from KAAPCWIN10 ([84.47.189.78]) by smtp.gmail.com with ESMTPSA id j74-v6sm148926lfi.97.2018.04.09.09.48.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Apr 2018 09:48:34 -0700 (PDT) From: "Alexander Kravchenko" To: "'FFmpeg development discussions and patches'" References: <01ab01d3ccf1$ff6b67d0$fe423770$@gmail.com> <01dc01d3cd91$8b2758c0$a1760a40$@gmail.com> <052224f4-e938-9c22-d2a3-dd8e39517ab8@jkqxz.net> <000101d3cf6d$a8527f00$f8f77d00$@gmail.com> <603166e9-d4f8-1db2-d603-fa6e850b2772@jkqxz.net> In-Reply-To: <603166e9-d4f8-1db2-d603-fa6e850b2772@jkqxz.net> Date: Mon, 9 Apr 2018 19:48:33 +0300 Message-ID: <004701d3d022$99422230$cbc66690$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Content-Language: en-us Thread-Index: AQJRN8lF2celsV/yjjet0q9G9fCIogEhXecKAq9NyMECL6/gXAHXBAU7AdZkTjqisF7EcA== Subject: Re: [FFmpeg-devel] [PATCH] lavc/amfenc: Retain a reference to D3D frames used as input during the encoding process X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hi, could you please review updated patch? Fixes according on Mark's review: * Macroses changed to functions * error level of AMF_RETURN_IF_FALSE changed to fatal (all cases it returns are fatal according on fatal error level description) * used AMF_RETURN_IF_FALSE for case if a frame reference has been completely lost (was just warning before) Hopefully this patch is ok enough to be applied :). FYI, near time I am going to send the following patches * cosmetic fixes * hwcontext_amf (to be reused in encoder and color space converter) * AMF colors space converter (input memory types: opencl, dx9, dx11, host; output memory types: opencl, dx9, dx11) --- libavcodec/amfenc.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++- libavcodec/amfenc.h | 5 ++- 2 files changed, 97 insertions(+), 2 deletions(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 89a10ff253..ea2c5acbbf 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -157,6 +157,9 @@ static int amf_init_context(AVCodecContext *avctx) AmfContext *ctx = avctx->priv_data; AMF_RESULT res = AMF_OK; + ctx->hwsurfaces_in_queue = 0; + ctx->hwsurfaces_in_queue_max = 16; + // configure AMF logger // the return of these functions indicates old state and do not affect behaviour ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); @@ -187,6 +190,7 @@ static int amf_init_context(AVCodecContext *avctx) if (!ctx->hw_frames_ctx) { return AVERROR(ENOMEM); } + ctx->hwsurfaces_in_queue_max = device_ctx->initial_pool_size - 1; } else { if(res == AMF_NOT_SUPPORTED) av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has D3D11 device which doesn't have D3D11VA interface, switching to default\n"); @@ -443,6 +447,75 @@ int ff_amf_encode_init(AVCodecContext *avctx) return ret; } +static AMF_RESULT amf_set_property_buffer(AMFSurface *object, const wchar_t *name, AMFBuffer *val) +{ + AMF_RESULT res; + AMFVariantStruct var; + res = AMFVariantInit(&var); + if (res == AMF_OK) { + AMFGuid guid_AMFInterface = IID_AMFInterface(); + AMFInterface *amf_interface; + res = val->pVtbl->QueryInterface(val, &guid_AMFInterface, (void**)&amf_interface); + + if (res == AMF_OK) { + res = AMFVariantAssignInterface(&var, amf_interface); + amf_interface->pVtbl->Release(amf_interface); + } + if (res == AMF_OK) { + res = object->pVtbl->SetProperty(object, name, var); + } + AMFVariantClear(&var); + } + return res; +} + +static AMF_RESULT amf_get_property_buffer(AMFData *object, const wchar_t *name, AMFBuffer **val) +{ + AMF_RESULT res; + AMFVariantStruct var; + res = AMFVariantInit(&var); + if (res == AMF_OK) { + res = object->pVtbl->GetProperty(object, name, &var); + if (res == AMF_OK) { + if (var.type == AMF_VARIANT_INTERFACE) { + AMFGuid guid_AMFBuffer = IID_AMFBuffer(); + AMFInterface *amf_interface = AMFVariantInterface(&var); + res = amf_interface->pVtbl->QueryInterface(amf_interface, &guid_AMFBuffer, (void**)val); + } else { + res = AMF_INVALID_DATA_TYPE; + } + } + AMFVariantClear(&var); + } + return res; +} + +static AMFBuffer *amf_create_buffer_with_frame_ref(const AVFrame *frame, AMFContext *context) +{ + AVFrame *frame_ref; + AMFBuffer *frame_ref_storage_buffer = NULL; + AMF_RESULT res; + + res = context->pVtbl->AllocBuffer(context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); + if (res == AMF_OK) { + frame_ref = av_frame_clone(frame); + if (frame_ref) { + memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); + } else { + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + frame_ref_storage_buffer = NULL; + } + } + return frame_ref_storage_buffer; +} + +static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffer) +{ + AVFrame *av_frame_ref; + memcpy(&av_frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(av_frame_ref)); + av_frame_free(&av_frame_ref); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); +} int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) { @@ -484,6 +557,8 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx == (AVHWDeviceContext*)ctx->hw_device_ctx->data) )) { + AMFBuffer *frame_ref_storage_buffer; + #if CONFIG_D3D11VA static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture @@ -496,6 +571,14 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); #endif + + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); + + res = amf_set_property_buffer(surface, L"av_frame_ref", frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "SetProperty failed for \"av_frame_ref\" with error %d\n", res); + ctx->hwsurfaces_in_queue++; + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); } else { res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); @@ -560,6 +643,15 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) ret = amf_copy_buffer(avctx, avpkt, buffer); buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer; + res = amf_get_property_buffer(data, L"av_frame_ref", &frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + ctx->hwsurfaces_in_queue--; + } + data->pVtbl->Release(data); AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); @@ -589,7 +681,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); } } - } else if (ctx->delayed_surface != NULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF)) { + } else if (ctx->delayed_surface != NULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { block_and_wait = 1; av_usleep(1000); // wait and poll again } diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 84f0aad2fa..0e4c6e4b89 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -62,6 +62,9 @@ typedef struct AmfContext { AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) + int hwsurfaces_in_queue; + int hwsurfaces_in_queue_max; + // helpers to handle async calls int delayed_drain; AMFSurface *delayed_surface; @@ -140,7 +143,7 @@ extern const enum AVPixelFormat ff_amf_pix_fmts[]; */ #define AMF_RETURN_IF_FALSE(avctx, exp, ret_value, /*message,*/ ...) \ if (!(exp)) { \ - av_log(avctx, AV_LOG_ERROR, __VA_ARGS__); \ + av_log(avctx, AV_LOG_FATAL, __VA_ARGS__); \ return ret_value; \ }