From patchwork Fri Apr 6 10:25:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kravchenko X-Patchwork-Id: 8344 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp501151jad; Fri, 6 Apr 2018 03:25:21 -0700 (PDT) X-Google-Smtp-Source: AIpwx49S8Gpv4gCymfVrW5qViZZqb5T7SX4WcuQl/lFjwwdzbYHXLt0mcs7i3IMVuDN24sIxeNGC X-Received: by 10.223.168.7 with SMTP id l7mr18355006wrc.77.1523010321130; Fri, 06 Apr 2018 03:25:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523010321; cv=none; d=google.com; s=arc-20160816; b=yMSRCHbPGtEau+vxyG8Go2qSqHbagamE8E61SOgJKTj3zA3MFp4y9WC/zCwyUnJGy9 ghMLfFBUI27En1RniQpPbuSRVilT+IcBIO+r1vDM0D1iUxCRDqNirxfoKnkKutOABKNg G2FasdxKQbNaww5vg571Mq7/CrIsCMJ2rsoCrTViDf7RNdSUruh6v7bRwJQEBG9ZaylG 8BPA/BV6HGLpTzsoT954Rd54rFmNInxkQ9WKfIoLxltavRMaKw7Ao52PffWZD6Zd+BKJ UoxWJvH43+2JbWAWmnufmLew+rE0uV4PAt8UdZp2hUAb79pEhU8RLvoeGbH43WFqwcOI JueQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:thread-index:content-language:mime-version :message-id:date:in-reply-to:references:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=4xsrj+Vc0vqLneYZqyhHamgdSfnBo3eV3R6dYkkCXBg=; b=p+GcUnUG4fTKmcmhY8+4HJOwnMcupBK1KpmVGo+tywTpwyb3NOcT7bwYBzz0WSBfsP 7TAjG9PFrJgPe79H32518hme6F9Pyyd27jmymD7pTstPtduBKuZeDAnL1KuGx9+3+JiP XSn9timF4GJj1F+MlIawswu4OMzNKpAOQvj86vusqB/fA2JW1Sjck2HTXssm2AIS7FQB rZgdo35efQkz39Uufn2gmYXxcVMdoGwVrSzINxFaeupm1xxn8QCF+5cSVhWSPsh4iPwY 13P4FmnxqymbnEBPv21TcgGNOmejhCgWZMihyCoT+kMkdEoGRdEBVxs6mFvbZisiLRGV f9YQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=W/nEDH+J; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j17si6881131wrh.394.2018.04.06.03.25.20; Fri, 06 Apr 2018 03:25:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=W/nEDH+J; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8E30D689940; Fri, 6 Apr 2018 13:24:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f173.google.com (mail-wr0-f173.google.com [209.85.128.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E165968987F for ; Fri, 6 Apr 2018 13:24:49 +0300 (EEST) Received: by mail-wr0-f173.google.com with SMTP id p53so1180270wrc.10 for ; Fri, 06 Apr 2018 03:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-transfer-encoding:content-language:thread-index; bh=2rVxs+so4KjC1cBfzyyUsoBV6WzlXuzd8EvhctSemtA=; b=W/nEDH+Jpgz/5IITAH8zGFqWjc0osFQ8l/SWJeRc4/0g5K+VadseLzkC2/FJJ/NLS2 Yny/IoshtOvut7brRTwgoEFBMj1E9Gd78rf/o+TLBMnWB0q9SOaSLoP3k8j5ryyYg2Lx t2E9PFhYD9fl9vhX/tBJpgoXJxvATZV59FlnZNXKf2l3ZMXw/TFFS3E+sRfIk29UGeTp x6n+NLkJk4lDUGJLd8dJidv60TtUgfifr1aZBMrJO+xzSnkCTee+OGQcObjk5bd3mowO o8pEdh/z1TWabcqUfwceB9q+9FpDjblNgYC8JNS8CiLAB+KMk7psVFcVT6m4aR0en26l UbrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:content-language :thread-index; bh=2rVxs+so4KjC1cBfzyyUsoBV6WzlXuzd8EvhctSemtA=; b=PDeyiUaF30/XkBXJUh4awZDaAkPQ95CEbpO8EffT/9zaWfV2Li70UZ2qCqU94Vin3z GtOB9xWMFG8FG2dlyuYUfOM3jwI0V3/IS8OKYrKvsW9pp4YrRtKY6f+PnTY4V/kxAYYh DsNhD4MFX+ZipJmJYg8+2teNDCo7SU6eAT0Y9nCXATAhP2u6puxNlEZTzQe2+fTL9JU/ D7DJNI9H7SjrGMowcbMxUahazIbpsZxDzZ+3yd9ww8xFbnU0cKPmUVugZcq2mXujOZnM 9wy3MJip46HagicMT1Jy0/AOtqtBw6xGXpZRPi9LyGSgs18jIhIWXBLklaxsnbJ4meRO mCtA== X-Gm-Message-State: ALQs6tA3MAcSqM4o9jv6ZIS0miSukWV1BRvtnVZV5ziZBe/13BRoAz3F tfv3lp6N09Jrb76UJGgKigKCGAyv X-Received: by 2002:a19:4d46:: with SMTP id a67-v6mr16434916lfb.36.1523010310871; Fri, 06 Apr 2018 03:25:10 -0700 (PDT) Received: from KAAMBWin10 ([188.227.65.188]) by smtp.gmail.com with ESMTPSA id s25-v6sm1976749lfc.21.2018.04.06.03.25.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Apr 2018 03:25:10 -0700 (PDT) From: "Alexander Kravchenko" To: "'FFmpeg development discussions and patches'" References: <01ab01d3ccf1$ff6b67d0$fe423770$@gmail.com> In-Reply-To: Date: Fri, 6 Apr 2018 13:25:10 +0300 Message-ID: <01dc01d3cd91$8b2758c0$a1760a40$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Content-Language: en-us Thread-Index: AQJRN8lF2celsV/yjjet0q9G9fCIogEhXecKou+mFuA= Subject: Re: [FFmpeg-devel] [PATCH] lavc/amfenc: Retain a reference to D3D frames used as input during the encoding process X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" > > This breaks the testcase described in > https://trac.ffmpeg.org/ticket/6990 which is basically the same as the > one you described in this patch. > > I get the following spammed repeatedly: > > [AVHWFramesContext @ 000000000502d340] Static surface pool size exceeded. > [mpeg2video @ 0000000002f8d400] get_buffer() failed > [mpeg2video @ 0000000002f8d400] thread_get_buffer() failed > [mpeg2video @ 0000000002f8d400] get_buffer() failed (-12 0000000000000000) > Error while decoding stream #0:1: Operation not permitted Hi, could you please review the following updated patch I added queue limit according initial pool size of incoming frame context. This solves the problem running this pipeline without -extra_hw_frames 16 option, but -extra_hw_frames option can be used to have bigger queue for encoder. --- libavcodec/amfenc.c | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++++- libavcodec/amfenc.h | 3 ++ 2 files changed, 99 insertions(+), 1 deletion(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 89a10ff253..eb7b20c252 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -157,6 +157,9 @@ static int amf_init_context(AVCodecContext *avctx) AmfContext *ctx = avctx->priv_data; AMF_RESULT res = AMF_OK; + ctx->hwsurfaces_in_queue = 0; + ctx->hwsurfaces_in_queue_max = 16; + // configure AMF logger // the return of these functions indicates old state and do not affect behaviour ctx->trace->pVtbl->EnableWriter(ctx->trace, AMF_TRACE_WRITER_DEBUG_OUTPUT, ctx->log_to_dbg != 0 ); @@ -187,6 +190,7 @@ static int amf_init_context(AVCodecContext *avctx) if (!ctx->hw_frames_ctx) { return AVERROR(ENOMEM); } + ctx->hwsurfaces_in_queue_max = device_ctx->initial_pool_size - 1; } else { if(res == AMF_NOT_SUPPORTED) av_log(avctx, AV_LOG_INFO, "avctx->hw_frames_ctx has D3D11 device which doesn't have D3D11VA interface, switching to default\n"); @@ -443,6 +447,73 @@ int ff_amf_encode_init(AVCodecContext *avctx) return ret; } +#define AMF_AV_QUERY_INTERFACE(res, from, InterfaceTypeTo, to) \ + { \ + AMFGuid guid_##InterfaceTypeTo = IID_##InterfaceTypeTo(); \ + res = from->pVtbl->QueryInterface(from, &guid_##InterfaceTypeTo, (void**)&to); \ + } + +#define AMF_AV_ASSIGN_PROPERTY_INTERFACE(res, pThis, name, val) \ + { \ + AMFInterface *amf_interface; \ + AMFVariantStruct var; \ + res = AMFVariantInit(&var); \ + if (res == AMF_OK) { \ + AMF_AV_QUERY_INTERFACE(res, val, AMFInterface, amf_interface)\ + if (res == AMF_OK) { \ + res = AMFVariantAssignInterface(&var, amf_interface); \ + amf_interface->pVtbl->Release(amf_interface); \ + } \ + if (res == AMF_OK) { \ + res = pThis->pVtbl->SetProperty(pThis, name, var); \ + } \ + AMFVariantClear(&var); \ + }\ + } + +#define AMF_AV_GET_PROPERTY_INTERFACE(res, pThis, name, TargetType, val) \ + { \ + AMFVariantStruct var; \ + res = AMFVariantInit(&var); \ + if (res == AMF_OK) { \ + res = pThis->pVtbl->GetProperty(pThis, name, &var); \ + if (res == AMF_OK) { \ + if (var.type == AMF_VARIANT_INTERFACE && AMFVariantInterface(&var)) { \ + AMF_AV_QUERY_INTERFACE(res, AMFVariantInterface(&var), TargetType, val); \ + } else { \ + res = AMF_INVALID_DATA_TYPE; \ + } \ + } \ + AMFVariantClear(&var); \ + }\ + } + +static AMFBuffer* amf_create_buffer_with_frame_ref(const AVFrame* frame, AMFContext *context) +{ + AVFrame *frame_ref; + AMFBuffer *frame_ref_storage_buffer = NULL; + AMF_RESULT res; + + res = context->pVtbl->AllocBuffer(context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); + if (res == AMF_OK) { + frame_ref = av_frame_clone(frame); + if (frame_ref) { + memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); + } else { + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + frame_ref_storage_buffer = NULL; + } + } + return frame_ref_storage_buffer; +} + +static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffer) +{ + AVFrame *av_frame_ref; + memcpy(&av_frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(av_frame_ref)); + av_frame_free(&av_frame_ref); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); +} int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) { @@ -484,6 +555,7 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx == (AVHWDeviceContext*)ctx->hw_device_ctx->data) )) { + AMFBuffer *frame_ref_storage_buffer; #if CONFIG_D3D11VA static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture @@ -496,6 +568,17 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); #endif + + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); + + AMF_AV_ASSIGN_PROPERTY_INTERFACE(res, surface, L"av_frame_ref", frame_ref_storage_buffer); + if (res != AMF_OK) { + surface->pVtbl->Release(surface); + } + ctx->hwsurfaces_in_queue++; + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "SetProperty failed for \"av_frame_ref\" with error %d\n", res); } else { res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); @@ -560,6 +643,18 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) ret = amf_copy_buffer(avctx, avpkt, buffer); buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer; + AMF_AV_GET_PROPERTY_INTERFACE(res, data, L"av_frame_ref", AMFBuffer, frame_ref_storage_buffer); + if (res == AMF_OK) { + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + ctx->hwsurfaces_in_queue--; + } else { + av_log(avctx, AV_LOG_WARNING, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + } + } + data->pVtbl->Release(data); AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); @@ -589,7 +684,7 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) av_log(avctx, AV_LOG_WARNING, "Data acquired but delayed drain submission got AMF_INPUT_FULL- should not happen\n"); } } - } else if (ctx->delayed_surface != NULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF)) { + } else if (ctx->delayed_surface != NULL || ctx->delayed_drain || (ctx->eof && res_query != AMF_EOF) || (ctx->hwsurfaces_in_queue >= ctx->hwsurfaces_in_queue_max)) { block_and_wait = 1; av_usleep(1000); // wait and poll again } diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index 84f0aad2fa..b1361842bd 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -62,6 +62,9 @@ typedef struct AmfContext { AVBufferRef *hw_device_ctx; ///< pointer to HW accelerator (decoder) AVBufferRef *hw_frames_ctx; ///< pointer to HW accelerator (frame allocator) + int hwsurfaces_in_queue; + int hwsurfaces_in_queue_max; + // helpers to handle async calls int delayed_drain; AMFSurface *delayed_surface;