From patchwork Wed Mar 28 11:58:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kravchenko X-Patchwork-Id: 8206 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp374054jad; Wed, 28 Mar 2018 05:04:09 -0700 (PDT) X-Google-Smtp-Source: AIpwx495ej0wOH5r40kdfyXjGjrr27vJKk8yG+vFvxnzzq+F+0wlZHRkja7hUdgJ9H+cPJr6sTh6 X-Received: by 10.28.114.9 with SMTP id n9mr2235647wmc.147.1522238649717; Wed, 28 Mar 2018 05:04:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522238649; cv=none; d=google.com; s=arc-20160816; b=rIfyZLFXliBGGlMpebdbNSLd48LXSVDae4adLpI28aod739XOoCIqMXKTJVIALgL9t 6F95jBJWJgqxb1Waj+7+m5aMGRjVfrMRlvg6NOy+3di0T5IyyxraQB/Xun1ZPLkCKYSs N/6aGI7OYIdCUULyZ3ei+ywfGYe/HNlrS7xXmLZu3O4UGR4gJYfL0dHx6J3VyokNqoeb dZ+44Ehz0Nd2hAR5mqPmKsfKeqRcWW11sA2x5GeB41vl5ZclL6gej5BmchDFjdD/DzkH AF2f+hMYB1rFsIcitKK9zJbyPghgMo38f545GfFtwzuQgGe45pwnygh5ghLdRBbylehz K3QA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:thread-index:content-language:mime-version :message-id:date:in-reply-to:references:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=uGPrIWYgZM9OXHtmL0cYD/dvHrg9+aU5oWuHsqeaNTE=; b=QN12vl4cnhp72L1gprh+7LBHplKwiKn5b7FMk5VFCbfQOHoePt5xmO+BqJ2/VoxYin dUhmcan/kWVU25KZ6MK8HefhnLXOGrpk7A4E6Rv0Si3I4HVYJopfNxPWNu2iCyrBjejx mcVCUfxBv5Rn7yxU/WS6XDth8wRFTJmFkd0rk6r97PF5vIU5qSoEHZefuPExmSwYgWbv HGgLTHf8OU8OfswQiJmulpqd9lRYWqWnZGVbqEn5tIZpUseUuD4qegy6PR9Z9OOkQdQG n7qAbkNc2iVYsyJ8n80QPlaTkZJMWEkDslMosrzcD4k0w9t+r/qT2SujlLb7pG0ii9aG z9cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=HxRaBJAK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id z109si2795846wrc.49.2018.03.28.05.04.09; Wed, 28 Mar 2018 05:04:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=HxRaBJAK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D57A3689BE4; Wed, 28 Mar 2018 15:03:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf0-f42.google.com (mail-lf0-f42.google.com [209.85.215.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2CC12680475 for ; Wed, 28 Mar 2018 15:03:44 +0300 (EEST) Received: by mail-lf0-f42.google.com with SMTP id j68-v6so3078570lfg.13 for ; Wed, 28 Mar 2018 05:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-transfer-encoding:content-language:thread-index; bh=WT3MPC3rKKQF4pJmUHz1Ok+lHsAiQ5BcLlniUG/mY48=; b=HxRaBJAKc3WtUzyUYe+se4yv1XQu60hlJSFpnsOVjq8Q3a9BE4N0TQhsrcgrShx7xJ jTFQckuGhs+yNPQXB8LI06sjeMZU/kjdqCaOc/Vchjdp3cpVxC3LkjGGpzDEJoXiEJZ9 K4yGM3eKo9CZJPHOt7D8KP3N+kFTkllY0bljrfHctHb4eELTxBXgBLJ3PiAMaG7JC/EM quI8ngm0HWvNVm93WNFwS0LBGhfSSlpfyeBob2pLNPFucbYQKgQQnG/sHI0zQCFqJzvJ DtkVs+Vs/uNzo/asgJh8FN7M8ldkaXlrbvMyEbQ9JdYyw0ZfBLLmXyq38oA99QkrAQQ9 Cw3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:content-language :thread-index; bh=WT3MPC3rKKQF4pJmUHz1Ok+lHsAiQ5BcLlniUG/mY48=; b=sV4LyL9uGuFP3Zi+hIpGXnDzqk/ugWMEU8ZNFVYmaay0Or7bCE3OZ3qG9PFiuTIu6g DtQu9LVzXzRdcoLjEWHs5DhxE3PjuBK70IbITELhqHhFPrsVI5/zOLLe6KkqPQBavR9I SWpOouR0eLcJSyRmEVxFOV9m2t8uEsYlWvwg0KG8EpGp+pp7Hge67fEjEVo2vvEUvibN FJdBKxHkFGoAc6xeuxr/mnPBZITq/po0oG3oHEXlRH7AzSjX+Ndx7SAsvcccj6BaEhmN Jff5DJn9AUnok2Uy5f81mUEuAgQfOTUwddepxMr10UDNoIw6Np0CZtLT7fG62IDmnZLg jFuQ== X-Gm-Message-State: AElRT7Fne1DvhMNDhISRcxVbhDdq0/XXne9of36+282yj1rRc3CL90he Xqn/JD2XLBW4x7JLuHm/xHoC+g8C X-Received: by 10.46.122.15 with SMTP id v15mr2206861ljc.7.1522238304682; Wed, 28 Mar 2018 04:58:24 -0700 (PDT) Received: from KAAMBWin10 ([188.227.65.188]) by smtp.gmail.com with ESMTPSA id q194-v6sm662066lfe.68.2018.03.28.04.58.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Mar 2018 04:58:24 -0700 (PDT) From: "Alexander Kravchenko" To: "'FFmpeg development discussions and patches'" References: <1006FD0F0FE06E479CEE8E19981AE0703680AA52@ORO-MBOX-05.luxoft.com> <002401d3c4ee$fada2fe0$f08e8fa0$@gmail.com> <03314621-c380-cce6-021f-1e90b2074507@jkqxz.net> <1006FD0F0FE06E479CEE8E19981AE0703680BA0F@ORO-MBOX-05.luxoft.com> <002c01d3c554$eedd94a0$cc98bde0$@gmail.com> In-Reply-To: <002c01d3c554$eedd94a0$cc98bde0$@gmail.com> Date: Wed, 28 Mar 2018 14:58:24 +0300 Message-ID: <011501d3c68c$13d825d0$3b887170$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Content-Language: en-us Thread-Index: AQEOIRNY0HgZF/j9DGUWfZUmLoStCgIQX00SAmPqzekB0KqT/gKHIymPA0YqTeSlECTf4A== Subject: [FFmpeg-devel] [PATCH] lavc/amfenc: Reference to input AVFrame (hwaccel) is retained during the encoding process X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello. I fixed couple of issues in previous patch. So if it is possible to use newer one, could you please review the following patch. Subject: amfenc: Retain a reference to D3D frames used as input during the encoding process --- libavcodec/amfenc.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 89a10ff253..4a8385f9e2 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -443,6 +443,75 @@ int ff_amf_encode_init(AVCodecContext *avctx) return ret; } +#define AMF_AV_QUERY_INTERFACE(res, from, InterfaceTypeTo, to) \ + { \ + AMFGuid guid_##InterfaceTypeTo = IID_##InterfaceTypeTo(); \ + res = from->pVtbl->QueryInterface(from, &guid_##InterfaceTypeTo, (void**)&to); \ + } + +#define AMF_AV_ASSIGN_PROPERTY_INTERFACE(res, pThis, name, val) \ + { \ + AMFInterface *amf_interface; \ + AMFVariantStruct var; \ + res = AMFVariantInit(&var); \ + if (res != AMF_OK) \ + return res; \ + if (res == AMF_OK) { \ + AMF_AV_QUERY_INTERFACE(res, val, AMFInterface, amf_interface)\ + } \ + if (res == AMF_OK) { \ + res = AMFVariantAssignInterface(&var, amf_interface); \ + amf_interface->pVtbl->Release(amf_interface); \ + } \ + if (res == AMF_OK) { \ + res = pThis->pVtbl->SetProperty(pThis, name, var); \ + } \ + AMFVariantClear(&var); \ + } + +#define AMF_AV_GET_PROPERTY_INTERFACE(res, pThis, name, TargetType, val) \ + { \ + AMFVariantStruct var; \ + res = AMFVariantInit(&var); \ + if (res != AMF_OK) \ + return res; \ + res = pThis->pVtbl->GetProperty(pThis, name, &var); \ + if (res == AMF_OK) { \ + if (var.type == AMF_VARIANT_INTERFACE && AMFVariantInterface(&var)) { \ + AMF_AV_QUERY_INTERFACE(res, AMFVariantInterface(&var), TargetType, val); \ + } else { \ + res = AMF_INVALID_DATA_TYPE; \ + } \ + } \ + AMFVariantClear(&var); \ + } + +static AMFBuffer* amf_create_buffer_with_frame_ref(const AVFrame* frame, AMFContext *context) +{ + AVFrame *frame_ref; + AMFBuffer *frame_ref_storage_buffer = NULL; + AMF_RESULT res; + + res = context->pVtbl->AllocBuffer(context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); + if (res == AMF_OK) { + frame_ref = av_frame_clone(frame); + if (frame_ref) { + memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); + } else { + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + frame_ref_storage_buffer = NULL; + } + } + return frame_ref_storage_buffer; +} + +static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffer) +{ + AVFrame *av_frame_ref; + memcpy(&av_frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(av_frame_ref)); + av_frame_free(&av_frame_ref); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); +} int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) { @@ -484,6 +553,7 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx == (AVHWDeviceContext*)ctx->hw_device_ctx->data) )) { + AMFBuffer *frame_ref_storage_buffer; #if CONFIG_D3D11VA static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture @@ -496,6 +566,16 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); #endif + + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); + + AMF_AV_ASSIGN_PROPERTY_INTERFACE(res, surface, L"av_frame_ref", frame_ref_storage_buffer); + if (res != AMF_OK) { + surface->pVtbl->Release(surface); + } + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "SetProperty failed for \"av_frame_ref\" with error %d\n", res); } else { res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); @@ -560,6 +640,17 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) ret = amf_copy_buffer(avctx, avpkt, buffer); buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer; + AMF_AV_GET_PROPERTY_INTERFACE(res, data, L"av_frame_ref", AMFBuffer, frame_ref_storage_buffer); + if (res == AMF_OK) { + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + } else { + av_log(avctx, AV_LOG_WARNING, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + } + } + data->pVtbl->Release(data); AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret);