From patchwork Fri Mar 23 10:20:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kravchenko, Alexander" X-Patchwork-Id: 8120 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp245071jad; Fri, 23 Mar 2018 03:20:20 -0700 (PDT) X-Google-Smtp-Source: AG47ELtyfTxPqYCrsWegqG+OdQcilcyEuiB+huFG1DGebz+Al2LFe5Zd3gaZ6FZqusP8yfnCUEi+ X-Received: by 10.28.172.135 with SMTP id v129mr7497131wme.144.1521800419670; Fri, 23 Mar 2018 03:20:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521800419; cv=none; d=google.com; s=arc-20160816; b=D8sBtPCYOlL1c+1wyIpwg2Ox5A04Pxk6LtOOAbOku3ikRpPDi54YXeX09/bxUolcxV ukaB6e0dHC/kuhffBgXMjLsNDR2O+svbIUL3PMRUx/w1+bfrqW+2aR1Mx3UooWsU5Di7 2bBNGKLrdBZsWa74OQJIdBamUY+tEVRUvs3x16yVYIwBdq/g0Li5i/VIJjIddseC3GZr kmZX8yHBWoGAYRT5RgN126S4XDaiGDxakIwE4ZpenyCyWVe19mKTebPB4FvCt910Tlqm h5/xP55DBGBi3UkfHdpepshzycUdgj24Z5YEANZFRYKaOM2GMdFu/B/AnQ7dqEPrAJjW r0LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:content-language:accept-language :message-id:date:thread-index:thread-topic:to:from:delivered-to :arc-authentication-results; bh=BpZD04SJK9QPmlaM8J7Yc/jjyfjP7SMR+3W4Mw93oHg=; b=R9BHKkU0k77Im8fGOuRpV/Kov2LOx67O43dAPDI5LZXX+AFJ1ApF5NR0N5O/KwbJNB XU4/srSjPQ/wNxFtoOc5g0J6sel2Q59jUiDUUVz9QcrvTN8A5hJ6vVKx3xoQJ17zv7cb uyu+kFBvciEtFP+2Mx629dwS5OrKys/21KW0aQhmEqvEbYjXxlyIIRCYBFVz4XkS4Fmb kqZmElB5iC83Bl+eHFtUtNFXNC59eSQgRo4nBRqHyBxOdPg9x5tbJhdRtuunYO9Ai7cB /yriv6BEpSIceAzbYxBRsfyrMsJaZgp8KAMjtYQBn8/4w3rrrN28qx6KRc4aPZUM1iG6 +Ywg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i125si5642855wma.57.2018.03.23.03.20.19; Fri, 23 Mar 2018 03:20:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 074EE689951; Fri, 23 Mar 2018 12:20:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from ORO-EDGE-01.luxoft.com (oro-edge-01.luxoft.com [109.166.244.21]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 15B71680847 for ; Fri, 23 Mar 2018 12:19:55 +0200 (EET) Received: from ORO-HUBT-02.luxoft.com (172.30.115.36) by oro-edge-01.luxoft.com (172.30.19.21) with Microsoft SMTP Server (TLS) id 14.3.382.0; Fri, 23 Mar 2018 12:20:06 +0200 Received: from ORO-MBOX-05.luxoft.com ([fe80::1137:8dc2:bce5:c3fc]) by ORO-HUBT-02.luxoft.com ([::1]) with mapi id 14.03.0382.000; Fri, 23 Mar 2018 12:20:09 +0200 From: "Kravchenko, Alexander" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH] AMF Encoder: fix issue with frame corruption if encoder connected directly to decoder Thread-Index: AdPCjrFjPLErY6CVS2iveGYmKIrBxQ== Date: Fri, 23 Mar 2018 10:20:08 +0000 Message-ID: <1006FD0F0FE06E479CEE8E19981AE0703680A9D4@ORO-MBOX-05.luxoft.com> Accept-Language: ru-RU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [188.227.65.188] MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] AMF Encoder: fix issue with frame corruption if encoder connected directly to decoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hello, I am collaborating with AMD to integrate and support AMF integration FFmpeg This patch solves issue with frame corruption if encoder connected directly to decoder Solution: storing frame reference while it is used during encoding From 0fae3679bae8e121ed7b997d7eabd533deb113fb Mon Sep 17 00:00:00 2001 From: Alexander Kravchenko Date: Fri, 23 Mar 2018 12:50:21 +0300 Subject: [PATCH] AMF Encoder: fix issue with frame corruption if encoder connected directly to decoder. Solution: storing frame reference while it is used during encoding --- libavcodec/amfenc.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 61 insertions(+), 3 deletions(-) -- 2.16.2.windows.1 ________________________________ This e-mail and any attachment(s) are intended only for the recipient(s) named above and others who have been specifically authorized to receive them. They may contain confidential information. If you are not the intended recipient, please do not read this email or its attachment(s). Furthermore, you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and then delete this e-mail and any attachment(s) or copies thereof from your system. Thank you. diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 89a10ff253..9ffc52b0b0 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -443,6 +443,41 @@ int ff_amf_encode_init(AVCodecContext *avctx) return ret; } +#define AV_AMF_QUERY_INTERFACE(res, from, InterfaceType, to ) \ + { \ + AMFGuid guid_##InterfaceType = IID_##InterfaceType(); \ + res = from->pVtbl->QueryInterface(from, &guid_##InterfaceType, (void**)&to); \ + } + +#define AV_AMF_ASSIGN_PROPERTY_INTERFACE(res, pThis, name, val ) \ + { \ + AMFInterface *amf_interface; \ + AV_AMF_QUERY_INTERFACE(res, val, AMFInterface, amf_interface)\ + if(res == AMF_OK) { \ + AMFVariantStruct var; \ + AMFVariantInit(&var); \ + AMFVariantAssignInterface(&var, amf_interface); \ + amf_interface->pVtbl->Release(amf_interface); \ + res = pThis->pVtbl->SetProperty(pThis, name, var); \ + AMFVariantClear(&var); \ + } \ + } + +#define AV_AMF_GET_PROPERTY_INTERFACE(res, pThis, name, target_type, val) \ + { \ + AMFVariantStruct var; \ + AMFVariantInit(&var); \ + res = pThis->pVtbl->GetProperty(pThis, name, &var); \ + if(res == AMF_OK) { \ + if(var.type == AMF_VARIANT_INTERFACE && var.pInterface) { \ + AMFGuid guid = IID_##target_type(); \ + res = var.pInterface->pVtbl->QueryInterface(var.pInterface, &guid, (void**)&val); \ + } else { \ + res = AMF_INVALID_DATA_TYPE; \ + } \ + AMFVariantClear(&var); \ + } \ + } int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) { @@ -484,6 +519,8 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx == (AVHWDeviceContext*)ctx->hw_device_ctx->data) )) { + AVFrame* frame_ref = av_frame_clone(frame); + AMFBuffer* frame_ref_storage_buffer; #if CONFIG_D3D11VA static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture @@ -496,6 +533,12 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); #endif + res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocBuffer() failed with error %d\n", res); + memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); + + AV_AMF_ASSIGN_PROPERTY_INTERFACE(res, surface, L"av_frame_ref", frame_ref_storage_buffer); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); } else { res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); @@ -555,11 +598,26 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) if (data) { // copy data to packet AMFBuffer* buffer; - AMFGuid guid = IID_AMFBuffer(); - data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface + AV_AMF_QUERY_INTERFACE(res, data, AMFBuffer, buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Invalid data type from encoder->QueryOutput, should be AMFBuffer, error %d\n", res); ret = amf_copy_buffer(avctx, avpkt, buffer); - buffer->pVtbl->Release(buffer); + + //try to get attached av_frame_ref and unref + if(data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer = NULL; + AVFrame *av_frame_ref; + + AV_AMF_GET_PROPERTY_INTERFACE(res, data, L"av_frame_ref", AMFBuffer, frame_ref_storage_buffer); + if(res == AMF_OK) { + memcpy(&av_frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(av_frame_ref)); + av_frame_unref(av_frame_ref); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + } else { + av_log(avctx, AV_LOG_WARNING, "av_frame_ref data attached to frame is corrupted\n"); + } + } + data->pVtbl->Release(data); AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret);