From patchwork Fri Mar 23 11:19:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kravchenko, Alexander" X-Patchwork-Id: 8121 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp300995jad; Fri, 23 Mar 2018 04:20:05 -0700 (PDT) X-Google-Smtp-Source: AG47ELuov+lGxdsvtEpvRhotqi4XZjhQsgxy0Rv4JzPwuoINH/0thOlo53toe2uoHdgGHiZ9F1gi X-Received: by 10.223.132.103 with SMTP id 94mr21649595wrf.275.1521804005888; Fri, 23 Mar 2018 04:20:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521804005; cv=none; d=google.com; s=arc-20160816; b=gJjtJ+wGZVWR7wCuNhz7Bwkmf5NmdDAUYlIYo5GptavZSe+XiBGZIqZT2HvscWwRsA eam2tDwDRXcItSM/LY5hJZkJRd7+ShTY2JCVlnBJH2Q6P/ItwmsPCxZQV3iE8NHeTDtA zTpOtNEykNWmfWil118NuHIvhrdqRNt7qgGrH0Bzidl6KKhMXYsiKwS3lu5kXv03u/Ir 3y4OcHt9qyUFPVHQBepbM4c3xXCt/Y/WJcaIF5s5fqLYULyAnS0npk/IeQ8kTFu8q43z gCi6MGo8O+SswAnmCCiEnll0YNLfMPnDN4XcBr330vUotWxRnPA1V+0N05+x1eZzieRv 5GrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:content-language:accept-language :message-id:date:thread-index:thread-topic:to:from:delivered-to :arc-authentication-results; bh=5e8Gzt/+ua7SVWZ0ZRlVeZLX2nHZDx/S/qctrsZ58Ew=; b=icZPLv64z8K6pf27kHqJKx6WWBcODIdl8Nyk9y000CwsmMyVz2AMiW20Dn2qhYQfzf Abfu4Mh6b6p9ND7hCoxFJ84QCZQn+ExOFO37z0qTZg6ZBnkU9B2Z4EqHTSNzIRedWgH7 v7pGwTGsjXPZrr3txxVWRg4C3pZytcGUctX3J7yz1Do85gXM+/nyH+laSjS4G1rxRXEc nCJBvdpVo5bGIGZwrY+npZ4W+fAaJnP3DWoa4+m+9Stp+QzQMac+Y2I6D7cjqYjEqI8j MCOP6ozts9fzrltJh9N71yeP2UNN4W2sTJf0dAOL5eCbnfSD+EV/MwexyndoJcQbIdQd 4SsA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f2si6488946wrg.475.2018.03.23.04.20.04; Fri, 23 Mar 2018 04:20:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A0CF76899EA; Fri, 23 Mar 2018 13:19:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from ORO-EDGE-03.luxoft.com (oro-edge-03.luxoft.com [109.166.244.23]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 05C09687991 for ; Fri, 23 Mar 2018 13:19:40 +0200 (EET) Received: from ORO-HUBT-02.luxoft.com (172.30.115.36) by oro-edge-03.luxoft.com (172.30.19.23) with Microsoft SMTP Server (TLS) id 14.3.382.0; Fri, 23 Mar 2018 13:19:54 +0200 Received: from ORO-MBOX-05.luxoft.com ([fe80::1137:8dc2:bce5:c3fc]) by ORO-HUBT-02.luxoft.com ([::1]) with mapi id 14.03.0382.000; Fri, 23 Mar 2018 13:19:55 +0200 From: "Kravchenko, Alexander" To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH] lavc/amfenc: Reference to input AVFrame (hwaccel) is retained during the encoding process Thread-Index: AdPCmLLPD8HrYYuBRTGvCUuE2dj3wg== Date: Fri, 23 Mar 2018 11:19:55 +0000 Message-ID: <1006FD0F0FE06E479CEE8E19981AE0703680AA52@ORO-MBOX-05.luxoft.com> Accept-Language: ru-RU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [188.227.65.188] MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] lavc/amfenc: Reference to input AVFrame (hwaccel) is retained during the encoding process X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" An additional reference to input AVFrame (hwaccel) is retained during the encoding process. This postpone reusing frames by decoder while they are used by encoder and prevents frame corruption Issue with frame corruption was reproduced using: ffmpeg.exe -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i input.h264 -an -c:v h264_amf output.mkv From: Alexander Kravchenko --- libavcodec/amfenc.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 61 insertions(+), 3 deletions(-) -- 2.16.2.windows.1 diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 89a10ff253..9ffc52b0b0 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -443,6 +443,41 @@ int ff_amf_encode_init(AVCodecContext *avctx) return ret; } +#define AV_AMF_QUERY_INTERFACE(res, from, InterfaceType, to ) \ + { \ + AMFGuid guid_##InterfaceType = IID_##InterfaceType(); \ + res = from->pVtbl->QueryInterface(from, &guid_##InterfaceType, (void**)&to); \ + } + +#define AV_AMF_ASSIGN_PROPERTY_INTERFACE(res, pThis, name, val ) \ + { \ + AMFInterface *amf_interface; \ + AV_AMF_QUERY_INTERFACE(res, val, AMFInterface, amf_interface)\ + if(res == AMF_OK) { \ + AMFVariantStruct var; \ + AMFVariantInit(&var); \ + AMFVariantAssignInterface(&var, amf_interface); \ + amf_interface->pVtbl->Release(amf_interface); \ + res = pThis->pVtbl->SetProperty(pThis, name, var); \ + AMFVariantClear(&var); \ + } \ + } + +#define AV_AMF_GET_PROPERTY_INTERFACE(res, pThis, name, target_type, val) \ + { \ + AMFVariantStruct var; \ + AMFVariantInit(&var); \ + res = pThis->pVtbl->GetProperty(pThis, name, &var); \ + if(res == AMF_OK) { \ + if(var.type == AMF_VARIANT_INTERFACE && var.pInterface) { \ + AMFGuid guid = IID_##target_type(); \ + res = var.pInterface->pVtbl->QueryInterface(var.pInterface, &guid, (void**)&val); \ + } else { \ + res = AMF_INVALID_DATA_TYPE; \ + } \ + AMFVariantClear(&var); \ + } \ + } int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) { @@ -484,6 +519,8 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx == (AVHWDeviceContext*)ctx->hw_device_ctx->data) )) { + AVFrame* frame_ref = av_frame_clone(frame); + AMFBuffer* frame_ref_storage_buffer; #if CONFIG_D3D11VA static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture @@ -496,6 +533,12 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); #endif + res = ctx->context->pVtbl->AllocBuffer(ctx->context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocBuffer() failed with error %d\n", res); + memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); + + AV_AMF_ASSIGN_PROPERTY_INTERFACE(res, surface, L"av_frame_ref", frame_ref_storage_buffer); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); } else { res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); @@ -555,11 +598,26 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) if (data) { // copy data to packet AMFBuffer* buffer; - AMFGuid guid = IID_AMFBuffer(); - data->pVtbl->QueryInterface(data, &guid, (void**)&buffer); // query for buffer interface + AV_AMF_QUERY_INTERFACE(res, data, AMFBuffer, buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_UNKNOWN, "Invalid data type from encoder->QueryOutput, should be AMFBuffer, error %d\n", res); ret = amf_copy_buffer(avctx, avpkt, buffer); - buffer->pVtbl->Release(buffer); + + //try to get attached av_frame_ref and unref + if(data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer = NULL; + AVFrame *av_frame_ref; + + AV_AMF_GET_PROPERTY_INTERFACE(res, data, L"av_frame_ref", AMFBuffer, frame_ref_storage_buffer); + if(res == AMF_OK) { + memcpy(&av_frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(av_frame_ref)); + av_frame_unref(av_frame_ref); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + } else { + av_log(avctx, AV_LOG_WARNING, "av_frame_ref data attached to frame is corrupted\n"); + } + } + data->pVtbl->Release(data); AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret);