From patchwork Thu Apr 5 15:23:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kravchenko X-Patchwork-Id: 8338 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp6178712jad; Thu, 5 Apr 2018 08:28:32 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+hachWaESYV1TSbSsFPUw/fiDS849LLZw+0PX92lxeT6zPDnkUr4cxiMBGhcjdjpLC/4pS X-Received: by 10.28.109.131 with SMTP id b3mr9880185wmi.145.1522942112107; Thu, 05 Apr 2018 08:28:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522942112; cv=none; d=google.com; s=arc-20160816; b=WGlTFUD3nxYZ7lEXOuO0yedimadB8RVWrK5eJVg6tFIHTZM3YOauppZ/WbLb6W6R9i sJ8QtqzWVZ2z7jh9qeWSxiugLkVxcfGvXvi4IpFgbLmWqCtu/qF+SStoG+mCL08ck/94 pP9ICb8OpHa1iiLQSjU9o3nRrdPB6Xdkqqlo4HxzDMVijw2hLPKwsEc5k8RN7aFRk1a/ lQHJLLBPxTUnH7/FrRTesOyKqSKICV4WEYCyDZ+PTI4kWHShJa4kZuda6i8IT347PS6o yI+LXJ3bg2+96Lp1FTOuJYrhaSsn/UQTvrIs3REaZHlxoziUYm+Ic1RG+//gcQKsOQ7z 5oPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:thread-index:content-language:mime-version :message-id:date:to:from:dkim-signature:delivered-to :arc-authentication-results; bh=FO1oLNudR5E7gDRS/uhbmSv3QLEbW/WdZbDuanVSdpo=; b=QBlynX9xyDR1r0cDz5LL+I/sk4NcWWY113V/dulaoleXAbloiO8AlvbVVBWSmBywo1 JL4Iqb15JWqloPKOdLjkSavf60Ksf5V+t+qZ/Chs6o/u7IAbYs7SW3pXuwUVqCHLTsdK YFA6rETCfZ2zSrsz8Dtb+zrr4CFpq4sAibz1fp2MVs+EqOG4F3wZ6FhqjQ3xw23Q5LhN IiRoJRRMjqcSXULCSES8KhZZ3mN3Ymiqh8M+YWFAq2MZ+5jC8MO6QqXKEDlutTRnIoTA 8i2WAmE7ZVG5dyavbVllRupkb7VKZSz5CTeNlphz1tdUx6DRxXSa4upSI4Y4iQ3HRuyV yhnQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=lKF5ptJ6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p15si4656385wmf.99.2018.04.05.08.28.31; Thu, 05 Apr 2018 08:28:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=lKF5ptJ6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EC8856898D4; Thu, 5 Apr 2018 18:28:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9715E6805E8 for ; Thu, 5 Apr 2018 18:28:01 +0300 (EEST) Received: by mail-wm0-f52.google.com with SMTP id t67so6888989wmt.0 for ; Thu, 05 Apr 2018 08:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding:content-language:thread-index; bh=eCdOmltCkPzM/Is3I/rzDg8sk+nmg48nXq9ULXVUDFg=; b=lKF5ptJ6VaAGZOjR0QlZIGxi8amdSqmtxRQWK/poy57+GaVHtrtsB9PhdO7Q6LDPfK Vzsx4tuz588iVZkLfcNSdrpq1WMnZiIPvfFjws8CCrJdZRt771nWo1AmBIyG02/Nnf6q DOFE1QmnXre1wdkzxE+SYyw+cR6MaRgX/nZ3uGobe7vlvz79jMyGUlCmW4Xmj2xgWVcG zYAt8IJtIe1E9x/TzPL+gFzXb+d3jJq1NnsXUf8akEtV+BFOwIldkbXDzgZwCiQMrfy4 Y+XnXQHZwF4JZPbdxmvgMoio6RHCgNJlSan1Z2+eaQ29N7eRn1st5UpOzqOPCp34M5+Y z1ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding:content-language:thread-index; bh=eCdOmltCkPzM/Is3I/rzDg8sk+nmg48nXq9ULXVUDFg=; b=scYkVny7bUKr9iC74do5zq5RktPUBVGDrSQMPt6+TeAE8vbzNmpCVBROBI5LHw/3BE ZlMwgvhQMeqf+nm6aSTNVbB1x17kXSYKW00Xh03BHYeA0u4ExcRG7BcJfTGltWqebgVw otDaQxuWTN1vzMeRJeFcW6DUwvvQ9tdpw0sBEWa7HUDcclYk3KzfOWYhXr5PbORMsrjl 6vSrs7D9Cl+IFHWlJ/sU3294fJBnIMitdVMAudJyQUEyysRwnfoJTw+A3+QzVViniDS/ /hlBDI6rV4AX1Km77w701Z/6TgWKaSl3lvWR3k6mVTYtBWWobdSFoy7J0Zd8jBOsd2VI MAGg== X-Gm-Message-State: ALQs6tC47hLAzc+GNpRThgDY3yYncWnZRXnplUgxHhXM1fE6U7O2c4lj BOFEJdQJndzstdRs22NES8pIBTC0 X-Received: by 10.46.58.16 with SMTP id h16mr4726279lja.146.1522941786698; Thu, 05 Apr 2018 08:23:06 -0700 (PDT) Received: from KAAMBWin10 ([188.227.65.188]) by smtp.gmail.com with ESMTPSA id f7-v6sm1565463lfb.56.2018.04.05.08.23.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Apr 2018 08:23:05 -0700 (PDT) From: "Alexander Kravchenko" To: "'FFmpeg development discussions and patches'" Date: Thu, 5 Apr 2018 18:23:06 +0300 Message-ID: <01ab01d3ccf1$ff6b67d0$fe423770$@gmail.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Content-Language: en-us Thread-Index: AdPM7YmWuYY1LjTfSjqsafnlEVesIw== Subject: [FFmpeg-devel] [PATCH] lavc/amfenc: Retain a reference to D3D frames used as input during the encoding process X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This fixes frame corruption issue when decoder started reusing frames while they are still in use of encoding process Issue with frame corruption was reproduced using: ffmpeg.exe -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i input.h264 -an -c:v h264_amf output.mkv Previous questions and answers > How many frames can end up queued inside the encoder here? 16 > Is there always an exact 1->1 correspondence between input frames and output packets? That is, is it guaranteed that no frames are > ever dropped, even in the low-latency modes? yes > These look even more like they should be functions rather than macros now? In particular, the returns in them interact badly with the code below. Macroses were used because they works with different types. I have removed returns from them (sorry, I missed this comment). --- libavcodec/amfenc.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture @@ -496,6 +564,16 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) // input HW surfaces can be vertically aligned by 16; tell AMF the real size surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); #endif + + frame_ref_storage_buffer = amf_create_buffer_with_frame_ref(frame, ctx->context); + AMF_RETURN_IF_FALSE(ctx, frame_ref_storage_buffer != NULL, AVERROR(ENOMEM), "create_buffer_with_frame_ref() returned NULL\n"); + + AMF_AV_ASSIGN_PROPERTY_INTERFACE(res, surface, L"av_frame_ref", frame_ref_storage_buffer); + if (res != AMF_OK) { + surface->pVtbl->Release(surface); + } + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "SetProperty failed for \"av_frame_ref\" with error %d\n", res); } else { res = ctx->context->pVtbl->AllocSurface(ctx->context, AMF_MEMORY_HOST, ctx->format, avctx->width, avctx->height, &surface); AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); @@ -560,6 +638,17 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) ret = amf_copy_buffer(avctx, avpkt, buffer); buffer->pVtbl->Release(buffer); + + if (data->pVtbl->HasProperty(data, L"av_frame_ref")) { + AMFBuffer *frame_ref_storage_buffer; + AMF_AV_GET_PROPERTY_INTERFACE(res, data, L"av_frame_ref", AMFBuffer, frame_ref_storage_buffer); + if (res == AMF_OK) { + amf_release_buffer_with_frame_ref(frame_ref_storage_buffer); + } else { + av_log(avctx, AV_LOG_WARNING, "GetProperty failed for \"av_frame_ref\" with error %d\n", res); + } + } + data->pVtbl->Release(data); AMF_RETURN_IF_FALSE(ctx, ret >= 0, ret, "amf_copy_buffer() failed with error %d\n", ret); diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 89a10ff253..084b06e56d 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -443,6 +443,73 @@ int ff_amf_encode_init(AVCodecContext *avctx) return ret; } +#define AMF_AV_QUERY_INTERFACE(res, from, InterfaceTypeTo, to) \ + { \ + AMFGuid guid_##InterfaceTypeTo = IID_##InterfaceTypeTo(); \ + res = from->pVtbl->QueryInterface(from, &guid_##InterfaceTypeTo, (void**)&to); \ + } + +#define AMF_AV_ASSIGN_PROPERTY_INTERFACE(res, pThis, name, val) \ + { \ + AMFInterface *amf_interface; \ + AMFVariantStruct var; \ + res = AMFVariantInit(&var); \ + if (res == AMF_OK) { \ + AMF_AV_QUERY_INTERFACE(res, val, AMFInterface, amf_interface)\ + if (res == AMF_OK) { \ + res = AMFVariantAssignInterface(&var, amf_interface); \ + amf_interface->pVtbl->Release(amf_interface); \ + } \ + if (res == AMF_OK) { \ + res = pThis->pVtbl->SetProperty(pThis, name, var); \ + } \ + AMFVariantClear(&var); \ + }\ + } + +#define AMF_AV_GET_PROPERTY_INTERFACE(res, pThis, name, TargetType, val) \ + { \ + AMFVariantStruct var; \ + res = AMFVariantInit(&var); \ + if (res == AMF_OK) { \ + res = pThis->pVtbl->GetProperty(pThis, name, &var); \ + if (res == AMF_OK) { \ + if (var.type == AMF_VARIANT_INTERFACE && AMFVariantInterface(&var)) { \ + AMF_AV_QUERY_INTERFACE(res, AMFVariantInterface(&var), TargetType, val); \ + } else { \ + res = AMF_INVALID_DATA_TYPE; \ + } \ + } \ + AMFVariantClear(&var); \ + }\ + } + +static AMFBuffer* amf_create_buffer_with_frame_ref(const AVFrame* frame, AMFContext *context) +{ + AVFrame *frame_ref; + AMFBuffer *frame_ref_storage_buffer = NULL; + AMF_RESULT res; + + res = context->pVtbl->AllocBuffer(context, AMF_MEMORY_HOST, sizeof(frame_ref), &frame_ref_storage_buffer); + if (res == AMF_OK) { + frame_ref = av_frame_clone(frame); + if (frame_ref) { + memcpy(frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), &frame_ref, sizeof(frame_ref)); + } else { + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); + frame_ref_storage_buffer = NULL; + } + } + return frame_ref_storage_buffer; +} + +static void amf_release_buffer_with_frame_ref(AMFBuffer *frame_ref_storage_buffer) +{ + AVFrame *av_frame_ref; + memcpy(&av_frame_ref, frame_ref_storage_buffer->pVtbl->GetNative(frame_ref_storage_buffer), sizeof(av_frame_ref)); + av_frame_free(&av_frame_ref); + frame_ref_storage_buffer->pVtbl->Release(frame_ref_storage_buffer); +} int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) { @@ -484,6 +551,7 @@ int ff_amf_send_frame(AVCodecContext *avctx, const AVFrame *frame) (ctx->hw_device_ctx && ((AVHWFramesContext*)frame->hw_frames_ctx->data)->device_ctx == (AVHWDeviceContext*)ctx->hw_device_ctx->data) )) { + AMFBuffer *frame_ref_storage_buffer; #if CONFIG_D3D11VA static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } };