From patchwork Fri Jan 26 19:18:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 7423 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.156.27 with SMTP id q27csp1057524jak; Fri, 26 Jan 2018 11:18:49 -0800 (PST) X-Google-Smtp-Source: AH8x227m7tGPgUHOKyReddmR6c9E1JYbyapgpJn7Ed9SwbtGaeqz9bWSOUV7ANKuvnAO7vdpnfPH X-Received: by 10.28.225.133 with SMTP id y127mr10824606wmg.55.1516994329631; Fri, 26 Jan 2018 11:18:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516994329; cv=none; d=google.com; s=arc-20160816; b=scwN3wjSK/M2yiByjA+WKSbJHled89y8e3O3pVSR9RaA8NCahyF16he1DMWmWao5GU IFxhOgUlVokTZ2D6I8VZKT2Pu0EWJQrXp+X18LURxDB9/Qzmt3HKkqyNJYDMaSNn4csA BJaVFsc+xdlbQSLBUeIGzJousYAyGWiLoL4Rb62izlqo7f7VbzplSrqjfjuHBVBwekg5 BX9QrGTqv1MAuJc9UU0YaxOBVC+b8beFh/Yl+9Mhtuic7Eht2SscW6mAvI5LnX+3XaWH K3Y4H4L+FbyuyJpacRbo7ndQZVlrJNgaNEqp2Zpe0ygZjPVlSR7kxNbG+Lgaoj1j1/yH h65Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=wksAeJ8lFEaQufcp+9fpLbhGGnhgY8HMzCtr6z7BIkU=; b=RbsXpTFKBcaKHmTrbyOCYg6INtSQ8dAbrXhi2/7O8TIozX3kxvtyeOa84xASrxLLH9 Gmq63PMdCHuAkZ3qWe4rZkoMva2C/8KpNd8DQqtovvgrPTV7xpN8u5R06epheT5Vm+45 4f42Qgg3qLtgHHMJ4W9s8XmpQQSDjiSii21yXTXNYzNa7piIUBE3V6apHDzEdKUc+Mh/ axWyTHT6QwvOQTWDHtz3NiiEymnqDyieq7AmU0DiSzJPqFNLPuQr4NPxo4pk2JQ7VBXF Q6rvqUibWCRPqEquv6rv2oPvmuhzHX+MN0uDQTpdKfZrzmslWjcZF1Qr2qU5Lm535vjX 1l1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=W1EHHQ6T; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v13si5798889wrg.448.2018.01.26.11.18.48; Fri, 26 Jan 2018 11:18:49 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=W1EHHQ6T; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DAA4C6808B4; Fri, 26 Jan 2018 21:18:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [5.9.118.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A05C96806C5 for ; Fri, 26 Jan 2018 21:18:37 +0200 (EET) Received: from localhost.localdomain (unknown [IPv6:2a02:8109:1600:48e0:ba97:5aff:fe10:ec69]) by btbn.de (Postfix) with ESMTPSA id 9AE9E22675; Fri, 26 Jan 2018 20:18:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rothenpieler.org; s=mail; t=1516994320; bh=7L5PA+viwM7cL1baI3bLXAv2PSazBYvyhCj6aHOUtKQ=; h=From:To:Cc:Subject:Date; b=W1EHHQ6Tz6I/gRxKyTJJMN0+HOIW36y5wrzkkxkUH0TRfwu3SwsnYE4SSVLaWcYKw uAolCYuiawWTR4p42H52lB5KVWNSp1A+xWLEh5VKNfdDBU3XTlna3Nyr0ZLh+38BJ+ j7xbM6XR6SnGg6PPP0efTCNn1DIcGcF8JmCC8sCY= From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Fri, 26 Jan 2018 20:18:30 +0100 Message-Id: <20180126191830.13533-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.15.1 Subject: [FFmpeg-devel] [PATCH] avcodec/nvenc: refcount input frame mappings X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" If some logic like vsync in ffmpeg.c duplicates frames, it might pass the same frame twice, which will result in a crash due it being effectively mapped and unmapped twice. Signed-off-by: Timo Rothenpieler --- libavcodec/nvenc.c | 39 +++++++++++++++++++++++---------------- libavcodec/nvenc.h | 2 +- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index 4a91d99720..0ecaa15162 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1389,12 +1389,9 @@ av_cold int ff_nvenc_encode_close(AVCodecContext *avctx) av_fifo_freep(&ctx->unused_surface_queue); if (ctx->surfaces && (avctx->pix_fmt == AV_PIX_FMT_CUDA || avctx->pix_fmt == AV_PIX_FMT_D3D11)) { - for (i = 0; i < ctx->nb_surfaces; ++i) { - if (ctx->surfaces[i].input_surface) { - p_nvenc->nvEncUnmapInputResource(ctx->nvencoder, ctx->surfaces[i].in_map.mappedResource); - } - } for (i = 0; i < ctx->nb_registered_frames; i++) { + if (ctx->registered_frames[i].mapped) + p_nvenc->nvEncUnmapInputResource(ctx->nvencoder, ctx->registered_frames[i].in_map.mappedResource); if (ctx->registered_frames[i].regptr) p_nvenc->nvEncUnregisterResource(ctx->nvencoder, ctx->registered_frames[i].regptr); } @@ -1629,19 +1626,23 @@ static int nvenc_upload_frame(AVCodecContext *avctx, const AVFrame *frame, if (res < 0) return res; - nvenc_frame->in_map.version = NV_ENC_MAP_INPUT_RESOURCE_VER; - nvenc_frame->in_map.registeredResource = ctx->registered_frames[reg_idx].regptr; - nv_status = p_nvenc->nvEncMapInputResource(ctx->nvencoder, &nvenc_frame->in_map); - if (nv_status != NV_ENC_SUCCESS) { - av_frame_unref(nvenc_frame->in_ref); - return nvenc_print_error(avctx, nv_status, "Error mapping an input resource"); + if (!ctx->registered_frames[reg_idx].mapped) { + ctx->registered_frames[reg_idx].in_map.version = NV_ENC_MAP_INPUT_RESOURCE_VER; + ctx->registered_frames[reg_idx].in_map.registeredResource = ctx->registered_frames[reg_idx].regptr; + nv_status = p_nvenc->nvEncMapInputResource(ctx->nvencoder, &ctx->registered_frames[reg_idx].in_map); + if (nv_status != NV_ENC_SUCCESS) { + av_frame_unref(nvenc_frame->in_ref); + return nvenc_print_error(avctx, nv_status, "Error mapping an input resource"); + } } - ctx->registered_frames[reg_idx].mapped = 1; + ctx->registered_frames[reg_idx].mapped += 1; + nvenc_frame->reg_idx = reg_idx; - nvenc_frame->input_surface = nvenc_frame->in_map.mappedResource; - nvenc_frame->format = nvenc_frame->in_map.mappedBufferFmt; + nvenc_frame->input_surface = ctx->registered_frames[reg_idx].in_map.mappedResource; + nvenc_frame->format = ctx->registered_frames[reg_idx].in_map.mappedBufferFmt; nvenc_frame->pitch = frame->linesize[0]; + return 0; } else { NV_ENC_LOCK_INPUT_BUFFER lockBufferParams = { 0 }; @@ -1793,9 +1794,15 @@ static int process_output_surface(AVCodecContext *avctx, AVPacket *pkt, NvencSur if (avctx->pix_fmt == AV_PIX_FMT_CUDA || avctx->pix_fmt == AV_PIX_FMT_D3D11) { - p_nvenc->nvEncUnmapInputResource(ctx->nvencoder, tmpoutsurf->in_map.mappedResource); + ctx->registered_frames[tmpoutsurf->reg_idx].mapped -= 1; + if (ctx->registered_frames[tmpoutsurf->reg_idx].mapped == 0) { + p_nvenc->nvEncUnmapInputResource(ctx->nvencoder, ctx->registered_frames[tmpoutsurf->reg_idx].in_map.mappedResource); + } else if (ctx->registered_frames[tmpoutsurf->reg_idx].mapped < 0) { + res = AVERROR_BUG; + goto error; + } + av_frame_unref(tmpoutsurf->in_ref); - ctx->registered_frames[tmpoutsurf->reg_idx].mapped = 0; tmpoutsurf->input_surface = NULL; } diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h index 2e51f1e946..ab6825f633 100644 --- a/libavcodec/nvenc.h +++ b/libavcodec/nvenc.h @@ -44,7 +44,6 @@ typedef struct NvencSurface { NV_ENC_INPUT_PTR input_surface; AVFrame *in_ref; - NV_ENC_MAP_INPUT_RESOURCE in_map; int reg_idx; int width; int height; @@ -131,6 +130,7 @@ typedef struct NvencContext int ptr_index; NV_ENC_REGISTERED_PTR regptr; int mapped; + NV_ENC_MAP_INPUT_RESOURCE in_map; } registered_frames[MAX_REGISTERED_FRAMES]; int nb_registered_frames;