From patchwork Mon Dec 5 22:25:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 39621 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp3614352pzb; Mon, 5 Dec 2022 14:26:24 -0800 (PST) X-Google-Smtp-Source: AA0mqf79f+ADXlUz9rCLWKcKuuxe8jO6q5j5oGFdZPMYTunwbVYBCUx0fsgqwH9VE+eBoO0nBGT7 X-Received: by 2002:a17:906:9709:b0:7c0:cb51:887c with SMTP id k9-20020a170906970900b007c0cb51887cmr12873469ejx.620.1670279183858; Mon, 05 Dec 2022 14:26:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670279183; cv=none; d=google.com; s=arc-20160816; b=oKedBXDTCceWr0AYpEbhUvYrXLFzzfFj5iwEAMAx1AQW1kwzh0Ik7BYEeLyDfr6TlA thryXqt1wBu4Np6dU2DopQ0eLbnIESI126NVZ+JJ1nVwlnztNjX03/GkUrYHe8Ho9IyD n44sbKc5g0BJNkL7s75yHyEom9hmVaHVwqmxDj7S1VQQKMXZhS+Ea7JmloKHLm/l7se3 h4u/6DEMhnYhW4qSUo8JZ4OSuXRNwF8UmJQdQ1uJbRbd+uiXnfyq22zxZ7tvYWq2gsKA MdrvR5HOIbwE0eKHayjoqRMG+i2Z60fYSXYZIlxqAvpJciDBA4LbjwAmfT62/ubjpMMG 9fpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=zOZ3kLLScdYf2bMVQqA1JjkNS/DbmIMxPIcEXJmmGu8=; b=VFtRbJfC6U3TpMTVEZg5pwSlJiwZpTWFPGhJ2oMvGhGLDlTDheb0vcHGVK+ac1aRQn nnyWyvVmsle94WIz9/FLYbA1PvETEi7kuh1k8x954oesbyfblfd9G6W+yt4/lDNWwdNJ O5hboJq9Gaib3quYs2XuVZqL+sG8nj1gU+F4I1E2QonQR6w4gGRkrWBjWSJQr62CiGg+ 7NmRFnRq+6z4OB+rhbTNbZFJ41F+xBYt+XfO+Fy4UI/igKOtG7RsB5xpKTIay8rYqd6z RRh2lLKpf24gMSJL4YjeRW1xMdbyvKxGOPZYheGqutzuYLpr3m7XpsPXiPxy7YGg6Mvc /ABQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=iwDRYggk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e20-20020a17090658d400b0078db594808dsi13845166ejs.588.2022.12.05.14.26.23; Mon, 05 Dec 2022 14:26:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=iwDRYggk; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A550F68BD00; Tue, 6 Dec 2022 00:26:20 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 34E3468BC45 for ; Tue, 6 Dec 2022 00:26:14 +0200 (EET) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id E26713632EC; Mon, 5 Dec 2022 23:26:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1670279173; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=pATlMYHR0YchVdgLhMJP4JNdFI/ZjTLyjxS6W/hXSe8=; b=iwDRYggk7Dn4eQEw2MOpJZh9wYp+TuKd+vr0dSCu6qPehb6y9jDnUQpm4/yuutnymt7mX/ oerHZcCsK4P2UxeG+5K+E14zbgObIJ8KMl+CjZi0cr3Y2Z9L6lZii0+RHAEBxZ9fRFSUYu jgVRM2x+ExcDOkOp0Td48fetUQ/6N5/3hmrowMme07qHI4mpNy4HCoxA1oAbM8HIbVABRK h+Cld8DkGRXqaikDYxx1CGx9ygtCtGPyglD4hZ4C16smAtSae04abNs7OJ3MgTpA8aTSwb NBrx0UPRV0rftWLzf//TZcUkGlXDNRN+a81d0pgXFgdrhrPzaCIpaFxVFqv++w== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Dec 2022 23:25:58 +0100 Message-Id: <20221205222559.107006-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc: add new minimize_copies hwaccel_flag X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NSEoUp7V6G/9 --- doc/APIchanges | 3 +++ libavcodec/avcodec.h | 11 +++++++++++ libavcodec/options_table.h | 1 + libavcodec/version.h | 2 +- 4 files changed, 16 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index ab7ce15fae..9c1f905a61 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,9 @@ libavutil: 2021-04-27 API changes, most recent first: +2022-12-xx - xxxxxxxxxx - lavc 59.54.101 - avcodec.h + Add AV_HWACCEL_FLAG_MINIMIZE_COPIES. + 2022-11-xx - xxxxxxxxxx - lavu 57.43.100 - tx.h Add AV_TX_FLOAT_DCT, AV_TX_DOUBLE_DCT and AV_TX_INT32_DCT. diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 3edd8e2636..a72551bb5e 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -2253,6 +2253,17 @@ typedef struct AVHWAccel { */ #define AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH (1 << 2) +/** + * Hardware acceleration can have a limited number of direct output surfaces. + * For some processing chains, this can be okay, but others will run into the + * limit and in turn produce very confusing errors. + * + * Thus, the hwaccel will by default make a safety copy. If a users really + * wants to minimize the amount of copies, they can set this flag and ensure + * their chain does not exhaust the surface pool. + */ +#define AV_HWACCEL_FLAG_MINIMIZE_COPIES (1 << 3) + /** * @} */ diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index cd02f5096f..05e8948bc2 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -399,6 +399,7 @@ static const AVOption avcodec_options[] = { {"ignore_level", "ignore level even if the codec level used is unknown or higher than the maximum supported level reported by the hardware driver", 0, AV_OPT_TYPE_CONST, { .i64 = AV_HWACCEL_FLAG_IGNORE_LEVEL }, INT_MIN, INT_MAX, V | D, "hwaccel_flags" }, {"allow_high_depth", "allow to output YUV pixel formats with a different chroma sampling than 4:2:0 and/or other than 8 bits per component", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_ALLOW_HIGH_DEPTH }, INT_MIN, INT_MAX, V | D, "hwaccel_flags"}, {"allow_profile_mismatch", "attempt to decode anyway if HW accelerated decoder's supported profiles do not exactly match the stream", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH }, INT_MIN, INT_MAX, V | D, "hwaccel_flags"}, +{"minimize_copies", "minimize number of frame copies at cost of risking surface pool exhaustion", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_MINIMIZE_COPIES }, INT_MIN, INT_MAX, V | D, "hwaccel_flags"}, {"extra_hw_frames", "Number of extra hardware frames to allocate for the user", OFFSET(extra_hw_frames), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, V|D }, {"discard_damaged_percentage", "Percentage of damaged samples to discard a frame", OFFSET(discard_damaged_percentage), AV_OPT_TYPE_INT, {.i64 = 95 }, 0, 100, V|D }, {NULL}, diff --git a/libavcodec/version.h b/libavcodec/version.h index d149bc6c46..9e66920593 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 54 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ From patchwork Mon Dec 5 22:25:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 39622 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp3614414pzb; Mon, 5 Dec 2022 14:26:32 -0800 (PST) X-Google-Smtp-Source: AA0mqf5sCWxsC3a81LTAuUD5/JXv0yQmC49iHjnR5Iyl+NE8Ul+ytPcBfu/rQdHUJPvNR7AJMrpQ X-Received: by 2002:a05:6402:1381:b0:468:5b78:6381 with SMTP id b1-20020a056402138100b004685b786381mr63781166edv.373.1670279191888; Mon, 05 Dec 2022 14:26:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670279191; cv=none; d=google.com; s=arc-20160816; b=QJSyM7rktGRDyNJ0RmwZ293Wmr+DIW+1a4bIUWGt2ezlFzJaQjU3Rrm7bYQM8N/o+H JHzV6S93gLZAFQlEXUJ9nOvpJfLJe2kZLxUiYsQ3NhOCw7Pn5s6Mj2vCXxSep3KDw8UC Gz2Sy0DXh5chmJcbd3wwbih6+rCqa7Sdauh+iFqb102x/5F7uixCdafd3MshMp23er8q uAqdGsOc6oxLk7yTkAG2ATzu4I+5ycTB8jeIbEH/NbmfaWk4mZya1w0LAa4xrJyvyN/r S+I2W0KHqxdG1QJY5A5KitRuf3fZY2X2ylPFH4Nrw6hLVbXF6WufA/fmKZmIrLHwOgN9 HlRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=ooecOgN4sABJCpy7J66908XgtV2MtHjV/PZiGF+0k8o=; b=HM2PkTLhCztJemZ/UVZqLUwVi3Tn81UGxSRbkhJ0U44i1P2KsHaHm3DXrncvsGJWug JW/DvNScok5GSUGfbGUMl+HxBr07vJHsrcm0hdfktvCGms+7oZMjUz/uxwcV/TuI2k1Z 311JivqEHiQIcRTPpZziAnxiI/0rZ8AaU9KBZXTUv8XaAS+Ucn0Up0C7DsfBpLzrCshg spm/TKplo3ft4KNR2RSOL0TK1JJ7jijWvt5p24PNKEEWpqG7Cw8zVQrPms6aB9UbaGk9 e/zzPQBgsY92pTCR4ciL7le0vn0AOK3zaUuHwlZFHYSVFDIoM1SDNxAcj40I7Wuyv9x1 71Hg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=aZF0FK0V; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t18-20020a056402525200b0046194b2dd53si639614edd.119.2022.12.05.14.26.31; Mon, 05 Dec 2022 14:26:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=aZF0FK0V; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9AD8568BD08; Tue, 6 Dec 2022 00:26:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32C3B68B1B3 for ; Tue, 6 Dec 2022 00:26:14 +0200 (EET) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 8528A3632ED; Mon, 5 Dec 2022 23:26:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1670279173; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fI2lYwCdSA/3SsQvFFOVCXzq0Wjv4qkfGy+vFzf+xOI=; b=aZF0FK0VjJzYGASVG9swwBbXEFNbU31bMfgs53tEP7eGQ4mdZFWvq/UoJvKed8qEBQFC4p HgXuQPc/CTugx3dH5PFriBHYFSbi9paoPg56O1H/xcjvJ346eTjdu1eBi/NNW4c9atUqKi Q0pvYz/gzVRdX5zIVTbGNk7Hi2bIjkwqLKReBy7sGve4JnA5BAgONFTIRMgY5rxpnzjzZx V1nW0v6Pqpd8vdpBGesqJIGTvCBxjam7mJzd65qrvsBSFjeAUclxuUBpuaY5fs7BseNkr2 9kOfhEQ4sCHQHEr9yj3FsGErF+NQGZ+n2KVdXNHTCks1zkQwxOKTic1Zg/jYVQ== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Dec 2022 23:25:59 +0100 Message-Id: <20221205222559.107006-2-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221205222559.107006-1-timo@rothenpieler.org> References: <20221205222559.107006-1-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avcodec/nvdec: make explicit copy of frames unless user requested otherwise X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wbArsxiL2eVi --- libavcodec/nvdec.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c index fbaedf0b6b..76ee395734 100644 --- a/libavcodec/nvdec.c +++ b/libavcodec/nvdec.c @@ -51,6 +51,8 @@ typedef struct NVDECDecoder { CudaFunctions *cudl; CuvidFunctions *cvdl; + + int minimize_copies; } NVDECDecoder; typedef struct NVDECFramePool { @@ -344,6 +346,8 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) int cuvid_codec_type, cuvid_chroma_format, chroma_444; int ret = 0; + int minimize_copies = !!(avctx->hwaccel_flags & AV_HWACCEL_FLAG_MINIMIZE_COPIES); + sw_desc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!sw_desc) return AVERROR_BUG; @@ -402,7 +406,7 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) params.CodecType = cuvid_codec_type; params.ChromaFormat = cuvid_chroma_format; params.ulNumDecodeSurfaces = frames_ctx->initial_pool_size; - params.ulNumOutputSurfaces = frames_ctx->initial_pool_size; + params.ulNumOutputSurfaces = minimize_copies ? frames_ctx->initial_pool_size : 1; ret = nvdec_decoder_create(&ctx->decoder_ref, frames_ctx->device_ref, ¶ms, avctx); if (ret < 0) { @@ -417,6 +421,7 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) } decoder = (NVDECDecoder*)ctx->decoder_ref->data; + decoder->minimize_copies = minimize_copies; decoder->real_hw_frames_ref = real_hw_frames_ref; real_hw_frames_ref = NULL; @@ -554,7 +559,11 @@ copy_fail: finish: CHECK_CU(decoder->cudl->cuCtxPopCurrent(&dummy)); - return ret; + + if (ret < 0 || decoder->minimize_copies) + return ret; + + return av_frame_make_writable(frame); } int ff_nvdec_start_frame(AVCodecContext *avctx, AVFrame *frame)