From patchwork Fri Dec 9 14:16:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 39665 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1371844pzb; Fri, 9 Dec 2022 06:16:18 -0800 (PST) X-Google-Smtp-Source: AA0mqf7OPNjMG84Wzlgl+laZ+FAJStM7h5Wjg9E5kyWHbqvPiv7ULbY/m5dotWA6bdU4syjiP79c X-Received: by 2002:a05:6402:43c7:b0:46c:e558:ce60 with SMTP id p7-20020a05640243c700b0046ce558ce60mr5280124edc.22.1670595378147; Fri, 09 Dec 2022 06:16:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670595378; cv=none; d=google.com; s=arc-20160816; b=iThEvGqLcGBczdgq2yZO+tMofKzqzXNUN5nHsKsju12JcJVt4ysuRHOHgtJMDt71th o4ciYPqibTx448/xK/TBIVLe89CCE8Ippstcf+zU6hcrSQc2GErKU/L383jeOgmpFJRc EWm2r5bSYIyhyc1m+WnQ+JWwfVSYCttbQcddfYPOm/Vr29rf8HLfUSujs9rz58ax19ZM c5H554UakIlTt9CFc2DNn0Bo4Ym6kCVeq+pKSRcSIiOH034JgRWtmLK4fUHteY0u7QiM pZ0V5LBDl1KjCA/SEiaEIVMoquz5qJ7vs2eFcjaH+/6B9Sz9+2oNm7VcahliBV44e5iq Y/XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=yCsHHssLoVnkjzlqS3qS/fnrQu9WGWpmSABjaZFLMAw=; b=w8Tf7zDxs6MVLlEcZkVDUAN2Ox6MJZ8IbaW6i++kyzMVBvxlHvgMCYpI/MrSihZxzB X15osJx48fuVroTfaN5qeSKDHB9C9+hOmw1DPN09K1TqHJM0fOxKvZVeNtoatUa00US6 zWvuZltReebZmImcrUuhLpmxTiODGMhPsLryKMnxH35Rl8+Nn53l6eg3T9Bo4An6n/BN Wi3UOrZJX4NMscISeRfS529UWQnlPXlPMG6hodM1T6L2rddTt8AOc8nANZ6NW2qEbj/H KtDqGbXsqxL+oZNc88v2PrGLwMW1OskmYwlCHAqVuzjOawi7GwNOmT/N6Zl6nxj2Ipyc tYqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=VosmyTYF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d12-20020a50ea8c000000b00469af6681b1si1430028edo.183.2022.12.09.06.16.16; Fri, 09 Dec 2022 06:16:18 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=VosmyTYF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EE28168BB65; Fri, 9 Dec 2022 16:16:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EF83968BB65 for ; Fri, 9 Dec 2022 16:16:05 +0200 (EET) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 008033952B7; Fri, 9 Dec 2022 15:16:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1670595365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WSrAhXVKAbc0A3rqWFLRHrRUCXKwi+xzpmxW5qyk9+4=; b=VosmyTYFJfMjP08mCkICQpZbyNjllwBKrUcXRpZ2LGwXZzaCXFc8MuRbHVZR2kFaM1sBNI 1ypftPENSTo72ANV+d1ggsb3XuScs+SbNmvBnbIlzdRPUnJ0mKYuTOxWjyilmt+egrBzth jCbNj5sGT8aSihez8QIEHN5uPNZEEVkAoBoY7z1hkGt6IxywZEHejeIk4ADeDVm0mqgCqp g/hZL5BAvPridkVfDlYV/kvZ+gxZ0iiyLGHwQ2NDB/SwxMyqFyfm6L6crn5bCEPDdIZKkG Msz1B4IlSh1XZIgWIz/OkfM0DSBhX37Hh4QCS5koLxnxEH2XDe29mZRD85wyEA== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Fri, 9 Dec 2022 15:16:16 +0100 Message-Id: <20221209141617.13641-1-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221205222559.107006-1-timo@rothenpieler.org> References: <20221205222559.107006-1-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] lavc: add new unsafe_output hwaccel_flag X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uSwwHRgFNBxS --- doc/APIchanges | 3 +++ libavcodec/avcodec.h | 16 ++++++++++++++++ libavcodec/options_table.h | 1 + libavcodec/version.h | 4 ++-- 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index ab7ce15fae..328028f293 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,9 @@ libavutil: 2021-04-27 API changes, most recent first: +2022-12-xx - xxxxxxxxxx - lavc 59.55.100 - avcodec.h + Add AV_HWACCEL_FLAG_UNSAFE_OUTPUT. + 2022-11-xx - xxxxxxxxxx - lavu 57.43.100 - tx.h Add AV_TX_FLOAT_DCT, AV_TX_DOUBLE_DCT and AV_TX_INT32_DCT. diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 3edd8e2636..0ac581d660 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -2253,6 +2253,22 @@ typedef struct AVHWAccel { */ #define AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH (1 << 2) +/** + * Some hardware decoders (namely nvdec) can either output direct decoder + * surfaces, or make an on-device copy and return said copy. + * There is a hard limit on how many decoder surfaces there can be, and it + * cannot be accurately guessed ahead of time. + * For some processing chains, this can be okay, but others will run into the + * limit and in turn produce very confusing errors that require fine tuning of + * more or less obscure options by the user, or in extreme cases cannot be + * resolved at all without inserting an avfilter that forces a copy. + * + * Thus, the hwaccel will by default make a copy for safety and resilience. + * If a users really wants to minimize the amount of copies, they can set this + * flag and ensure their processing chain does not exhaust the surface pool. + */ +#define AV_HWACCEL_FLAG_UNSAFE_OUTPUT (1 << 3) + /** * @} */ diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index cd02f5096f..7924ca6144 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -399,6 +399,7 @@ static const AVOption avcodec_options[] = { {"ignore_level", "ignore level even if the codec level used is unknown or higher than the maximum supported level reported by the hardware driver", 0, AV_OPT_TYPE_CONST, { .i64 = AV_HWACCEL_FLAG_IGNORE_LEVEL }, INT_MIN, INT_MAX, V | D, "hwaccel_flags" }, {"allow_high_depth", "allow to output YUV pixel formats with a different chroma sampling than 4:2:0 and/or other than 8 bits per component", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_ALLOW_HIGH_DEPTH }, INT_MIN, INT_MAX, V | D, "hwaccel_flags"}, {"allow_profile_mismatch", "attempt to decode anyway if HW accelerated decoder's supported profiles do not exactly match the stream", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH }, INT_MIN, INT_MAX, V | D, "hwaccel_flags"}, +{"unsafe_output", "allow potentially unsafe hwaccel frame output that might require special care to process successfully", 0, AV_OPT_TYPE_CONST, {.i64 = AV_HWACCEL_FLAG_UNSAFE_OUTPUT }, INT_MIN, INT_MAX, V | D, "hwaccel_flags"}, {"extra_hw_frames", "Number of extra hardware frames to allocate for the user", OFFSET(extra_hw_frames), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, V|D }, {"discard_damaged_percentage", "Percentage of damaged samples to discard a frame", OFFSET(discard_damaged_percentage), AV_OPT_TYPE_INT, {.i64 = 95 }, 0, 100, V|D }, {NULL}, diff --git a/libavcodec/version.h b/libavcodec/version.h index 9e66920593..9f42f09f4e 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,8 +29,8 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 54 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MINOR 55 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ From patchwork Fri Dec 9 14:16:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 39666 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1371988pzb; Fri, 9 Dec 2022 06:16:27 -0800 (PST) X-Google-Smtp-Source: AA0mqf7sWNmiiAO7ufbclByQyxEd4OE8Zo3paQkqJNOeG9c/aEU98bQWwPGqOM6mi5biDilDiRj5 X-Received: by 2002:aa7:cd78:0:b0:46c:be9f:7c3b with SMTP id ca24-20020aa7cd78000000b0046cbe9f7c3bmr5389248edb.19.1670595386848; Fri, 09 Dec 2022 06:16:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670595386; cv=none; d=google.com; s=arc-20160816; b=jUrWMF5Xpduu349uSEIRilipEJ7WBXIptDRmkL12y/hAL6Xg0xBu9cJCB004IKRlh1 FX0IEXhF7KvkkTxraU9tP0f0CgJL+JLiZWxnsCJfXzFtzbEbJEx91FQHe/zesDSHiogQ 6BKBEX3qbQXhWqb7bcEvMT8VCIdzSHlqNUMhGSvQ2oU97Ue4Rw/3+YV+faQhE7K6rARz foRqcEi7ge7uE4a2QuZA/Nnstju1Fq1fPsIMjMQE/OgQzlVpow50WZbHiInJYGnyqhiN KW1AJxhU0W643In4lTeeD2U0qDEtKlXTiAuRf0hhCaTj+ogOYAIVWpQ2yXfHkTj6NeZw UfgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=63p2k9ytie2fEzTqDaA5l0c8jVpP8zZVzZwwB3ictkU=; b=GLdnRnT5o762O2b/0lhLQzFb8SSUSUx0McaQXJzYvwvKhGv0B7Yh9EIojvr8iFSmyJ CVq3jd+iiqPWdWes/6rZt9Ix933CSYdfHyJu3pEJSk63ApoDZTa0v/5BRmlRJzlcDyew 07Bklb/Y5EHqADxNWsuGAV/P2+zu1I//iGAaYXubiJX85XcJfqYFv2QsOm/5Bz3x7fii fi0wckwqGF1khu/3R3vt0D0KyR9xWmYNVGkwUMQzRqZUcklMlYDmezkvfxyvhYpMzv3k qjBGqJHKBwrEcqGdMpCEYajGFFgc+Ugltp/AISW8KMlQ2CkRQS65uEYB5BieBVSc8Uhi +cdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=Vl2OOFs9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e7-20020a056402148700b0046b1f708941si1312754edv.556.2022.12.09.06.16.26; Fri, 09 Dec 2022 06:16:26 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=Vl2OOFs9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1375C68BC95; Fri, 9 Dec 2022 16:16:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F26D668BB82 for ; Fri, 9 Dec 2022 16:16:05 +0200 (EET) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 32A743952B8; Fri, 9 Dec 2022 15:16:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1670595365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bhL+XOKF6bZwTLv/M3VQrXibnsR4nZE2sYuiRdZq66I=; b=Vl2OOFs9ujEiPiYX+Ks8WGExeqGjPkak7ue3l5pYDTsdBNT+8M4x0p08fJWaSxNXnIOnMX ygTjHN2WjKHrXkVeyESYLcs81XT86QnimI2R717sxGkFIZoTvLbLSqcUFrMIodE/SgmANf 89nthD3NrEsnXt7v6OeOCupVN84ixc4LDSfhJ1QUZdp6sjzhCg1fpWvM2bAK4+yyG95T+M T3ZyGISliVe5dMCobHL8+WaVpwMxKkUsJBV9Z6Elqrwi4Zy2pxTeW8LJGCEKmBNd4UMSGz YJnH4Y+KCEQDOJxxrGh03jbD/JIuhmC+wsNXGP/Kv3UBQKIMGlerRoG2+QMKXg== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Fri, 9 Dec 2022 15:16:17 +0100 Message-Id: <20221209141617.13641-2-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221209141617.13641-1-timo@rothenpieler.org> References: <20221205222559.107006-1-timo@rothenpieler.org> <20221209141617.13641-1-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/2] avcodec/nvdec: make explicit copy of frames unless user requested otherwise X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QuOHn7HhSjVh --- libavcodec/nvdec.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c index fbaedf0b6b..a477449d14 100644 --- a/libavcodec/nvdec.c +++ b/libavcodec/nvdec.c @@ -51,6 +51,8 @@ typedef struct NVDECDecoder { CudaFunctions *cudl; CuvidFunctions *cvdl; + + int unsafe_output; } NVDECDecoder; typedef struct NVDECFramePool { @@ -344,6 +346,8 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) int cuvid_codec_type, cuvid_chroma_format, chroma_444; int ret = 0; + int unsafe_output = !!(avctx->hwaccel_flags & AV_HWACCEL_FLAG_UNSAFE_OUTPUT); + sw_desc = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!sw_desc) return AVERROR_BUG; @@ -402,7 +406,7 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) params.CodecType = cuvid_codec_type; params.ChromaFormat = cuvid_chroma_format; params.ulNumDecodeSurfaces = frames_ctx->initial_pool_size; - params.ulNumOutputSurfaces = frames_ctx->initial_pool_size; + params.ulNumOutputSurfaces = unsafe_output ? frames_ctx->initial_pool_size : 1; ret = nvdec_decoder_create(&ctx->decoder_ref, frames_ctx->device_ref, ¶ms, avctx); if (ret < 0) { @@ -417,6 +421,7 @@ int ff_nvdec_decode_init(AVCodecContext *avctx) } decoder = (NVDECDecoder*)ctx->decoder_ref->data; + decoder->unsafe_output = unsafe_output; decoder->real_hw_frames_ref = real_hw_frames_ref; real_hw_frames_ref = NULL; @@ -554,7 +559,11 @@ copy_fail: finish: CHECK_CU(decoder->cudl->cuCtxPopCurrent(&dummy)); - return ret; + + if (ret < 0 || decoder->unsafe_output) + return ret; + + return av_frame_make_writable(frame); } int ff_nvdec_start_frame(AVCodecContext *avctx, AVFrame *frame)