From patchwork Mon Jan 2 21:26:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Koshevoy X-Patchwork-Id: 2026 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp4112939vsb; Mon, 2 Jan 2017 13:27:42 -0800 (PST) X-Received: by 10.194.81.233 with SMTP id d9mr62837066wjy.118.1483392462120; Mon, 02 Jan 2017 13:27:42 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t10si70984564wmb.123.2017.01.02.13.27.41; Mon, 02 Jan 2017 13:27:42 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6B9CB689D6B; Mon, 2 Jan 2017 23:27:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f65.google.com (mail-pg0-f65.google.com [74.125.83.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 04C3A689DDD for ; Mon, 2 Jan 2017 23:27:29 +0200 (EET) Received: by mail-pg0-f65.google.com with SMTP id n5so31412255pgh.3 for ; Mon, 02 Jan 2017 13:27:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/Kx2/otbRJM2w3K0K1epOsJKsZmpcw7eM0Eg+e7FEVY=; b=t6xV3T+m3tALr1W5mAxCncuhv3W6KJdP0kQIDEvymVJ0UXTdD3EE1j7/pBmIFELIIm DlfE3z2tzCyxJS8s6+d9Rszfwf/GZEkGYgiebWS7ZKUsWZqRlyrWV8N+DeV3iGDsPx3b O0D5yuhA8xwL+xREbwN3WqKCbQPUZE09JBqqi6cS8L42mllvJ/Omwab9W9sQ0FuxjMPs BD4y8sUyoI2sgoLB6NBwJkrQnphxRiT5d9SHYxLtTszjkJfp99fOYcDI8vYA+ocrjW7v a+Gg9SiDmdQ/Qj8rOK21psPfNp2N21LYObBH3KaF0WbBFGdCODATIfO4IXxR69FaD3/7 hICw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/Kx2/otbRJM2w3K0K1epOsJKsZmpcw7eM0Eg+e7FEVY=; b=eMpWDLtNSdMVWz/WwzTT1TQOqt5C4+ahx8Bn7gVOVqm0h3ulUsg2PTOllKD08aSXys LFSDqQg9q0t6EsP7RW+Cy0uBUosF86ij2nlHp3rkh1wSythWXTV+3Zo+aUIU5ciPVSvN 1FFxcIMbwkIycmQGz+doUFvqa15lL/bNtpItINWDngD9yTnTt1C9HelatLkwLRsQd+r1 Fmx9CmBBoeu2ZEvq9t2j5ka3IVMnJK6DHQfW9YTF1BecxBek9G+ZXmbFFp0o15ZaTU+z E+JIeTR1a7PHdu+i7S/Y+Xg/2Q9UEQsGCcMqvZgCxEFEHP02VdG1TI2g1yu8MFlBGOw7 w7hQ== X-Gm-Message-State: AIkVDXJ5AlgQRLNdFepcs3466gXFHwzIQU8tUgJ734fsh61qO0CmC4PUEJzZA4pEwucAPw== X-Received: by 10.98.131.67 with SMTP id h64mr54925316pfe.86.1483392451839; Mon, 02 Jan 2017 13:27:31 -0800 (PST) Received: from homestead.aragog.com ([136.60.170.51]) by smtp.gmail.com with ESMTPSA id d1sm134391511pfb.76.2017.01.02.13.27.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 02 Jan 2017 13:27:31 -0800 (PST) From: pkoshevoy@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Mon, 2 Jan 2017 14:26:45 -0700 Message-Id: <1483392405-9969-1-git-send-email-pkoshevoy@gmail.com> X-Mailer: git-send-email 2.6.6 In-Reply-To: References: Subject: [FFmpeg-devel] [PATCH] lavc/cuvid: fail early if GPU can't handle given video parameters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Pavel Koshevoy MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Pavel Koshevoy Evidently CUVID doesn't support decoding 422 or 444 chroma formats, and only a limited set of resolutions per codec are supported. Given that stream resolution and pixel format are typically known as a result of probing it is better to use this info during avcodec_open2 call and fail early in case the video parameters are not supported, rather than failing later during avcodec_send_packet calls. This problem surfaced when trying to decode 5120x2700 h246 video on Geforce GT 730, or when decoding 422 mpeg2 stream on same GPU -- avcodec_open2 succeeded but decoding failed. --- libavcodec/cuvid.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 53 insertions(+), 5 deletions(-) diff --git a/libavcodec/cuvid.c b/libavcodec/cuvid.c index 8fc713d..febdd71 100644 --- a/libavcodec/cuvid.c +++ b/libavcodec/cuvid.c @@ -612,7 +612,11 @@ static av_cold int cuvid_decode_end(AVCodecContext *avctx) return 0; } -static int cuvid_test_dummy_decoder(AVCodecContext *avctx, CUVIDPARSERPARAMS *cuparseinfo) +static int cuvid_test_dummy_decoder(AVCodecContext *avctx, + const CUVIDPARSERPARAMS *cuparseinfo, + cudaVideoChromaFormat probed_chroma_format, + int probed_width, + int probed_height) { CuvidContext *ctx = avctx->priv_data; CUVIDDECODECREATEINFO cuinfo; @@ -622,11 +626,11 @@ static int cuvid_test_dummy_decoder(AVCodecContext *avctx, CUVIDPARSERPARAMS *cu memset(&cuinfo, 0, sizeof(cuinfo)); cuinfo.CodecType = cuparseinfo->CodecType; - cuinfo.ChromaFormat = cudaVideoChromaFormat_420; + cuinfo.ChromaFormat = probed_chroma_format; cuinfo.OutputFormat = cudaVideoSurfaceFormat_NV12; - cuinfo.ulWidth = 1280; - cuinfo.ulHeight = 720; + cuinfo.ulWidth = probed_width; + cuinfo.ulHeight = probed_height; cuinfo.ulTargetWidth = cuinfo.ulWidth; cuinfo.ulTargetHeight = cuinfo.ulHeight; @@ -653,6 +657,36 @@ static int cuvid_test_dummy_decoder(AVCodecContext *avctx, CUVIDPARSERPARAMS *cu return 0; } +static int convert_to_cuda_video_chroma_format(enum AVPixelFormat pix_fmt, + cudaVideoChromaFormat *out) +{ + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt); + if (!(out && desc && + (desc->nb_components == 1 || desc->nb_components == 3) && + (desc->log2_chroma_w < 2 && desc->log2_chroma_h < 2))) + { + return AVERROR(EINVAL); + } + + if (desc->nb_components == 1) + { + *out = cudaVideoChromaFormat_Monochrome; + } + else if (desc->flags == AV_PIX_FMT_FLAG_PLANAR) + { + *out = ((desc->log2_chroma_w == 0) ? cudaVideoChromaFormat_444 : + (desc->log2_chroma_h == 0) ? cudaVideoChromaFormat_422 : + cudaVideoChromaFormat_420); + } + else + { + return AVERROR(EINVAL); + } + + // unfortunately, 420 is the only one that works: + return (*out == cudaVideoChromaFormat_420) ? 0 : AVERROR_EXTERNAL; +} + static av_cold int cuvid_decode_init(AVCodecContext *avctx) { CuvidContext *ctx = avctx->priv_data; @@ -663,12 +697,23 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) CUcontext cuda_ctx = NULL; CUcontext dummy; const AVBitStreamFilter *bsf; + cudaVideoChromaFormat probed_chroma_format; + int probed_width; + int probed_height; int ret = 0; enum AVPixelFormat pix_fmts[3] = { AV_PIX_FMT_CUDA, AV_PIX_FMT_NV12, AV_PIX_FMT_NONE }; + ret = convert_to_cuda_video_chroma_format(avctx->pix_fmt, &probed_chroma_format); + if (ret < 0) { + // pixel format is not supported: + return ret; + } + probed_width = avctx->coded_width ? avctx->coded_width : 1280; + probed_height = avctx->coded_height ? avctx->coded_height : 720; + // Accelerated transcoding scenarios with 'ffmpeg' require that the // pix_fmt be set to AV_PIX_FMT_CUDA early. The sw_pix_fmt, and the // pix_fmt for non-accelerated transcoding, do not need to be correct @@ -824,7 +869,10 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) if (ret < 0) goto error; - ret = cuvid_test_dummy_decoder(avctx, &ctx->cuparseinfo); + ret = cuvid_test_dummy_decoder(avctx, &ctx->cuparseinfo, + probed_chroma_format, + probed_width, + probed_height); if (ret < 0) goto error;