diff mbox series

[FFmpeg-devel,v2] avcodec/cuvidddec: Guess pixel format based on probed bit depth and chroma sampling

Message ID CAJZRZVkuSRo=bBMprdUhzc=3oCPpn84HMSVmNhzPTrZ5JDKNpg@mail.gmail.com
State New
Headers show
Series [FFmpeg-devel,v2] avcodec/cuvidddec: Guess pixel format based on probed bit depth and chroma sampling | expand

Checks

Context Check Description
yinshiyou/commit_msg_loongarch64 warning The first line of the commit message must start with a context terminated by a colon and a space, for example "lavu/opt: " or "doc: ".
andriy/commit_msg_x86 warning The first line of the commit message must start with a context terminated by a colon and a space, for example "lavu/opt: " or "doc: ".
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished

Commit Message

Roman Arzumanyan Aug. 2, 2024, 1:31 p.m. UTC
Hello world,

Please find v2 of the previous patch attached.
Both bit depth and chroma sampling (420 and 444 as supported by Nvdec) are
now taken into account when selecting sw_pix_fmt.

Comments

Timo Rothenpieler Aug. 2, 2024, 1:57 p.m. UTC | #1
On 02/08/2024 15:31, Roman Arzumanyan wrote:
> Hello world,
> 
> Please find v2 of the previous patch attached.
> Both bit depth and chroma sampling (420 and 444 as supported by Nvdec) are
> now taken into account when selecting sw_pix_fmt.

It's actually AV_PIX_FMT_YUV444P16 for both 10 and 12 bit, due to how 
the bits are aligned opposite to what ffmpeg expects in 
AV_PIX_FMT_YUV444P10.

No need to resend for that, will ammend locally.
Roman Arzumanyan Aug. 2, 2024, 2:05 p.m. UTC | #2
Thanks a lot Timo !

Just for my own education: in my application I follow the usual sequence of
API calls to init hw device context and open the codec. What's the right
place to probe the input? I was under the impression that it's done within
avformat_find_stream_info().

пт, 2 авг. 2024 г. в 16:57, Timo Rothenpieler <timo@rothenpieler.org>:

> On 02/08/2024 15:31, Roman Arzumanyan wrote:
> > Hello world,
> >
> > Please find v2 of the previous patch attached.
> > Both bit depth and chroma sampling (420 and 444 as supported by Nvdec)
> are
> > now taken into account when selecting sw_pix_fmt.
>
> It's actually AV_PIX_FMT_YUV444P16 for both 10 and 12 bit, due to how
> the bits are aligned opposite to what ffmpeg expects in
> AV_PIX_FMT_YUV444P10.
>
> No need to resend for that, will ammend locally.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
Timo Rothenpieler Aug. 2, 2024, 3:32 p.m. UTC | #3
On 02.08.2024 16:05, Roman Arzumanyan wrote:
> Thanks a lot Timo !
> 
> Just for my own education: in my application I follow the usual sequence of
> API calls to init hw device context and open the codec. What's the right
> place to probe the input? I was under the impression that it's done within
> avformat_find_stream_info().

ffmpeg.c calls avcodec_parameters_to_context() on the codecpars got from 
the demuxer/probing phase.
I'm not actually sure where the probing itself happens.
diff mbox series

Patch

From a0c0c8497e75987ae771a466c9f0fce5c3ef106c Mon Sep 17 00:00:00 2001
From: Roman Arzumanyan <r.arzumanyan@visionlabs.ai>
Date: Thu, 1 Aug 2024 16:35:22 +0300
Subject: [PATCH] Guess pixel format based on bit depth and chroma sampling:

---
 libavcodec/cuviddec.c | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
index f88ad75e88..a7185fecbf 100644
--- a/libavcodec/cuviddec.c
+++ b/libavcodec/cuviddec.c
@@ -834,22 +834,41 @@  static av_cold int cuvid_decode_init(AVCodecContext *avctx)
     int ret = 0;
 
     enum AVPixelFormat pix_fmts[3] = { AV_PIX_FMT_CUDA,
-                                       AV_PIX_FMT_NV12,
+                                       AV_PIX_FMT_NONE,
                                        AV_PIX_FMT_NONE };
 
     int probed_width = avctx->coded_width ? avctx->coded_width : 1280;
     int probed_height = avctx->coded_height ? avctx->coded_height : 720;
-    int probed_bit_depth = 8;
+    int probed_bit_depth = 8, is_yuv444 = 0;
 
     const AVPixFmtDescriptor *probe_desc = av_pix_fmt_desc_get(avctx->pix_fmt);
     if (probe_desc && probe_desc->nb_components)
         probed_bit_depth = probe_desc->comp[0].depth;
 
+    if (probe_desc && !probe_desc->log2_chroma_w && !probe_desc->log2_chroma_h)
+        is_yuv444 = 1;
+
+    // Pick pixel format based on bit depth and chroma sampling.
+    // Only 420 and 444 sampling are supported by HW so far, no need to check for 422.
+    switch (probed_bit_depth) {
+    case 8:
+        pix_fmts[1] = is_yuv444 ? AV_PIX_FMT_YUV444P : AV_PIX_FMT_NV12;
+        break;        
+    case 10:
+        pix_fmts[1] = is_yuv444 ? AV_PIX_FMT_YUV444P10 : AV_PIX_FMT_P010;
+        break;
+    case 12:
+        pix_fmts[1] = is_yuv444 ? AV_PIX_FMT_YUV444P16 : AV_PIX_FMT_P016;
+        break;
+    default:
+        break;
+    }
+
     ctx->pkt = avctx->internal->in_pkt;
     // Accelerated transcoding scenarios with 'ffmpeg' require that the
     // pix_fmt be set to AV_PIX_FMT_CUDA early. The sw_pix_fmt, and the
     // pix_fmt for non-accelerated transcoding, do not need to be correct
-    // but need to be set to something. We arbitrarily pick NV12.
+    // but need to be set to something.
     ret = ff_get_format(avctx, pix_fmts);
     if (ret < 0) {
         av_log(avctx, AV_LOG_ERROR, "ff_get_format failed: %d\n", ret);
-- 
2.34.1