[FFmpeg-devel] (for discussion): ffmpeg: prefer cuvid decoders when use option -cuvid

Submitted by Miroslav Slugeň on Feb. 12, 2017, 8:37 p.m.

Details

Message ID 58A0C78E.4070103@email.cz
State New
Headers show

Commit Message

Miroslav Slugeň Feb. 12, 2017, 8:37 p.m.
This patch is for discussion only, not ready to commit yet and maybe 
newer will be.

We were facing issue when using -hwaccel cuvid we have to also specify 
input decoder like -c:v XXXX_cuvid for every input and input video 
format was sometimes mpeg2/h264/hevc. So this is my FIX/HACK to only 
specify -cuvid and ffmpeg will pick cuvid decoder for any supported input.

I don't know correct solution for this yet.

Comments

Mark Thompson Feb. 12, 2017, 9:20 p.m.
On 12/02/17 20:37, Miroslav Slugeň wrote:
> This patch is for discussion only, not ready to commit yet and maybe newer will be.
> 
> We were facing issue when using -hwaccel cuvid we have to also specify input decoder like -c:v XXXX_cuvid for every input and input video format was sometimes mpeg2/h264/hevc. So this is my FIX/HACK to only specify -cuvid and ffmpeg will pick cuvid decoder for any supported input.
> 
> I don't know correct solution for this yet.

Adding global variables to libraries to mess with their internals is not an acceptable solution to anything.

The correct solution to this problem is to write a real cuvid hwaccel, which works within the existing decoder to offer decoding of streams which it supports without changing the behaviour at all in normal software cases (compare the behaviour of cuvid (standalone decoder) with dxva2, vaapi or vdpau (full hwaccels inside the normal decoder)).

An alternative solution for your specific case would be to disable the normal H.264, MPEG-2, etc. decoders in your build, such that the cuvid decoder appears first in the list and would always be picked for any given stream.  (This of course would also remove support for the wider set of streams which the libavcodec decoders support, such as H.264 at higher big depths, though given that your patch here also has that effect I assume you aren't particularly concerned about that case.)

- Mark
wm4 Feb. 13, 2017, 4:08 a.m.
On Sun, 12 Feb 2017 21:20:12 +0000
Mark Thompson <sw@jkqxz.net> wrote:

> On 12/02/17 20:37, Miroslav Slugeň wrote:
> > This patch is for discussion only, not ready to commit yet and maybe newer will be.
> > 
> > We were facing issue when using -hwaccel cuvid we have to also specify input decoder like -c:v XXXX_cuvid for every input and input video format was sometimes mpeg2/h264/hevc. So this is my FIX/HACK to only specify -cuvid and ffmpeg will pick cuvid decoder for any supported input.
> > 
> > I don't know correct solution for this yet.  
> 
> Adding global variables to libraries to mess with their internals is not an acceptable solution to anything.
> 
> The correct solution to this problem is to write a real cuvid hwaccel, which works within the existing decoder to offer decoding of streams which it supports without changing the behaviour at all in normal software cases (compare the behaviour of cuvid (standalone decoder) with dxva2, vaapi or vdpau (full hwaccels inside the normal decoder)).
> 
> An alternative solution for your specific case would be to disable the normal H.264, MPEG-2, etc. decoders in your build, such that the cuvid decoder appears first in the list and would always be picked for any given stream.  (This of course would also remove support for the wider set of streams which the libavcodec decoders support, such as H.264 at higher big depths, though given that your patch here also has that effect I assume you aren't particularly concerned about that case.)

What's the problem with just specifying the correct decoder? Both API
and ffmpeg.c allow doing this.

Although what I don't like is that API users need to do not-so-clean
things to find the right decoder, e.g. relying on the naming
conventions, and assuming you can get the the cuda wrapper by
concatenating the codec and API names e.g. h264 -> "h264_cuda".
Maybe adding decoder and API fields to AVHWAccel would be nice.
Then ffmpeg.c could get an option to select codecs by API too.
Miroslav Slugeň Feb. 13, 2017, 8:08 a.m.
Dne 13.2.2017 v 05:08 wm4 napsal(a):
> On Sun, 12 Feb 2017 21:20:12 +0000
> Mark Thompson <sw@jkqxz.net> wrote:
>
>> On 12/02/17 20:37, Miroslav Slugeň wrote:
>>> This patch is for discussion only, not ready to commit yet and maybe newer will be.
>>>
>>> We were facing issue when using -hwaccel cuvid we have to also specify input decoder like -c:v XXXX_cuvid for every input and input video format was sometimes mpeg2/h264/hevc. So this is my FIX/HACK to only specify -cuvid and ffmpeg will pick cuvid decoder for any supported input.
>>>
>>> I don't know correct solution for this yet.
>> Adding global variables to libraries to mess with their internals is not an acceptable solution to anything.
>>
>> The correct solution to this problem is to write a real cuvid hwaccel, which works within the existing decoder to offer decoding of streams which it supports without changing the behaviour at all in normal software cases (compare the behaviour of cuvid (standalone decoder) with dxva2, vaapi or vdpau (full hwaccels inside the normal decoder)).
>>
>> An alternative solution for your specific case would be to disable the normal H.264, MPEG-2, etc. decoders in your build, such that the cuvid decoder appears first in the list and would always be picked for any given stream.  (This of course would also remove support for the wider set of streams which the libavcodec decoders support, such as H.264 at higher big depths, though given that your patch here also has that effect I assume you aren't particularly concerned about that case.)
> What's the problem with just specifying the correct decoder? Both API
> and ffmpeg.c allow doing this.
>
> Although what I don't like is that API users need to do not-so-clean
> things to find the right decoder, e.g. relying on the naming
> conventions, and assuming you can get the the cuda wrapper by
> concatenating the codec and API names e.g. h264 -> "h264_cuda".
> Maybe adding decoder and API fields to AVHWAccel would be nice.
> Then ffmpeg.c could get an option to select codecs by API too.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Problem is that when you are using for example 200 input streams you 
have to always specify correct input format h264/mpeg2/nvenc. Also when 
you are using -hwaccel cuvid you have to specify it too, otherwise there 
is error: CUVID hwaccel requested, but impossible to achieve.

M.
Hendrik Leppkes Feb. 13, 2017, 8:11 a.m.
On Mon, Feb 13, 2017 at 9:08 AM, Miroslav Slugeň <thunder.m@email.cz> wrote:
> Problem is that when you are using for example 200 input streams you have to
> always specify correct input format h264/mpeg2/nvenc. Also when you are
> using -hwaccel cuvid you have to specify it too, otherwise there is error:
> CUVID hwaccel requested, but impossible to achieve.
>

You can just script that and then its not really that huge of an extra
effort to specify it. The patch is beyond terrible, and I don't see
any good way to implement something like this either without some
thorough API design to be able to select decoders in a smart way..

- Hendrik
Miroslav Slugeň Feb. 13, 2017, 8:26 a.m.
Dne 13.2.2017 v 09:11 Hendrik Leppkes napsal(a):
> On Mon, Feb 13, 2017 at 9:08 AM, Miroslav Slugeň <thunder.m@email.cz> wrote:
>> Problem is that when you are using for example 200 input streams you have to
>> always specify correct input format h264/mpeg2/nvenc. Also when you are
>> using -hwaccel cuvid you have to specify it too, otherwise there is error:
>> CUVID hwaccel requested, but impossible to achieve.
>>
> You can just script that and then its not really that huge of an extra
> effort to specify it. The patch is beyond terrible, and I don't see
> any good way to implement something like this either without some
> thorough API design to be able to select decoders in a smart way..
>
> - Hendrik
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
I know it was 1 hour hack :)

Correct behavior should be:

1. when you use "-hwaccel cuvid" ffmpeg should try cuvid decoders first
2. alias in decoders "-c:v cuvid" and then ffmpeg should try only cuvid 
decoders

This should be same for other hwaccels.

M.

Patch hide | download patch | download mbox

From 08e448c036166b645a0364c0a2a6b2b1dbdd869d Mon Sep 17 00:00:00 2001
From: Miroslav Slugen <thunder.m@email.cz>
Date: Sun, 12 Feb 2017 21:32:45 +0100
Subject: [PATCH 1/1] ffmpeg: prefer cuvid decoders when use option -cuvid

---
 ffmpeg.c               |  8 +++++++
 ffmpeg_opt.c           |  2 ++
 libavcodec/allcodecs.c | 57 ++++++++++++++++++++++++++++++++++++++++++--------
 libavcodec/avcodec.h   |  2 ++
 4 files changed, 60 insertions(+), 9 deletions(-)

diff --git a/ffmpeg.c b/ffmpeg.c
index 06570c0..a47dcd2 100644
--- a/ffmpeg.c
+++ b/ffmpeg.c
@@ -4557,6 +4557,14 @@  int main(int argc, char **argv)
         argv++;
     }
 
+    avcodec_cuvid = 0;
+    for (i = 1; i < argc; i++) {
+        if (!strcmp(argv[i], "-cuvid")) {
+            avcodec_cuvid = 1;
+            break;
+        }
+    }
+
     avcodec_register_all();
 #if CONFIG_AVDEVICE
     avdevice_register_all();
diff --git a/ffmpeg_opt.c b/ffmpeg_opt.c
index 6a47d32..46e440c 100644
--- a/ffmpeg_opt.c
+++ b/ffmpeg_opt.c
@@ -3582,6 +3582,8 @@  const OptionDef options[] = {
         "audio bitrate (please use -b:a)", "bitrate" },
     { "b",            OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT,            { .func_arg = opt_bitrate },
         "video bitrate (please use -b:v)", "bitrate" },
+    { "cuvid",          OPT_BOOL | OPT_EXPERT,                                   { &avcodec_cuvid },
+        "enable cuvid acceleration for input" },
     { "hwaccel",          OPT_VIDEO | OPT_STRING | HAS_ARG | OPT_EXPERT |
                           OPT_SPEC | OPT_INPUT,                                  { .off = OFFSET(hwaccels) },
         "use HW accelerated decoding", "hwaccel name" },
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 3680129..e613728 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -58,6 +58,36 @@ 
             av_register_codec_parser(&ff_##x##_parser);                 \
     }
 
+int avcodec_cuvid = 0;
+
+static void avcodec_register_cuvid_h264(void) {
+    REGISTER_DECODER(H264_CUVID,        h264_cuvid);
+}
+static void avcodec_register_cuvid_hevc(void) {
+    REGISTER_DECODER(HEVC_CUVID,       hevc_cuvid);
+}
+static void avcodec_register_cuvid_mjpeg(void) {
+    REGISTER_DECODER(MJPEG_CUVID,      mjpeg_cuvid);
+}
+static void avcodec_register_cuvid_mpeg1(void) {
+    REGISTER_DECODER(MPEG1_CUVID,      mpeg1_cuvid);
+}
+static void avcodec_register_cuvid_mpeg2(void) {
+    REGISTER_DECODER(MPEG2_CUVID,      mpeg2_cuvid);
+}
+static void avcodec_register_cuvid_mpeg4(void) {
+    REGISTER_DECODER(MPEG4_CUVID,      mpeg4_cuvid);
+}
+static void avcodec_register_cuvid_vc1(void) {
+    REGISTER_DECODER(VC1_CUVID,        vc1_cuvid);
+}
+static void avcodec_register_cuvid_vp8(void) {
+    REGISTER_DECODER(VP8_CUVID,        vp8_cuvid);
+}
+static void avcodec_register_cuvid_vp9(void) {
+    REGISTER_DECODER(VP9_CUVID,        vp9_cuvid);
+}
+
 void avcodec_register_all(void)
 {
     static int initialized;
@@ -202,6 +232,7 @@  void avcodec_register_all(void)
     REGISTER_ENCDEC (H263,              h263);
     REGISTER_DECODER(H263I,             h263i);
     REGISTER_ENCDEC (H263P,             h263p);
+    if (avcodec_cuvid) avcodec_register_cuvid_h264();
     REGISTER_DECODER(H264,              h264);
     REGISTER_DECODER(H264_CRYSTALHD,    h264_crystalhd);
     REGISTER_DECODER(H264_MEDIACODEC,   h264_mediacodec);
@@ -212,6 +243,7 @@  void avcodec_register_all(void)
     REGISTER_DECODER(H264_VDPAU,        h264_vdpau);
 #endif
     REGISTER_ENCDEC (HAP,               hap);
+    if (avcodec_cuvid) avcodec_register_cuvid_hevc();
     REGISTER_DECODER(HEVC,              hevc);
     REGISTER_DECODER(HEVC_QSV,          hevc_qsv);
     REGISTER_DECODER(HNM4_VIDEO,        hnm4_video);
@@ -237,6 +269,7 @@  void avcodec_register_all(void)
     REGISTER_DECODER(MAGICYUV,          magicyuv);
     REGISTER_DECODER(MDEC,              mdec);
     REGISTER_DECODER(MIMIC,             mimic);
+    if (avcodec_cuvid) avcodec_register_cuvid_mjpeg();
     REGISTER_ENCDEC (MJPEG,             mjpeg);
     REGISTER_DECODER(MJPEGB,            mjpegb);
     REGISTER_DECODER(MMVIDEO,           mmvideo);
@@ -244,8 +277,11 @@  void avcodec_register_all(void)
 #if FF_API_XVMC
     REGISTER_DECODER(MPEG_XVMC,         mpeg_xvmc);
 #endif /* FF_API_XVMC */
+    avcodec_register_cuvid_mpeg1();
     REGISTER_ENCDEC (MPEG1VIDEO,        mpeg1video);
+    avcodec_register_cuvid_mpeg2();
     REGISTER_ENCDEC (MPEG2VIDEO,        mpeg2video);
+    avcodec_register_cuvid_mpeg4();
     REGISTER_ENCDEC (MPEG4,             mpeg4);
     REGISTER_DECODER(MPEG4_CRYSTALHD,   mpeg4_crystalhd);
     REGISTER_DECODER(MPEG4_MMAL,        mpeg4_mmal);
@@ -344,6 +380,7 @@  void avcodec_register_all(void)
     REGISTER_ENCDEC (V410,              v410);
     REGISTER_DECODER(VB,                vb);
     REGISTER_DECODER(VBLE,              vble);
+    avcodec_register_cuvid_vc1();
     REGISTER_DECODER(VC1,               vc1);
     REGISTER_DECODER(VC1_CRYSTALHD,     vc1_crystalhd);
 #if FF_API_VDPAU
@@ -362,7 +399,9 @@  void avcodec_register_all(void)
     REGISTER_DECODER(VP6A,              vp6a);
     REGISTER_DECODER(VP6F,              vp6f);
     REGISTER_DECODER(VP7,               vp7);
+    if (avcodec_cuvid) avcodec_register_cuvid_vp8();
     REGISTER_DECODER(VP8,               vp8);
+    if (avcodec_cuvid) avcodec_register_cuvid_vp9();
     REGISTER_DECODER(VP9,               vp9);
     REGISTER_DECODER(VQA,               vqa);
     REGISTER_DECODER(WEBP,              webp);
@@ -642,7 +681,7 @@  void avcodec_register_all(void)
     /* external libraries, that shouldn't be used by default if one of the
      * above is available */
     REGISTER_ENCDEC (LIBOPENH264,       libopenh264);
-    REGISTER_DECODER(H264_CUVID,        h264_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_h264();
     REGISTER_ENCODER(H264_NVENC,        h264_nvenc);
     REGISTER_ENCODER(H264_OMX,          h264_omx);
     REGISTER_ENCODER(H264_QSV,          h264_qsv);
@@ -653,25 +692,25 @@  void avcodec_register_all(void)
     REGISTER_ENCODER(NVENC_H264,        nvenc_h264);
     REGISTER_ENCODER(NVENC_HEVC,        nvenc_hevc);
 #endif
-    REGISTER_DECODER(HEVC_CUVID,        hevc_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_hevc();
     REGISTER_DECODER(HEVC_MEDIACODEC,   hevc_mediacodec);
     REGISTER_ENCODER(HEVC_NVENC,        hevc_nvenc);
     REGISTER_ENCODER(HEVC_QSV,          hevc_qsv);
     REGISTER_ENCODER(HEVC_VAAPI,        hevc_vaapi);
     REGISTER_ENCODER(LIBKVAZAAR,        libkvazaar);
-    REGISTER_DECODER(MJPEG_CUVID,       mjpeg_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_mjpeg();
     REGISTER_ENCODER(MJPEG_VAAPI,       mjpeg_vaapi);
-    REGISTER_DECODER(MPEG1_CUVID,       mpeg1_cuvid);
-    REGISTER_DECODER(MPEG2_CUVID,       mpeg2_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_mpeg1();
+    if (!avcodec_cuvid) avcodec_register_cuvid_mpeg2();
     REGISTER_ENCODER(MPEG2_QSV,         mpeg2_qsv);
     REGISTER_ENCODER(MPEG2_VAAPI,       mpeg2_vaapi);
-    REGISTER_DECODER(MPEG4_CUVID,       mpeg4_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_mpeg4();
     REGISTER_DECODER(MPEG4_MEDIACODEC,  mpeg4_mediacodec);
-    REGISTER_DECODER(VC1_CUVID,         vc1_cuvid);
-    REGISTER_DECODER(VP8_CUVID,         vp8_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_vc1();
+    if (!avcodec_cuvid) avcodec_register_cuvid_vp8();
     REGISTER_DECODER(VP8_MEDIACODEC,    vp8_mediacodec);
     REGISTER_ENCODER(VP8_VAAPI,         vp8_vaapi);
-    REGISTER_DECODER(VP9_CUVID,         vp9_cuvid);
+    if (!avcodec_cuvid) avcodec_register_cuvid_vp9();
     REGISTER_DECODER(VP9_MEDIACODEC,    vp9_mediacodec);
 
     /* parsers */
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 3e161ea..09c1835 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -6256,6 +6256,8 @@  const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
  */
 AVCPBProperties *av_cpb_properties_alloc(size_t *size);
 
+extern int avcodec_cuvid;
+
 /**
  * @}
  */
-- 
2.1.4