[FFmpeg-devel] avcodec/videotoolbox: fix kVTCouldNotFindVideoDecoderErr trying to decode HEVC on iOS

Submitted by Aman Gupta on April 19, 2018, 10:34 p.m.

Details

Message ID 20180419223401.72955-1-ffmpeg@tmm1.net
State New
Headers show

Commit Message

Aman Gupta April 19, 2018, 10:34 p.m.
From: Aman Gupta <aman@tmm1.net>

Older iOS devices don't have a hardware HEVC decoder, but the
software decoder offered by VideoToolbox is well-optimized and
performs much better than the ffmpeg decoder.
---
 libavcodec/videotoolbox.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

Comments

Rodger Combs April 20, 2018, 8:43 a.m.
If there was a way to indicate this to consumers, or expose an option to turn this off, I'd say it should have that... but there's no good infrastructure to do that in an hwaccel, so whatever.
One nit; otherwise LGTM.

> On Apr 19, 2018, at 17:34, Aman Gupta <ffmpeg@tmm1.net> wrote:
> 
> From: Aman Gupta <aman@tmm1.net>
> 
> Older iOS devices don't have a hardware HEVC decoder, but the
> software decoder offered by VideoToolbox is well-optimized and
> performs much better than the ffmpeg decoder.
> ---
> libavcodec/videotoolbox.c | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c
> index 57b6698e1b..8671608a35 100644
> --- a/libavcodec/videotoolbox.c
> +++ b/libavcodec/videotoolbox.c
> @@ -36,6 +36,9 @@
> #ifndef kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder
> #  define kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder CFSTR("RequireHardwareAcceleratedVideoDecoder")
> #endif
> +#ifndef kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder
> +#  define kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder CFSTR("EnableHardwareAcceleratedVideoDecoder")
> +#endif
> 
> #if !HAVE_KCMVIDEOCODECTYPE_HEVC
> enum { kCMVideoCodecType_HEVC = 'hvc1' };
> @@ -709,7 +712,9 @@ static CFDictionaryRef videotoolbox_decoder_config_create(CMVideoCodecType codec
>                                                                    &kCFTypeDictionaryValueCallBacks);
> 
>     CFDictionarySetValue(config_info,
> -                         kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder,
> +                         codec_type == kCMVideoCodecType_HEVC ?
> +                            kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder :
> +                            kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder,
>                          kCFBooleanTrue);
> 
>     CFMutableDictionaryRef avc_info;
> @@ -833,6 +838,9 @@ static int videotoolbox_start(AVCodecContext *avctx)
>     case kVTVideoDecoderUnsupportedDataFormatErr:
>         av_log(avctx, AV_LOG_VERBOSE, "VideoToolbox does not support this format.\n");
>         return AVERROR(ENOSYS);
> +    case kVTCouldNotFindVideoDecoderErr:
> +        av_log(avctx, AV_LOG_VERBOSE, "VideoToolbox decoder for this format not found.\n");
> +        return AVERROR(ENOSYS);
>     case kVTVideoDecoderMalfunctionErr:
>         av_log(avctx, AV_LOG_VERBOSE, "VideoToolbox malfunction.\n");
>         return AVERROR(EINVAL);
> @@ -842,7 +850,7 @@ static int videotoolbox_start(AVCodecContext *avctx)
>     case 0:
>         return 0;
>     default:
> -        av_log(avctx, AV_LOG_VERBOSE, "Unknown VideoToolbox session creation error %u\n", (unsigned)status);
> +        av_log(avctx, AV_LOG_VERBOSE, "Unknown VideoToolbox session creation error %d\n", (int)status);

Bit tangential; should be its own commit. Maybe just combine the two logging changes.

>         return AVERROR_UNKNOWN;
>     }
> }
> -- 
> 2.14.2
>
wm4 April 20, 2018, 9:56 a.m.
On Fri, 20 Apr 2018 03:43:30 -0500
Rodger Combs <rodger.combs@gmail.com> wrote:

> If there was a way to indicate this to consumers, or expose an option to turn this off, I'd say it should have that... but there's no good infrastructure to do that in an hwaccel, so whatever.
> One nit; otherwise LGTM.
> 

I guess it's OK as long as the Apple decoder really is faster than
FFmpeg with frame threading, and it supports all features a hw decoder
also would.

The whole point of hwaccel is to get a speed advantage over the
internal decoder.

Some mechanism to indicate that a sw decoder is used (even if it's
"better") would still be good.

Patch hide | download patch | download mbox

diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c
index 57b6698e1b..8671608a35 100644
--- a/libavcodec/videotoolbox.c
+++ b/libavcodec/videotoolbox.c
@@ -36,6 +36,9 @@ 
 #ifndef kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder
 #  define kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder CFSTR("RequireHardwareAcceleratedVideoDecoder")
 #endif
+#ifndef kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder
+#  define kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder CFSTR("EnableHardwareAcceleratedVideoDecoder")
+#endif
 
 #if !HAVE_KCMVIDEOCODECTYPE_HEVC
 enum { kCMVideoCodecType_HEVC = 'hvc1' };
@@ -709,7 +712,9 @@  static CFDictionaryRef videotoolbox_decoder_config_create(CMVideoCodecType codec
                                                                    &kCFTypeDictionaryValueCallBacks);
 
     CFDictionarySetValue(config_info,
-                         kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder,
+                         codec_type == kCMVideoCodecType_HEVC ?
+                            kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder :
+                            kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder,
                          kCFBooleanTrue);
 
     CFMutableDictionaryRef avc_info;
@@ -833,6 +838,9 @@  static int videotoolbox_start(AVCodecContext *avctx)
     case kVTVideoDecoderUnsupportedDataFormatErr:
         av_log(avctx, AV_LOG_VERBOSE, "VideoToolbox does not support this format.\n");
         return AVERROR(ENOSYS);
+    case kVTCouldNotFindVideoDecoderErr:
+        av_log(avctx, AV_LOG_VERBOSE, "VideoToolbox decoder for this format not found.\n");
+        return AVERROR(ENOSYS);
     case kVTVideoDecoderMalfunctionErr:
         av_log(avctx, AV_LOG_VERBOSE, "VideoToolbox malfunction.\n");
         return AVERROR(EINVAL);
@@ -842,7 +850,7 @@  static int videotoolbox_start(AVCodecContext *avctx)
     case 0:
         return 0;
     default:
-        av_log(avctx, AV_LOG_VERBOSE, "Unknown VideoToolbox session creation error %u\n", (unsigned)status);
+        av_log(avctx, AV_LOG_VERBOSE, "Unknown VideoToolbox session creation error %d\n", (int)status);
         return AVERROR_UNKNOWN;
     }
 }