diff mbox

[FFmpeg-devel,1/2] avcodec: Add metadata to identify hardware backed codecs

Message ID 20171128044601.5963-2-philipl@overt.org
State New
Headers show

Commit Message

Philip Langdale Nov. 28, 2017, 4:46 a.m. UTC
If hardware acceleration is implemented through an HWAccel, then it's
easy to recognise, but some hardware implementations are best suited
to implementation as full decoders, and these are not easy to identify.
Clients typically need hardcoded lists of codecs, and usually need to
rely on related codecs using a particular naming convention.

To make it easier to discover these codecs dynamically, we can
introduce an explicit AV_CODEC_CAP_HARDWARE that indicates the codec
is hardware-backed. We also introduce a 'provider' field into AVCodec
that allows for an identifier to be attached to these codecs for
simple matching. The provider could also be used to identify the
use of an external software library.

Unfortunately, we know of at least one case where an external interface
might silently end up using a separate software implementation (libmfx)
and so we introduce AV_CODEC_CAP_MAYBE_NOT_HARDWARE so say that you
can't trust it 100%. It's silly but what can you do?

The expected use-case here is allowing a client application to
enumerate hardware decoding options for a given format and to allow
a user to select one based on a recognisable identifier (the provider).

Signed-off-by: Philip Langdale <philipl@overt.org>
---
 Changelog            |  1 +
 doc/APIchanges       |  5 +++++
 libavcodec/avcodec.h | 19 +++++++++++++++++++
 3 files changed, 25 insertions(+)

Comments

wm4 Nov. 29, 2017, 2:31 p.m. UTC | #1
On Mon, 27 Nov 2017 20:46:00 -0800
Philip Langdale <philipl@overt.org> wrote:

> If hardware acceleration is implemented through an HWAccel, then it's
> easy to recognise, but some hardware implementations are best suited
> to implementation as full decoders, and these are not easy to identify.
> Clients typically need hardcoded lists of codecs, and usually need to
> rely on related codecs using a particular naming convention.
> 
> To make it easier to discover these codecs dynamically, we can
> introduce an explicit AV_CODEC_CAP_HARDWARE that indicates the codec
> is hardware-backed. We also introduce a 'provider' field into AVCodec
> that allows for an identifier to be attached to these codecs for
> simple matching. The provider could also be used to identify the
> use of an external software library.
> 
> Unfortunately, we know of at least one case where an external interface
> might silently end up using a separate software implementation (libmfx)
> and so we introduce AV_CODEC_CAP_MAYBE_NOT_HARDWARE so say that you
> can't trust it 100%. It's silly but what can you do?
> 
> The expected use-case here is allowing a client application to
> enumerate hardware decoding options for a given format and to allow
> a user to select one based on a recognisable identifier (the provider).
> 
> Signed-off-by: Philip Langdale <philipl@overt.org>
> ---
>  Changelog            |  1 +
>  doc/APIchanges       |  5 +++++
>  libavcodec/avcodec.h | 19 +++++++++++++++++++
>  3 files changed, 25 insertions(+)
> 
> diff --git a/Changelog b/Changelog
> index 76d6fad56b..a99edbf4b7 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -21,6 +21,7 @@ version <next>:
>  - video mix filter
>  - video normalize filter
>  - audio lv2 wrapper filter
> +- Added codec metadata to identify hardware backed decoders
>  
>  
>  version 3.4:
> diff --git a/doc/APIchanges b/doc/APIchanges
> index 44a740e51f..7856cb13a1 100644
> --- a/doc/APIchanges
> +++ b/doc/APIchanges
> @@ -15,6 +15,11 @@ libavutil:     2017-10-21
>  
>  API changes, most recent first:
>  
> +2017-12-xx - xxxxxxx - lavc 58.7.100 - avcodec.h
> +  Add AV_CODEC_CAP_HARDWARE
> +  Add AV_CODEC_CAP_MAYBE_NOT_HARDWARE
> +  Add AVCodec.provider
> +
>  2017-xx-xx - xxxxxxx - lavc 58.6.100 - avcodec.h
>    Add const to AVCodecContext.hwaccel.
>  
> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> index c1e68b1d13..04cfa48665 100644
> --- a/libavcodec/avcodec.h
> +++ b/libavcodec/avcodec.h
> @@ -1036,6 +1036,16 @@ typedef struct RcOverride{
>   * choice for probing.
>   */
>  #define AV_CODEC_CAP_AVOID_PROBING       (1 << 17)
> +/**
> + * Codec is backed by a hardware implementation. Typically used to
> + * identify a non-hwaccel hardware decoder.
> + */
> +#define AV_CODEC_CAP_HARDWARE            (1 << 18)
> +/**
> + * Codec is normally backed by a hardware implementation, but the
> + * external abstraction could be software backed.
> + */
> +#define AV_CODEC_CAP_MAYBE_NOT_HARDWARE  (1 << 19)

One could argue that every hw-backed decoder might fallback to software
fully or partially. For example, Intel is known to have hybrid drivers
(that sometimes even tend to be slower than libavcodec), and
videotoolbox has a software decoder (though we disable it). So I'm not
sure if this flag is really necessary, but I'm not opposed either.

>  /**
>   * Codec is intra only.
>   */
> @@ -3475,6 +3485,15 @@ typedef struct AVCodec {
>       * The user can only access this field via avcodec_get_hw_config().
>       */
>      const struct AVCodecHWConfigInternal **hw_configs;
> +
> +    /**
> +     * String that helps identify the external provider for the codec.
> +     * For example, if some external hardware decoder provides support
> +     * for a variety of codecs, they could each set the same provider
> +     * value to signify their relationship. Could also be used to
> +     * identify the use of an external software implementation.
> +     */
> +   const char *provider;
>  } AVCodec;
>  
>  #if FF_API_CODEC_GET_SET

Generally LGTM. It's pretty useless if it doesn't land in libav,
though. Can you post it there too?
diff mbox

Patch

diff --git a/Changelog b/Changelog
index 76d6fad56b..a99edbf4b7 100644
--- a/Changelog
+++ b/Changelog
@@ -21,6 +21,7 @@  version <next>:
 - video mix filter
 - video normalize filter
 - audio lv2 wrapper filter
+- Added codec metadata to identify hardware backed decoders
 
 
 version 3.4:
diff --git a/doc/APIchanges b/doc/APIchanges
index 44a740e51f..7856cb13a1 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -15,6 +15,11 @@  libavutil:     2017-10-21
 
 API changes, most recent first:
 
+2017-12-xx - xxxxxxx - lavc 58.7.100 - avcodec.h
+  Add AV_CODEC_CAP_HARDWARE
+  Add AV_CODEC_CAP_MAYBE_NOT_HARDWARE
+  Add AVCodec.provider
+
 2017-xx-xx - xxxxxxx - lavc 58.6.100 - avcodec.h
   Add const to AVCodecContext.hwaccel.
 
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index c1e68b1d13..04cfa48665 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1036,6 +1036,16 @@  typedef struct RcOverride{
  * choice for probing.
  */
 #define AV_CODEC_CAP_AVOID_PROBING       (1 << 17)
+/**
+ * Codec is backed by a hardware implementation. Typically used to
+ * identify a non-hwaccel hardware decoder.
+ */
+#define AV_CODEC_CAP_HARDWARE            (1 << 18)
+/**
+ * Codec is normally backed by a hardware implementation, but the
+ * external abstraction could be software backed.
+ */
+#define AV_CODEC_CAP_MAYBE_NOT_HARDWARE  (1 << 19)
 /**
  * Codec is intra only.
  */
@@ -3475,6 +3485,15 @@  typedef struct AVCodec {
      * The user can only access this field via avcodec_get_hw_config().
      */
     const struct AVCodecHWConfigInternal **hw_configs;
+
+    /**
+     * String that helps identify the external provider for the codec.
+     * For example, if some external hardware decoder provides support
+     * for a variety of codecs, they could each set the same provider
+     * value to signify their relationship. Could also be used to
+     * identify the use of an external software implementation.
+     */
+   const char *provider;
 } AVCodec;
 
 #if FF_API_CODEC_GET_SET