diff mbox series

[FFmpeg-devel,1/2] qsvdec: add support for HW_DEVICE_CTX method

Message ID 20210623030346.2025933-1-haihao.xiang@intel.com
State New
Headers show
Series [FFmpeg-devel,1/2] qsvdec: add support for HW_DEVICE_CTX method
Related show

Checks

Context Check Description
andriy/x86_make success Make finished
andriy/x86_make_fate success Make fate finished
andriy/PPC64_make success Make finished
andriy/PPC64_make_fate success Make fate finished

Commit Message

Haihao Xiang June 23, 2021, 3:03 a.m. UTC
This allows user set hw_device_ctx instead of hw_frames_ctx for QSV
decoders, hence we may remove the ad-hoc libmfx setup code from FFmpeg.

"-hwaccel_output_format format" is applied to QSV decoders after
removing the ad-hoc libmfx code. In order to keep compatibility with old
commandlines, the default format is set to AV_PIX_FMT_QSV, but this
behavior will be removed in the future. Please set "-hwaccel_output_format qsv"
explicitly if AV_PIX_FMT_QSV is expected.

The normal device stuff works for QSV decoders now, user may use
"-init_hw_device args" to initialise device and "-hwaccel_device
devicename" to select a device for QSV decoders. "-qsv_device device"
which was added for workarounding device selection in the ad-hoc libmfx
code is deprecated and will be removed from FFmpeg.

For example:

$> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device
qsv=hw@va -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -

/dev/dri/renderD128 is actually open for h264_qsv decoder in the above
command without this patch. After applying this patch, /dev/dri/card0
is used.

$> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device
qsv=hw@va -hwaccel_device hw -hwaccel qsv -c:v h264_qsv -i input.h264
-f null -

device hw of type qsv is not usable in the above command without this
patch. After applying this patch, this command works as expected.
---
 fftools/Makefile     |   1 -
 fftools/ffmpeg.h     |   1 -
 fftools/ffmpeg_hw.c  |  12 +++++
 fftools/ffmpeg_opt.c |  56 ++++++++++++++++++++--
 fftools/ffmpeg_qsv.c | 110 -------------------------------------------
 libavcodec/qsvdec.c  |  31 +++++++++++-
 6 files changed, 93 insertions(+), 118 deletions(-)
 delete mode 100644 fftools/ffmpeg_qsv.c

Comments

Soft Works June 23, 2021, 4:41 p.m. UTC | #1
-----Original Message-----
From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Haihao Xiang
Sent: Mittwoch, 23. Juni 2021 05:04
To: ffmpeg-devel@ffmpeg.org
Cc: Haihao Xiang <haihao.xiang@intel.com>
Subject: [FFmpeg-devel] [PATCH 1/2] qsvdec: add support for HW_DEVICE_CTX method

This allows user set hw_device_ctx instead of hw_frames_ctx for QSV decoders, hence we may remove the ad-hoc libmfx setup code from FFmpeg.

"-hwaccel_output_format format" is applied to QSV decoders after removing the ad-hoc libmfx code. In order to keep compatibility with old commandlines, the default format is set to AV_PIX_FMT_QSV, but this behavior will be removed in the future. Please set "-hwaccel_output_format qsv"
explicitly if AV_PIX_FMT_QSV is expected.

The normal device stuff works for QSV decoders now, user may use "-init_hw_device args" to initialise device and "-hwaccel_device devicename" to select a device for QSV decoders. "-qsv_device device"
which was added for workarounding device selection in the ad-hoc libmfx code is deprecated and will be removed from FFmpeg.

For example:

$> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device qsv=hw@va -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -

/dev/dri/renderD128 is actually open for h264_qsv decoder in the above command without this patch. After applying this patch, /dev/dri/card0 is used.

$> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device qsv=hw@va -hwaccel_device hw -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -

device hw of type qsv is not usable in the above command without this patch. After applying this patch, this command works as expected.
---

Haihao,

I think, from all the not-yet-merged QSV patches, this is the most important one.
I'll try to find some time to review shortly.

Is this patch independent of your other submitted patches?

softworkz
Haihao Xiang June 24, 2021, 1:33 a.m. UTC | #2
On Wed, 2021-06-23 at 16:41 +0000, Soft Works wrote:
> 
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Haihao Xiang
> Sent: Mittwoch, 23. Juni 2021 05:04
> To: ffmpeg-devel@ffmpeg.org
> Cc: Haihao Xiang <haihao.xiang@intel.com>
> Subject: [FFmpeg-devel] [PATCH 1/2] qsvdec: add support for HW_DEVICE_CTX
> method
> 
> This allows user set hw_device_ctx instead of hw_frames_ctx for QSV decoders,
> hence we may remove the ad-hoc libmfx setup code from FFmpeg.
> 
> "-hwaccel_output_format format" is applied to QSV decoders after removing the
> ad-hoc libmfx code. In order to keep compatibility with old commandlines, the
> default format is set to AV_PIX_FMT_QSV, but this behavior will be removed in
> the future. Please set "-hwaccel_output_format qsv"
> explicitly if AV_PIX_FMT_QSV is expected.
> 
> The normal device stuff works for QSV decoders now, user may use "-
> init_hw_device args" to initialise device and "-hwaccel_device devicename" to
> select a device for QSV decoders. "-qsv_device device"
> which was added for workarounding device selection in the ad-hoc libmfx code
> is deprecated and will be removed from FFmpeg.
> 
> For example:
> 
> $> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device qsv=hw@va
> -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -
> 
> /dev/dri/renderD128 is actually open for h264_qsv decoder in the above command
> without this patch. After applying this patch, /dev/dri/card0 is used.
> 
> $> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device qsv=hw@va
> -hwaccel_device hw -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -
> 
> device hw of type qsv is not usable in the above command without this patch.
> After applying this patch, this command works as expected.
> ---
> 
> Haihao,
> 
> I think, from all the not-yet-merged QSV patches, this is the most important
> one.
> I'll try to find some time to review shortly.

Thanks for helping to review this patch, and any comment is welcome. 

> 
> Is this patch independent of your other submitted patches?

Yes, this patch is independent of other patches and you may apply it to the
latest master branch directly. 

BTW the fix in 
http://ffmpeg.org/pipermail/ffmpeg-devel/2021-February/276695.html is no longer
needed after applying this patch. The command  works for me if using
-init_hw_device qsv=qsv@intel to create the connection between vaapi and QSVdevices. 

Regards
Haihao

> 
> softworkz
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/fftools/Makefile b/fftools/Makefile
index 5affaa3f56..5234932ab0 100644
--- a/fftools/Makefile
+++ b/fftools/Makefile
@@ -10,7 +10,6 @@  ALLAVPROGS   = $(AVBASENAMES:%=%$(PROGSSUF)$(EXESUF))
 ALLAVPROGS_G = $(AVBASENAMES:%=%$(PROGSSUF)_g$(EXESUF))
 
 OBJS-ffmpeg                        += fftools/ffmpeg_opt.o fftools/ffmpeg_filter.o fftools/ffmpeg_hw.o
-OBJS-ffmpeg-$(CONFIG_LIBMFX)       += fftools/ffmpeg_qsv.o
 ifndef CONFIG_VIDEOTOOLBOX
 OBJS-ffmpeg-$(CONFIG_VDA)          += fftools/ffmpeg_videotoolbox.o
 endif
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index e9d30fbd67..27856f95f4 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -60,7 +60,6 @@  enum HWAccelID {
     HWACCEL_AUTO,
     HWACCEL_GENERIC,
     HWACCEL_VIDEOTOOLBOX,
-    HWACCEL_QSV,
 };
 
 typedef struct HWAccel {
diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c
index fc4a5d31d6..043e4c5863 100644
--- a/fftools/ffmpeg_hw.c
+++ b/fftools/ffmpeg_hw.c
@@ -339,6 +339,18 @@  int hw_device_setup_for_decode(InputStream *ist)
         } else if (ist->hwaccel_id == HWACCEL_GENERIC) {
             type = ist->hwaccel_device_type;
             dev = hw_device_get_by_type(type);
+
+            // When "-qsv_device device" is used, an internal QSV device named
+            // as "__qsv_device" is created and another QSV device is created
+            // if "-init_hw_device qsv=name@name" is used. There are 2 QSV devices
+            // if both "-qsv_device device" and "-init_hw_device qsv=name@name"
+            // are used, hw_device_get_by_type(AV_HWDEVICE_TYPE_QSV) returns NULL.
+            // To keep back-compatibility with the removed ad-hoc libmfx setup code,
+            // call hw_device_get_by_name("__qsv_device") to select the internal QSV
+            // device. This will be removed once -qsv_device is no longer supported.
+            if (!dev && type == AV_HWDEVICE_TYPE_QSV)
+                dev = hw_device_get_by_name("__qsv_device");
+
             if (!dev)
                 err = hw_device_init_from_type(type, NULL, &dev);
         } else {
diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
index a63bed54cf..1f4d9dbcfa 100644
--- a/fftools/ffmpeg_opt.c
+++ b/fftools/ffmpeg_opt.c
@@ -136,9 +136,6 @@  static const char *const opt_name_enc_time_bases[]            = {"enc_time_base"
 const HWAccel hwaccels[] = {
 #if CONFIG_VIDEOTOOLBOX
     { "videotoolbox", videotoolbox_init, HWACCEL_VIDEOTOOLBOX, AV_PIX_FMT_VIDEOTOOLBOX },
-#endif
-#if CONFIG_LIBMFX
-    { "qsv",   qsv_init,   HWACCEL_QSV,   AV_PIX_FMT_QSV },
 #endif
     { 0 },
 };
@@ -569,6 +566,49 @@  static int opt_vaapi_device(void *optctx, const char *opt, const char *arg)
 }
 #endif
 
+#if CONFIG_QSV
+static int opt_qsv_device(void *optctx, const char *opt, const char *arg)
+{
+#if CONFIG_VAAPI
+    const char *prefix = "vaapi=__qsv_child_device:";
+#elif CONFIG_DXVA2
+    const char *prefix = "dxva2=__qsv_child_device:";
+#else
+    const char *prefix = NULL;
+#endif
+    int err = 0;
+    char *tmp = NULL;
+
+    if (prefix) {
+        av_log(NULL, AV_LOG_WARNING,
+            "WARNING: Please use \"-init_hw_device args\" to initialise QSV "
+            "device and \"-hwaccel_device devicename\" to select QSV device "
+            "for QSV decoders. \"-qsv_device device\" is DEPRECATED and will "
+            "be removed in the future. Please do not use \"__qsv_child_device\" "
+            "or \"__qsv_device\" as device name when both \"-qsv_device device\" "
+            "and \"-init_hw_device args\" are used.\n");
+
+        tmp = av_asprintf("%s%s", prefix, arg);
+
+        if (!tmp)
+            return AVERROR(ENOMEM);
+
+        err = hw_device_init_from_string(tmp, NULL);
+
+        if (err)
+            goto error;
+
+        err = hw_device_init_from_string("qsv=__qsv_device@__qsv_child_device", NULL);
+    } else
+        err = AVERROR(ENOSYS);
+
+error:
+    av_free(tmp);
+    return err;
+}
+
+#endif
+
 static int opt_init_hw_device(void *optctx, const char *opt, const char *arg)
 {
     if (!strcmp(arg, "list")) {
@@ -894,6 +934,12 @@  static void add_input_streams(OptionsContext *o, AVFormatContext *ic)
                     "with old commandlines. This behaviour is DEPRECATED and will be removed "
                     "in the future. Please explicitly set \"-hwaccel_output_format cuda\".\n");
                 ist->hwaccel_output_format = AV_PIX_FMT_CUDA;
+            } else if (!hwaccel_output_format && hwaccel && !strcmp(hwaccel, "qsv")) {
+                av_log(NULL, AV_LOG_WARNING,
+                    "WARNING: defaulting hwaccel_output_format to qsv for compatibility "
+                    "with old commandlines. This behaviour is DEPRECATED and will be removed "
+                    "in the future. Please explicitly set \"-hwaccel_output_format qsv\".\n");
+                ist->hwaccel_output_format = AV_PIX_FMT_QSV;
             } else if (hwaccel_output_format) {
                 ist->hwaccel_output_format = av_get_pix_fmt(hwaccel_output_format);
                 if (ist->hwaccel_output_format == AV_PIX_FMT_NONE) {
@@ -3805,8 +3851,8 @@  const OptionDef options[] = {
 #endif
 
 #if CONFIG_QSV
-    { "qsv_device", HAS_ARG | OPT_STRING | OPT_EXPERT, { &qsv_device },
-        "set QSV hardware device (DirectX adapter index, DRM path or X11 display name)", "device"},
+    { "qsv_device", HAS_ARG | OPT_EXPERT, { .func_arg = opt_qsv_device },
+        "set QSV hardware device (deprecated, use the normal device stuff; DirectX adapter index, DRM path or X11 display name)", "device"},
 #endif
 
     { "init_hw_device", HAS_ARG | OPT_EXPERT, { .func_arg = opt_init_hw_device },
diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c
deleted file mode 100644
index 960c88b69d..0000000000
--- a/fftools/ffmpeg_qsv.c
+++ /dev/null
@@ -1,110 +0,0 @@ 
-/*
- * This file is part of FFmpeg.
- *
- * FFmpeg is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * FFmpeg is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- *
- * You should have received a copy of the GNU Lesser General Public
- * License along with FFmpeg; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include <mfx/mfxvideo.h>
-#include <stdlib.h>
-
-#include "libavutil/dict.h"
-#include "libavutil/hwcontext.h"
-#include "libavutil/hwcontext_qsv.h"
-#include "libavutil/mem.h"
-#include "libavutil/opt.h"
-#include "libavcodec/qsv.h"
-
-#include "ffmpeg.h"
-
-static AVBufferRef *hw_device_ctx;
-char *qsv_device = NULL;
-
-static int qsv_get_buffer(AVCodecContext *s, AVFrame *frame, int flags)
-{
-    InputStream *ist = s->opaque;
-
-    return av_hwframe_get_buffer(ist->hw_frames_ctx, frame, 0);
-}
-
-static void qsv_uninit(AVCodecContext *s)
-{
-    InputStream *ist = s->opaque;
-    av_buffer_unref(&ist->hw_frames_ctx);
-}
-
-static int qsv_device_init(InputStream *ist)
-{
-    int err;
-    AVDictionary *dict = NULL;
-
-    if (qsv_device) {
-        err = av_dict_set(&dict, "child_device", qsv_device, 0);
-        if (err < 0)
-            return err;
-    }
-
-    err = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_QSV,
-                                 ist->hwaccel_device, dict, 0);
-    if (err < 0) {
-        av_log(NULL, AV_LOG_ERROR, "Error creating a QSV device\n");
-        goto err_out;
-    }
-
-err_out:
-    if (dict)
-        av_dict_free(&dict);
-
-    return err;
-}
-
-int qsv_init(AVCodecContext *s)
-{
-    InputStream *ist = s->opaque;
-    AVHWFramesContext *frames_ctx;
-    AVQSVFramesContext *frames_hwctx;
-    int ret;
-
-    if (!hw_device_ctx) {
-        ret = qsv_device_init(ist);
-        if (ret < 0)
-            return ret;
-    }
-
-    av_buffer_unref(&ist->hw_frames_ctx);
-    ist->hw_frames_ctx = av_hwframe_ctx_alloc(hw_device_ctx);
-    if (!ist->hw_frames_ctx)
-        return AVERROR(ENOMEM);
-
-    frames_ctx   = (AVHWFramesContext*)ist->hw_frames_ctx->data;
-    frames_hwctx = frames_ctx->hwctx;
-
-    frames_ctx->width             = FFALIGN(s->coded_width,  32);
-    frames_ctx->height            = FFALIGN(s->coded_height, 32);
-    frames_ctx->format            = AV_PIX_FMT_QSV;
-    frames_ctx->sw_format         = s->sw_pix_fmt;
-    frames_ctx->initial_pool_size = 64 + s->extra_hw_frames;
-    frames_hwctx->frame_type      = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET;
-
-    ret = av_hwframe_ctx_init(ist->hw_frames_ctx);
-    if (ret < 0) {
-        av_log(NULL, AV_LOG_ERROR, "Error initializing a QSV frame pool\n");
-        return ret;
-    }
-
-    ist->hwaccel_get_buffer = qsv_get_buffer;
-    ist->hwaccel_uninit     = qsv_uninit;
-
-    return 0;
-}
diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
index 622750927c..1ee9944aa2 100644
--- a/libavcodec/qsvdec.c
+++ b/libavcodec/qsvdec.c
@@ -99,7 +99,7 @@  static const AVCodecHWConfigInternal *const qsv_hw_configs[] = {
         .public = {
             .pix_fmt     = AV_PIX_FMT_QSV,
             .methods     = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX |
-                           AV_CODEC_HW_CONFIG_METHOD_AD_HOC,
+                           AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX,
             .device_type = AV_HWDEVICE_TYPE_QSV,
         },
         .hwaccel = NULL,
@@ -248,6 +248,35 @@  static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel
         q->nb_ext_buffers = user_ctx->nb_ext_buffers;
     }
 
+    if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret == AV_PIX_FMT_QSV) {
+        AVHWFramesContext *hwframes_ctx;
+        AVQSVFramesContext *frames_hwctx;
+
+        avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx);
+
+        if (!avctx->hw_frames_ctx) {
+            av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n");
+            return AVERROR(ENOMEM);
+        }
+
+        hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data;
+        frames_hwctx = hwframes_ctx->hwctx;
+        hwframes_ctx->width             = FFALIGN(avctx->coded_width,  32);
+        hwframes_ctx->height            = FFALIGN(avctx->coded_height, 32);
+        hwframes_ctx->format            = AV_PIX_FMT_QSV;
+        hwframes_ctx->sw_format         = avctx->sw_pix_fmt;
+        hwframes_ctx->initial_pool_size = 64 + avctx->extra_hw_frames;
+        frames_hwctx->frame_type        = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET;
+
+        ret = av_hwframe_ctx_init(avctx->hw_frames_ctx);
+
+        if (ret < 0) {
+            av_log(NULL, AV_LOG_ERROR, "Error initializing a QSV frame pool\n");
+            av_buffer_unref(&avctx->hw_frames_ctx);
+            return ret;
+        }
+    }
+
     if (avctx->hw_frames_ctx) {
         AVHWFramesContext    *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data;
         AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx;