From patchwork Thu Jul 29 07:04:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xiang, Haihao" X-Patchwork-Id: 29115 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp5553121ios; Thu, 29 Jul 2021 00:04:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwJi2qpmA0L7sduoYpyETWzHg8RuBXbO2HT1eIo5tjLBheLoWYwvDNaZ3q/fkFgi8tC2ROO X-Received: by 2002:a17:906:56ca:: with SMTP id an10mr3325396ejc.90.1627542279584; Thu, 29 Jul 2021 00:04:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627542279; cv=none; d=google.com; s=arc-20160816; b=RcalXssPPDmE7PrsC48SB21pmtakVGYsnIRpsefmvToX2ae4fKK1hGaQRf0EoSFBlF Lwy1z8f5VWc3apNSjgmiTuTMGqFvYoq7RrUY16WMeBy4GL5U0qs3SEMoVYeNCas4dAqJ ZR1GxgJb1gHpIOIHx+kOAAJS3dy5PgOO5Dvr92747C9wsZ1UFdCscerOUwcMT9nc/rXp rN/piR6c+EKYCtkQm6xukZ7D6k3bt7+11HU1gIkiz1a/kjslYShRAs51pYwC2l3r7lqR /Wlc3MmYpO5zeXtYvxW/aOiVb1lEW6rdz/i0kMlfXYCtHc6JX7Atroz0EcHmDsaNra13 O/Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:delivered-to; bh=XP1mQQBxdJ1LyRdGIfJSMMX4C7+6fLUROnqCUKrzqio=; b=DBMfCSrYYGlHkLpMD74MpffhxXC7zevnsKUlRkNkSHPExVfnPm/QRhwZt3Par9DGft n1zC0VVNAyC/X61bFHq+cILvzJotoZp4Y4NLZ7FNz0HNMPLpJYGEfBHvfCOikN4RTLG+ 99CkZVkJsoPm/wkKz58h66ov6s9jJs3bqaC8NQpagP1vMOkMWSD+OY845iB5k4YlSymu 2owUT9j1ITbn94jY5d3xeq3D/aCz+37/w6DjbqnkvxENyqLhbC061Y7wHh2ke1D9KQAV I1lncbltO8/LqeEoIa/LATdvFLzVkds4/5wvj210Mh0fX8IlRQy8HEhAgbW16uQx+44M Ok8w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p12si2599262ejy.596.2021.07.29.00.04.38; Thu, 29 Jul 2021 00:04:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2410A68A28C; Thu, 29 Jul 2021 10:04:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 86F7D680549 for ; Thu, 29 Jul 2021 10:04:26 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10059"; a="273897527" X-IronPort-AV: E=Sophos;i="5.84,278,1620716400"; d="scan'208";a="273897527" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2021 00:04:22 -0700 X-IronPort-AV: E=Sophos;i="5.84,278,1620716400"; d="scan'208";a="456752446" Received: from xhh-dg164.sh.intel.com ([10.239.159.167]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2021 00:04:21 -0700 From: Haihao Xiang To: ffmpeg-devel@ffmpeg.org Date: Thu, 29 Jul 2021 15:04:11 +0800 Message-Id: <20210729070412.1250-1-haihao.xiang@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH v2 1/2] qsvdec: add support for HW_DEVICE_CTX method X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Haihao Xiang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: A77mbu6zry/r This allows user set hw_device_ctx instead of hw_frames_ctx for QSV decoders, hence we may remove the ad-hoc libmfx setup code from FFmpeg. "-hwaccel_output_format format" is applied to QSV decoders after removing the ad-hoc libmfx code. To keep compatibility with old commandlines, the default format is set to AV_PIX_FMT_QSV. User should set "-hwaccel_output_format qsv" explicitly if AV_PIX_FMT_QSV is expected. User may use the normal way to set device for QSV decoders now. "-qsv_device device" is deprecated and will be removed from FFmpeg. For example: $> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device qsv=hw@va -hwaccel qsv -c:v h264_qsv -i input.h264 -f null - /dev/dri/renderD128 is actually open for h264_qsv decoder without this patch. After applying this patch, /dev/dri/card0 is used. --- v2: Rebase the patchset against the latest master branch and resolve a conflict. fftools/Makefile | 1 - fftools/ffmpeg.h | 1 - fftools/ffmpeg_hw.c | 12 +++++ fftools/ffmpeg_opt.c | 54 +++++++++++++++++++-- fftools/ffmpeg_qsv.c | 109 ------------------------------------------- libavcodec/qsvdec.c | 31 +++++++++++- 6 files changed, 92 insertions(+), 116 deletions(-) delete mode 100644 fftools/ffmpeg_qsv.c diff --git a/fftools/Makefile b/fftools/Makefile index 5affaa3f56..5234932ab0 100644 --- a/fftools/Makefile +++ b/fftools/Makefile @@ -10,7 +10,6 @@ ALLAVPROGS = $(AVBASENAMES:%=%$(PROGSSUF)$(EXESUF)) ALLAVPROGS_G = $(AVBASENAMES:%=%$(PROGSSUF)_g$(EXESUF)) OBJS-ffmpeg += fftools/ffmpeg_opt.o fftools/ffmpeg_filter.o fftools/ffmpeg_hw.o -OBJS-ffmpeg-$(CONFIG_LIBMFX) += fftools/ffmpeg_qsv.o ifndef CONFIG_VIDEOTOOLBOX OBJS-ffmpeg-$(CONFIG_VDA) += fftools/ffmpeg_videotoolbox.o endif diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index d9c0628657..d2dd7ca092 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -61,7 +61,6 @@ enum HWAccelID { HWACCEL_AUTO, HWACCEL_GENERIC, HWACCEL_VIDEOTOOLBOX, - HWACCEL_QSV, }; typedef struct HWAccel { diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index fc4a5d31d6..043e4c5863 100644 --- a/fftools/ffmpeg_hw.c +++ b/fftools/ffmpeg_hw.c @@ -339,6 +339,18 @@ int hw_device_setup_for_decode(InputStream *ist) } else if (ist->hwaccel_id == HWACCEL_GENERIC) { type = ist->hwaccel_device_type; dev = hw_device_get_by_type(type); + + // When "-qsv_device device" is used, an internal QSV device named + // as "__qsv_device" is created and another QSV device is created + // if "-init_hw_device qsv=name@name" is used. There are 2 QSV devices + // if both "-qsv_device device" and "-init_hw_device qsv=name@name" + // are used, hw_device_get_by_type(AV_HWDEVICE_TYPE_QSV) returns NULL. + // To keep back-compatibility with the removed ad-hoc libmfx setup code, + // call hw_device_get_by_name("__qsv_device") to select the internal QSV + // device. This will be removed once -qsv_device is no longer supported. + if (!dev && type == AV_HWDEVICE_TYPE_QSV) + dev = hw_device_get_by_name("__qsv_device"); + if (!dev) err = hw_device_init_from_type(type, NULL, &dev); } else { diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 1b43bab9fc..4ddc6c939b 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -137,9 +137,6 @@ static const char *const opt_name_enc_time_bases[] = {"enc_time_base" const HWAccel hwaccels[] = { #if CONFIG_VIDEOTOOLBOX { "videotoolbox", videotoolbox_init, HWACCEL_VIDEOTOOLBOX, AV_PIX_FMT_VIDEOTOOLBOX }, -#endif -#if CONFIG_LIBMFX - { "qsv", qsv_init, HWACCEL_QSV, AV_PIX_FMT_QSV }, #endif { 0 }, }; @@ -571,6 +568,49 @@ static int opt_vaapi_device(void *optctx, const char *opt, const char *arg) } #endif +#if CONFIG_QSV +static int opt_qsv_device(void *optctx, const char *opt, const char *arg) +{ +#if CONFIG_VAAPI + const char *prefix = "vaapi=__qsv_child_device:"; +#elif CONFIG_DXVA2 + const char *prefix = "dxva2=__qsv_child_device:"; +#else + const char *prefix = NULL; +#endif + int err = 0; + char *tmp = NULL; + + if (prefix) { + av_log(NULL, AV_LOG_WARNING, + "WARNING: Please use \"-init_hw_device args\" to initialise QSV " + "device and \"-hwaccel_device devicename\" to select QSV device " + "for QSV decoders. \"-qsv_device device\" is DEPRECATED and will " + "be removed in the future. Please do not use \"__qsv_child_device\" " + "or \"__qsv_device\" as device name when both \"-qsv_device device\" " + "and \"-init_hw_device args\" are used.\n"); + + tmp = av_asprintf("%s%s", prefix, arg); + + if (!tmp) + return AVERROR(ENOMEM); + + err = hw_device_init_from_string(tmp, NULL); + + if (err) + goto error; + + err = hw_device_init_from_string("qsv=__qsv_device@__qsv_child_device", NULL); + } else + err = AVERROR(ENOSYS); + +error: + av_free(tmp); + return err; +} + +#endif + static int opt_init_hw_device(void *optctx, const char *opt, const char *arg) { if (!strcmp(arg, "list")) { @@ -898,6 +938,12 @@ static void add_input_streams(OptionsContext *o, AVFormatContext *ic) "with old commandlines. This behaviour is DEPRECATED and will be removed " "in the future. Please explicitly set \"-hwaccel_output_format cuda\".\n"); ist->hwaccel_output_format = AV_PIX_FMT_CUDA; + } else if (!hwaccel_output_format && hwaccel && !strcmp(hwaccel, "qsv")) { + av_log(NULL, AV_LOG_WARNING, + "WARNING: defaulting hwaccel_output_format to qsv for compatibility " + "with old commandlines. This behaviour is DEPRECATED and will be removed " + "in the future. Please explicitly set \"-hwaccel_output_format qsv\".\n"); + ist->hwaccel_output_format = AV_PIX_FMT_QSV; } else if (hwaccel_output_format) { ist->hwaccel_output_format = av_get_pix_fmt(hwaccel_output_format); if (ist->hwaccel_output_format == AV_PIX_FMT_NONE) { @@ -3825,7 +3871,7 @@ const OptionDef options[] = { #endif #if CONFIG_QSV - { "qsv_device", HAS_ARG | OPT_STRING | OPT_EXPERT, { &qsv_device }, + { "qsv_device", HAS_ARG | OPT_EXPERT, { .func_arg = opt_qsv_device }, "set QSV hardware device (DirectX adapter index, DRM path or X11 display name)", "device"}, #endif diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c deleted file mode 100644 index 7bd999d310..0000000000 --- a/fftools/ffmpeg_qsv.c +++ /dev/null @@ -1,109 +0,0 @@ -/* - * This file is part of FFmpeg. - * - * FFmpeg is free software; you can redistribute it and/or - * modify it under the terms of the GNU Lesser General Public - * License as published by the Free Software Foundation; either - * version 2.1 of the License, or (at your option) any later version. - * - * FFmpeg is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with FFmpeg; if not, write to the Free Software - * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#include -#include - -#include "libavutil/dict.h" -#include "libavutil/hwcontext.h" -#include "libavutil/hwcontext_qsv.h" -#include "libavutil/opt.h" -#include "libavcodec/qsv.h" - -#include "ffmpeg.h" - -static AVBufferRef *hw_device_ctx; -char *qsv_device = NULL; - -static int qsv_get_buffer(AVCodecContext *s, AVFrame *frame, int flags) -{ - InputStream *ist = s->opaque; - - return av_hwframe_get_buffer(ist->hw_frames_ctx, frame, 0); -} - -static void qsv_uninit(AVCodecContext *s) -{ - InputStream *ist = s->opaque; - av_buffer_unref(&ist->hw_frames_ctx); -} - -static int qsv_device_init(InputStream *ist) -{ - int err; - AVDictionary *dict = NULL; - - if (qsv_device) { - err = av_dict_set(&dict, "child_device", qsv_device, 0); - if (err < 0) - return err; - } - - err = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_QSV, - ist->hwaccel_device, dict, 0); - if (err < 0) { - av_log(NULL, AV_LOG_ERROR, "Error creating a QSV device\n"); - goto err_out; - } - -err_out: - if (dict) - av_dict_free(&dict); - - return err; -} - -int qsv_init(AVCodecContext *s) -{ - InputStream *ist = s->opaque; - AVHWFramesContext *frames_ctx; - AVQSVFramesContext *frames_hwctx; - int ret; - - if (!hw_device_ctx) { - ret = qsv_device_init(ist); - if (ret < 0) - return ret; - } - - av_buffer_unref(&ist->hw_frames_ctx); - ist->hw_frames_ctx = av_hwframe_ctx_alloc(hw_device_ctx); - if (!ist->hw_frames_ctx) - return AVERROR(ENOMEM); - - frames_ctx = (AVHWFramesContext*)ist->hw_frames_ctx->data; - frames_hwctx = frames_ctx->hwctx; - - frames_ctx->width = FFALIGN(s->coded_width, 32); - frames_ctx->height = FFALIGN(s->coded_height, 32); - frames_ctx->format = AV_PIX_FMT_QSV; - frames_ctx->sw_format = s->sw_pix_fmt; - frames_ctx->initial_pool_size = 64 + s->extra_hw_frames; - frames_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; - - ret = av_hwframe_ctx_init(ist->hw_frames_ctx); - if (ret < 0) { - av_log(NULL, AV_LOG_ERROR, "Error initializing a QSV frame pool\n"); - return ret; - } - - ist->hwaccel_get_buffer = qsv_get_buffer; - ist->hwaccel_uninit = qsv_uninit; - - return 0; -} diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 622750927c..1ee9944aa2 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -99,7 +99,7 @@ static const AVCodecHWConfigInternal *const qsv_hw_configs[] = { .public = { .pix_fmt = AV_PIX_FMT_QSV, .methods = AV_CODEC_HW_CONFIG_METHOD_HW_FRAMES_CTX | - AV_CODEC_HW_CONFIG_METHOD_AD_HOC, + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, .device_type = AV_HWDEVICE_TYPE_QSV, }, .hwaccel = NULL, @@ -248,6 +248,35 @@ static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel q->nb_ext_buffers = user_ctx->nb_ext_buffers; } + if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret == AV_PIX_FMT_QSV) { + AVHWFramesContext *hwframes_ctx; + AVQSVFramesContext *frames_hwctx; + + avctx->hw_frames_ctx = av_hwframe_ctx_alloc(avctx->hw_device_ctx); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n"); + return AVERROR(ENOMEM); + } + + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; + frames_hwctx = hwframes_ctx->hwctx; + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32); + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32); + hwframes_ctx->format = AV_PIX_FMT_QSV; + hwframes_ctx->sw_format = avctx->sw_pix_fmt; + hwframes_ctx->initial_pool_size = 64 + avctx->extra_hw_frames; + frames_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; + + ret = av_hwframe_ctx_init(avctx->hw_frames_ctx); + + if (ret < 0) { + av_log(NULL, AV_LOG_ERROR, "Error initializing a QSV frame pool\n"); + av_buffer_unref(&avctx->hw_frames_ctx); + return ret; + } + } + if (avctx->hw_frames_ctx) { AVHWFramesContext *frames_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx->data; AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx;