From patchwork Tue Jan 26 02:32:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25181 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3BE2B44BD9F for ; Tue, 26 Jan 2021 04:36:07 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 21381689735; Tue, 26 Jan 2021 04:36:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 85DB268815C for ; Tue, 26 Jan 2021 04:35:59 +0200 (EET) IronPort-SDR: SfrLLqNxYF5QZDoq9li9NzO35b5tMQaoPSb7FoTtmHPMlzaUKAIJjzYoZWwXmvkzMypafe6zBT Bu2NBhJEuBIg== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015214" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015214" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:35:57 -0800 IronPort-SDR: TQxz9/Dy12E5Lxkd2DYecMA0oWRm8LJq7mOhrM+9l/mL+kHeLPF2Gg2l9Lj+HaP5zK0ifarnho UWXte/VeTUtg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361796989" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:35:56 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:04 +0800 Message-Id: <20210126023213.146185-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/10] libavcodec/qsvenc.c: add max_frame_size support for hevc_qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin The max_frame_size parameter is set only when codec is h264. Now I add hevc in that conditional statement. Signed-off-by: Wenbin Chen --- libavcodec/qsvenc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 566a5c8552..0bca4f30e3 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -691,7 +691,7 @@ FF_ENABLE_DEPRECATION_WARNINGS q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extco; #if QSV_HAVE_CO2 - if (avctx->codec_id == AV_CODEC_ID_H264) { + if (avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id == AV_CODEC_ID_HEVC) { if (q->int_ref_type >= 0) q->extco2.IntRefType = q->int_ref_type; if (q->int_ref_cycle_size >= 0) From patchwork Tue Jan 26 02:32:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25182 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4D62044BD9F for ; Tue, 26 Jan 2021 04:36:09 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2D1F268920B; Tue, 26 Jan 2021 04:36:09 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1B07B68816E for ; Tue, 26 Jan 2021 04:36:00 +0200 (EET) IronPort-SDR: oLJbg4nm+DRfUh/bugDTxmPDsn2UW1zQ72wpF+bgutjPOeVzZS5A1+jX3asipOF297qrCxPbDy HvCS+WeXv91w== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015216" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015216" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:35:59 -0800 IronPort-SDR: v5Mdr1PxWRvOhOrDADAKGL8Z+2IxleRxe31Cmg+xviMRsXhH39xBDzyeEOUIrfJveyaWDZ6Vvd m3ue1F1RKdLQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797014" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:35:58 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:05 +0800 Message-Id: <20210126023213.146185-2-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/10] libavcodec/qsvenc: add DisableDeblockingIdc support for qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin MediaSDK already has a flag to control deblocking (DisableDeblockingIdc). Add dblk_idc parameter in ffmpeg to expose this flag to user. Sigend-off-by: Wenbin Chen --- libavcodec/qsvenc.c | 6 ++++++ libavcodec/qsvenc.h | 3 +++ 2 files changed, 9 insertions(+) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 0bca4f30e3..2434feb660 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -295,6 +295,8 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, av_log(avctx, AV_LOG_VERBOSE, "FrameRateExtD: %"PRIu32"; FrameRateExtN: %"PRIu32" \n", info->FrameInfo.FrameRateExtD, info->FrameInfo.FrameRateExtN); + av_log(avctx, AV_LOG_VERBOSE, "DisableDeblockingIdc: %"PRIu32" \n", co2->DisableDeblockingIdc); + } static int select_rc_mode(AVCodecContext *avctx, QSVEncContext *q) @@ -710,6 +712,10 @@ FF_ENABLE_DEPRECATION_WARNINGS if (q->max_slice_size >= 0) q->extco2.MaxSliceSize = q->max_slice_size; #endif +#if QSV_HAVE_DISABLEDEBLOCKIDC + if(q->dblk_idc >= 0) + q->extco2.DisableDeblockingIdc = q->dblk_idc; +#endif #if QSV_HAVE_TRELLIS if (avctx->trellis >= 0) diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 6d305f87dd..3720320789 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -44,6 +44,7 @@ #define QSV_HAVE_TRELLIS QSV_VERSION_ATLEAST(1, 8) #define QSV_HAVE_MAX_SLICE_SIZE QSV_VERSION_ATLEAST(1, 9) +#define QSV_HAVE_DISABLEDEBLOCKIDC QSV_VERSION_ATLEAST(1, 9) #define QSV_HAVE_BREF_TYPE QSV_VERSION_ATLEAST(1, 8) #define QSV_HAVE_LA QSV_VERSION_ATLEAST(1, 7) @@ -97,6 +98,7 @@ { "b_strategy", "Strategy to choose between I/P/B-frames", OFFSET(qsv.b_strategy), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "forced_idr", "Forcing I frames as IDR frames", OFFSET(qsv.forced_idr), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, \ { "low_power", "enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.)", OFFSET(qsv.low_power), AV_OPT_TYPE_BOOL, { .i64 = 0}, 0, 1, VE},\ +{ "dblk_idc", "value of DisableDeblockingIdc (default is 0), in range [0,2]", OFFSET(qsv.dblk_idc), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 2, VE}, \ extern const AVCodecHWConfigInternal *const ff_qsv_enc_hw_configs[]; @@ -167,6 +169,7 @@ typedef struct QSVEncContext { int rdo; int max_frame_size; int max_slice_size; + int dblk_idc; int tile_cols; int tile_rows; From patchwork Tue Jan 26 02:32:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25183 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4DB4244BD9F for ; Tue, 26 Jan 2021 04:36:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 336CE689C62; Tue, 26 Jan 2021 04:36:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AEE1F6883D3 for ; Tue, 26 Jan 2021 04:36:03 +0200 (EET) IronPort-SDR: v1ovIo1mA9MaHSmwV8/P7w1EdReY8Vhw9Wpkp+TpF1SvDufPx9xA/fYNRaUr2TyEV54x657kHl rGzy2USA7zCw== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015217" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015217" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:02 -0800 IronPort-SDR: VuZ2USYTg8++1wmHbFx8SbGqdoYIYDcpj+19HcSpRFIRC+MEC2dO9yU4wb5S6j2XwsXdDkRXQ5 10k8HhkPfEag== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797043" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:00 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:06 +0800 Message-Id: <20210126023213.146185-3-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/10] libavcodec/qsvenc: add low latency P-pyramid support for qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Add low latency P-pyramid support for qsv, and it relates to a new command line parameter "-p_strategy". To enable this flag, user also need to set "-bf" to 0. P-strategy has two modes "1-simple" and "2-pyramid". The details of the two models refer to https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#PRefType Signed-off-by Wenbin Chen --- libavcodec/qsvenc.c | 30 ++++++++++++++++++++++++++++++ libavcodec/qsvenc.h | 2 ++ 2 files changed, 32 insertions(+) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 2434feb660..a811d06329 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -268,6 +268,14 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, case MFX_B_REF_PYRAMID: av_log(avctx, AV_LOG_VERBOSE, "pyramid"); break; default: av_log(avctx, AV_LOG_VERBOSE, "auto"); break; } + + av_log(avctx, AV_LOG_VERBOSE, "; PRefType: "); + switch (co3->PRefType) { + case MFX_P_REF_DEFAULT: av_log(avctx, AV_LOG_VERBOSE, "default"); break; + case MFX_P_REF_SIMPLE: av_log(avctx, AV_LOG_VERBOSE, "simple"); break; + case MFX_P_REF_PYRAMID: av_log(avctx, AV_LOG_VERBOSE, "pyramid"); break; + default: av_log(avctx, AV_LOG_VERBOSE, "unknown"); break; + } av_log(avctx, AV_LOG_VERBOSE, "\n"); #endif @@ -783,6 +791,28 @@ FF_ENABLE_DEPRECATION_WARNINGS #if QSV_HAVE_CO3 q->extco3.Header.BufferId = MFX_EXTBUFF_CODING_OPTION3; q->extco3.Header.BufferSz = sizeof(q->extco3); + + switch (q->p_strategy) { + case 0: + q->extco3.PRefType = MFX_P_REF_DEFAULT; + break; + case 1: + q->extco3.PRefType = MFX_P_REF_SIMPLE; + break; + case 2: + q->extco3.PRefType = MFX_P_REF_PYRAMID; + break; + default: + q->extco3.PRefType = MFX_P_REF_DEFAULT; + av_log(avctx, AV_LOG_WARNING, "invalid p_strategy, set to default\n"); + break; + } + if ((q->extco3.PRefType == MFX_P_REF_SIMPLE || + q->extco3.PRefType == MFX_P_REF_PYRAMID) && + avctx->max_b_frames != 0) { + av_log(avctx, AV_LOG_WARNING, "Please set max_b_frames(-bf) to 0 to enable P-pyramid\n"); + } + #if QSV_HAVE_GPB if (avctx->codec_id == AV_CODEC_ID_HEVC) q->extco3.GPB = q->gpb ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 3720320789..7da6e0a50f 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -95,6 +95,7 @@ { "extbrc", "Extended bitrate control", OFFSET(qsv.extbrc), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "adaptive_i", "Adaptive I-frame placement", OFFSET(qsv.adaptive_i), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "adaptive_b", "Adaptive B-frame placement", OFFSET(qsv.adaptive_b), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ +{ "p_strategy", "enable P-pyramid 0-default 1-simple 2-pyramid", OFFSET(qsv.p_strategy), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, VE }, \ { "b_strategy", "Strategy to choose between I/P/B-frames", OFFSET(qsv.b_strategy), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "forced_idr", "Forcing I frames as IDR frames", OFFSET(qsv.forced_idr), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, \ { "low_power", "enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.)", OFFSET(qsv.low_power), AV_OPT_TYPE_BOOL, { .i64 = 0}, 0, 1, VE},\ @@ -185,6 +186,7 @@ typedef struct QSVEncContext { int adaptive_i; int adaptive_b; int b_strategy; + int p_strategy; int cavlc; int int_ref_type; From patchwork Tue Jan 26 02:32:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25184 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 78D6144BD9F for ; Tue, 26 Jan 2021 04:36:13 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5ED0068A0F0; Tue, 26 Jan 2021 04:36:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7091F689E13 for ; Tue, 26 Jan 2021 04:36:05 +0200 (EET) IronPort-SDR: Vd90ljdemXRrNGIE82hIJ/zaWg0CMFaMi5hHgupRu3YcZDH6y4mwqbirV/sMfrJdu1xKMb2XGW JkoZggjMvG+g== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015222" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015222" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:04 -0800 IronPort-SDR: NoX/aS8jqlZBPLX4sFyI9+ehScLte7e8ffu/iCNBer/eFoLDhuSulfsBEnqZ0kKEfW70K0FN1M Px2Qq5nMSIMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797067" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:02 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:07 +0800 Message-Id: <20210126023213.146185-4-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/10] libavcodec/qsvdec.c: extract frame packing arrangement data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Use h264_sei to parse SEI data got from MediaSDK. Extract frame packing arrangement information from SEI data and config AVStereo3D side data for decoded frame. Sigend-off-by: Wenbin Chen --- libavcodec/qsv_internal.h | 2 + libavcodec/qsvdec.c | 149 ++++++++++++++++++++++++++++++++++++++ libavcodec/qsvdec.h | 6 ++ 3 files changed, 157 insertions(+) diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6b2fbbe252..aa3da97c99 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -52,6 +52,8 @@ #define QSV_MAX_ENC_PAYLOAD 2 // # of mfxEncodeCtrl payloads supported +#define QSV_PAYLOAD_SIZE 1024 + #define QSV_VERSION_ATLEAST(MAJOR, MINOR) \ (MFX_VERSION_MAJOR > (MAJOR) || \ MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index d10f90a0db..5cb49cfb4a 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -36,10 +36,12 @@ #include "libavutil/pixfmt.h" #include "libavutil/time.h" #include "libavutil/imgutils.h" +#include "libavutil/stereo3d.h" #include "avcodec.h" #include "internal.h" #include "decode.h" +#include "get_bits.h" #include "qsv.h" #include "qsv_internal.h" #include "qsvdec.h" @@ -398,6 +400,147 @@ static QSVFrame *find_frame(QSVContext *q, mfxFrameSurface1 *surf) return NULL; } +static int h264_decode_fpa(H264SEIFramePacking *fpa, AVFrame *frame) +{ + if (!fpa || !frame) { + return AVERROR(EINVAL); + } + + if (!fpa->arrangement_cancel_flag && + fpa->arrangement_type <= 6 && + fpa->content_interpretation_type > 0 && + fpa->content_interpretation_type < 3) { + AVStereo3D *stereo = av_stereo3d_create_side_data(frame); + if (stereo) { + switch (fpa->arrangement_type) { + case 0: + stereo->type = AV_STEREO3D_CHECKERBOARD; + break; + case 1: + stereo->type = AV_STEREO3D_COLUMNS; + break; + case 2: + stereo->type = AV_STEREO3D_LINES; + break; + case 3: + if (fpa->quincunx_sampling_flag) + stereo->type = AV_STEREO3D_SIDEBYSIDE_QUINCUNX; + else + stereo->type = AV_STEREO3D_SIDEBYSIDE; + break; + case 4: + stereo->type = AV_STEREO3D_TOPBOTTOM; + break; + case 5: + stereo->type = AV_STEREO3D_FRAMESEQUENCE; + if (fpa->current_frame_is_frame0_flag) + stereo->view = AV_STEREO3D_VIEW_LEFT; + else + stereo->view = AV_STEREO3D_VIEW_RIGHT; + break; + case 6: + stereo->type = AV_STEREO3D_2D; + break; + } + + if (fpa->content_interpretation_type == 2) + stereo->flags = AV_STEREO3D_FLAG_INVERT; + } + } + return 0; +} + +static int h264_parse_side_data(AVCodecContext *avctx, QSVContext *q, AVFrame *frame) +{ + GetBitContext gb_payload; + uint8_t *sei_buffer; + int sei_buffer_index; + int ret; + + /* remove emulation prevention bytes */ + sei_buffer = (uint8_t *)av_mallocz(q->payload.NumBit / 8); + if (!sei_buffer) { + av_freep(&sei_buffer); + return AVERROR(ENOMEM); + } + sei_buffer_index = 0; + for (int i = 0; i < q->payload.NumBit / 8; i++) { + if (q->payload.Data[i] == 3) + i++; + sei_buffer[sei_buffer_index] = q->payload.Data[i]; + sei_buffer_index += 1; + } + + ret = init_get_bits8(&gb_payload, sei_buffer, sei_buffer_index+1); + if (ret < 0) { + av_freep(&sei_buffer); + return ret; + } + + ret = ff_h264_sei_decode(&q->sei, &gb_payload, &q->ps, avctx); + if (ret < 0) { + av_freep(&sei_buffer); + return ret; + } + + switch (q->payload.Type) { + case SEI_TYPE_FRAME_PACKING_ARRANGEMENT: + ret = h264_decode_fpa(&q->sei.frame_packing, frame); + break; + default: + break; + } + + av_freep(&sei_buffer); + return ret; +} + +static int extract_frame_side_data(AVCodecContext *avctx, QSVContext *q, AVFrame *frame) +{ + mfxU64 ts; + mfxStatus sts; + int ret; + + if (q->payload.BufSize == 0) { + q->payload.Data = av_mallocz(QSV_PAYLOAD_SIZE); + if (!q->payload.Data) { + av_freep(&q->payload.Data); + return AVERROR(ENOMEM); + } + q->payload.BufSize = QSV_PAYLOAD_SIZE; + } + + sts = MFX_ERR_NONE; + while (sts == MFX_ERR_NONE) { + + sts = MFXVideoDECODE_GetPayload(q->session, &ts, &q->payload); + + if (sts == MFX_ERR_NOT_ENOUGH_BUFFER) { + av_log(avctx, AV_LOG_WARNING, "Space for SEI is not enough. One SEI will be skipped\n"); + continue; + } else if (sts != MFX_ERR_NONE || q->payload.NumBit == 0) { + break; + } + + if (q->payload.Type != SEI_TYPE_FRAME_PACKING_ARRANGEMENT) + continue; + + switch (avctx->codec_id) { + case AV_CODEC_ID_H264: + ret = h264_parse_side_data(avctx, q, frame); + break; + default: + break; + } + + if (ret < 0) { + av_log(avctx, AV_LOG_WARNING, "parse side data failed\n"); + break; + } + } + return ret; +} + static int qsv_decode(AVCodecContext *avctx, QSVContext *q, AVFrame *frame, int *got_frame, AVPacket *avpkt) @@ -501,6 +644,10 @@ static int qsv_decode(AVCodecContext *avctx, QSVContext *q, outsurf = &out_frame->surface; + ret = extract_frame_side_data(avctx, q, frame); + if (ret < 0) + av_log(avctx, AV_LOG_WARNING, "Extracting side from packet failed\n"); + #if FF_API_PKT_PTS FF_DISABLE_DEPRECATION_WARNINGS frame->pkt_pts = outsurf->Data.TimeStamp; @@ -564,6 +711,8 @@ int ff_qsv_decode_close(QSVContext *q) av_buffer_unref(&q->frames_ctx.mids_buf); av_buffer_pool_uninit(&q->pool); + av_freep(&q->payload.Data); + return 0; } diff --git a/libavcodec/qsvdec.h b/libavcodec/qsvdec.h index f3b7344cba..282943b00c 100644 --- a/libavcodec/qsvdec.h +++ b/libavcodec/qsvdec.h @@ -34,6 +34,7 @@ #include "avcodec.h" #include "hwconfig.h" +#include "h264_sei.h" #include "qsv_internal.h" typedef struct QSVContext { @@ -70,8 +71,13 @@ typedef struct QSVContext { char *load_plugins; + mfxPayload payload; + mfxExtBuffer **ext_buffers; int nb_ext_buffers; + + H264SEIContext sei; + H264ParamSets ps; } QSVContext; extern const AVCodecHWConfigInternal *const ff_qsv_hw_configs[]; From patchwork Tue Jan 26 02:32:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25185 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B387644BD9F for ; Tue, 26 Jan 2021 04:36:14 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9510568A132; Tue, 26 Jan 2021 04:36:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 10572689E13 for ; Tue, 26 Jan 2021 04:36:06 +0200 (EET) IronPort-SDR: lDuWWg/4c0h+TTc7RoRgkASRjuglCwXkpFrLtmt+FfDENQdKPpvbO/iQlrtqAPZ1AdD6M1uCWG 0OcmUww9pzBw== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015223" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015223" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:06 -0800 IronPort-SDR: /jd5s66lD1GEb7KPVk1SzlWr5pOxd3NRC/nYE8FZYRHHLXzaRLmrTD1AvIjBUUnFv0PPCC4OAj 7R0Ka9poRlSA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797094" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:04 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:08 +0800 Message-Id: <20210126023213.146185-5-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/10] libavcodec/qsvenc: Add transform skip for hevc_qsv X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Add transform skip option for hevc_qsv. Command line option is "transform_skip". By enabling this option, the transform_skip_enabled_flag in PPS will be set to 1. This option is only supported in the platform newer than ICL. Signed-off-by Wenbin Chen --- libavcodec/qsvenc.c | 10 ++++++++-- libavcodec/qsvenc.h | 1 + libavcodec/qsvenc_hevc.c | 3 +++ 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index a811d06329..f2fcd09ae0 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -304,7 +304,9 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, info->FrameInfo.FrameRateExtD, info->FrameInfo.FrameRateExtN); av_log(avctx, AV_LOG_VERBOSE, "DisableDeblockingIdc: %"PRIu32" \n", co2->DisableDeblockingIdc); - +#if QSV_VERSION_ATLEAST(1, 26) + av_log(avctx, AV_LOG_VERBOSE, "TransformSkip: %s \n", print_threestate(co3->TransformSkip)); +#endif } static int select_rc_mode(AVCodecContext *avctx, QSVEncContext *q) @@ -813,10 +815,14 @@ FF_ENABLE_DEPRECATION_WARNINGS av_log(avctx, AV_LOG_WARNING, "Please set max_b_frames(-bf) to 0 to enable P-pyramid\n"); } + if (avctx->codec_id == AV_CODEC_ID_HEVC) { +#if QSV_VERSION_ATLEAST(1, 26) + q->extco3.TransformSkip = q->transform_skip ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; +#endif #if QSV_HAVE_GPB - if (avctx->codec_id == AV_CODEC_ID_HEVC) q->extco3.GPB = q->gpb ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; #endif + } q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extco3; #endif } diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 7da6e0a50f..d1de914724 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -197,6 +197,7 @@ typedef struct QSVEncContext { int repeat_pps; int low_power; int gpb; + int transform_skip; int a53_cc; diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 347f30655e..07a7e3b2be 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -246,6 +246,9 @@ static const AVOption options[] = { { "tile_cols", "Number of columns for tiled encoding", OFFSET(qsv.tile_cols), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, { "tile_rows", "Number of rows for tiled encoding", OFFSET(qsv.tile_rows), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, +#if QSV_VERSION_ATLEAST(1, 26) + { "transform_skip", "Turn this option ON to enable transformskip", OFFSET(qsv.transform_skip), AV_OPT_TYPE_INT, { .i64 = -1}, -1, 1, VE}, +#endif { NULL }, }; From patchwork Tue Jan 26 02:32:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25187 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 15D5C44BD9F for ; Tue, 26 Jan 2021 04:36:18 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF78468A2B1; Tue, 26 Jan 2021 04:36:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B1CA268A1D4 for ; Tue, 26 Jan 2021 04:36:09 +0200 (EET) IronPort-SDR: RXqPDqspkG5P5bj+wj1is3AcfKLi2qdPj+AUr4ilW2s2RxZ51L0syy4aH+Ut3QI/fN4qGreEsM xEBgDE2UpSfg== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015226" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015226" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:08 -0800 IronPort-SDR: vBtLia5WpPJY/CH4aXvqUfgwtfFq1oYsG852cFUHBvgSkQr2WFbZS7XNK+3vY03lp9rS3pL4ya WpfQ5hAcYaUw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797117" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:07 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:09 +0800 Message-Id: <20210126023213.146185-6-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/10] libavcodec/qsvenc.c: add ROI support to qsv encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Use the mfxEncoderCtrl parameter to enable ROI. Get side data "AVRegionOfInterest" from filter "addroi" and use it to configure "mfxExtEncoderROI" which is the MediaSDK's ROI configuration. It is the first time to use encoderCtrl feature in ffmpeg-qsv, so add allocate and free process as well. Sigend-off-by: Wenbin Chen --- libavcodec/qsv_internal.h | 3 ++ libavcodec/qsvenc.c | 74 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+) diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index aa3da97c99..d846a2741a 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -51,6 +51,9 @@ #define ASYNC_DEPTH_DEFAULT 4 // internal parallelism #define QSV_MAX_ENC_PAYLOAD 2 // # of mfxEncodeCtrl payloads supported +#define QSV_MAX_ENC_EXTPARAM 2 + +#define QSV_MAX_ROI_NUM 256 #define QSV_PAYLOAD_SIZE 1024 diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index f2fcd09ae0..addc61bf89 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1291,6 +1291,13 @@ static void clear_unused_frames(QSVEncContext *q) QSVFrame *cur = q->work_frames; while (cur) { if (cur->used && !cur->surface.Data.Locked) { + + for (int i = 0; i < cur->enc_ctrl.NumExtParam; i++) { + if (cur->enc_ctrl.ExtParam[i]) + av_freep(&(cur->enc_ctrl.ExtParam[i])); + } + cur->enc_ctrl.NumExtParam = 0; + free_encoder_ctrl_payloads(&cur->enc_ctrl); if (cur->frame->format == AV_PIX_FMT_QSV) { av_frame_unref(cur->frame); @@ -1333,6 +1340,12 @@ static int get_free_frame(QSVEncContext *q, QSVFrame **f) av_freep(&frame); return AVERROR(ENOMEM); } + frame->enc_ctrl.ExtParam = av_mallocz(sizeof(mfxExtBuffer*) * QSV_MAX_ENC_EXTPARAM); + if (!frame->enc_ctrl.ExtParam) { + av_free(frame->enc_ctrl.Payload); + av_freep(&frame); + return AVERROR(ENOMEM); + } *last = frame; *f = frame; @@ -1436,6 +1449,7 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, const AVFrame *frame) { AVPacket new_pkt = { 0 }; + AVFrameSideData *sd; mfxBitstream *bs; #if QSV_VERSION_ATLEAST(1, 26) mfxExtAVCEncodedFrameInfo *enc_info; @@ -1446,6 +1460,7 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, mfxSyncPoint *sync = NULL; QSVFrame *qsv_frame = NULL; mfxEncodeCtrl* enc_ctrl = NULL; + mfxExtEncoderROI *enc_roi = NULL; int ret; if (frame) { @@ -1480,6 +1495,8 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, bs->Data = new_pkt.data; bs->MaxLength = new_pkt.size; + enc_info = NULL; + enc_buf = NULL; #if QSV_VERSION_ATLEAST(1, 26) if (avctx->codec_id == AV_CODEC_ID_H264) { enc_info = av_mallocz(sizeof(*enc_info)); @@ -1502,6 +1519,62 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, q->set_encode_ctrl_cb(avctx, frame, &qsv_frame->enc_ctrl); } +#if QSV_VERSION_ATLEAST(1, 22) + sd = NULL; + if (frame) { + sd = av_frame_get_side_data(frame, + AV_FRAME_DATA_REGIONS_OF_INTEREST); + } + if (sd) { + AVRegionOfInterest *roi; + uint32_t roi_size; + int nb_roi, i; + + enc_roi = (mfxExtEncoderROI *)av_mallocz(sizeof(*enc_roi)); + if (!enc_roi) { + av_packet_unref(&new_pkt); + if (bs) + av_freep(bs); + if (enc_info) + av_freep(&enc_info); + if (enc_buf) + av_freep(&enc_buf); + return AVERROR(ENOMEM); + } + roi = (AVRegionOfInterest *)sd->data; + roi_size = roi->self_size; + av_assert0(roi_size && sd->size % roi_size == 0); + nb_roi = sd->size / roi_size; + if (nb_roi > QSV_MAX_ROI_NUM) { + av_log(avctx, AV_LOG_WARNING, "More ROIs set than " + "supported by driver (%d > %d).\n", + nb_roi, QSV_MAX_ROI_NUM); + nb_roi = QSV_MAX_ROI_NUM; + } + enc_roi->Header.BufferId = MFX_EXTBUFF_ENCODER_ROI; + enc_roi->Header.BufferSz = sizeof(*enc_roi); + enc_roi->NumROI = nb_roi; + enc_roi->ROIMode = MFX_ROI_MODE_QP_DELTA; + // For overlapping regions, the first in the array takes priority. + for (i = 0; i < nb_roi; i++) { + roi = (AVRegionOfInterest*)(sd->data + roi_size * i); + av_assert0(roi->qoffset.den != 0); + + enc_roi->ROI[i].Top = roi->top & 0xfffffff0; + enc_roi->ROI[i].Bottom = roi->bottom & 0xfffffff0; + enc_roi->ROI[i].Left = (roi->left & 0xfffffff0) + 0x10; + enc_roi->ROI[i].Right = (roi->right & 0xfffffff0) + 0x10; + enc_roi->ROI[i].DeltaQP = + roi->qoffset.num * 51 / roi->qoffset.den; + av_log(avctx, AV_LOG_DEBUG, "ROI: (%d,%d)-(%d,%d) -> %+d.\n", + roi->top, roi->left, roi->bottom, roi->right, + enc_roi->ROI[i].DeltaQP); + } + enc_ctrl->ExtParam[enc_ctrl->NumExtParam] = (mfxExtBuffer *)enc_roi; + enc_ctrl->NumExtParam += 1; + } +#endif + sync = av_mallocz(sizeof(*sync)); if (!sync) { av_freep(&bs); @@ -1670,6 +1743,7 @@ int ff_qsv_enc_close(AVCodecContext *avctx, QSVEncContext *q) q->work_frames = cur->next; av_frame_free(&cur->frame); av_free(cur->enc_ctrl.Payload); + av_free(cur->enc_ctrl.ExtParam); av_freep(&cur); cur = q->work_frames; } From patchwork Tue Jan 26 02:32:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25186 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 045F844BD9F for ; Tue, 26 Jan 2021 04:36:16 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DE93068A148; Tue, 26 Jan 2021 04:36:15 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 365B7689E8A for ; Tue, 26 Jan 2021 04:36:11 +0200 (EET) IronPort-SDR: 5VVkfpM4FuUS9CK5hsS3fy32BZThEmI8UIeYRB6jfM0wNQbfpHqT2QOJpAFc87ToerimpuE6Vu yz454DG4oJ+g== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015231" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015231" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:11 -0800 IronPort-SDR: aMFH2TdnvR0B/7vU2BX6j2N3ELeD7pbit3vamfQX/SBSsM8izRjehcwL3fDmXGE4G+KzjAFQTD 7kTUnO6bMk8A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797140" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:08 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:10 +0800 Message-Id: <20210126023213.146185-7-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/10] libavocdec/qsvenc_hevc: encode RGB format rawvideo X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Add support for hevc_qsv to input RGB format frame. It will transform frame to yuv inside MediaSDK instead of using auto scale. Hevc_qsv supports directly encoding BGRA and X2RGB10Le format. X2RGB10Le is only supported in VDENC (-low_power 1). The X2RGB10Le correspond to the A2RGB20 format in MediaSDK, and for A2RGB10 the two MSBs is A channels while this two bits in X2RGB10Le is undefined. MediaSDK team is discussing about this, so I leave an interface here. Sigend-of-by: Wenbin Chen --- libavcodec/qsv.c | 16 ++++++++++++++++ libavcodec/qsvenc.c | 19 +++++++++++++++++-- libavcodec/qsvenc_hevc.c | 6 ++++++ 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 6e3154e1a3..7ade53e278 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -196,6 +196,12 @@ int ff_qsv_print_warning(void *log_ctx, mfxStatus err, enum AVPixelFormat ff_qsv_map_fourcc(uint32_t fourcc) { switch (fourcc) { +#if QSV_VERSION_ATLEAST(1, 9) + case MFX_FOURCC_A2RGB10: return AV_PIX_FMT_X2RGB10LE; +#endif +#if QSV_VERSION_ATLEAST(1, 17) + case MFX_FOURCC_RGB4: return AV_PIX_FMT_BGRA; +#endif case MFX_FOURCC_NV12: return AV_PIX_FMT_NV12; case MFX_FOURCC_P010: return AV_PIX_FMT_P010; case MFX_FOURCC_P8: return AV_PIX_FMT_PAL8; @@ -222,6 +228,16 @@ int ff_qsv_map_pixfmt(enum AVPixelFormat format, uint32_t *fourcc) *fourcc = MFX_FOURCC_P010; return AV_PIX_FMT_P010; #if CONFIG_VAAPI +#if QSV_VERSION_ATLEAST(1, 9) + case AV_PIX_FMT_BGRA: + *fourcc = MFX_FOURCC_RGB4; + return AV_PIX_FMT_BGRA; +#endif +#if QSV_VERSION_ATLEAST(1, 17) + case AV_PIX_FMT_X2RGB10LE: + *fourcc = MFX_FOURCC_A2RGB10; + return AV_PIX_FMT_X2RGB10LE; +#endif case AV_PIX_FMT_YUV422P: case AV_PIX_FMT_YUYV422: *fourcc = MFX_FOURCC_YUY2; diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index addc61bf89..4b345aca0a 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -613,6 +613,13 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) brc_param_multiplier = (FFMAX(FFMAX3(target_bitrate_kbps, max_bitrate_kbps, buffer_size_in_kilobytes), initial_delay_in_kilobytes) + 0x10000) / 0x10000; +#if QSV_VERSION_ATLEAST(1, 9) + if (sw_format == AV_PIX_FMT_X2RGB10LE && q->low_power != 1) { + av_log(avctx, AV_LOG_ERROR, "Only VDENC support encoding x2rgb10\n"); + return AVERROR(EINVAL); + } +#endif + switch (q->param.mfx.RateControlMethod) { case MFX_RATECONTROL_CBR: case MFX_RATECONTROL_VBR: @@ -1422,8 +1429,16 @@ static int submit_frame(QSVEncContext *q, const AVFrame *frame, qf->surface.Info.PicStruct |= MFX_PICSTRUCT_FRAME_TRIPLING; qf->surface.Data.PitchLow = qf->frame->linesize[0]; - qf->surface.Data.Y = qf->frame->data[0]; - qf->surface.Data.UV = qf->frame->data[1]; + if (frame->format == AV_PIX_FMT_X2RGB10LE || + frame->format == AV_PIX_FMT_BGRA) { + qf->surface.Data.B = qf->frame->data[0]; + qf->surface.Data.G = qf->frame->data[0] + 1; + qf->surface.Data.R = qf->frame->data[0] + 2; + qf->surface.Data.A = qf->frame->data[0] + 3; + } else { + qf->surface.Data.Y = qf->frame->data[0]; + qf->surface.Data.UV = qf->frame->data[1]; + } } qf->surface.Data.TimeStamp = av_rescale_q(frame->pts, q->avctx->time_base, (AVRational){1, 90000}); diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 07a7e3b2be..18e110d140 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -289,6 +289,12 @@ AVCodec ff_hevc_qsv_encoder = { .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12, AV_PIX_FMT_P010, AV_PIX_FMT_QSV, +#if QSV_VERSION_ATLEAST(1, 17) + AV_PIX_FMT_BGRA, +#endif +#if QSV_VERSION_ATLEAST(1, 9) + AV_PIX_FMT_X2RGB10LE, +#endif AV_PIX_FMT_NONE }, .priv_class = &class, .defaults = qsv_enc_defaults, From patchwork Tue Jan 26 02:32:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25188 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EDDA144BD9F for ; Tue, 26 Jan 2021 04:36:18 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D34B468A30C; Tue, 26 Jan 2021 04:36:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A2D03689E29 for ; Tue, 26 Jan 2021 04:36:15 +0200 (EET) IronPort-SDR: vLSIs62cLN0ERjAamEIRx4a4PD2LRy2O0HLW/aj6yeuGEE48Yb/nFkMejVWeqS87M88eb0WkR6 BLPfEAJgRCww== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015236" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015236" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:14 -0800 IronPort-SDR: pIcPDmj0quL1UuBDl7SbNJde3iwJOf/9VTdexxAsx1THhFyQTprqZNlEpFt85/SSXoRxzCKByL U7UQa+/VsOKQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797176" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:12 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:11 +0800 Message-Id: <20210126023213.146185-8-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/10] libavcodec/qsvenc: add horizontal intra refresh and refresh cycle dist X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Add an new intra refresh type: "horizontal", and an new param ref_cycle_dist. This param specify the distance between the beginnings of the intra-refresh cycles in frames. Signed-off-by: Wenbin Chen --- libavcodec/qsvenc.c | 13 +++++++++++-- libavcodec/qsvenc.h | 1 + libavcodec/qsvenc_h264.c | 5 ++++- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 4b345aca0a..d16f1094c8 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -217,8 +217,10 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, #if QSV_HAVE_CO2 av_log(avctx, AV_LOG_VERBOSE, - "RecoveryPointSEI: %s IntRefType: %"PRIu16"; IntRefCycleSize: %"PRIu16"; IntRefQPDelta: %"PRId16"\n", - print_threestate(co->RecoveryPointSEI), co2->IntRefType, co2->IntRefCycleSize, co2->IntRefQPDelta); + "RecoveryPointSEI: %s IntRefType: %"PRIu16"; IntRefCycleSize: %"PRIu16 + "; IntRefQPDelta: %"PRId16"; IntRefCycleDist: %"PRId16"\n", + print_threestate(co->RecoveryPointSEI), co2->IntRefType, co2->IntRefCycleSize, + co2->IntRefQPDelta, co3->IntRefCycleDist); av_log(avctx, AV_LOG_VERBOSE, "MaxFrameSize: %d; ", co2->MaxFrameSize); #if QSV_HAVE_MAX_SLICE_SIZE @@ -822,6 +824,13 @@ FF_ENABLE_DEPRECATION_WARNINGS av_log(avctx, AV_LOG_WARNING, "Please set max_b_frames(-bf) to 0 to enable P-pyramid\n"); } +#if QSV_VERSION_ATLEAST(1, 16) + if (avctx->codec_id == AV_CODEC_ID_H264) { + if (q->int_ref_cycle_dist) + q->extco3.IntRefCycleDist = q->int_ref_cycle_dist; + } +#endif + if (avctx->codec_id == AV_CODEC_ID_HEVC) { #if QSV_VERSION_ATLEAST(1, 26) q->extco3.TransformSkip = q->transform_skip ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index d1de914724..99befefde1 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -192,6 +192,7 @@ typedef struct QSVEncContext { int int_ref_type; int int_ref_cycle_size; int int_ref_qp_delta; + int int_ref_cycle_dist; int recovery_point_sei; int repeat_pps; diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index ddafc45ec3..47583621b4 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -129,10 +129,13 @@ static const AVOption options[] = { { "int_ref_type", "Intra refresh type", OFFSET(qsv.int_ref_type), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE, "int_ref_type" }, { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, .flags = VE, "int_ref_type" }, { "vertical", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, .flags = VE, "int_ref_type" }, + { "horizontal", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 2 }, .flags = VE, "int_ref_type" }, { "int_ref_cycle_size", "Number of frames in the intra refresh cycle", OFFSET(qsv.int_ref_cycle_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE }, { "int_ref_qp_delta", "QP difference for the refresh MBs", OFFSET(qsv.int_ref_qp_delta), AV_OPT_TYPE_INT, { .i64 = INT16_MIN }, INT16_MIN, INT16_MAX, VE }, { "recovery_point_sei", "Insert recovery point SEI messages", OFFSET(qsv.recovery_point_sei), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, - +#if QSV_VERSION_ATLEAST(1, 16) + { "int_ref_cycle_dist", "Distance between the beginnings of the intra-refresh cycles in frames", OFFSET(qsv.int_ref_cycle_dist), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT16_MAX, VE }, +#endif { "profile", NULL, OFFSET(qsv.profile), AV_OPT_TYPE_INT, { .i64 = MFX_PROFILE_UNKNOWN }, 0, INT_MAX, VE, "profile" }, { "unknown" , NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_PROFILE_UNKNOWN }, INT_MIN, INT_MAX, VE, "profile" }, { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_PROFILE_AVC_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, From patchwork Tue Jan 26 02:32:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25189 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C907744BD9F for ; Tue, 26 Jan 2021 04:36:19 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B06D068A255; Tue, 26 Jan 2021 04:36:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2347768A2C6 for ; Tue, 26 Jan 2021 04:36:17 +0200 (EET) IronPort-SDR: 4AbB2YdeHHSgUQ78uifUJsS54/3F6diaeVxbPrbGEddzgrjyWuIEO6AAGfIM9qISuk6VEOtrTR 9B4dzy8ZeUEA== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015240" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015240" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:16 -0800 IronPort-SDR: riFwX20LfkNZUojHMMAAJZcg539xAcauzodlMNwWIS9cYXZiZ0SAH9xTH+xc6ZPyGLoT6WraPQ LVdJXGpfLCUg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797202" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:15 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:12 +0800 Message-Id: <20210126023213.146185-9-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/10] libavcodec/qsvdec: using suggested num to set init_pool_size X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin The init_pool_size is set to be 64 and it is too many. Use IOSurfQuery to get NumFrameSuggest which is the suggested number of frame that needed to be allocated when initial the decoder Considering the hevc_qsv encoder's async is 4 (default) and max_b_frames is 8 (default) and decoder may followed by VPP, use NumFrameSuggest + 16 to set init_pool_size. Sigend-off-by Wenbin Chen --- fftools/ffmpeg_qsv.c | 9 ++++++++- libavcodec/qsvdec.c | 28 +++++++++++++++++++++++++++- 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c index 960c88b69d..3862dc1e7d 100644 --- a/fftools/ffmpeg_qsv.c +++ b/fftools/ffmpeg_qsv.c @@ -74,6 +74,7 @@ int qsv_init(AVCodecContext *s) InputStream *ist = s->opaque; AVHWFramesContext *frames_ctx; AVQSVFramesContext *frames_hwctx; + int suggest_pool_size; int ret; if (!hw_device_ctx) { @@ -82,6 +83,12 @@ int qsv_init(AVCodecContext *s) return ret; } + suggest_pool_size = 0; + if (ist->hw_frames_ctx) { + frames_ctx = (AVHWFramesContext *)ist->hw_frames_ctx->data; + suggest_pool_size = frames_ctx->initial_pool_size; + } + av_buffer_unref(&ist->hw_frames_ctx); ist->hw_frames_ctx = av_hwframe_ctx_alloc(hw_device_ctx); if (!ist->hw_frames_ctx) @@ -94,7 +101,7 @@ int qsv_init(AVCodecContext *s) frames_ctx->height = FFALIGN(s->coded_height, 32); frames_ctx->format = AV_PIX_FMT_QSV; frames_ctx->sw_format = s->sw_pix_fmt; - frames_ctx->initial_pool_size = 64 + s->extra_hw_frames; + frames_ctx->initial_pool_size = suggest_pool_size + 16 + s->extra_hw_frames; frames_hwctx->frame_type = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET; ret = av_hwframe_ctx_init(ist->hw_frames_ctx); diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 5cb49cfb4a..fba7d9cee6 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -720,6 +720,7 @@ int ff_qsv_process_data(AVCodecContext *avctx, QSVContext *q, AVFrame *frame, int *got_frame, AVPacket *pkt) { int ret; + int first_init; mfxVideoParam param = { 0 }; enum AVPixelFormat pix_fmt = AV_PIX_FMT_NV12; @@ -738,12 +739,20 @@ int ff_qsv_process_data(AVCodecContext *avctx, QSVContext *q, if (!avctx->coded_height) avctx->coded_height = 720; + first_init = 0; + if (!q->session) + first_init = 1; + ret = qsv_decode_header(avctx, q, pkt, pix_fmt, ¶m); if (ret >= 0 && (q->orig_pix_fmt != ff_qsv_map_fourcc(param.mfx.FrameInfo.FourCC) || avctx->coded_width != param.mfx.FrameInfo.Width || - avctx->coded_height != param.mfx.FrameInfo.Height)) { + avctx->coded_height != param.mfx.FrameInfo.Height || + first_init == 1)) { AVPacket zero_pkt = {0}; + mfxFrameAllocRequest request; + AVHWFramesContext *hw_frames_ctx; + memset(&request, 0, sizeof(request)); if (q->buffered_count) { q->reinit_flag = 1; @@ -758,6 +767,23 @@ int ff_qsv_process_data(AVCodecContext *avctx, QSVContext *q, avctx->coded_width = param.mfx.FrameInfo.Width; avctx->coded_height = param.mfx.FrameInfo.Height; + if (first_init == 0) { + hw_frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; + hw_frames_ctx->initial_pool_size = 0; + ret = qsv_decode_preinit(avctx, q, pix_fmt, ¶m); + if (ret < 0) + goto reinit_fail; + } + + ret = MFXVideoDECODE_QueryIOSurf(q->session, ¶m, &request); + if (ret < 0) + return ff_qsv_print_error(avctx, ret, "Error querying IO surface"); + + if (avctx->hw_frames_ctx) { + hw_frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data; + hw_frames_ctx->initial_pool_size = request.NumFrameSuggested; + } + ret = qsv_decode_preinit(avctx, q, pix_fmt, ¶m); if (ret < 0) goto reinit_fail; From patchwork Tue Jan 26 02:32:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenbin Chen X-Patchwork-Id: 25190 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2010D44BD9F for ; Tue, 26 Jan 2021 04:36:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 048AD68A33F; Tue, 26 Jan 2021 04:36:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 42B9368A2BB for ; Tue, 26 Jan 2021 04:36:19 +0200 (EET) IronPort-SDR: 7FbL69pOsqp95Ev4ZGAo8JERN3+i3SZscfrkMt2Qvk9DSCD18qd9DxCCKDQEv+zT3A/hXP1EC/ lfWowMGGYjfg== X-IronPort-AV: E=McAfee;i="6000,8403,9875"; a="159015241" X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="159015241" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Jan 2021 18:36:18 -0800 IronPort-SDR: 8qkA7nV1ZBQDbYqMzg7qPFOcVcmIVy+XeuS55YuqOoIvKGJ7FfXqs0S5oG8LSInE9J1/5nO0tF EtoNBRAZAqSg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,375,1602572400"; d="scan'208";a="361797239" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.3]) by fmsmga008.fm.intel.com with ESMTP; 25 Jan 2021 18:36:17 -0800 From: wenbin.chen@intel.com To: ffmpeg-devel@ffmpeg.org Date: Tue, 26 Jan 2021 10:32:13 +0800 Message-Id: <20210126023213.146185-10-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126023213.146185-1-wenbin.chen@intel.com> References: <20210126023213.146185-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/10] libavcodec/qsvdec: move unref before get_format X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wenbinc-Bin Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Wenbinc-Bin Qsv decoder needs to call preinit() twice to get NumFrameSuggest to config init_pool_size. The hw_frame_ctx is released at the second time calling preinit(). The hw_frame_ctx is AVBufferRef, so it release the source when the last unref is called. When the old context is released, the new one is already created, and somehow the device memory is stil be taken, even if the destory memory function is called. Moving unref before creating new context can solve this problem. Signed-off-by Wenbin Chen --- libavcodec/qsvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index fba7d9cee6..23494c217d 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -179,6 +179,8 @@ static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel pix_fmt, /* system memory format obtained from bitstream parser */ AV_PIX_FMT_NONE }; + av_buffer_unref(&q->frames_ctx.mids_buf); + av_buffer_unref(&q->frames_ctx.hw_frames_ctx); ret = ff_get_format(avctx, pix_fmts); if (ret < 0) { q->orig_pix_fmt = avctx->pix_fmt = AV_PIX_FMT_NONE;