From patchwork Thu Oct 13 08:25:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 38713 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp225657pzb; Thu, 13 Oct 2022 01:30:10 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6wYQ68t5OF66vdd+b1JAGWtXy5rF51cTFoopLtCFWBV2Wl3unBesTE+7excB9554pXYYuf X-Received: by 2002:a05:6402:448b:b0:43b:5ec6:8863 with SMTP id er11-20020a056402448b00b0043b5ec68863mr30721432edb.377.1665649810343; Thu, 13 Oct 2022 01:30:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665649810; cv=none; d=google.com; s=arc-20160816; b=Sc/qAJqiu3KPpg3QnoLGEou0GGJ2/TUtzIrS+OE2Iaz5ySbY6EQn/JOQfLpAtaEa4C 9VEcZEflkS0RRW9y0PeBZ01Jpi2wjMXkJ67CAoSSvyyEQ93Tr75+cj4VF67AyBBDRGYd uOERTytMOHg8j0xQgo3xs0OyEHgSnLC/NDr8Q20sScpGDWsrmSpwxEdjrlLvBidHHjSc +Q563OfaGn0ZVwSuQA8BP7G6+TlFbDcpRuxmOBEJ2vVVALe8j4RE2L9t8UH30WxvZTXg aNfjMI5b1vhEKH7ZbR9S+GW9w2dhqlwfjdR3OeV65J3XuWjhSj703hyvPQA4ZCtPID7g 8LnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=qnkqo5kkwTibxTGIvY8DT6q4On2dH9lWGNLnsSjToIc=; b=ZwykHttoMQWkI4OlDfo93R5mwySBMLWSOJI++b6j8ctvfCozEwY2KM4bOyPPxjUNyS Xcvx3tl1JcXi00onIhHoTSV/ABeAnqW0NRdvbbmP4l7K9HBguSwmlXBhQoVGTNGmNMZg Ri9a8OXLBjtAJku3+s/qzjGndsKpRL0YyU/I+Hi2/izLXUir/5QyAFH/KiIC0XT3XWvT FBLXmsMKTfY0Wig5/zuYbQkAXu1zXo0E3BUUPUXLqSGOm2UZyQLeU4vs+Q6vvda6VbJ4 /xxGmJPGMybLuyyjYLantUXInocJ4d5I16oakZnZsD8LH9Yj8HzU20ZQ+vsYNj0NrSKj RcjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="gbWPvrC/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jg35-20020a170907972300b007879bb732c6si17974591ejc.712.2022.10.13.01.30.09; Thu, 13 Oct 2022 01:30:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="gbWPvrC/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4319168BD3D; Thu, 13 Oct 2022 11:30:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 536AD68BC0B for ; Thu, 13 Oct 2022 11:29:59 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665649804; x=1697185804; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=d3ZKBnHcWgg09W0dc4ZKB64Ur3VxmH78LRXYDRKmISk=; b=gbWPvrC/2IGTU+WuTkEWB72uKYvM9Y+aNfXfPHh4524p9p73qnbJFolj l9BVjGX+G6H81Qp5Aja5Vn6jEWMQzFBfIHw+9GW8wYSFcQhneYtXYUU6e 1UHFuZ4RMRRXVvqaaskam7qhMLLCJC7C8jgT2/owaogggxeX4XXJPVAKZ FfQ4FqnO3GnFX1zH/0vVo56y9/WkIzyaTFWzJ/Iv5Rm5zbXJKwNkV8k4f G+StHKumT/cEUuDSlpLZdDkvTF/jDJFXTa1VEyewYJqcge9r3Zb1nIexk pwXoZti28UhhpygaYppj4ak5nBR4URTICx5dd6qhOeE3K/jLHXbcrSrvo w==; X-IronPort-AV: E=McAfee;i="6500,9779,10498"; a="303760625" X-IronPort-AV: E=Sophos;i="5.95,180,1661842800"; d="scan'208";a="303760625" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Oct 2022 01:29:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10498"; a="658095401" X-IronPort-AV: E=Sophos;i="5.95,180,1661842800"; d="scan'208";a="658095401" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by orsmga008.jf.intel.com with ESMTP; 13 Oct 2022 01:29:56 -0700 From: wenbin.chen-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Thu, 13 Oct 2022 16:25:23 +0800 Message-Id: <20221013082524.47409-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] libavcodec/qsvenc_av1: add av1_qsv encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: w2AO0eAkxKtz From: Wenbin Chen It is available only when libvpl is enabled. MSDK doesn't support av1 encoding. sample command: ffmpeg -f rawvideo -pix_fmt nv12 -s 1920x1080 -i input.yuv \ -c:v av1_qsv output.ivf Signed-off-by: Wenbin Chen Signed-off-by: Haihao Xiang --- configure | 2 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/qsvenc.c | 196 +++++++++++++++++++++++++++++++++++++++- libavcodec/qsvenc.h | 7 +- libavcodec/qsvenc_av1.c | 156 ++++++++++++++++++++++++++++++++ 6 files changed, 358 insertions(+), 5 deletions(-) create mode 100644 libavcodec/qsvenc_av1.c diff --git a/configure b/configure index f3fd91f592..7c4fef6cb0 100755 --- a/configure +++ b/configure @@ -3269,6 +3269,8 @@ vp9_qsv_encoder_select="qsvenc" vp9_v4l2m2m_decoder_deps="v4l2_m2m vp9_v4l2_m2m" wmv3_crystalhd_decoder_select="crystalhd" av1_qsv_decoder_select="qsvdec" +av1_qsv_encoder_select="qsvenc" +av1_qsv_encoder_deps="libvpl" # parsers aac_parser_select="adts_header mpeg4audio" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 37b63cadc2..77deaafe98 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -244,6 +244,7 @@ OBJS-$(CONFIG_AURA_DECODER) += cyuv.o OBJS-$(CONFIG_AURA2_DECODER) += aura.o OBJS-$(CONFIG_AV1_DECODER) += av1dec.o OBJS-$(CONFIG_AV1_CUVID_DECODER) += cuviddec.o +OBJS-$(CONFIG_AV1_QSV_ENCODER) += qsvenc_av1.o OBJS-$(CONFIG_AVRN_DECODER) += avrndec.o OBJS-$(CONFIG_AVRP_DECODER) += r210dec.o OBJS-$(CONFIG_AVRP_ENCODER) += r210enc.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index cfeb01ac1c..57e53437dc 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -827,6 +827,7 @@ extern const FFCodec ff_libaom_av1_decoder; extern const FFCodec ff_av1_decoder; extern const FFCodec ff_av1_cuvid_decoder; extern const FFCodec ff_av1_qsv_decoder; +extern const FFCodec ff_av1_qsv_encoder; extern const FFCodec ff_libopenh264_encoder; extern const FFCodec ff_libopenh264_decoder; extern const FFCodec ff_h264_amf_encoder; diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index dc5479d0f3..fd3b9d5cbe 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -82,6 +82,14 @@ static const struct profile_names vp9_profiles[] = { { MFX_PROFILE_VP9_3, "vp9 3" }, }; +static const struct profile_names av1_profiles[] = { +#if QSV_VERSION_ATLEAST(1, 34) + { MFX_PROFILE_AV1_MAIN, "av1 main" }, + { MFX_PROFILE_AV1_HIGH, "av1 high" }, + { MFX_PROFILE_AV1_PRO, "av1 professional" }, +#endif +}; + typedef struct QSVPacket { AVPacket pkt; mfxSyncPoint *sync; @@ -114,6 +122,11 @@ static const char *print_profile(enum AVCodecID codec_id, mfxU16 profile) num_profiles = FF_ARRAY_ELEMS(vp9_profiles); break; + case AV_CODEC_ID_AV1: + profiles = av1_profiles; + num_profiles = FF_ARRAY_ELEMS(av1_profiles); + break; + default: return "unknown"; } @@ -429,6 +442,88 @@ static void dump_video_mjpeg_param(AVCodecContext *avctx, QSVEncContext *q) info->FrameInfo.FrameRateExtD, info->FrameInfo.FrameRateExtN); } +#if QSV_HAVE_EXT_AV1_PARAM +static void dump_video_av1_param(AVCodecContext *avctx, QSVEncContext *q, + mfxExtBuffer **coding_opts) +{ + mfxInfoMFX *info = &q->param.mfx; + mfxExtAV1TileParam *av1_tile_param = (mfxExtAV1TileParam *)coding_opts[0]; + mfxExtAV1BitstreamParam *av1_bs_param = (mfxExtAV1BitstreamParam *)coding_opts[1]; + mfxExtCodingOption2 *co2 = (mfxExtCodingOption2*)coding_opts[2]; + mfxExtCodingOption3 *co3 = (mfxExtCodingOption3*)coding_opts[3]; + + av_log(avctx, AV_LOG_VERBOSE, "profile: %s; level: %"PRIu16"\n", + print_profile(avctx->codec_id, info->CodecProfile), info->CodecLevel); + + av_log(avctx, AV_LOG_VERBOSE, + "GopPicSize: %"PRIu16"; GopRefDist: %"PRIu16"; GopOptFlag:%s%s; IdrInterval: %"PRIu16"\n", + info->GopPicSize, info->GopRefDist, + info->GopOptFlag & MFX_GOP_CLOSED ? " closed" : "", + info->GopOptFlag & MFX_GOP_STRICT ? " strict" : "", + info->IdrInterval); + + av_log(avctx, AV_LOG_VERBOSE, "TargetUsage: %"PRIu16"; RateControlMethod: %s\n", + info->TargetUsage, print_ratecontrol(info->RateControlMethod)); + + if (info->RateControlMethod == MFX_RATECONTROL_CBR || + info->RateControlMethod == MFX_RATECONTROL_VBR) + av_log(avctx, AV_LOG_VERBOSE, + "BufferSizeInKB: %"PRIu16"; InitialDelayInKB: %"PRIu16"; TargetKbps: %"PRIu16"; MaxKbps: %"PRIu16"; BRCParamMultiplier: %"PRIu16"\n", + info->BufferSizeInKB, info->InitialDelayInKB, info->TargetKbps, info->MaxKbps, info->BRCParamMultiplier); + else if (info->RateControlMethod == MFX_RATECONTROL_CQP) + av_log(avctx, AV_LOG_VERBOSE, "QPI: %"PRIu16"; QPP: %"PRIu16"; QPB: %"PRIu16"\n", + info->QPI, info->QPP, info->QPB); + else if (info->RateControlMethod == MFX_RATECONTROL_ICQ) + av_log(avctx, AV_LOG_VERBOSE, "ICQQuality: %"PRIu16"\n", info->ICQQuality); + else + av_log(avctx, AV_LOG_VERBOSE, "Unsupported ratecontrol method: %d \n", info->RateControlMethod); + + av_log(avctx, AV_LOG_VERBOSE, "NumRefFrame: %"PRIu16"\n", info->NumRefFrame); + + av_log(avctx, AV_LOG_VERBOSE, + "IntRefType: %"PRIu16"; IntRefCycleSize: %"PRIu16 + "; IntRefQPDelta: %"PRId16"; IntRefCycleDist: %"PRId16"\n", + co2->IntRefType, co2->IntRefCycleSize, + co2->IntRefQPDelta, co3->IntRefCycleDist); + + av_log(avctx, AV_LOG_VERBOSE, "MaxFrameSize: %d;\n", co2->MaxFrameSize); + + av_log(avctx, AV_LOG_VERBOSE, + "BitrateLimit: %s; MBBRC: %s; ExtBRC: %s\n", + print_threestate(co2->BitrateLimit), print_threestate(co2->MBBRC), + print_threestate(co2->ExtBRC)); + + av_log(avctx, AV_LOG_VERBOSE, "VDENC: %s\n", print_threestate(info->LowPower)); + + switch (co2->BRefType) { + case MFX_B_REF_OFF: av_log(avctx, AV_LOG_VERBOSE, "BRefType: off\n"); break; + case MFX_B_REF_PYRAMID: av_log(avctx, AV_LOG_VERBOSE, "BRefType: pyramid\n"); break; + default: av_log(avctx, AV_LOG_VERBOSE, "BRefType: auto\n"); break; + } + + switch (co3->PRefType) { + case MFX_P_REF_DEFAULT: av_log(avctx, AV_LOG_VERBOSE, "PRefType: default\n"); break; + case MFX_P_REF_SIMPLE: av_log(avctx, AV_LOG_VERBOSE, "PRefType: simple\n"); break; + case MFX_P_REF_PYRAMID: av_log(avctx, AV_LOG_VERBOSE, "PRefType: pyramid\n"); break; + default: av_log(avctx, AV_LOG_VERBOSE, "PRefType: unknown\n"); break; + } + + av_log(avctx, AV_LOG_VERBOSE, + "MinQPI: %"PRIu8"; MaxQPI: %"PRIu8"; MinQPP: %"PRIu8"; MaxQPP: %"PRIu8"; MinQPB: %"PRIu8"; MaxQPB: %"PRIu8"\n", + co2->MinQPI, co2->MaxQPI, co2->MinQPP, co2->MaxQPP, co2->MinQPB, co2->MaxQPB); + + av_log(avctx, AV_LOG_VERBOSE, "FrameRateExtD: %"PRIu32"; FrameRateExtN: %"PRIu32" \n", + info->FrameInfo.FrameRateExtD, info->FrameInfo.FrameRateExtN); + + av_log(avctx, AV_LOG_VERBOSE, + "NumTileRows: %"PRIu16"; NumTileColumns: %"PRIu16"; NumTileGroups: %"PRIu16"\n", + av1_tile_param->NumTileRows, av1_tile_param->NumTileColumns, av1_tile_param->NumTileGroups); + + av_log(avctx, AV_LOG_VERBOSE, "WriteIVFHeaders: %s \n", + print_threestate(av1_bs_param->WriteIVFHeaders)); +} +#endif + static int select_rc_mode(AVCodecContext *avctx, QSVEncContext *q) { const char *rc_desc; @@ -741,10 +836,15 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) break; case MFX_RATECONTROL_CQP: quant = avctx->global_quality / FF_QP2LAMBDA; - - q->param.mfx.QPI = av_clip(quant * fabs(avctx->i_quant_factor) + avctx->i_quant_offset, 0, 51); - q->param.mfx.QPP = av_clip(quant, 0, 51); - q->param.mfx.QPB = av_clip(quant * fabs(avctx->b_quant_factor) + avctx->b_quant_offset, 0, 51); + if (avctx->codec_id == AV_CODEC_ID_AV1) { + q->param.mfx.QPI = av_clip_uintp2(quant * fabs(avctx->i_quant_factor) + avctx->i_quant_offset, 8); + q->param.mfx.QPP = av_clip_uintp2(quant, 8); + q->param.mfx.QPB = av_clip_uintp2(quant * fabs(avctx->b_quant_factor) + avctx->b_quant_offset, 8); + } else { + q->param.mfx.QPI = av_clip(quant * fabs(avctx->i_quant_factor) + avctx->i_quant_offset, 0, 51); + q->param.mfx.QPP = av_clip(quant, 0, 51); + q->param.mfx.QPB = av_clip(quant * fabs(avctx->b_quant_factor) + avctx->b_quant_offset, 0, 51); + } q->old_global_quality = avctx->global_quality; q->old_i_quant_factor = avctx->i_quant_factor; q->old_i_quant_offset = avctx->i_quant_offset; @@ -895,6 +995,20 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) q->extco2.Header.BufferId = MFX_EXTBUFF_CODING_OPTION2; q->extco2.Header.BufferSz = sizeof(q->extco2); + q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extco2; + } else if (avctx->codec_id == AV_CODEC_ID_AV1) { + if (q->extbrc >= 0) + q->extco2.ExtBRC = q->extbrc ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; + if (q->b_strategy >= 0) + q->extco2.BRefType = q->b_strategy ? MFX_B_REF_PYRAMID : MFX_B_REF_OFF; + if (q->adaptive_i >= 0) + q->extco2.AdaptiveI = q->adaptive_i ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; + if (q->adaptive_b >= 0) + q->extco2.AdaptiveB = q->adaptive_b ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; + + q->extco2.Header.BufferId = MFX_EXTBUFF_CODING_OPTION2; + q->extco2.Header.BufferSz = sizeof(q->extco2); + q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extco2; } @@ -972,6 +1086,27 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extvp9param; } +#if QSV_HAVE_EXT_AV1_PARAM + if (avctx->codec_id == AV_CODEC_ID_AV1) { + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 2, 5)) { + q->extav1tileparam.Header.BufferId = MFX_EXTBUFF_AV1_TILE_PARAM; + q->extav1tileparam.Header.BufferSz = sizeof(q->extav1tileparam); + q->extav1tileparam.NumTileColumns = q->tile_cols; + q->extav1tileparam.NumTileRows = q->tile_rows; + q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extav1tileparam; + + q->extav1bsparam.Header.BufferId = MFX_EXTBUFF_AV1_BITSTREAM_PARAM; + q->extav1bsparam.Header.BufferSz = sizeof(q->extav1bsparam); + q->extav1bsparam.WriteIVFHeaders = MFX_CODINGOPTION_OFF; + q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extav1bsparam; + } else { + av_log(avctx, AV_LOG_ERROR, + "This version of runtime doesn't support AV1 encoding\n"); + return AVERROR_UNKNOWN; + } + } +#endif + if (avctx->codec_id == AV_CODEC_ID_HEVC) { q->exthevctiles.Header.BufferId = MFX_EXTBUFF_HEVC_TILES; q->exthevctiles.Header.BufferSz = sizeof(q->exthevctiles); @@ -1082,6 +1217,56 @@ static int qsv_retrieve_enc_vp9_params(AVCodecContext *avctx, QSVEncContext *q) return 0; } +static int qsv_retrieve_enc_av1_params(AVCodecContext *avctx, QSVEncContext *q) +{ +#if QSV_HAVE_EXT_AV1_PARAM + int ret = 0; + mfxExtAV1TileParam av1_extend_tile_buf = { + .Header.BufferId = MFX_EXTBUFF_AV1_TILE_PARAM, + .Header.BufferSz = sizeof(av1_extend_tile_buf), + }; + mfxExtAV1BitstreamParam av1_bs_param = { + .Header.BufferId = MFX_EXTBUFF_AV1_BITSTREAM_PARAM, + .Header.BufferSz = sizeof(av1_bs_param), + }; + + mfxExtCodingOption2 co2 = { + .Header.BufferId = MFX_EXTBUFF_CODING_OPTION2, + .Header.BufferSz = sizeof(co2), + }; + + mfxExtCodingOption3 co3 = { + .Header.BufferId = MFX_EXTBUFF_CODING_OPTION3, + .Header.BufferSz = sizeof(co3), + }; + + mfxExtBuffer *ext_buffers[] = { + (mfxExtBuffer*)&av1_extend_tile_buf, + (mfxExtBuffer*)&av1_bs_param, + (mfxExtBuffer*)&co2, + (mfxExtBuffer*)&co3, + }; + + if (!QSV_RUNTIME_VERSION_ATLEAST(q->ver, 2, 5)) { + av_log(avctx, AV_LOG_ERROR, + "This version of runtime doesn't support AV1 encoding\n"); + return AVERROR_UNKNOWN; + } + + q->param.ExtParam = ext_buffers; + q->param.NumExtParam = FF_ARRAY_ELEMS(ext_buffers); + + ret = MFXVideoENCODE_GetVideoParam(q->session, &q->param); + if (ret < 0) + return ff_qsv_print_error(avctx, ret, + "Error calling GetVideoParam"); + + q->packet_size = q->param.mfx.BufferSizeInKB * q->param.mfx.BRCParamMultiplier * 1000; + dump_video_av1_param(avctx, q, ext_buffers); +#endif + return 0; +} + static int qsv_retrieve_enc_params(AVCodecContext *avctx, QSVEncContext *q) { AVCPBProperties *cpb_props; @@ -1420,6 +1605,9 @@ int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q) case AV_CODEC_ID_VP9: ret = qsv_retrieve_enc_vp9_params(avctx, q); break; + case AV_CODEC_ID_AV1: + ret = qsv_retrieve_enc_av1_params(avctx, q); + break; default: ret = qsv_retrieve_enc_params(avctx, q); break; diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index d77bc0aee1..eb79db9871 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -39,6 +39,7 @@ #include "qsv_internal.h" #define QSV_HAVE_EXT_VP9_TILES QSV_VERSION_ATLEAST(1, 29) +#define QSV_HAVE_EXT_AV1_PARAM QSV_VERSION_ATLEAST(2, 5) #if defined(_WIN32) || defined(__CYGWIN__) #define QSV_HAVE_AVBR 1 @@ -154,6 +155,10 @@ typedef struct QSVEncContext { #endif mfxExtHEVCTiles exthevctiles; mfxExtVP9Param extvp9param; +#if QSV_HAVE_EXT_AV1_PARAM + mfxExtAV1TileParam extav1tileparam; + mfxExtAV1BitstreamParam extav1bsparam; +#endif #if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; @@ -163,7 +168,7 @@ typedef struct QSVEncContext { mfxExtVideoSignalInfo extvsi; - mfxExtBuffer *extparam_internal[5 + (QSV_HAVE_MF * 2)]; + mfxExtBuffer *extparam_internal[5 + (QSV_HAVE_MF * 2) + QSV_HAVE_EXT_AV1_PARAM * 2]; int nb_extparam_internal; mfxExtBuffer **extparam; diff --git a/libavcodec/qsvenc_av1.c b/libavcodec/qsvenc_av1.c new file mode 100644 index 0000000000..bb9ad16927 --- /dev/null +++ b/libavcodec/qsvenc_av1.c @@ -0,0 +1,156 @@ +/* + * Intel MediaSDK QSV based AV1 encoder + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + + +#include +#include + +#include + +#include "libavutil/common.h" +#include "libavutil/opt.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "bsf.h" +#include "qsv.h" +#include "qsvenc.h" + +typedef struct QSVAV1EncContext { + AVClass *class; + AVBSFContext *extra_data_bsf; + QSVEncContext qsv; +} QSVAV1EncContext; + +static av_cold int qsv_enc_init(AVCodecContext *avctx) +{ + QSVAV1EncContext *q = avctx->priv_data; + int ret; + + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { + const AVBitStreamFilter *filter = av_bsf_get_by_name("extract_extradata"); + if (!filter) { + av_log(avctx, AV_LOG_ERROR, "Cannot get extract_extradata bitstream filter\n"); + return AVERROR_BUG; + } + ret = av_bsf_alloc(filter, &q->extra_data_bsf); + if (ret < 0) + return ret; + ret = avcodec_parameters_from_context(q->extra_data_bsf->par_in, avctx); + if (ret < 0) + return ret; + ret = av_bsf_init(q->extra_data_bsf); + if (ret < 0) + return ret; + } + + return ff_qsv_enc_init(avctx, &q->qsv); +} + +static int qsv_enc_frame(AVCodecContext *avctx, AVPacket *pkt, + const AVFrame *frame, int *got_packet) +{ + QSVAV1EncContext *q = avctx->priv_data; + int ret; + + ret = ff_qsv_encode(avctx, &q->qsv, pkt, frame, got_packet); + if (ret < 0) + return ret; + + if (*got_packet && avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { + ret = av_bsf_send_packet(q->extra_data_bsf, pkt); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "extract_extradata filter " + "failed to send input packet\n"); + return ret; + } + + ret = av_bsf_receive_packet(q->extra_data_bsf, pkt); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "extract_extradata filter " + "failed to receive output packet\n"); + return ret; + } + } + + return ret; +} + +static av_cold int qsv_enc_close(AVCodecContext *avctx) +{ + QSVAV1EncContext *q = avctx->priv_data; + + av_bsf_free(&q->extra_data_bsf); + + return ff_qsv_enc_close(avctx, &q->qsv); +} + +#define OFFSET(x) offsetof(QSVAV1EncContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM +static const AVOption options[] = { + QSV_COMMON_OPTS + QSV_OPTION_B_STRATEGY + QSV_OPTION_ADAPTIVE_I + QSV_OPTION_ADAPTIVE_B + QSV_OPTION_EXTBRC + { "profile", NULL, OFFSET(qsv.profile), AV_OPT_TYPE_INT, { .i64 = MFX_PROFILE_UNKNOWN }, 0, INT_MAX, VE, "profile" }, + { "unknown" , NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_PROFILE_UNKNOWN }, INT_MIN, INT_MAX, VE, "profile" }, + { "main" , NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_PROFILE_AV1_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "tile_cols", "Number of columns for tiled encoding", OFFSET(qsv.tile_cols), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, + { "tile_rows", "Number of rows for tiled encoding", OFFSET(qsv.tile_rows), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, + { "look_ahead_depth", "Depth of look ahead in number frames, available when extbrc option is enabled", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE }, + { NULL }, +}; + +static const AVClass class = { + .class_name = "av1_qsv encoder", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const FFCodecDefault qsv_enc_defaults[] = { + { "b", "1M" }, + { "g", "-1" }, + { "bf", "-1" }, + { "refs", "0" }, + { NULL }, +}; + +FFCodec ff_av1_qsv_encoder = { + .p.name = "av1_qsv", + .p.long_name = NULL_IF_CONFIG_SMALL("AV1 (Intel Quick Sync Video acceleration)"), + .priv_data_size = sizeof(QSVAV1EncContext), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_AV1, + .init = qsv_enc_init, + FF_CODEC_ENCODE_CB(qsv_enc_frame), + .close = qsv_enc_close, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HYBRID, + .p.pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_QSV, + AV_PIX_FMT_NONE }, + .p.priv_class = &class, + .defaults = qsv_enc_defaults, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, + .p.wrapper_name = "qsv", + .hw_configs = ff_qsv_enc_hw_configs, +};