From patchwork Mon Sep 23 07:10:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 51729 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:d154:0:b0:48e:c0f8:d0de with SMTP id bt20csp2276391vqb; Mon, 23 Sep 2024 00:14:21 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCV3zW2DX8hAAzlyPCsfjPPExH+2MMf0mJWPAGhH68uIHuFwennuAVQQtWhF8vt5yQqBaVkZKBYVpHSDeFUbch8x@gmail.com X-Google-Smtp-Source: AGHT+IEZ74fKe2UbbYvsTiw3c64jBhTUG8vlsKk2UP6ofOz1urB1AW6dXJmCQ6dLNraRwz5VmZop X-Received: by 2002:a05:6512:ea8:b0:533:4722:ebbe with SMTP id 2adb3069b0e04-536ac2f2b4cmr5144861e87.26.1727075660676; Mon, 23 Sep 2024 00:14:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1727075660; cv=none; d=google.com; s=arc-20240605; b=LfP75ot4tk3PQbOJPHL4exMfaLaPOChLScdaUB0SNBC5xRekpDqVq2u6SYF6AJXS6l NQuVEdYF7LaDDXsjr9irpZUFdznP6Xob6WER4cOk+nnshb4n9SOZT48hn5u89mps4MCl zYgFV7VzdcgtYQL7j+BR8HsoXWN+IcrMroCk5EfrGNxCA6FBjj8XJxFjY41FVR7pbUP5 Jn1r+Nk1HyQRQW5oznl2M8luG0+peq5W415D0PfFAA8GwiuSr4unh8BOljAE7FcW+FgG uXUkE+YEy9CVQBqJW9K/7gVTqPukDFFhCGcKTV60Ii0EisEm9/ZaJzUBmp1OcqPYqf/c pKnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=J59RnuDrt861gI5Ufr6LE01MoLUt/+we2SCGj6SFElI=; fh=i4ESP4ZRFDcfYfwXKWpOXjc2YhmIGuOsCfZUnwNO0gc=; b=ZDh792xWHnGPFmFQalO2/FcYpae3XdgxAvDysEnUMKCnA5Y0w40q/Hspgl1ZBTkysB AZjhLmdJkWZ+ou20WHOK9zJEHpxIKTFk57c17tCgRkIxnq3mK6E7+GH4bBRSkiDpzW/t 5crzRl9BMVo8/nT58PxcPAaOQbMBDwHTodZOa4Rj3B4CmbgxxJQUBEGcK3DQ6jnPzaWo Ukm8m3lrnWSBBHQ2Gy4X/i2Cy+4SwKns1KbKax7uHUG+sj3+J4GTu/XHQ7aA6mLtjpei pEl40MmI8x6lfcWpybjAnEiL5WmbZ+jXeCZhjWVSlzXv+q3RRYljRN210xWIEyuoMxxb q4nQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=XZYUmm4j; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c42bc9079fsi13586476a12.437.2024.09.23.00.14.19; Mon, 23 Sep 2024 00:14:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=XZYUmm4j; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B7BBE68DB63; Mon, 23 Sep 2024 10:07:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1011D68DB73 for ; Mon, 23 Sep 2024 10:07:43 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727075269; x=1758611269; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Dsrxwu1yCkSfRQiXnKYLqAtotUlRhnQh5YOupTFJA/Y=; b=XZYUmm4jAt3kVXRMcFG527aUoqVC+/hxLpx19BYAgaNpLq1An/VBx3zK Ww/Zs07jK/3KQAj5bdPCySlPnIF/Zoj0Uis3txxKRZJlXhI6r3X3dJ5EJ zXocWdd/GvMF1kIEsya7QvxcZ8BAHxsY8Kal8Ko836EX83yxOqIVtXRx0 6fYiv4jAFQFUc8d4CaQ74HZMEAYoO3MFHe0La677HeB1vAox+L6QQcH1F aP1qdVIvhZCIab4IfuRkTskifm1blWue5u+4wfwseG6nSXcrf9tK2ZI6C 23EZ/K4azCIyow11QffTW1YqUhgacAnBg9/AK9tazLBj2nIj3TDmR35qG A==; X-CSE-ConnectionGUID: JE5cp3sZTYSzq0g1bVGN7A== X-CSE-MsgGUID: peAdGMffTs6EYsovf+0oeg== X-IronPort-AV: E=McAfee;i="6700,10204,11202"; a="43525026" X-IronPort-AV: E=Sophos;i="6.10,250,1719903600"; d="scan'208";a="43525026" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Sep 2024 00:07:41 -0700 X-CSE-ConnectionGUID: HdvwPKlkTDKpZZEcBPn4HQ== X-CSE-MsgGUID: /JukKuStQs+o6L5o7dSj0w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,250,1719903600"; d="scan'208";a="70560522" Received: from feiwan1-desk3.sh.intel.com ([10.238.208.39]) by fmviesa007.fm.intel.com with ESMTP; 23 Sep 2024 00:07:40 -0700 From: fei.w.wang-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 23 Sep 2024 15:10:05 +0800 Message-Id: <20240923071008.1599696-2-fei.w.wang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240923071008.1599696-1-fei.w.wang@intel.com> References: <20240923071008.1599696-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/5] libavcodec/qsvenc: enable Screen Content Tool Encode for AV1 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: fei.w.wang@intel.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FXWMjGPZfO+M From: Fei Wang Screen Content Tool provides Intra Block Copy and Palette Mode when encoding. Signed-off-by: Fei Wang --- doc/encoders.texi | 6 ++++++ libavcodec/qsvenc.c | 43 +++++++++++++++++++++++++++++++++++++++++ libavcodec/qsvenc.h | 9 ++++++++- libavcodec/qsvenc_av1.c | 4 ++++ 4 files changed, 61 insertions(+), 1 deletion(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 6094434ed4..85be976a46 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -4121,6 +4121,12 @@ than zero, then for I frames the value set by max_frame_size is ignored. @item @var{max_frame_size_p} Maximum encoded frame size for P frames in bytes. If this value is set as larger than zero, then for P frames the value set by max_frame_size is ignored. + +@item @var{palette_mode} +Use palette mode in Screen Content Tool. + +@item @var{intrabc} +Use intra block copy in Screen Content Tool. @end table @section snow diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 9af9932035..dfef15fa5a 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -519,6 +519,9 @@ static void dump_video_av1_param(AVCodecContext *avctx, QSVEncContext *q, mfxExtAV1BitstreamParam *av1_bs_param = (mfxExtAV1BitstreamParam *)coding_opts[1]; mfxExtCodingOption2 *co2 = (mfxExtCodingOption2*)coding_opts[2]; mfxExtCodingOption3 *co3 = (mfxExtCodingOption3*)coding_opts[3]; +#if QSV_HAVE_EXT_AV1_SCC + mfxExtAV1ScreenContentTools *scc = (mfxExtAV1ScreenContentTools*)coding_opts[4]; +#endif av_log(avctx, AV_LOG_VERBOSE, "profile: %s; level: %"PRIu16"\n", print_profile(avctx->codec_id, info->CodecProfile), info->CodecLevel); @@ -591,6 +594,14 @@ static void dump_video_av1_param(AVCodecContext *avctx, QSVEncContext *q, print_threestate(av1_bs_param->WriteIVFHeaders)); av_log(avctx, AV_LOG_VERBOSE, "LowDelayBRC: %s\n", print_threestate(co3->LowDelayBRC)); av_log(avctx, AV_LOG_VERBOSE, "MaxFrameSize: %d;\n", co2->MaxFrameSize); + +#if QSV_HAVE_EXT_AV1_SCC + if (scc) { + av_log(avctx, AV_LOG_VERBOSE, + "Palette: %s; IntraBlockCopy: %s\n", + print_threestate(scc->Palette), print_threestate(scc->IntraBlockCopy)); + } +#endif } #endif @@ -1339,6 +1350,28 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) } #endif +#if QSV_HAVE_EXT_AV1_SCC + if (q->palette_mode || q->intrabc) { + if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 2, 13)) { + if (q->param.mfx.CodecId != MFX_CODEC_AV1) { + av_log(avctx, AV_LOG_ERROR, "Not supported encoder for Screen Content Tool Encode. " + "Supported: av1_qsv \n"); + return AVERROR_UNKNOWN; + } + + q->extsccparam.Header.BufferId = MFX_EXTBUFF_AV1_SCREEN_CONTENT_TOOLS; + q->extsccparam.Header.BufferSz = sizeof(q->extsccparam); + q->extsccparam.Palette = q->palette_mode ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; + q->extsccparam.IntraBlockCopy = q->intrabc ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; + q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer *)&q->extsccparam; + } else { + av_log(avctx, AV_LOG_ERROR, + "This version of runtime doesn't support Screen Content Tool Encode\n"); + return AVERROR_UNKNOWN; + } + } +#endif + if (!check_enc_param(avctx,q)) { av_log(avctx, AV_LOG_ERROR, "some encoding parameters are not supported by the QSV " @@ -1446,11 +1479,21 @@ static int qsv_retrieve_enc_av1_params(AVCodecContext *avctx, QSVEncContext *q) .Header.BufferSz = sizeof(co3), }; +#if QSV_HAVE_EXT_AV1_SCC + mfxExtAV1ScreenContentTools scc_buf = { + .Header.BufferId = MFX_EXTBUFF_AV1_SCREEN_CONTENT_TOOLS, + .Header.BufferSz = sizeof(scc_buf), + }; +#endif + mfxExtBuffer *ext_buffers[] = { (mfxExtBuffer*)&av1_extend_tile_buf, (mfxExtBuffer*)&av1_bs_param, (mfxExtBuffer*)&co2, (mfxExtBuffer*)&co3, +#if QSV_HAVE_EXT_AV1_SCC + (mfxExtBuffer*)&scc_buf, +#endif }; if (!QSV_RUNTIME_VERSION_ATLEAST(q->ver, 2, 5)) { diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 559d40c919..1e1474d37c 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -38,6 +38,7 @@ #define QSV_HAVE_EXT_VP9_TILES QSV_VERSION_ATLEAST(1, 29) #define QSV_HAVE_EXT_AV1_PARAM QSV_VERSION_ATLEAST(2, 5) +#define QSV_HAVE_EXT_AV1_SCC QSV_VERSION_ATLEAST(2, 13) #if defined(_WIN32) || defined(__CYGWIN__) #define QSV_HAVE_AVBR 1 @@ -193,10 +194,14 @@ typedef struct QSVEncContext { #if QSV_HAVE_AC mfxExtAlphaChannelEncCtrl extaplhachannelparam; #endif +#if QSV_HAVE_EXT_AV1_SCC + mfxExtAV1ScreenContentTools extsccparam; +#endif mfxExtVideoSignalInfo extvsi; - mfxExtBuffer *extparam_internal[5 + (QSV_HAVE_MF * 2) + (QSV_HAVE_EXT_AV1_PARAM * 2) + QSV_HAVE_HE + QSV_HAVE_AC]; + mfxExtBuffer *extparam_internal[5 + (QSV_HAVE_MF * 2) + (QSV_HAVE_EXT_AV1_PARAM * 2) + + QSV_HAVE_HE + QSV_HAVE_AC + QSV_HAVE_EXT_AV1_SCC]; int nb_extparam_internal; mfxExtBuffer **extparam_str; @@ -326,6 +331,8 @@ typedef struct QSVEncContext { AVDictionary *qsv_params; int alpha_encode; + int palette_mode; + int intrabc; } QSVEncContext; int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q); diff --git a/libavcodec/qsvenc_av1.c b/libavcodec/qsvenc_av1.c index a86b409bed..4f035f3d83 100644 --- a/libavcodec/qsvenc_av1.c +++ b/libavcodec/qsvenc_av1.c @@ -189,6 +189,10 @@ static const AVOption options[] = { { "tile_cols", "Number of columns for tiled encoding", OFFSET(qsv.tile_cols), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, { "tile_rows", "Number of rows for tiled encoding", OFFSET(qsv.tile_rows), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, { "look_ahead_depth", "Depth of look ahead in number frames, available when extbrc option is enabled", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE }, +#if QSV_HAVE_EXT_AV1_SCC + { "palette_mode", "Enable palette mode of Screen Content Tool for encoding", OFFSET(qsv.palette_mode), AV_OPT_TYPE_BOOL, { .i64 = 0}, 0, 1, VE}, + { "intrabc", "Enable intra block copy of Screen Content Tool for encoding", OFFSET(qsv.intrabc), AV_OPT_TYPE_BOOL, { .i64 = 0}, 0, 1, VE}, +#endif { NULL }, };