From patchwork Wed Nov 2 08:11:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 39110 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp664850pzb; Wed, 2 Nov 2022 01:17:22 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5n5lcVzF+HvwYsdJnmdSEgExdvfctfVhFht7xVpOY5Pb6oH03CUPla6B1mAxt50CbxD1pD X-Received: by 2002:a17:906:3b48:b0:7ad:fa67:1daa with SMTP id h8-20020a1709063b4800b007adfa671daamr3897168ejf.653.1667377042617; Wed, 02 Nov 2022 01:17:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667377042; cv=none; d=google.com; s=arc-20160816; b=GGbXrPqusXjWhknCEvdZP4tTiIewUtaLGGJwIVef0sRRTFlgbqBTDeo1vayEHcdMv5 GLUqbx9DZVvejMBNbemaD8M45xG/0aRELjlvkIzvpo4uniuy7F6sHzd4cdFnlDQEfOOt ouVqwMFc0+SEes1FP5x8yEVVCitb9SNV5of8PzRIY/GxFM6Vtm4P2sO7c5EViKDYM5yY llZyu4UpcML5FGwfdXpYSa7no44awYPE7dJjwPwFfUVlDP7qEBA2vfxw4JDM30PFSFeJ s2YfNRjwS5LmCFIK/prrK1SWGpQBXguaUiZDsupPjiJSVBSjC1k0EHGjjFuXDd9cmhEB 861w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=xPiPrDsaLfy6cqlSjyfiFEt+aIcMn7aveVzvpZSrJ6s=; b=H+qgKEg9Hil33/6d3wCuEvT09ZTiKmXAvl5i61O4lRJDJHoQCoTP0L1gz30bSkAkLC 6WZzbOVyZ+g5vydnhhJxvvxGuk1agWNEYr3X0rwRUYk7NEdFsCefH5aWadXq1HGAPZ8Q Uo3B3xAcvSjtx1IcIKZeEIptAkno19URoZAR23HoywVeZVL4/03XUB2mqfpC9ypKIqFm GoFSXB1tujCgloTRW8g9IQm5yl+iXiO4/VGaVbEFvh2zua6NvxjyubgwvcEe+8BcXs5f R4RcLZu+mrRxj1Az+lT82NXv52b6u/hYWRU7fJ5evKqH7RPIlau2qM6RZpDuA1oU0OHe 7HdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=IBGG75iV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x10-20020a50ba8a000000b0045a3b951e8bsi12453214ede.194.2022.11.02.01.17.21; Wed, 02 Nov 2022 01:17:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=IBGG75iV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5E45368BEAF; Wed, 2 Nov 2022 10:17:18 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D98C868BE57 for ; Wed, 2 Nov 2022 10:17:10 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667377036; x=1698913036; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=v4UlCsRsYfipxADDl7G9m+AkvG1pgctoBFVB/x3e1HQ=; b=IBGG75iV+EZkod5TQ+zVtxfvSe/OBmrQ9uncZw1pY/C8lGHTIdPR8wx6 RWQZbeJkCSKy2fDJ6ANluaEBrQWfuS2IrZRaOqSPXWxlFi7AThYPphJrQ 5dmIORmprtfxiDRRYxK0ZO/ID79X+wMpc19DBLio13cUvuzRFm7fjSxhH LmIP5T6jU9gbxbHavgBLrDzH1xmj2mydpmHAjlLjc0gPE7gh2qgItLINk i0P7lT2c40GyMn7O06Cfo+IhcXmE1Hn4+/a29rj5vImp8Vip3cww5zsCF BqYU9dr7e4XirvNEl8F5IunHpDX9Pk0RnqUhkwbSxkfNk27eTDKawtTvE g==; X-IronPort-AV: E=McAfee;i="6500,9779,10518"; a="395655356" X-IronPort-AV: E=Sophos;i="5.95,232,1661842800"; d="scan'208";a="395655356" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Nov 2022 01:17:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10518"; a="759466582" X-IronPort-AV: E=Sophos;i="5.95,232,1661842800"; d="scan'208";a="759466582" Received: from wenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.4]) by orsmga004.jf.intel.com with ESMTP; 02 Nov 2022 01:16:59 -0700 From: wenbin.chen-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 2 Nov 2022 16:11:51 +0800 Message-Id: <20221102081151.803641-1-wenbin.chen@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] libavcodec/qsvenc: Add skip_frame support to qsvenc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 38VgKMHGtPHv From: Wenbin Chen Add skip_frame support to qsvenc. Use per-frame metadata "qsv_skip_frame" to control it. skip_frame option defines the behavior of qsv_skip_frame. no_skip: Frame skipping is disabled. insert_dummy: Encoder inserts into bitstream frame where all macroblocks are encoded as skipped. insert_nothing: Similar to insert_dummy, but encoder inserts nothing. The skipped frames are still used in brc. For example, gop still include skipped frames, and the frames after skipped frames will be larger in size. brc_only: skip_frame metadata indicates the number of missed frames before the current frame. Signed-off-by: Wenbin Chen --- doc/encoders.texi | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/qsvenc.c | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/qsvenc.h | 13 +++++++++++++ libavcodec/qsvenc_h264.c | 1 + libavcodec/qsvenc_hevc.c | 1 + 5 files changed, 87 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 53dd02fd28..59f39d18f6 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3564,6 +3564,24 @@ bitrate, @var{target_bitrate}, within the accuracy range @var{avbr_accuracy}, after a @var{avbr_Convergence} period. This method does not follow HRD and the instant bitrate is not capped or padded. +@item @var{skip_frame} +Use per-frame metadata "qsv_skip_frame" to skip frame when encoding. This option +defines the usage of this metadata. +@table @samp +@item no_skip +Frame skipping is disabled. +@item insert_dummy +Encoder inserts into bitstream frame where all macroblocks are encoded as +skipped. +@item insert_nothing +Similar to insert_dummy, but encoder inserts nothing into bitstream. The skipped +frames are still used in brc. For example, gop still include skipped frames, and +the frames after skipped frames will be larger in size. +@item brc_only +skip_frame metadata indicates the number of missed frames before the current +frame. +@end table + @end table @subsection HEVC Options @@ -3742,6 +3760,24 @@ bitrate, @var{target_bitrate}, within the accuracy range @var{avbr_accuracy}, after a @var{avbr_Convergence} period. This method does not follow HRD and the instant bitrate is not capped or padded. +@item @var{skip_frame} +Use per-frame metadata "qsv_skip_frame" to skip frame when encoding. This option +defines the usage of this metadata. +@table @samp +@item no_skip +Frame skipping is disabled. +@item insert_dummy +Encoder inserts into bitstream frame where all macroblocks are encoded as +skipped. +@item insert_nothing +Similar to insert_dummy, but encoder inserts nothing into bitstream. The skipped +frames are still used in brc. For example, gop still include skipped frames, and +the frames after skipped frames will be larger in size. +@item brc_only +skip_frame metadata indicates the number of missed frames before the current +frame. +@end table + @end table @subsection MPEG2 Options diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 0db774ea63..4bfa65c575 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -329,6 +329,22 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, "MinQPI: %"PRIu8"; MaxQPI: %"PRIu8"; MinQPP: %"PRIu8"; MaxQPP: %"PRIu8"; MinQPB: %"PRIu8"; MaxQPB: %"PRIu8"\n", co2->MinQPI, co2->MaxQPI, co2->MinQPP, co2->MaxQPP, co2->MinQPB, co2->MaxQPB); av_log(avctx, AV_LOG_VERBOSE, "DisableDeblockingIdc: %"PRIu32" \n", co2->DisableDeblockingIdc); + + switch (co2->SkipFrame) { + case MFX_SKIPFRAME_NO_SKIP: + av_log(avctx, AV_LOG_VERBOSE, "SkipFrame: no_skip\n"); + break; + case MFX_SKIPFRAME_INSERT_DUMMY: + av_log(avctx, AV_LOG_VERBOSE, "SkipFrame: insert_dummy\n"); + break; + case MFX_SKIPFRAME_INSERT_NOTHING: + av_log(avctx, AV_LOG_VERBOSE, "SkipFrame: insert_nothing\n"); + break; + case MFX_SKIPFRAME_BRC_ONLY: + av_log(avctx, AV_LOG_VERBOSE, "SkipFrame: brc_only\n"); + break; + default: break; + } } if (co3) { @@ -991,6 +1007,8 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) q->old_max_qp_b = q->max_qp_b; if (q->mbbrc >= 0) q->extco2.MBBRC = q->mbbrc ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; + if (q->skip_frame >= 0) + q->extco2.SkipFrame = q->skip_frame; q->extco2.Header.BufferId = MFX_EXTBUFF_CODING_OPTION2; q->extco2.Header.BufferSz = sizeof(q->extco2); @@ -1911,6 +1929,19 @@ static int set_roi_encode_ctrl(AVCodecContext *avctx, const AVFrame *frame, return 0; } +static void set_skip_frame_encode_ctrl(AVCodecContext *avctx, const AVFrame *frame, + mfxEncodeCtrl *enc_ctrl) +{ + AVDictionaryEntry* skip_frame_dict = NULL; + if (!frame->metadata) + return; + skip_frame_dict = av_dict_get(frame->metadata, "qsv_skip_frame", NULL, 0); + if (!skip_frame_dict) + return; + enc_ctrl->SkipFrame = strtol(skip_frame_dict->value, NULL, 10); + return; +} + static int update_qp(AVCodecContext *avctx, QSVEncContext *q) { int updated = 0, new_qp = 0; @@ -2282,6 +2313,11 @@ static int encode_frame(AVCodecContext *avctx, QSVEncContext *q, if (ret < 0) goto free; } + if ((avctx->codec_id == AV_CODEC_ID_H264 || + avctx->codec_id == AV_CODEC_ID_H265) && + q->skip_frame != MFX_SKIPFRAME_NO_SKIP && + enc_ctrl && QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 13)) + set_skip_frame_encode_ctrl(avctx, frame, enc_ctrl); pkt.sync = av_mallocz(sizeof(*pkt.sync)); if (!pkt.sync) diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index eb79db9871..d46575ef78 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -127,6 +127,18 @@ { "avbr_accuracy", "Accuracy of the AVBR ratecontrol (unit of tenth of percent)", OFFSET(qsv.avbr_accuracy), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, \ { "avbr_convergence", "Convergence of the AVBR ratecontrol (unit of 100 frames)", OFFSET(qsv.avbr_convergence), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, UINT16_MAX, VE }, +#define QSV_OPTION_SKIP_FRAME \ +{ "skip_frame", "Allow frame skipping", OFFSET(qsv.skip_frame), AV_OPT_TYPE_INT, { .i64 = MFX_SKIPFRAME_NO_SKIP }, \ + MFX_SKIPFRAME_NO_SKIP, MFX_SKIPFRAME_BRC_ONLY, VE, "skip_frame" }, \ +{ "no_skip", "Frame skipping is disabled", \ + 0, AV_OPT_TYPE_CONST, { .i64 = MFX_SKIPFRAME_NO_SKIP }, .flags = VE, "skip_frame" }, \ +{ "insert_dummy", "Encoder inserts into bitstream frame where all macroblocks are encoded as skipped", \ + 0, AV_OPT_TYPE_CONST, { .i64 = MFX_SKIPFRAME_INSERT_DUMMY }, .flags = VE, "skip_frame" }, \ +{ "insert_nothing", "Encoder inserts nothing into bitstream", \ + 0, AV_OPT_TYPE_CONST, { .i64 = MFX_SKIPFRAME_INSERT_NOTHING }, .flags = VE, "skip_frame" }, \ +{ "brc_only", "skip_frame metadata indicates the number of missed frames before the current frame", \ + 0, AV_OPT_TYPE_CONST, { .i64 = MFX_SKIPFRAME_BRC_ONLY }, .flags = VE, "skip_frame" }, + extern const AVCodecHWConfigInternal *const ff_qsv_enc_hw_configs[]; typedef int SetEncodeCtrlCB (AVCodecContext *avctx, @@ -285,6 +297,7 @@ typedef struct QSVEncContext { int old_rc_max_rate; // This is used for SEI Timing reset int old_pic_timing_sei; + int skip_frame; } QSVEncContext; int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q); diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index 11aaabbd1b..0ff7356346 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -116,6 +116,7 @@ static const AVOption options[] = { QSV_OPTION_MAX_MIN_QP QSV_OPTION_SCENARIO QSV_OPTION_AVBR + QSV_OPTION_SKIP_FRAME { "cavlc", "Enable CAVLC", OFFSET(qsv.cavlc), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, #if QSV_HAVE_VCM diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 28d2cfcae1..95cf417f67 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -317,6 +317,7 @@ static const AVOption options[] = { QSV_OPTION_ADAPTIVE_B QSV_OPTION_SCENARIO QSV_OPTION_AVBR + QSV_OPTION_SKIP_FRAME { "idr_interval", "Distance (in I-frames) between IDR frames", OFFSET(qsv.idr_interval), AV_OPT_TYPE_INT, { .i64 = 0 }, -1, INT_MAX, VE, "idr_interval" }, { "begin_only", "Output an IDR-frame only at the beginning of the stream", 0, AV_OPT_TYPE_CONST, { .i64 = -1 }, 0, 0, VE, "idr_interval" },