From patchwork Mon Jan 24 02:59:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Wenbin" X-Patchwork-Id: 33758 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp2375385iog; Sun, 23 Jan 2022 19:02:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJzFkIAuWgrNq17b/jjUWj0vbvruISxiIWnG6aR52UjkDqzTjQYoDg+gkWMg8xdk8+UVcjbK X-Received: by 2002:a17:906:9601:: with SMTP id s1mr1984538ejx.713.1642993327452; Sun, 23 Jan 2022 19:02:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1642993327; cv=none; d=google.com; s=arc-20160816; b=goGPRzrMB1/SP7oXJOsjj6t0hma+yKwNhKRT0T729cTrMOgHxUYDiusUTj21HXl1o8 3k0u1Zg50Q5/fFtM6xT84UiK2KbzAOJMHBB/5LNI27AKVYqPhzb0Vrm92OWF17Ptf0yB b87EV9M/VlGpZr9gntbvMDmyd0vmUFGoW2L3VgmCaR/G7uU8VAHFvvvZk487DGW7oNRT 2HVsrKKUTi953TtbGwmhUF0Da0WfRrebcbSft8lINzUdK2RdneerkxCiRd1kgrzPXtGz lyxeo2DEevJ03Du/reKTqCml/7oCPU4vjQy2gcZtk0UNA2bhCydScsCzhajZCoV9eQJ/ rWkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=v7uUyjpvwbKkmzII4NMrcBzyFjJsIg5vJzNksBnnW6Q=; b=Ag5oanKi6Wq7Yci8XwWAgBHbk7mrw8F/wB7STbY8ghu+kDz7+ELiZdQVHg7Z80CdWC wgxtzepqERTVqZ+sPeW7/Ub3qexhCKg6ZY668cA25zAXkUutdgtpNShKP1Ozrc2FSV9/ EB9tDsbBs7HyA9PUIWYDYBpdjEFISfPTNw0gr+hQx72U9w33tj8Ic0vMVLLPvUkw7HZ5 WdJK7UsfqPdP9X1kvdYgkoaTVkaK3Dwj0+rzLnXCnJzbe5v41aubSZ72mhi3FZ4Dpd1L cVMOzKEQsvCvwXNhkgHbvvz9ijEjMF6ZpOo6pY7LNKFXY+VCkKE8QbV0XtelcEzvXCxb olAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=dDAwkk3K; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hd13si763059ejc.889.2022.01.23.19.02.07; Sun, 23 Jan 2022 19:02:07 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=dDAwkk3K; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AC42068B03C; Mon, 24 Jan 2022 05:01:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CF6B568B037 for ; Mon, 24 Jan 2022 05:01:43 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1642993309; x=1674529309; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=2iI0XzgPW/NXImLG9LE1SW5LwHWjiX8Ptp8FYVnneYU=; b=dDAwkk3KZOSwfSVl4gp/QQTQO3D1Px8+2HFwWmq17VrmMQlFPDl6JAiD uyW0awBQFeRI+dc/rrEHGk0fcnqePKOKP8Rv457m8/NFZl+KlLZxrZ0SN gvInLl7ZkF+VopvYv5qAdvHBuZqW+fpotOxMccH+9O0PfPHdOufe3ZaUD rSarp9wlwuz06hbounyT98psmx1w0zBNW8QNbOUxf6gyvI5mPWUjN37Ip rvaClrvCtyJsqjOF7iuBD2QBc60Uef01rkL3ELsOHrvTrNnQdmCrDk0gV CB29L/pl0/YUGJoBvNXezp1ZQLtO0zLEnB76jDH6FfYXFsthAo2QQfwjH g==; X-IronPort-AV: E=McAfee;i="6200,9189,10236"; a="246168135" X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="246168135" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2022 19:01:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,311,1635231600"; d="scan'208";a="673480639" Received: from chenwenbin-z390-aorus-ultra.sh.intel.com ([10.239.35.110]) by fmsmga001.fm.intel.com with ESMTP; 23 Jan 2022 19:01:34 -0800 From: Wenbin Chen To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Jan 2022 10:59:55 +0800 Message-Id: <20220124025955.204168-3-wenbin.chen@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220124025955.204168-1-wenbin.chen@intel.com> References: <20220124025955.204168-1-wenbin.chen@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/3] libavcodec/qsvenc: Add intra refresh to hevc_qsv and add new intra refresh parameter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: D5q8xNjg0o/k Add intra refresh support to hevc_qsv as well. Add an new intra refresh type: "horizontal", and an new param ref_cycle_dist. This param specify the distance between the beginnings of the intra-refresh cycles in frames. Signed-off-by: Wenbin Chen --- doc/encoders.texi | 26 +++++++++++++++++++++++++- libavcodec/qsvenc.c | 23 ++++++++++++++--------- libavcodec/qsvenc.h | 1 + libavcodec/qsvenc_h264.c | 7 +++++-- libavcodec/qsvenc_hevc.c | 9 +++++++++ 5 files changed, 54 insertions(+), 12 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 6c1c4df57a..4e35e50e4d 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3344,7 +3344,8 @@ Specifies intra refresh type. The major goal of intra refresh is improvement of error resilience without significant impact on encoded bitstream size caused by I frames. The SDK encoder achieves this by encoding part of each frame in refresh cycle using intra MBs. @var{none} means no refresh. @var{vertical} means -vertical refresh, by column of MBs. +vertical refresh, by column of MBs. To enable intra refresh, B frame should be +set to 0. @item @var{int_ref_cycle_size} Specifies number of pictures within refresh cycle starting from 2. 0 and 1 are @@ -3355,6 +3356,9 @@ Specifies QP difference for inserted intra MBs. This is signed value in [-51, 51] range if target encoding bit-depth for luma samples is 8 and this range is [-63, 63] for 10 bit-depth or [-75, 75] for 12 bit-depth respectively. +@item @var{int_ref_cycle_dist} +Distance between the beginnings of the intra-refresh cycles in frames. + @item @var{profile} @table @samp @item unknown @@ -3463,6 +3467,26 @@ Insert picture timing SEI with pic_struct_syntax element. @item @var{transform_skip} Turn this option ON to enable transformskip. It is supported on platform equal or newer than ICL. + +@item @var{int_ref_type} +Specifies intra refresh type. The major goal of intra refresh is improvement of +error resilience without significant impact on encoded bitstream size caused by +I frames. The SDK encoder achieves this by encoding part of each frame in +refresh cycle using intra MBs. @var{none} means no refresh. @var{vertical} means +vertical refresh, by column of MBs. To enable intra refresh, B frame should be +set to 0. + +@item @var{int_ref_cycle_size} +Specifies number of pictures within refresh cycle starting from 2. 0 and 1 are +invalid values. + +@item @var{int_ref_qp_delta} +Specifies QP difference for inserted intra MBs. This is signed value in +[-51, 51] range if target encoding bit-depth for luma samples is 8 and this +range is [-63, 63] for 10 bit-depth or [-75, 75] for 12 bit-depth respectively. + +@item @var{int_ref_cycle_dist} +Distance between the beginnings of the intra-refresh cycles in frames. @end table @subsection MPEG2 Options diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index a8d876d6d9..af1529936e 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -267,8 +267,10 @@ static void dump_video_param(AVCodecContext *avctx, QSVEncContext *q, #if QSV_HAVE_CO2 av_log(avctx, AV_LOG_VERBOSE, - "RecoveryPointSEI: %s IntRefType: %"PRIu16"; IntRefCycleSize: %"PRIu16"; IntRefQPDelta: %"PRId16"\n", - print_threestate(co->RecoveryPointSEI), co2->IntRefType, co2->IntRefCycleSize, co2->IntRefQPDelta); + "RecoveryPointSEI: %s IntRefType: %"PRIu16"; IntRefCycleSize: %"PRIu16 + "; IntRefQPDelta: %"PRId16"; IntRefCycleDist: %"PRId16"\n", + print_threestate(co->RecoveryPointSEI), co2->IntRefType, co2->IntRefCycleSize, + co2->IntRefQPDelta, co3->IntRefCycleDist); av_log(avctx, AV_LOG_VERBOSE, "MaxFrameSize: %d; ", co2->MaxFrameSize); #if QSV_HAVE_MAX_SLICE_SIZE @@ -865,13 +867,6 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) #if QSV_HAVE_CO2 if (avctx->codec_id == AV_CODEC_ID_H264) { - if (q->int_ref_type >= 0) - q->extco2.IntRefType = q->int_ref_type; - if (q->int_ref_cycle_size >= 0) - q->extco2.IntRefCycleSize = q->int_ref_cycle_size; - if (q->int_ref_qp_delta != INT16_MIN) - q->extco2.IntRefQPDelta = q->int_ref_qp_delta; - if (q->bitrate_limit >= 0) q->extco2.BitrateLimit = q->bitrate_limit ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; if (q->mbbrc >= 0) @@ -900,6 +895,12 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) q->extco2.ExtBRC = q->extbrc ? MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF; if (q->max_frame_size >= 0) q->extco2.MaxFrameSize = q->max_frame_size; + if (q->int_ref_type >= 0) + q->extco2.IntRefType = q->int_ref_type; + if (q->int_ref_cycle_size >= 0) + q->extco2.IntRefCycleSize = q->int_ref_cycle_size; + if (q->int_ref_qp_delta != INT16_MIN) + q->extco2.IntRefQPDelta = q->int_ref_qp_delta; #if QSV_HAVE_MAX_SLICE_SIZE if (q->max_slice_size >= 0) q->extco2.MaxSliceSize = q->max_slice_size; @@ -973,6 +974,10 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) av_log(avctx, AV_LOG_WARNING, "Please set max_b_frames(-bf) to 0 to enable P-pyramid\n"); } +#endif +#if QSV_VERSION_ATLEAST(1, 16) + if (q->int_ref_cycle_dist >= 0) + q->extco3.IntRefCycleDist = q->int_ref_cycle_dist; #endif } diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 65f035045c..5fa0b6f5ba 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -195,6 +195,7 @@ typedef struct QSVEncContext { int int_ref_type; int int_ref_cycle_size; int int_ref_qp_delta; + int int_ref_cycle_dist; int recovery_point_sei; int repeat_pps; diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index 9f127607b7..bec3633268 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -126,13 +126,16 @@ static const AVOption options[] = { { "4x" , NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_LOOKAHEAD_DS_4x }, INT_MIN, INT_MAX, VE, "look_ahead_downsampling" }, #endif - { "int_ref_type", "Intra refresh type", OFFSET(qsv.int_ref_type), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE, "int_ref_type" }, + { "int_ref_type", "Intra refresh type. B frames should be set to 0.", OFFSET(qsv.int_ref_type), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE, "int_ref_type" }, { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, .flags = VE, "int_ref_type" }, { "vertical", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, .flags = VE, "int_ref_type" }, + { "horizontal", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 2 }, .flags = VE, "int_ref_type" }, { "int_ref_cycle_size", "Number of frames in the intra refresh cycle", OFFSET(qsv.int_ref_cycle_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE }, { "int_ref_qp_delta", "QP difference for the refresh MBs", OFFSET(qsv.int_ref_qp_delta), AV_OPT_TYPE_INT, { .i64 = INT16_MIN }, INT16_MIN, INT16_MAX, VE }, { "recovery_point_sei", "Insert recovery point SEI messages", OFFSET(qsv.recovery_point_sei), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, - +#if QSV_VERSION_ATLEAST(1, 16) + { "int_ref_cycle_dist", "Distance between the beginnings of the intra-refresh cycles in frames", OFFSET(qsv.int_ref_cycle_dist), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT16_MAX, VE }, +#endif { "profile", NULL, OFFSET(qsv.profile), AV_OPT_TYPE_INT, { .i64 = MFX_PROFILE_UNKNOWN }, 0, INT_MAX, VE, "profile" }, { "unknown" , NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_PROFILE_UNKNOWN }, INT_MIN, INT_MAX, VE, "profile" }, { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = MFX_PROFILE_AVC_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index 91a316f9c8..5cac141c4d 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -258,6 +258,15 @@ static const AVOption options[] = { #if QSV_VERSION_ATLEAST(1, 26) { "transform_skip", "Turn this option ON to enable transformskip", OFFSET(qsv.transform_skip), AV_OPT_TYPE_INT, { .i64 = -1}, -1, 1, VE}, #endif + { "int_ref_type", "Intra refresh type. B frames should be set to 0", OFFSET(qsv.int_ref_type), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE, "int_ref_type" }, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, .flags = VE, "int_ref_type" }, + { "vertical", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, .flags = VE, "int_ref_type" }, + { "horizontal", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 2 }, .flags = VE, "int_ref_type" }, + { "int_ref_cycle_size", "Number of frames in the intra refresh cycle", OFFSET(qsv.int_ref_cycle_size), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, UINT16_MAX, VE }, + { "int_ref_qp_delta", "QP difference for the refresh MBs", OFFSET(qsv.int_ref_qp_delta), AV_OPT_TYPE_INT, { .i64 = INT16_MIN }, INT16_MIN, INT16_MAX, VE }, +#if QSV_VERSION_ATLEAST(1, 16) + { "int_ref_cycle_dist", "Distance between the beginnings of the intra-refresh cycles in frames", OFFSET(qsv.int_ref_cycle_dist), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT16_MAX, VE }, +#endif { NULL }, };