From patchwork Fri Mar 11 09:00:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 34681 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp1511673nkb; Fri, 11 Mar 2022 01:08:37 -0800 (PST) X-Google-Smtp-Source: ABdhPJzbRcd9LqsX/qlthkf3nfJ6MDPo3xrnujfLtBmjsMwpnMNIxWmrRFlAlygsGjTHbkyWCCd9 X-Received: by 2002:a05:6402:1941:b0:413:2822:9c8 with SMTP id f1-20020a056402194100b00413282209c8mr8054398edz.13.1646989716892; Fri, 11 Mar 2022 01:08:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1646989716; cv=none; d=google.com; s=arc-20160816; b=B7jcuROo1Un1mpS3Gmg7reXt/O4gHBJRo/nj/gpfy+SGtEq5sXiU9Pqu826jsAcl7v VidQk/EGaij58tIVws0c3XW0S7I4B5vSo7KpeI07Z5Nfm/tPAwnhg1uNOKv04lwnKU8Z opb45bLo/1OuNQn3r2CPmQOeJVTBcYGHsn3CFoefQ8nh4FM34Vt58l2Bxq45u/Hbt5Ci tY1qaCEKJtHa2KCuS9LVU2emM7Ef/SN2DnAMlJC6sV5sSmY+tEzKK+QzzQLoE5UWXar2 0G+IujVXFNT7chmJa/4CjepCrAEKJ0GSZNjCle4cBPQny8SJHO9w+3Hc6TdjnmbJEYio h4WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=BXeH/6yjdkS7EmCfBqJVYc+03gVJci0O9BI5RJzCQ1g=; b=xkGUpDYdGypqhWHZ0Nj7jM+VPNH9AYtMOhxYyHizj9YP3YT4hFM20hZUPEm0feyuTM OpW6B6N9yP/0Jq08tonoSt+pyvaOqaM4Ft5iwkkvms2H+GYjKo5re6i1FckeCSOOjKr9 D63TPJm/xC98MBVBZs6ZaMGVaG3c32KO+iZ2b2T6l6j88hGuArL/yBVdSJvs4kumahIW dmTKrBY96WA6FV626KC755YeVeUE3xzjKLcEEpTBJYAWEttJq9nZ9UOk7ielWGQlh33z Ck95CUHbrFvhbWM2NvxGEC3ZbYS8Vg5n/7+D+OovfOl4SXhEsdjqt+8Ank9gEeQyrnda MAxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=eSyZ0GMe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q8-20020a170906770800b006b7faebc433si4480408ejm.853.2022.03.11.01.08.33; Fri, 11 Mar 2022 01:08:36 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=eSyZ0GMe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BCCCC68A61A; Fri, 11 Mar 2022 11:08:26 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D2F5168B279 for ; Fri, 11 Mar 2022 11:08:19 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646989705; x=1678525705; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=TSOKZTYpzwP+aZzGn0Ln+8YNXEsOz9rGM6GgoqGz2EA=; b=eSyZ0GMe502FIv5uDDXpZxhkIZBCqPQEolCPMF4p0lmOhMDBGYyy1m8A UsVi+3nWbjx6exT42FotEjXnx8EYQtL7SkS6weweG3aNOAZt9oIeWu51p qKCfJpQvgLuFRlYXxlL3kDTBb5B8c/AMfrWGZo9QowPa09iwWZeQmfmKg WsbJVg0aAWSU0ZIYBbPX8mh0x/c0Aq+zBR5PJSGmEdUOgi+8kWgW1lPAc wlt/NnLq1tCqBXa5WaBv+p9wNwtMPtMCBPFoaks6Sy4vOs0cbvTldqFiq d4YCf7m5Qs/9J3PbLe1nrXrl6UFZMyGrmrrKe9IaMevzdo+3RwKB2lfsu w==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="255483333" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="255483333" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:06:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="644872426" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga004.jf.intel.com with ESMTP; 11 Mar 2022 01:06:39 -0800 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Mar 2022 17:00:43 +0800 Message-Id: <20220311090046.262777-1-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/4] lavc/vaapi_encode_h265: Add GPB frame support for hevc_vaapi X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang , Linjie Fu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ryuKYabL5NFg From: Linjie Fu Use GPB frames to replace regular P/B frames if backend driver does not support it. - GPB: Generalized P and B picture. Regular P/B frames replaced by B frames with previous-predict only, L0 == L1. Normal B frames still have 2 different ref_lists and allow bi-prediction Signed-off-by: Linjie Fu Signed-off-by: Fei Wang --- update: 1. Add b to gpb. 2. Optimise debug message. libavcodec/vaapi_encode.c | 74 +++++++++++++++++++++++++++++++--- libavcodec/vaapi_encode.h | 2 + libavcodec/vaapi_encode_h265.c | 24 ++++++++++- 3 files changed, 93 insertions(+), 7 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 3bf379b1a0..bdba9726b2 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -848,9 +848,13 @@ static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, pic->b_depth = current_depth; vaapi_encode_add_ref(avctx, pic, start, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); + if (!ctx->b_to_gpb) + vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); + else + vaapi_encode_add_ref(avctx, pic, end, 0, 1, 0); + for (ref = end->refs[1]; ref; ref = ref->refs[1]) vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); } @@ -871,8 +875,11 @@ static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, vaapi_encode_add_ref(avctx, pic, pic, 0, 1, 0); vaapi_encode_add_ref(avctx, pic, start, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); + if (!ctx->b_to_gpb) + vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); + else + vaapi_encode_add_ref(avctx, pic, end, 0, 1, 0); for (ref = end->refs[1]; ref; ref = ref->refs[1]) vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); @@ -1845,6 +1852,51 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) ref_l1 = attr.value >> 16 & 0xffff; } + ctx->p_to_gpb = 0; + ctx->b_to_gpb = 0; + +#if VA_CHECK_VERSION(1, 9, 0) + if (!(ctx->codec->flags & FLAG_INTRA_ONLY || + avctx->gop_size <= 1)) { + attr = (VAConfigAttrib) { VAConfigAttribPredictionDirection }; + vas = vaGetConfigAttributes(ctx->hwctx->display, + ctx->va_profile, + ctx->va_entrypoint, + &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_WARNING, "Failed to query prediction direction " + "attribute: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR_EXTERNAL; + } else if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + av_log(avctx, AV_LOG_VERBOSE, "Driver does not report any additional " + "prediction constraints.\n"); + } else { + if (((ref_l0 > 0 || ref_l1 > 0) && !(attr.value & VA_PREDICTION_DIRECTION_PREVIOUS)) || + ((ref_l1 == 0) && (attr.value & (VA_PREDICTION_DIRECTION_FUTURE | VA_PREDICTION_DIRECTION_BI_NOT_EMPTY)))) { + av_log(avctx, AV_LOG_ERROR, "Driver report incorrect prediction " + "direction attribute.\n"); + return AVERROR_EXTERNAL; + } + + if (!(attr.value & VA_PREDICTION_DIRECTION_FUTURE)) { + if (ref_l0 > 0 && ref_l1 > 0) { + ctx->b_to_gpb = 1; + av_log(avctx, AV_LOG_VERBOSE, "Driver support previous prediction " + "only for B-frames.\n"); + } + } + + if (attr.value & VA_PREDICTION_DIRECTION_BI_NOT_EMPTY) { + if (ref_l0 > 0 && ref_l1 > 0) { + ctx->p_to_gpb = 1; + av_log(avctx, AV_LOG_VERBOSE, "Driver does not support P-frames, " + "replacing them with previous prediction only B-frames.\n"); + } + } + } + } +#endif + if (ctx->codec->flags & FLAG_INTRA_ONLY || avctx->gop_size <= 1) { av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); @@ -1855,14 +1907,24 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) return AVERROR(EINVAL); } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || ref_l1 < 1 || avctx->max_b_frames < 1) { - av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); + if (ctx->p_to_gpb) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " + "(supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); ctx->gop_size = avctx->gop_size; ctx->p_per_i = INT_MAX; ctx->b_per_p = 0; } else { - av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); + if (ctx->p_to_gpb) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " + "(supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); ctx->gop_size = avctx->gop_size; ctx->p_per_i = INT_MAX; ctx->b_per_p = avctx->max_b_frames; diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index b41604a883..4ce1df0c6f 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -328,6 +328,8 @@ typedef struct VAAPIEncodeContext { int idr_counter; int gop_counter; int end_of_stream; + int p_to_gpb; + int b_to_gpb; // Whether the driver supports ROI at all. int roi_allowed; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index e1dc53dfa9..25b9a707d2 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -717,7 +717,10 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, hpic->pic_type = 1; } else { VAAPIEncodePicture *irap_ref; - av_assert0(pic->refs[0] && pic->refs[1]); + if (!ctx->b_to_gpb) + av_assert0(pic->refs[0] && pic->refs[1]); + else + av_assert0(pic->refs[0]); for (irap_ref = pic; irap_ref; irap_ref = irap_ref->refs[1]) { if (irap_ref->type == PICTURE_TYPE_I) break; @@ -886,6 +889,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, VAAPIEncodePicture *pic, VAAPIEncodeSlice *slice) { + VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; VAAPIEncodeH265Picture *hpic = pic->priv_data; const H265RawSPS *sps = &priv->raw_sps; @@ -908,6 +912,9 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, sh->slice_type = hpic->slice_type; + if (sh->slice_type == HEVC_SLICE_P && ctx->p_to_gpb) + sh->slice_type = HEVC_SLICE_B; + sh->slice_pic_order_cnt_lsb = hpic->pic_order_cnt & (1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4)) - 1; @@ -1066,11 +1073,26 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); vslice->ref_pic_list0[0] = vpic->reference_frames[0]; + if (ctx->p_to_gpb && pic->type == PICTURE_TYPE_P) + // Reference for GPB B-frame, L0 == L1 + vslice->ref_pic_list1[0] = vpic->reference_frames[0]; } if (pic->nb_refs >= 2) { // Forward reference for B-frame. av_assert0(pic->type == PICTURE_TYPE_B); vslice->ref_pic_list1[0] = vpic->reference_frames[1]; + if (ctx->b_to_gpb && pic->type == PICTURE_TYPE_B) + // Reference for GPB B-frame, L0 == L1 + vslice->ref_pic_list1[0] = vpic->reference_frames[0]; + } + + if ((pic->type == PICTURE_TYPE_P && ctx->p_to_gpb) || + (pic->type == PICTURE_TYPE_B && ctx->b_to_gpb)) { + vslice->slice_type = HEVC_SLICE_B; + for (i = 0; i < FF_ARRAY_ELEMS(vslice->ref_pic_list0); i++) { + vslice->ref_pic_list1[i].picture_id = vslice->ref_pic_list0[i].picture_id; + vslice->ref_pic_list1[i].flags = vslice->ref_pic_list0[i].flags; + } } return 0; From patchwork Fri Mar 11 09:00:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 34682 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp1511786nkb; Fri, 11 Mar 2022 01:08:46 -0800 (PST) X-Google-Smtp-Source: ABdhPJwG0tPHjlwMfHOf+7k8NtV/eei8osnQBk3S6fFzLGvj7no3si6VuodCRmxWQAIUVWDXXYuC X-Received: by 2002:a05:6402:4390:b0:416:a29c:660c with SMTP id o16-20020a056402439000b00416a29c660cmr7948854edc.149.1646989726703; Fri, 11 Mar 2022 01:08:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1646989726; cv=none; d=google.com; s=arc-20160816; b=UcvRJHvcxeZxGvllBjsTI2eQb+82Aua2bbpMhSufyhaUZB1DkW16X1uwLzlonMpL11 z3H5EdNf2kRb1RZ/5C0xja1PdfH9kyOIugEoImVGmlE+lrczV9mT4/IKH4TSswI6hqgx vfnaS5O8/4p2ltOBw0dtma8+iFx7HYoD/NzZPibzLEwoMx4u7pR3WEsLJz73U9cZ+XEj 86U6NoBA4klfpdsCDOt3X2MtoDRQutOMDR6OjlwbLgznsGRrshUgUUlhXnVDVB7mgg02 /gC2zQDpqXY/jrhCN1uQY1f+/GJxYBgC45s3ccrEGfWIUF0CQKIygUD6e70ozFRICUTq 1V0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=+Jpj+I0DqeSqqf2C4HPd71gOgapkqzmUKAhW8sDOGXk=; b=kVCTbq3wt/lhWILmQUlCGIaUZ6CuRMbYV5O0BrIT9xP9mpYbuEDz/Ob2gMUxZY/5bf qGXD/Tc80cp/TzMB1LQ9t/mwUGMZroicsUj+3SsRsiqeW8+22W9+1UWO5U2JSDHKIC1T 2sZjGR7NcQtFFpPWnJbLDvfXiZ2PP9qfu2XGgOef+NVO6OSNnGz+fewEBIG9iKNsJpZR WhPkox0O5yxQho6jDC9eoQCp3jvTwLH/TsbEYWOE0GqqEODYylf8cpKx7SfccLi9+cAm m5Yw0BRYM36FAQEVgTC8tDwSqFaUvN3RgOZiHlqhAF5bTy+HZZx+kglUPEBazD3KJuVR D2fQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ONbeqRKG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ky9-20020a170907778900b006d0dee152d3si4729261ejc.101.2022.03.11.01.08.44; Fri, 11 Mar 2022 01:08:46 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ONbeqRKG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CDE7868B29A; Fri, 11 Mar 2022 11:08:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5FD5168B27A for ; Fri, 11 Mar 2022 11:08:21 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646989706; x=1678525706; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f6laOK4p5uvJpZWlfxG4KBE3b78CEaGBQfBLCKJtuRU=; b=ONbeqRKG5quNQGeImvAMMk3xZznqDAYzXULTPE/iA6CUpMOnqyO8cLkF FbxlWKaIebmb3OiAS724nEaqZ7UU2l+uZ/m5J4k6XgPsaWCc2omUuLle2 OHgkHIfWm3BQic6w1X7KPeF213LbEArsl3WYxlDdmnFT3wDAM9lyS3xwX BHPOnCOldveaucJ9MiyUSCmfRAYiRdY7UUfS8drKseB+geGBzveuNy/j2 1kBrtYg3z0DjhIfKZ7WLGnucz80uEu+0YR5AZhb3z0dCiJhqiij6M2m2X R4KSpZqegtNnlfnmCiF8slqI2CSGQGKnygn6nqhGCvSwhQQo4BJlQCcDf g==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="255483337" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="255483337" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:06:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="644872455" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga004.jf.intel.com with ESMTP; 11 Mar 2022 01:06:40 -0800 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Mar 2022 17:00:44 +0800 Message-Id: <20220311090046.262777-2-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220311090046.262777-1-fei.w.wang@intel.com> References: <20220311090046.262777-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/4] vaapi_encode: Move block size calculation after entrypoint selection X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson , Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: gHcNtU+hrszM From: Mark Thompson The block size can be dependent on the profile and entrypoint selected. It defaults to 16x16, with codecs able to override this choice with their own function. Signed-off-by: Fei Wang --- libavcodec/vaapi_encode.c | 16 ++++++++++++++++ libavcodec/vaapi_encode.h | 7 +++++++ libavcodec/vaapi_encode_h265.c | 32 ++++++++++++++++++++++++++------ libavcodec/vaapi_encode_mjpeg.c | 18 +++++++++++++++--- libavcodec/vaapi_encode_mpeg2.c | 3 --- libavcodec/vaapi_encode_vp8.c | 3 --- libavcodec/vaapi_encode_vp9.c | 16 ++++++++++++---- 7 files changed, 76 insertions(+), 19 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index bdba9726b2..d0aebad681 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -2074,6 +2074,8 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return 0; } + av_assert0(ctx->slice_block_height > 0 && ctx->slice_block_width > 0); + ctx->slice_block_rows = (avctx->height + ctx->slice_block_height - 1) / ctx->slice_block_height; ctx->slice_block_cols = (avctx->width + ctx->slice_block_width - 1) / @@ -2463,6 +2465,20 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; + if (ctx->codec->get_encoder_caps) { + err = ctx->codec->get_encoder_caps(avctx); + if (err < 0) + goto fail; + } else { + // Assume 16x16 blocks. + ctx->surface_width = FFALIGN(avctx->width, 16); + ctx->surface_height = FFALIGN(avctx->height, 16); + if (ctx->codec->flags & FLAG_SLICE_CONTROL) { + ctx->slice_block_width = 16; + ctx->slice_block_height = 16; + } + } + err = vaapi_encode_init_rate_control(avctx); if (err < 0) goto fail; diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 4ce1df0c6f..8a0b9a1e48 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -377,6 +377,13 @@ typedef struct VAAPIEncodeType { // factor depending on RC mode. int default_quality; + // Determine encode parameters like block sizes for surface alignment + // and slices. This may need to query the profile and entrypoint, + // which will be available when this function is called. If not set, + // assume that all blocks are 16x16 and that surfaces should be + // aligned to match this. + int (*get_encoder_caps)(AVCodecContext *avctx); + // Perform any extra codec-specific configuration after the // codec context is initialised (set up the private data and // add any necessary global parameters). diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 25b9a707d2..099a68cc1b 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -55,6 +55,10 @@ typedef struct VAAPIEncodeH265Picture { typedef struct VAAPIEncodeH265Context { VAAPIEncodeContext common; + // Encoder features. + uint32_t ctu_size; + uint32_t min_cb_size; + // User options. int qp; int aud; @@ -1098,6 +1102,27 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, return 0; } +static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeH265Context *priv = avctx->priv_data; + + if (!priv->ctu_size) { + priv->ctu_size = 32; + priv->min_cb_size = 16; + } + av_log(avctx, AV_LOG_VERBOSE, "Using CTU size %dx%d, " + "min CB size %dx%d.\n", priv->ctu_size, priv->ctu_size, + priv->min_cb_size, priv->min_cb_size); + + ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); + ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); + + ctx->slice_block_width = ctx->slice_block_height = priv->ctu_size; + + return 0; +} + static av_cold int vaapi_encode_h265_configure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -1167,6 +1192,7 @@ static const VAAPIEncodeType vaapi_encode_type_h265 = { .default_quality = 25, + .get_encoder_caps = &vaapi_encode_h265_get_encoder_caps, .configure = &vaapi_encode_h265_configure, .picture_priv_data_size = sizeof(VAAPIEncodeH265Picture), @@ -1212,12 +1238,6 @@ static av_cold int vaapi_encode_h265_init(AVCodecContext *avctx) VA_ENC_PACKED_HEADER_SLICE | // Slice headers. VA_ENC_PACKED_HEADER_MISC; // SEI - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); - - // CTU size is currently hard-coded to 32. - ctx->slice_block_width = ctx->slice_block_height = 32; - if (priv->qp > 0) ctx->explicit_qp = priv->qp; diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index 6206b23e5f..87a03f181b 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -434,6 +434,20 @@ static int vaapi_encode_mjpeg_init_slice_params(AVCodecContext *avctx, return 0; } +static av_cold int vaapi_encode_mjpeg_get_encoder_caps(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + const AVPixFmtDescriptor *desc; + + desc = av_pix_fmt_desc_get(ctx->input_frames->sw_format); + av_assert0(desc); + + ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); + ctx->surface_height = FFALIGN(avctx->height, 8 << desc->log2_chroma_h); + + return 0; +} + static av_cold int vaapi_encode_mjpeg_configure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -483,6 +497,7 @@ static const VAAPIEncodeType vaapi_encode_type_mjpeg = { .flags = FLAG_CONSTANT_QUALITY_ONLY | FLAG_INTRA_ONLY, + .get_encoder_caps = &vaapi_encode_mjpeg_get_encoder_caps, .configure = &vaapi_encode_mjpeg_configure, .default_quality = 80, @@ -509,9 +524,6 @@ static av_cold int vaapi_encode_mjpeg_init(AVCodecContext *avctx) ctx->desired_packed_headers = VA_ENC_PACKED_HEADER_RAW_DATA; - ctx->surface_width = FFALIGN(avctx->width, 8); - ctx->surface_height = FFALIGN(avctx->height, 8); - return ff_vaapi_encode_init(avctx); } diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index d084d8c4b9..f8809b08ab 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -623,9 +623,6 @@ static av_cold int vaapi_encode_mpeg2_init(AVCodecContext *avctx) ctx->desired_packed_headers = VA_ENC_PACKED_HEADER_SEQUENCE | VA_ENC_PACKED_HEADER_PICTURE; - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); - return ff_vaapi_encode_init(avctx); } diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index 7d4578f674..ab3c84a618 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -210,9 +210,6 @@ static av_cold int vaapi_encode_vp8_init(AVCodecContext *avctx) // adding them anyway. ctx->desired_packed_headers = 0; - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); - return ff_vaapi_encode_init(avctx); } diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index be706e3bd6..837adaa825 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -184,6 +184,17 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, return 0; } +static av_cold int vaapi_encode_vp9_get_encoder_caps(AVCodecContext *avctx) +{ + VAAPIEncodeContext *ctx = avctx->priv_data; + + // Surfaces must be aligned to 64x64 superblock boundaries. + ctx->surface_width = FFALIGN(avctx->width, 64); + ctx->surface_height = FFALIGN(avctx->height, 64); + + return 0; +} + static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -231,6 +242,7 @@ static const VAAPIEncodeType vaapi_encode_type_vp9 = { .picture_priv_data_size = sizeof(VAAPIEncodeVP9Picture), + .get_encoder_caps = &vaapi_encode_vp9_get_encoder_caps, .configure = &vaapi_encode_vp9_configure, .sequence_params_size = sizeof(VAEncSequenceParameterBufferVP9), @@ -251,10 +263,6 @@ static av_cold int vaapi_encode_vp9_init(AVCodecContext *avctx) // can write its own headers and there is no metadata to include. ctx->desired_packed_headers = 0; - // Surfaces must be aligned to superblock boundaries. - ctx->surface_width = FFALIGN(avctx->width, 64); - ctx->surface_height = FFALIGN(avctx->height, 64); - return ff_vaapi_encode_init(avctx); } From patchwork Fri Mar 11 09:00:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 34680 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp1511547nkb; Fri, 11 Mar 2022 01:08:24 -0800 (PST) X-Google-Smtp-Source: ABdhPJyfTWLWjPR+ghXLer5ASpYV8Dqt5yQU7M3g5D6mC+vJ6Hv4IrAs9xPmLvLcH60u3nfj98aJ X-Received: by 2002:a17:906:6d09:b0:6b9:2e0e:5bdd with SMTP id m9-20020a1709066d0900b006b92e0e5bddmr7508532ejr.246.1646989703948; Fri, 11 Mar 2022 01:08:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1646989703; cv=none; d=google.com; s=arc-20160816; b=K4kobkzNYr5r87qloYgwxf0U5bMlExeh2Bi8G12qD7DptlvxEDFutyuIKaMgrFlmG8 ykHMMaWuEh5HRprAZGmCgBTgy2ZVi/HeRQ0UqTRJJvgzVBGJfemC21X7X5s4JfYSWnN5 PjtEEB8XOT96axEgcBQ4WOb210hXnC5jqiNV7K+lx0Vo41N5m3BQzqNFgRxDrq+CCW7s tsdfHAZM75seiJarur/tkB4qj1eCILVRiS5NG6Z+uKbuOqr1xgbXX1L/aWCg2QQCcNUD 4mF5xAYxpCSXf4N78dEq9V6imy4rL+D4ODESMYQWOoYpIuHzUWxs2Yt9/geYZV/efZoc RFgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=G2KoUUbpO1w/VlN8NvB4l9uaIO/xEM/9R6bKrIhh6Nc=; b=zfxVgAPq6ERpQ5Pmi4r412GGP/8yr7wSQQDRvUAZzmgUusU0W/+I+Wli08OokyX3/F qPdcaW3FodbwTI5QzndlAbWFWkgqzIkcdYN3fZM/Nr8Dy5ke/xR22y/x2wqaFTcy/4Lr VFP3ijKctnwhD0SzMx/cgMHiCbVVC3pTsw9wtFDKwQZRq0CuGvdgQYrIIgYPEFxQcIJR F5S283qV1SdjPJ9lahbsrrp2t7VChO+gM3DMqhYiTZStAEnY5RvXfmgJ/Cp5Ahc1xkNf hgblV0/On5x9/n7XfxbnRIQQAaZ6Ot/2rwVWjdehh/SRuOJfactcHBwF8SYu31TKEkeR /sTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HqiCi86A; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 8-20020a170906024800b006ac3d8b2389si5048718ejl.847.2022.03.11.01.08.23; Fri, 11 Mar 2022 01:08:23 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HqiCi86A; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C4F0A68B27B; Fri, 11 Mar 2022 11:08:20 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 47A1D68B276 for ; Fri, 11 Mar 2022 11:08:13 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646989698; x=1678525698; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=05AFo1ISAkYtLxurthTKjGH8KcFyk6Hbhco3nIUnSXE=; b=HqiCi86AUnSKplGawPHdfYAY0FOeFhYORVvy1m6Cs8NYmTqhIFssFvuu C49aa738/FhgzoHv92hceqdDZy0kqxqaQb/sND4xRB+0ELy6J4lhNI1+y fsl+dA6ZjMgit+ZDGIZP1XF5nfMry2Pl+ZFh/tl/cmAQdp2COlreIcdCE JXvBAYxXwPOdW6oIn22HqHT/R1mLaItIi0Dou979yRgTSDJ4quFMghmQa FPWqFBoHEbSMDr8hf1lymCeBNvQfDbIzZShv+La1SVjnK0tZDSKd/Ta7z Tr4v/iXEOfayFW5hnaRchf3xEtbidgF6AJHq83V4TiqkydkXl52AhRknU w==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="255483344" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="255483344" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:06:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="644872471" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga004.jf.intel.com with ESMTP; 11 Mar 2022 01:06:42 -0800 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Mar 2022 17:00:45 +0800 Message-Id: <20220311090046.262777-3-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220311090046.262777-1-fei.w.wang@intel.com> References: <20220311090046.262777-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 3/4] vaapi_encode_h265: Explicitly set and correct some flags X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson , Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: CSpaK20fWCqy From: Mark Thompson max_14bit_constraint_flag should be set if the bit depth is not greater than 14 (currently always true). one_picture_only_flag should not be set because we don't support the still picture profiles. general_profile_compatibility_flag should be set according to general_profile_idc instead of bit depth. Signed-off-by: Fei Wang --- libavcodec/vaapi_encode_h265.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 099a68cc1b..e486e64eb9 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -303,17 +303,21 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) ptl->general_profile_idc = avctx->profile; ptl->general_tier_flag = priv->tier; - if (chroma_format == 1) { - ptl->general_profile_compatibility_flag[1] = bit_depth == 8; - ptl->general_profile_compatibility_flag[2] = bit_depth <= 10; + ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1; + + if (ptl->general_profile_compatibility_flag[1]) + ptl->general_profile_compatibility_flag[2] = 1; + if (ptl->general_profile_compatibility_flag[3]) { + ptl->general_profile_compatibility_flag[1] = 1; + ptl->general_profile_compatibility_flag[2] = 1; } - ptl->general_profile_compatibility_flag[4] = 1; ptl->general_progressive_source_flag = 1; ptl->general_interlaced_source_flag = 0; ptl->general_non_packed_constraint_flag = 1; ptl->general_frame_only_constraint_flag = 1; + ptl->general_max_14bit_constraint_flag = bit_depth <= 14; ptl->general_max_12bit_constraint_flag = bit_depth <= 12; ptl->general_max_10bit_constraint_flag = bit_depth <= 10; ptl->general_max_8bit_constraint_flag = bit_depth == 8; @@ -323,6 +327,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) ptl->general_max_monochrome_constraint_flag = chroma_format == 0; ptl->general_intra_constraint_flag = ctx->gop_size == 1; + ptl->general_one_picture_only_constraint_flag = 0; ptl->general_lower_bit_rate_constraint_flag = 1; From patchwork Fri Mar 11 09:00:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 34679 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:3486:0:0:0:0 with SMTP id ek6csp1511533nkb; Fri, 11 Mar 2022 01:08:23 -0800 (PST) X-Google-Smtp-Source: ABdhPJwlqD9HE6ljxXG/PCioVoH8ExJ6mYdjKTv7P5nKOX75g3NdvUtJURPbJL14l/4F1OLzZ6D3 X-Received: by 2002:a17:907:6da2:b0:6da:94a9:b7f3 with SMTP id sb34-20020a1709076da200b006da94a9b7f3mr7735803ejc.692.1646989702788; Fri, 11 Mar 2022 01:08:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1646989702; cv=none; d=google.com; s=arc-20160816; b=siv8inNjt1DGLLlLCVXTd4BbDs9mIDUe5r+dwI1C1ZsfUKN/aQGkAy5i67QdE1I84o 7oivIGtVfMK2z5K3Ne31fBo5MGbrFLvGbLFFf6fen8M3a6rM0+rJP04NJ7/I6N9zy19f TCdxtAVhGjN7rElno6OVZobPmGfZyz78ZrATFxsqw/x7cayUXro58jiOxDdCsNPYXYSz cZYNEtW3yoeeadjeGugd0UckN5t4/s8u63Uxl2HiPpWpdXvi+MqEYixVSfKlMInBxkSm gSvQs9ubjUSwzRNTtNi0eHwal4dPn/onMnB+EOrRlZo0l96K8Yu8p9a2+NlQeYc+4Wwr qR1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=RCa+UjI07ofKU9wUQ8YQCIlLzBTmEIl6SwPNST+M+PI=; b=IPi6p9SbrZLDzpQSjy/nw7EySIfqFTfOs3O+7zZhAwcIpJps69v3l/FkPq9GqSlcQV wdsLB82dr4CyZY/tHwYd9HvzH0uGKLdqii07hy1VMIn5ZmJu6CVjnsAh5zNZ3Sn1TICO 3YQZKOyNDu9gfhf7YCTPTiZE3vC+3tJtrm58EdjCFTvZVAuxf3dptw1jM21Sro5x8SDI fAHVEeVmlQwpWrIS6gzfGxTlVcYQRorhCSuMxp+rSE/t+LpZdkPqWrOKgGoZHh/3xD1U FgnMeXwUELeMSQ8RoVEp5ff1DZ1HDa1XIWH44H56/jVVKi1XjAu6Lq7A38rXxvKFnaTz WsnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=KLvQm4P0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ka3-20020a170907990300b006db63b1d103si4288169ejc.989.2022.03.11.01.08.03; Fri, 11 Mar 2022 01:08:22 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=KLvQm4P0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C6DD768B249; Fri, 11 Mar 2022 11:07:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0511868B178 for ; Fri, 11 Mar 2022 11:07:51 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646989677; x=1678525677; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ETpSrqQ+AT/J/2C4E604taC7nXswxDLTupB9qSRs/Hg=; b=KLvQm4P0AXNI05jDDDqo1akRZIdl32ws5vYeV9hDkz1gMq/Oo9ricl82 /zG9tkNaDbqSDHyo+W1WUuc9wBYOlFkyKHhM4qDrL1G3Bj8OmkAZWmF+Y M9rkT0lFjns/5rN9g12XKWhrw8AzZ8meTjCOpwIJa1OSAvjNhmhQQrcvf pzseH+oS+tzKQT3VEJCjPTZgVbZ+GDwBkp1ReY+nJb8ko9YWxjq55nDZp 8sUQ2jRuTbrCkraevAR46mQP102lPxoPSTf9Gga72WWC9KDnIEoU1c5MX kqgM/r2mrci9wkSEPxPMGejv046Hd1xi8c0b5fhQ9uMm1iMy0mFMiERDM A==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="236137662" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="236137662" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2022 01:06:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="644872489" Received: from t.sh.intel.com ([10.239.159.147]) by orsmga004.jf.intel.com with ESMTP; 11 Mar 2022 01:06:43 -0800 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 11 Mar 2022 17:00:46 +0800 Message-Id: <20220311090046.262777-4-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220311090046.262777-1-fei.w.wang@intel.com> References: <20220311090046.262777-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 4/4] vaapi_encode_h265: Query encoding block sizes and features X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson , Fei Wang Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: B/nodC12BVjt From: Mark Thompson Signed-off-by: Fei Wang --- libavcodec/vaapi_encode_h265.c | 114 +++++++++++++++++++++++++++++++-- 1 file changed, 108 insertions(+), 6 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index e486e64eb9..11eca5989b 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -56,6 +56,9 @@ typedef struct VAAPIEncodeH265Context { VAAPIEncodeContext common; // Encoder features. + uint32_t va_features; + // Block size info. + uint32_t va_bs; uint32_t ctu_size; uint32_t min_cb_size; @@ -427,9 +430,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) vps->vps_max_latency_increase_plus1[i]; } - // These have to come from the capabilities of the encoder. We have no - // way to query them, so just hardcode parameters which work on the Intel - // driver. + // These values come from the capabilities of the first encoder + // implementation in the i965 driver on Intel Skylake. They may + // fail badly with other platforms or drivers. // CTB size from 8x8 to 32x32. sps->log2_min_luma_coding_block_size_minus3 = 0; sps->log2_diff_max_min_luma_coding_block_size = 2; @@ -447,6 +450,42 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) sps->pcm_enabled_flag = 0; +// update sps setting according to queried result +#if VA_CHECK_VERSION(1, 13, 0) + if (priv->va_features) { + VAConfigAttribValEncHEVCFeatures features = { .value = priv->va_features }; + + // Enable feature if get queried result is VA_FEATURE_SUPPORTED | VA_FEATURE_REQUIRED + sps->amp_enabled_flag = + !!features.bits.amp; + sps->sample_adaptive_offset_enabled_flag = + !!features.bits.sao; + sps->sps_temporal_mvp_enabled_flag = + !!features.bits.temporal_mvp; + sps->pcm_enabled_flag = + !!features.bits.pcm; + } + + if (priv->va_bs) { + VAConfigAttribValEncHEVCBlockSizes bs = { .value = priv->va_bs }; + sps->log2_min_luma_coding_block_size_minus3 = + ff_ctz(priv->min_cb_size) - 3; + sps->log2_diff_max_min_luma_coding_block_size = + ff_ctz(priv->ctu_size) - ff_ctz(priv->min_cb_size); + + sps->log2_min_luma_transform_block_size_minus2 = + bs.bits.log2_min_luma_transform_block_size_minus2; + sps->log2_diff_max_min_luma_transform_block_size = + bs.bits.log2_max_luma_transform_block_size_minus2 - + bs.bits.log2_min_luma_transform_block_size_minus2; + + sps->max_transform_hierarchy_depth_inter = + bs.bits.max_max_transform_hierarchy_depth_inter; + sps->max_transform_hierarchy_depth_intra = + bs.bits.max_max_transform_hierarchy_depth_intra; + } +#endif + // STRPSs should ideally be here rather than defined individually in // each slice, but the structure isn't completely fixed so for now // don't bother. @@ -539,6 +578,23 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) pps->cu_qp_delta_enabled_flag = (ctx->va_rc_mode != VA_RC_CQP); pps->diff_cu_qp_delta_depth = 0; +// update pps setting according to queried result +#if VA_CHECK_VERSION(1, 13, 0) + if (priv->va_features) { + VAConfigAttribValEncHEVCFeatures features = { .value = priv->va_features }; + if (ctx->va_rc_mode != VA_RC_CQP) + pps->cu_qp_delta_enabled_flag = + !!features.bits.cu_qp_delta; + + pps->transform_skip_enabled_flag = + !!features.bits.transform_skip; + // set diff_cu_qp_delta_depth as its max value if cu_qp_delta enabled. Otherwise + // 0 will make cu_qp_delta invalid. + if (pps->cu_qp_delta_enabled_flag) + pps->diff_cu_qp_delta_depth = sps->log2_diff_max_min_luma_coding_block_size; + } +#endif + if (ctx->tile_rows && ctx->tile_cols) { int uniform_spacing; @@ -640,8 +696,8 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) .coded_buf = VA_INVALID_ID, - .collocated_ref_pic_index = 0xff, - + .collocated_ref_pic_index = sps->sps_temporal_mvp_enabled_flag ? + 0 : 0xff, .last_picture = 0, .pic_init_qp = pps->init_qp_minus26 + 26, @@ -674,6 +730,8 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) .entropy_coding_sync_enabled_flag = pps->entropy_coding_sync_enabled_flag, .loop_filter_across_tiles_enabled_flag = pps->loop_filter_across_tiles_enabled_flag, + .pps_loop_filter_across_slices_enabled_flag = + pps->pps_loop_filter_across_slices_enabled_flag, .scaling_list_data_present_flag = (sps->sps_scaling_list_data_present_flag | pps->pps_scaling_list_data_present_flag), .screen_content_flag = 0, @@ -1004,10 +1062,13 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, sh->num_long_term_sps = 0; sh->num_long_term_pics = 0; + // when this flag is not present, it is inerred to 1. + sh->collocated_from_l0_flag = 1; sh->slice_temporal_mvp_enabled_flag = sps->sps_temporal_mvp_enabled_flag; if (sh->slice_temporal_mvp_enabled_flag) { - sh->collocated_from_l0_flag = sh->slice_type == HEVC_SLICE_B; + if (sh->slice_type == HEVC_SLICE_B) + sh->collocated_from_l0_flag = 1; sh->collocated_ref_idx = 0; } @@ -1112,6 +1173,47 @@ static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; +#if VA_CHECK_VERSION(1, 13, 0) + { + VAConfigAttribValEncHEVCBlockSizes block_size; + VAConfigAttrib attr; + VAStatus vas; + + attr.type = VAConfigAttribEncHEVCFeatures; + vas = vaGetConfigAttributes(ctx->hwctx->display, ctx->va_profile, + ctx->va_entrypoint, &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query encoder " + "features, using guessed defaults.\n"); + return AVERROR_EXTERNAL; + } else if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + av_log(avctx, AV_LOG_WARNING, "Driver does not advertise " + "encoder features, using guessed defaults.\n"); + } else { + priv->va_features = attr.value; + } + + attr.type = VAConfigAttribEncHEVCBlockSizes; + vas = vaGetConfigAttributes(ctx->hwctx->display, ctx->va_profile, + ctx->va_entrypoint, &attr, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query encoder " + "block size, using guessed defaults.\n"); + return AVERROR_EXTERNAL; + } else if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { + av_log(avctx, AV_LOG_WARNING, "Driver does not advertise " + "encoder block size, using guessed defaults.\n"); + } else { + priv->va_bs = block_size.value = attr.value; + + priv->ctu_size = + 1 << block_size.bits.log2_max_coding_tree_block_size_minus3 + 3; + priv->min_cb_size = + 1 << block_size.bits.log2_min_luma_coding_block_size_minus3 + 3; + } + } +#endif + if (!priv->ctu_size) { priv->ctu_size = 32; priv->min_cb_size = 16;