From patchwork Thu Sep 12 08:23:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51545 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:14c:b0:48e:c0f8:d0de with SMTP id h12csp778716vqi; Thu, 12 Sep 2024 01:49:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUe3pBIhS2icm645v0vIKxPxLl0e4/f84hzu2xRHaYNXc8LDQAN1P8YbTmeOgYp4XGiXnsMDu/zpe+kjSmA/oor@gmail.com X-Google-Smtp-Source: AGHT+IHGw75/AdFenneOxAEqP44awCVJdpgMtwzkeDUTsrD+xQmv4coh+tKhKsW8A/cxJjoX2v0J X-Received: by 2002:a05:6512:3d8e:b0:530:aeea:27e1 with SMTP id 2adb3069b0e04-53678fec5bfmr1069350e87.50.1726130952410; Thu, 12 Sep 2024 01:49:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726130952; cv=none; d=google.com; s=arc-20240605; b=bzPXJ1iHVOYPuu0JaREWAzrFdnYJGnol30XRtdelR68bYrXJVBnPJu0zSXGxfzNcGw eViEOfsW7y3rnm7RjIoVrh04vWbCTwSkM+HBIpE7yPcpIFORplvWfqk5fHW9h5TnrYzS T4lhhViX5Vr8rA5GZqtZSz/mS54AgGK2wzeSPU9RWyqHLLRQj1+s81ehqP7N3JD82Bgz b+hhrXXF6uDNs6Ubx7wjxAy9K5YUfjxGwGqT/zF2p3yeXecDl5esVEImiYPy2olhVSvA Js3cdftGFQU2tXVVaJusZBGuKNux2SZ0eIdlGbt6+Xta+ygV5lOsvMjYr04AoUHOKkKL q7Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to :delivered-to; bh=TwTpvTmC/5wFp6YOjH/xkmbkWA91p/TTEamyAbohAOw=; fh=Ont/7iEF8fqtg5HC8SyRso6lu6K4YqkD/WYfFN9cWQs=; b=KpV49sYFi7oV6GaEhN7GJxb02TxS+xniATZ4YIJhT47XYHqHZOWTbSSSGLs91SjsI6 JXdPNS6axgsbjzI7eINEKow5Js/1Zk5RrSqCmhTOzgojVtxT3tF//JQ2TiwfCwukgJhF M11rvgCUZXIICS07gjnMpOFZ39JiCiZVemKWeE/2R+B9wj+7ooeE78LDR0hZDZ40EHf1 3AEldsDHI72Pv4ps2piY1ySAGiW7blgrzocSq45l5qHo4e2gJ8iFdsUWaA5gPig4EeTf keLWy9Ol9Ituhczj7NjUvm3UuXMwbVuUNLkj3Z5Rug5QE0yuT4FEoLiHd107iLd5DbjA lRrw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c3ebda84c0si7941044a12.495.2024.09.12.01.49.12; Thu, 12 Sep 2024 01:49:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 61C4F68DE04; Thu, 12 Sep 2024 11:23:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 14C4168D98A for ; Thu, 12 Sep 2024 11:23:42 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Thu, 12 Sep 2024 10:23:16 +0200 Message-ID: <20240912082334.925402-1-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] cbs_h265: add raw filler encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne , Dave Airlie Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NA7bbxB6woRh From: Dave Airlie --- libavcodec/cbs_h2645.c | 17 ++++++++++++++++ libavcodec/cbs_h265.h | 6 ++++++ libavcodec/cbs_h265_syntax_template.c | 29 +++++++++++++++++++++++++++ 3 files changed, 52 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 4abd9e0c2e..1969ad8116 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1055,6 +1055,14 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, } break; + case HEVC_NAL_FD_NUT: + { + err = cbs_h265_read_filler(ctx, &gbc, unit->content); + if (err < 0) + return err; + } + break; + case HEVC_NAL_SEI_PREFIX: case HEVC_NAL_SEI_SUFFIX: { @@ -1497,6 +1505,14 @@ static int cbs_h265_write_nal_unit(CodedBitstreamContext *ctx, } break; + case HEVC_NAL_FD_NUT: + { + err = cbs_h265_write_filler(ctx, pbc, unit->content); + if (err < 0) + return err; + } + break; + case HEVC_NAL_SEI_PREFIX: case HEVC_NAL_SEI_SUFFIX: { @@ -2006,6 +2022,7 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { CBS_UNIT_TYPE_INTERNAL_REF(HEVC_NAL_PPS, H265RawPPS, extension_data.data), CBS_UNIT_TYPE_POD(HEVC_NAL_AUD, H265RawAUD), + CBS_UNIT_TYPE_POD(HEVC_NAL_FD_NUT, H265RawFiller), // Slices of non-IRAP pictures. CBS_UNIT_RANGE_INTERNAL_REF(HEVC_NAL_TRAIL_N, HEVC_NAL_RASL_R, diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index 892a35bd22..26a5a34fe9 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -741,6 +741,12 @@ typedef struct H265RawSEI { SEIRawMessageList message_list; } H265RawSEI; +typedef struct H265RawFiller { + H265RawNALUnitHeader nal_unit_header; + + uint32_t filler_size; +} H265RawFiller; + typedef struct CodedBitstreamH265Context { // Reader/writer context in common with the H.264 implementation. CodedBitstreamH2645Context common; diff --git a/libavcodec/cbs_h265_syntax_template.c b/libavcodec/cbs_h265_syntax_template.c index 9f0281b8e8..1c11514435 100644 --- a/libavcodec/cbs_h265_syntax_template.c +++ b/libavcodec/cbs_h265_syntax_template.c @@ -2364,3 +2364,32 @@ static int FUNC(sei)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } + + +static int FUNC(filler)(CodedBitstreamContext *ctx, RWContext *rw, + H265RawFiller *current) +{ + int err; + + HEADER("Filler Data"); + + CHECK(FUNC(nal_unit_header)(ctx, rw, ¤t->nal_unit_header, + HEVC_NAL_FD_NUT)); + +#ifdef READ + while (show_bits(rw, 8) == 0xff) { + fixed(8, ff_byte, 0xff); + ++current->filler_size; + } +#else + { + uint32_t i; + for (i = 0; i < current->filler_size; i++) + fixed(8, ff_byte, 0xff); + } +#endif + + CHECK(FUNC(rbsp_trailing_bits)(ctx, rw)); + + return 0; +} From patchwork Thu Sep 12 08:23:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51547 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:14c:b0:48e:c0f8:d0de with SMTP id h12csp820180vqi; Thu, 12 Sep 2024 03:19:11 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXzMw3CBCh7EylPUPSO6JyBiz+vEyyQ3bnt7WxjwqBBKBVdVyEshUh9dx4a9qaxly+xBDyaSaGDI2oQ6sG8T1rl@gmail.com X-Google-Smtp-Source: AGHT+IFJlg0sKV1QXuzRtysNwCMIFyzq81XeZjZOkQqAFqeoBV0tE/l0wpUzgqzzWcFavB2eiI3b X-Received: by 2002:a05:6512:12c8:b0:533:46cc:a71e with SMTP id 2adb3069b0e04-53678fec522mr1377970e87.54.1726136351050; Thu, 12 Sep 2024 03:19:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726136351; cv=none; d=google.com; s=arc-20240605; b=iO1GoMAX1RoJqDorhd5yVeuZmzmxuG8Fz0hr4u1kwQeYoUK6R2TcK/zB72zyn7AuaJ JGvuaijkti5IxR19YdzJiSXqtRBrlh1rK6F1Tw8eOFQuCi2d1tsazv65tIYgRmiyef7i J6HULWQeAJKF55aJ3I0zagT7PV5Yeomkcpg/rEFiAWPUtAtzYNCtk70Tr4142uqHPFfP kYK0P295g62uBwKzvZ7cUZDL/TcNDSsNTroAIG7zr1Gx5uuqEUFOywgSyB3Bkf8csKr9 S4nUm5j1PIqL7/iWM4/Z+EDOtY6I+yUCNYJfQJd3J3cFl9uIBZF2m7nP/4F5LjsrFEDV PYCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=LuS+13CmyJqGNYIqhOzj+sGviFECWC8QHdzPxQLodvQ=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=Atbq2rjvuY8OufnPLBa50L1ZSUiO38Jwn0INfSKOhMM5JTcUSxzKSbhoSLA3hL536J nYi9CU/mZW7urZPxdlV1uf0woHIoIAQYb9RUhil0+zTBDkYQUHa48TuLOKdT8pnU/Baa vTR8POwfaOiWrqFrss+W7qkv4af61C92IrcQx0E66zh3RYmpCbSR1ubhVwUvcfY5157V kWXkCcUxKuOn2jUHYHiuS9JJz8SlrdUAHUzp8D3uFhYMKVu3dqlYhAWhtl9jt596z/yB bZYoWDlSj7Vux9bEghXts6nD7EHKb0MB8/O0N1X4UL/1VJXcH59W1PDalV1VIIDh+HHp 0Feg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-5365f8fecb9si3854899e87.443.2024.09.12.03.19.10; Thu, 12 Sep 2024 03:19:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9028568DE09; Thu, 12 Sep 2024 11:23:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 11BE668D5CA for ; Thu, 12 Sep 2024 11:23:42 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Thu, 12 Sep 2024 10:23:17 +0200 Message-ID: <20240912082334.925402-2-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240912082334.925402-1-dev@lynne.ee> References: <20240912082334.925402-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/3] hw_base_encode_h265: split off SPS/PPS/VPS generation from VAAPI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fh2w2/GXb4jN --- libavcodec/hw_base_encode_h265.c | 351 +++++++++++++++++++++++++++++++ libavcodec/hw_base_encode_h265.h | 56 +++++ 2 files changed, 407 insertions(+) create mode 100644 libavcodec/hw_base_encode_h265.c create mode 100644 libavcodec/hw_base_encode_h265.h diff --git a/libavcodec/hw_base_encode_h265.c b/libavcodec/hw_base_encode_h265.c new file mode 100644 index 0000000000..dceec8aba9 --- /dev/null +++ b/libavcodec/hw_base_encode_h265.c @@ -0,0 +1,351 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "hw_base_encode_h265.h" + +#include "h2645data.h" +#include "h265_profile_level.h" + +#include "libavutil/pixdesc.h" + +int ff_hw_base_encode_init_params_h265(FFHWBaseEncodeContext *base_ctx, + AVCodecContext *avctx, + FFHWBaseEncodeH265 *common, + FFHWBaseEncodeH265Opts *opts) +{ + H265RawVPS *vps = &common->raw_vps; + H265RawSPS *sps = &common->raw_sps; + H265RawPPS *pps = &common->raw_pps; + H265RawProfileTierLevel *ptl = &vps->profile_tier_level; + H265RawVUI *vui = &sps->vui; + + const AVPixFmtDescriptor *desc; + int chroma_format, bit_depth; + int i; + + memset(vps, 0, sizeof(*vps)); + memset(sps, 0, sizeof(*sps)); + memset(pps, 0, sizeof(*pps)); + + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); + av_assert0(desc); + if (desc->nb_components == 1) { + chroma_format = 0; + } else { + if (desc->log2_chroma_w == 1 && desc->log2_chroma_h == 1) { + chroma_format = 1; + } else if (desc->log2_chroma_w == 1 && desc->log2_chroma_h == 0) { + chroma_format = 2; + } else if (desc->log2_chroma_w == 0 && desc->log2_chroma_h == 0) { + chroma_format = 3; + } else { + av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " + "%s is not supported.\n", desc->name); + return AVERROR(EINVAL); + } + } + bit_depth = desc->comp[0].depth; + + + // VPS + + vps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_VPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + vps->vps_video_parameter_set_id = 0; + + vps->vps_base_layer_internal_flag = 1; + vps->vps_base_layer_available_flag = 1; + vps->vps_max_layers_minus1 = 0; + vps->vps_max_sub_layers_minus1 = 0; + vps->vps_temporal_id_nesting_flag = 1; + + ptl->general_profile_space = 0; + ptl->general_profile_idc = avctx->profile; + ptl->general_tier_flag = opts->tier; + + ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1; + + if (ptl->general_profile_compatibility_flag[1]) + ptl->general_profile_compatibility_flag[2] = 1; + if (ptl->general_profile_compatibility_flag[3]) { + ptl->general_profile_compatibility_flag[1] = 1; + ptl->general_profile_compatibility_flag[2] = 1; + } + + ptl->general_progressive_source_flag = 1; + ptl->general_interlaced_source_flag = 0; + ptl->general_non_packed_constraint_flag = 1; + ptl->general_frame_only_constraint_flag = 1; + + ptl->general_max_14bit_constraint_flag = bit_depth <= 14; + ptl->general_max_12bit_constraint_flag = bit_depth <= 12; + ptl->general_max_10bit_constraint_flag = bit_depth <= 10; + ptl->general_max_8bit_constraint_flag = bit_depth == 8; + + ptl->general_max_422chroma_constraint_flag = chroma_format <= 2; + ptl->general_max_420chroma_constraint_flag = chroma_format <= 1; + ptl->general_max_monochrome_constraint_flag = chroma_format == 0; + + ptl->general_intra_constraint_flag = base_ctx->gop_size == 1; + ptl->general_one_picture_only_constraint_flag = 0; + + ptl->general_lower_bit_rate_constraint_flag = 1; + + if (avctx->level != AV_LEVEL_UNKNOWN) { + ptl->general_level_idc = avctx->level; + } else { + const H265LevelDescriptor *level; + + level = ff_h265_guess_level(ptl, avctx->bit_rate, + base_ctx->surface_width, base_ctx->surface_height, + opts->nb_slices, opts->tile_rows, opts->tile_cols, + (base_ctx->b_per_p > 0) + 1); + if (level) { + av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); + ptl->general_level_idc = level->level_idc; + } else { + av_log(avctx, AV_LOG_VERBOSE, "Stream will not conform to " + "any normal level; using level 8.5.\n"); + ptl->general_level_idc = 255; + // The tier flag must be set in level 8.5. + ptl->general_tier_flag = 1; + } + } + + vps->vps_sub_layer_ordering_info_present_flag = 0; + vps->vps_max_dec_pic_buffering_minus1[0] = base_ctx->max_b_depth + 1; + vps->vps_max_num_reorder_pics[0] = base_ctx->max_b_depth; + vps->vps_max_latency_increase_plus1[0] = 0; + + vps->vps_max_layer_id = 0; + vps->vps_num_layer_sets_minus1 = 0; + vps->layer_id_included_flag[0][0] = 1; + + vps->vps_timing_info_present_flag = 1; + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { + vps->vps_num_units_in_tick = avctx->framerate.den; + vps->vps_time_scale = avctx->framerate.num; + vps->vps_poc_proportional_to_timing_flag = 1; + vps->vps_num_ticks_poc_diff_one_minus1 = 0; + } else { + vps->vps_num_units_in_tick = avctx->time_base.num; + vps->vps_time_scale = avctx->time_base.den; + vps->vps_poc_proportional_to_timing_flag = 0; + } + vps->vps_num_hrd_parameters = 0; + + + // SPS + + sps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_SPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + sps->sps_video_parameter_set_id = vps->vps_video_parameter_set_id; + + sps->sps_max_sub_layers_minus1 = vps->vps_max_sub_layers_minus1; + sps->sps_temporal_id_nesting_flag = vps->vps_temporal_id_nesting_flag; + + sps->profile_tier_level = vps->profile_tier_level; + + sps->sps_seq_parameter_set_id = 0; + + sps->chroma_format_idc = chroma_format; + sps->separate_colour_plane_flag = 0; + + sps->pic_width_in_luma_samples = base_ctx->surface_width; + sps->pic_height_in_luma_samples = base_ctx->surface_height; + + if (avctx->width != base_ctx->surface_width || + avctx->height != base_ctx->surface_height) { + sps->conformance_window_flag = 1; + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = + (base_ctx->surface_width - avctx->width) >> desc->log2_chroma_w; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = + (base_ctx->surface_height - avctx->height) >> desc->log2_chroma_h; + } else { + sps->conformance_window_flag = 0; + } + + sps->bit_depth_luma_minus8 = bit_depth - 8; + sps->bit_depth_chroma_minus8 = bit_depth - 8; + + sps->log2_max_pic_order_cnt_lsb_minus4 = 8; + + sps->sps_sub_layer_ordering_info_present_flag = + vps->vps_sub_layer_ordering_info_present_flag; + for (i = 0; i <= sps->sps_max_sub_layers_minus1; i++) { + sps->sps_max_dec_pic_buffering_minus1[i] = + vps->vps_max_dec_pic_buffering_minus1[i]; + sps->sps_max_num_reorder_pics[i] = + vps->vps_max_num_reorder_pics[i]; + sps->sps_max_latency_increase_plus1[i] = + vps->vps_max_latency_increase_plus1[i]; + } + + // These values come from the capabilities of the first encoder + // implementation in the i965 driver on Intel Skylake. They may + // fail badly with other platforms or drivers. + // CTB size from 8x8 to 32x32. + sps->log2_min_luma_coding_block_size_minus3 = 0; + sps->log2_diff_max_min_luma_coding_block_size = 2; + // Transform size from 4x4 to 32x32. + sps->log2_min_luma_transform_block_size_minus2 = 0; + sps->log2_diff_max_min_luma_transform_block_size = 3; + // Full transform hierarchy allowed (2-5). + sps->max_transform_hierarchy_depth_inter = 3; + sps->max_transform_hierarchy_depth_intra = 3; + // AMP works. + sps->amp_enabled_flag = 1; + // SAO and temporal MVP do not work. + sps->sample_adaptive_offset_enabled_flag = 0; + sps->sps_temporal_mvp_enabled_flag = 0; + + sps->pcm_enabled_flag = 0; + + // STRPSs should ideally be here rather than defined individually in + // each slice, but the structure isn't completely fixed so for now + // don't bother. + sps->num_short_term_ref_pic_sets = 0; + sps->long_term_ref_pics_present_flag = 0; + + sps->vui_parameters_present_flag = 1; + + if (avctx->sample_aspect_ratio.num != 0 && + avctx->sample_aspect_ratio.den != 0) { + int num, den, i; + av_reduce(&num, &den, avctx->sample_aspect_ratio.num, + avctx->sample_aspect_ratio.den, 65535); + for (i = 0; i < FF_ARRAY_ELEMS(ff_h2645_pixel_aspect); i++) { + if (num == ff_h2645_pixel_aspect[i].num && + den == ff_h2645_pixel_aspect[i].den) { + vui->aspect_ratio_idc = i; + break; + } + } + if (i >= FF_ARRAY_ELEMS(ff_h2645_pixel_aspect)) { + vui->aspect_ratio_idc = 255; + vui->sar_width = num; + vui->sar_height = den; + } + vui->aspect_ratio_info_present_flag = 1; + } + + // Unspecified video format, from table E-2. + vui->video_format = 5; + vui->video_full_range_flag = + avctx->color_range == AVCOL_RANGE_JPEG; + vui->colour_primaries = avctx->color_primaries; + vui->transfer_characteristics = avctx->color_trc; + vui->matrix_coefficients = avctx->colorspace; + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED || + avctx->color_trc != AVCOL_TRC_UNSPECIFIED || + avctx->colorspace != AVCOL_SPC_UNSPECIFIED) + vui->colour_description_present_flag = 1; + if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED || + vui->colour_description_present_flag) + vui->video_signal_type_present_flag = 1; + + if (avctx->chroma_sample_location != AVCHROMA_LOC_UNSPECIFIED) { + vui->chroma_loc_info_present_flag = 1; + vui->chroma_sample_loc_type_top_field = + vui->chroma_sample_loc_type_bottom_field = + avctx->chroma_sample_location - 1; + } + + vui->vui_timing_info_present_flag = 1; + vui->vui_num_units_in_tick = vps->vps_num_units_in_tick; + vui->vui_time_scale = vps->vps_time_scale; + vui->vui_poc_proportional_to_timing_flag = vps->vps_poc_proportional_to_timing_flag; + vui->vui_num_ticks_poc_diff_one_minus1 = vps->vps_num_ticks_poc_diff_one_minus1; + vui->vui_hrd_parameters_present_flag = 0; + + vui->bitstream_restriction_flag = 1; + vui->motion_vectors_over_pic_boundaries_flag = 1; + vui->restricted_ref_pic_lists_flag = 1; + vui->max_bytes_per_pic_denom = 0; + vui->max_bits_per_min_cu_denom = 0; + vui->log2_max_mv_length_horizontal = 15; + vui->log2_max_mv_length_vertical = 15; + + + // PPS + + pps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_PPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + pps->pps_pic_parameter_set_id = 0; + pps->pps_seq_parameter_set_id = sps->sps_seq_parameter_set_id; + + pps->num_ref_idx_l0_default_active_minus1 = 0; + pps->num_ref_idx_l1_default_active_minus1 = 0; + + pps->init_qp_minus26 = opts->fixed_qp_idr - 26; + + pps->cu_qp_delta_enabled_flag = opts->cu_qp_delta_enabled_flag; + pps->diff_cu_qp_delta_depth = 0; + + if (opts->tile_rows && opts->tile_cols) { + int uniform_spacing; + + pps->tiles_enabled_flag = 1; + pps->num_tile_columns_minus1 = opts->tile_cols - 1; + pps->num_tile_rows_minus1 = opts->tile_rows - 1; + + // Test whether the spacing provided matches the H.265 uniform + // spacing, and set the flag if it does. + uniform_spacing = 1; + for (i = 0; i <= pps->num_tile_columns_minus1 && + uniform_spacing; i++) { + if (opts->col_width[i] != + (i + 1) * opts->slice_block_cols / opts->tile_cols - + i * opts->slice_block_cols / opts->tile_cols) + uniform_spacing = 0; + } + for (i = 0; i <= pps->num_tile_rows_minus1 && + uniform_spacing; i++) { + if (opts->row_height[i] != + (i + 1) * opts->slice_block_rows / opts->tile_rows - + i * opts->slice_block_rows / opts->tile_rows) + uniform_spacing = 0; + } + pps->uniform_spacing_flag = uniform_spacing; + + for (i = 0; i <= pps->num_tile_columns_minus1; i++) + pps->column_width_minus1[i] = opts->col_width[i] - 1; + for (i = 0; i <= pps->num_tile_rows_minus1; i++) + pps->row_height_minus1[i] = opts->row_height[i] - 1; + + pps->loop_filter_across_tiles_enabled_flag = 1; + } + + pps->pps_loop_filter_across_slices_enabled_flag = 1; + + return 0; +} diff --git a/libavcodec/hw_base_encode_h265.h b/libavcodec/hw_base_encode_h265.h new file mode 100644 index 0000000000..468f91c9b6 --- /dev/null +++ b/libavcodec/hw_base_encode_h265.h @@ -0,0 +1,56 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_HW_BASE_ENCODE_H265_H +#define AVCODEC_HW_BASE_ENCODE_H265_H + +#include "hw_base_encode.h" +#include "cbs_h265.h" + +typedef struct FFHWBaseEncodeH265 { + H265RawVPS raw_vps; + H265RawSPS raw_sps; + H265RawPPS raw_pps; + + int dpb_frames; +} FFHWBaseEncodeH265; + +typedef struct FFHWBaseEncodeH265Opts { + int tier; + int fixed_qp_idr; + int cu_qp_delta_enabled_flag; + + int tile_rows; + int tile_cols; + + int nb_slices; + int slice_block_rows; + int slice_block_cols; + + // Tile width of the i-th column. + int col_width[22]; + // Tile height of i-th row. + int row_height[22]; +} FFHWBaseEncodeH265Opts; + +int ff_hw_base_encode_init_params_h265(FFHWBaseEncodeContext *base_ctx, + AVCodecContext *avctx, + FFHWBaseEncodeH265 *common, + FFHWBaseEncodeH265Opts *opts); + +#endif /* AVCODEC_HW_BASE_ENCODE_H265_H */ From patchwork Thu Sep 12 08:23:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51542 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:14c:b0:48e:c0f8:d0de with SMTP id h12csp768143vqi; Thu, 12 Sep 2024 01:23:54 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW2h2/830ESXItuJuAPy7gyNzDRZwsjQdXp6NxTf+U8Jj9HbozJG0zm9AK8bIvFJ6tQqHo6+c713cwLiSSzgx5+@gmail.com X-Google-Smtp-Source: AGHT+IE9pwpzJg4v/3CQ83KobIKA6h7zTXCpLbbr4JgaOlKjclC20BtP0IvzfrmN06pYi/G42yR9 X-Received: by 2002:a05:6512:3d09:b0:536:53c2:817d with SMTP id 2adb3069b0e04-53678feb18bmr1335634e87.45.1726129434582; Thu, 12 Sep 2024 01:23:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726129434; cv=none; d=google.com; s=arc-20240605; b=Skucd6o90VrT9dD+eMwWPEf6Nb3VgCIu2BzuypHGzq1vTyf8CcvJ6yK4vJQbWW7gV2 G5ElVoQxo0Y10GrCWsPCydC5V13nVQr4V+/ht/E9LLtOORx5xohB2RBTP2ylTzQa7UYW GeARNPi+49vgwYKvVRxGUFpIs4ICP20I0ADub+0SImOmI9Zx3AMcqMq1yLjW9nwAAw00 cgIHkFzEYcrNvLoPDn0rjoztfKbcsxaeCcwgt1nOtqbKcA+t4ytXvmZ4NiYy8sC0nK+U o2igj/X6QS4ayroL9Qv3m+kZ+VICVnqbduOLECuMhOU77ZTngNHLudDKtJ+t7dVr6O46 YZiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=uuboOQWIqGVYNLUSCPaWnJe46xot3V08BhZUALkLnh4=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=ScXmvdP0VFLJldpx68UD8T2NDo9S1EjRCMO751VuU04oEXOfQHu4RElajmV62kY2uw GdaOSNWeHlZAAqdN42n1aTtifrfVQ1WoK44Xr89L8FBmJtlV/MsOSUdsThcXXP1XJ68n uhaoD4Y5g1/Nx70TDvrYSeggdi3AKde5z9gM0E5HnDqDImdmfGAvEmpIeSlPLe/kNacO 75WhcKle8S03nDoBkz4gliAIogp+6VVoJoowI2ddM2C5vMd7AdtRA6lG8SiRGKiSnCwZ eN+lj3puJ68J4GCXsT8UrNGdPVLQjhtb++AFwlW83Zbluq8oQ2V8me7DIUG+5t9DEOJS vIow==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-5365f867094si3937373e87.43.2024.09.12.01.23.53; Thu, 12 Sep 2024 01:23:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3289568D5CA; Thu, 12 Sep 2024 11:23:49 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1965368D999 for ; Thu, 12 Sep 2024 11:23:42 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Thu, 12 Sep 2024 10:23:18 +0200 Message-ID: <20240912082334.925402-3-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240912082334.925402-1-dev@lynne.ee> References: <20240912082334.925402-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/3] lavc: add hevc_vulkan hardware encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: l+HDJ9BMH6gn This commit adds a Vulkan hardware HEVC encoder, with full support of the spec - I, P, and B-frames. --- configure | 1 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 1 + libavcodec/vulkan_encode_h264.c | 14 +- libavcodec/vulkan_encode_h265.c | 1870 +++++++++++++++++++++++++++++++ libavcodec/vulkan_hevc.c | 25 +- libavcodec/vulkan_video.c | 29 + libavcodec/vulkan_video.h | 2 + 8 files changed, 1916 insertions(+), 29 deletions(-) create mode 100644 libavcodec/vulkan_encode_h265.c diff --git a/configure b/configure index da36419f2d..8fbf3772a8 100755 --- a/configure +++ b/configure @@ -3388,6 +3388,7 @@ hevc_rkmpp_decoder_deps="rkmpp" hevc_rkmpp_decoder_select="hevc_mp4toannexb_bsf" hevc_vaapi_encoder_deps="VAEncPictureParameterBufferHEVC" hevc_vaapi_encoder_select="atsc_a53 cbs_h265 vaapi_encode" +hevc_vulkan_encoder_select="atsc_a53 cbs_h265 vulkan_encode" hevc_v4l2m2m_decoder_deps="v4l2_m2m hevc_v4l2_m2m" hevc_v4l2m2m_decoder_select="hevc_mp4toannexb_bsf" hevc_v4l2m2m_encoder_deps="v4l2_m2m hevc_v4l2_m2m" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 502be8b09b..82df93fff4 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -452,6 +452,9 @@ OBJS-$(CONFIG_HEVC_QSV_ENCODER) += qsvenc_hevc.o hevc/ps_enc.o OBJS-$(CONFIG_HEVC_RKMPP_DECODER) += rkmppdec.o OBJS-$(CONFIG_HEVC_VAAPI_ENCODER) += vaapi_encode_h265.o h265_profile_level.o \ h2645data.o +OBJS-$(CONFIG_HEVC_VULKAN_ENCODER) += vulkan_encode.o vulkan_encode_h265.o \ + hw_base_encode.o hw_base_encode_h265.o \ + h265_profile_level.o h2645data.o OBJS-$(CONFIG_HEVC_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_HEVC_V4L2M2M_ENCODER) += v4l2_m2m_enc.o OBJS-$(CONFIG_HEVC_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index cfd929b81f..aa0fc47647 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -862,6 +862,7 @@ extern const FFCodec ff_hevc_qsv_encoder; extern const FFCodec ff_hevc_v4l2m2m_encoder; extern const FFCodec ff_hevc_vaapi_encoder; extern const FFCodec ff_hevc_videotoolbox_encoder; +extern const FFCodec ff_hevc_vulkan_encoder; extern const FFCodec ff_libkvazaar_encoder; extern const FFCodec ff_mjpeg_cuvid_decoder; extern const FFCodec ff_mjpeg_qsv_encoder; diff --git a/libavcodec/vulkan_encode_h264.c b/libavcodec/vulkan_encode_h264.c index aa72fd74fb..bbdc5a66a3 100644 --- a/libavcodec/vulkan_encode_h264.c +++ b/libavcodec/vulkan_encode_h264.c @@ -1460,20 +1460,20 @@ static av_cold int vulkan_encode_h264_init(AVCodecContext *avctx) enc->caps.maxLevelIdc); av_log(avctx, AV_LOG_VERBOSE, " maxSliceCount: %i\n", enc->caps.maxSliceCount); - av_log(avctx, AV_LOG_VERBOSE, " max(P/B)PictureL0ReferenceCount: %i P's; %i B's\n", + av_log(avctx, AV_LOG_VERBOSE, " max(P/B)PictureL0ReferenceCount: %i P's; %i B's\n", enc->caps.maxPPictureL0ReferenceCount, enc->caps.maxBPictureL0ReferenceCount); - av_log(avctx, AV_LOG_VERBOSE, " maxL1ReferenceCount: %i\n", + av_log(avctx, AV_LOG_VERBOSE, " maxL1ReferenceCount: %i\n", enc->caps.maxL1ReferenceCount); - av_log(avctx, AV_LOG_VERBOSE, " maxTemporalLayerCount: %i\n", + av_log(avctx, AV_LOG_VERBOSE, " maxTemporalLayerCount: %i\n", enc->caps.maxTemporalLayerCount); - av_log(avctx, AV_LOG_VERBOSE, " expectDyadicTemporalLayerPattern: %i\n", + av_log(avctx, AV_LOG_VERBOSE, " expectDyadicTemporalLayerPattern: %i\n", enc->caps.expectDyadicTemporalLayerPattern); - av_log(avctx, AV_LOG_VERBOSE, " min/max Qp: [%i, %i]\n", + av_log(avctx, AV_LOG_VERBOSE, " min/max Qp: [%i, %i]\n", enc->caps.maxQp, enc->caps.minQp); - av_log(avctx, AV_LOG_VERBOSE, " prefersGopRemainingFrames: %i\n", + av_log(avctx, AV_LOG_VERBOSE, " prefersGopRemainingFrames: %i\n", enc->caps.prefersGopRemainingFrames); - av_log(avctx, AV_LOG_VERBOSE, " requiresGopRemainingFrames: %i\n", + av_log(avctx, AV_LOG_VERBOSE, " requiresGopRemainingFrames: %i\n", enc->caps.requiresGopRemainingFrames); err = init_enc_options(avctx); diff --git a/libavcodec/vulkan_encode_h265.c b/libavcodec/vulkan_encode_h265.c new file mode 100644 index 0000000000..c639053c22 --- /dev/null +++ b/libavcodec/vulkan_encode_h265.c @@ -0,0 +1,1870 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/opt.h" +#include "libavutil/mem.h" + +#include "cbs.h" +#include "cbs_h265.h" +#include "atsc_a53.h" +#include "libavutil/mastering_display_metadata.h" + +#include "codec_internal.h" +#include "version.h" +#include "hw_base_encode_h265.h" + +#include "vulkan_encode.h" + +enum UnitElems { + UNIT_AUD = 1 << 0, + UNIT_SEI_MASTERING_DISPLAY = 1 << 1, + UNIT_SEI_CONTENT_LIGHT_LEVEL = 1 << 2, + UNIT_SEI_A53_CC = 1 << 3, +}; + +const FFVulkanEncodeDescriptor ff_vk_enc_h265_desc = { + .codec_id = AV_CODEC_ID_H265, + .encode_extension = FF_VK_EXT_VIDEO_ENCODE_H265, + .encode_op = VK_VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR, + .ext_props = { + .extensionName = VK_STD_VULKAN_VIDEO_CODEC_H265_ENCODE_EXTENSION_NAME, + .specVersion = VK_STD_VULKAN_VIDEO_CODEC_H265_ENCODE_SPEC_VERSION, + }, +}; + +typedef struct VulkanEncodeH265Picture { + int frame_num; + int64_t last_idr_frame; + uint16_t idr_pic_id; + int primary_pic_type; + int slice_type; + int pic_order_cnt; + int pic_type; + + enum UnitElems units_needed; + + VkVideoEncodeH265RateControlInfoKHR vkrc_info; + VkVideoEncodeH265RateControlLayerInfoKHR vkrc_layer_info; + + StdVideoEncodeH265PictureInfo h265pic_info; + VkVideoEncodeH265PictureInfoKHR vkh265pic_info; + + StdVideoEncodeH265WeightTable slice_wt; + StdVideoEncodeH265SliceSegmentHeader slice_hdr; + VkVideoEncodeH265NaluSliceSegmentInfoKHR vkslice; + + StdVideoEncodeH265ReferenceInfo h265dpb_info; + VkVideoEncodeH265DpbSlotInfoKHR vkh265dpb_info; + + StdVideoEncodeH265ReferenceListsInfo ref_list_info; + StdVideoEncodeH265LongTermRefPics l_rps; + StdVideoH265ShortTermRefPicSet s_rps; +} VulkanEncodeH265Picture; + +typedef struct VulkanEncodeH265Context { + FFVulkanEncodeContext common; + + int tier; + + FFHWBaseEncodeH265 units; + FFHWBaseEncodeH265Opts unit_opts; + + enum UnitElems unit_elems; + + uint8_t fixed_qp_idr; + uint8_t fixed_qp_p; + uint8_t fixed_qp_b; + + uint64_t hrd_buffer_size; + uint64_t initial_buffer_fullness; + + VkVideoEncodeH265ProfileInfoKHR profile; + + VkVideoEncodeH265CapabilitiesKHR caps; + VkVideoEncodeH265QualityLevelPropertiesKHR quality_props; + + CodedBitstreamContext *cbs; + CodedBitstreamFragment current_access_unit; + + H265RawAUD raw_aud; + + SEIRawMasteringDisplayColourVolume sei_mastering_display; + SEIRawContentLightLevelInfo sei_content_light_level; + SEIRawUserDataRegistered sei_a53cc; + void *sei_a53cc_data; +} VulkanEncodeH265Context; + +static int init_pic_rc(AVCodecContext *avctx, FFHWBaseEncodePicture *pic, + VkVideoEncodeRateControlInfoKHR *rc_info, + VkVideoEncodeRateControlLayerInfoKHR *rc_layer) +{ + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + VulkanEncodeH265Picture *hp = pic->codec_priv; + + hp->vkrc_info = (VkVideoEncodeH265RateControlInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_RATE_CONTROL_INFO_KHR, + .flags = VK_VIDEO_ENCODE_H265_RATE_CONTROL_REFERENCE_PATTERN_FLAT_BIT_KHR | + VK_VIDEO_ENCODE_H265_RATE_CONTROL_REGULAR_GOP_BIT_KHR, + .idrPeriod = ctx->base.gop_size, + .gopFrameCount = ctx->base.gop_size, + .consecutiveBFrameCount = FFMAX(ctx->base.b_per_p - 1, 0), + .subLayerCount = 0, + }; + + rc_info->pNext = &hp->vkrc_info; + rc_info->virtualBufferSizeInMs = 1000; + rc_info->initialVirtualBufferSizeInMs = 500; + + if (rc_info->rateControlMode > VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR) { + hp->vkrc_layer_info = (VkVideoEncodeH265RateControlLayerInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_RATE_CONTROL_LAYER_INFO_KHR, + + .useMinQp = avctx->qmin > 0, + .minQp.qpI = avctx->qmin > 0 ? avctx->qmin : 0, + .minQp.qpP = avctx->qmin > 0 ? avctx->qmin : 0, + .minQp.qpB = avctx->qmin > 0 ? avctx->qmin : 0, + + .useMaxQp = avctx->qmax > 0, + .maxQp.qpI = avctx->qmax > 0 ? avctx->qmax : 0, + .maxQp.qpP = avctx->qmax > 0 ? avctx->qmax : 0, + .maxQp.qpB = avctx->qmax > 0 ? avctx->qmax : 0, + + .useMaxFrameSize = 0, + }; + rc_layer->pNext = &hp->vkrc_layer_info; + hp->vkrc_info.subLayerCount = 1; + } + + return 0; +} + +static int vk_enc_h265_update_pic_info(AVCodecContext *avctx, + FFHWBaseEncodePicture *pic) +{ + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + VulkanEncodeH265Picture *hp = pic->codec_priv; + FFHWBaseEncodePicture *prev = pic->prev; + VulkanEncodeH265Picture *hprev = prev ? prev->codec_priv : NULL; + + if (pic->type == FF_HW_PICTURE_TYPE_IDR) { + av_assert0(pic->display_order == pic->encode_order); + + hp->last_idr_frame = pic->display_order; + + hp->slice_type = STD_VIDEO_H265_SLICE_TYPE_I; + hp->pic_type = STD_VIDEO_H265_PICTURE_TYPE_IDR; + } else { + av_assert0(prev); + hp->last_idr_frame = hprev->last_idr_frame; + + if (pic->type == FF_HW_PICTURE_TYPE_I) { + hp->slice_type = STD_VIDEO_H265_SLICE_TYPE_I; + hp->pic_type = STD_VIDEO_H265_PICTURE_TYPE_I; + } else if (pic->type == FF_HW_PICTURE_TYPE_P) { + av_assert0(pic->refs[0]); + hp->slice_type = STD_VIDEO_H265_SLICE_TYPE_P; + hp->pic_type = STD_VIDEO_H265_PICTURE_TYPE_P; + } else { + FFHWBaseEncodePicture *irap_ref; + av_assert0(pic->refs[0][0] && pic->refs[1][0]); + for (irap_ref = pic; irap_ref; irap_ref = irap_ref->refs[1][0]) { + if (irap_ref->type == FF_HW_PICTURE_TYPE_I) + break; + } + hp->slice_type = STD_VIDEO_H265_SLICE_TYPE_B; + hp->pic_type = STD_VIDEO_H265_PICTURE_TYPE_B; + } + } + hp->pic_order_cnt = pic->display_order - hp->last_idr_frame; + + hp->units_needed = 0; + + if (enc->unit_elems & UNIT_AUD) { + hp->units_needed |= UNIT_AUD; + enc->raw_aud = (H265RawAUD) { + .nal_unit_header = { + .nal_unit_type = HEVC_NAL_AUD, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }, + .pic_type = hp->pic_type, + }; + } + + // Only look for the metadata on I/IDR frame on the output. We + // may force an IDR frame on the output where the medadata gets + // changed on the input frame. + if ((enc->unit_elems & UNIT_SEI_MASTERING_DISPLAY) && + (pic->type == FF_HW_PICTURE_TYPE_I || pic->type == FF_HW_PICTURE_TYPE_IDR)) { + AVFrameSideData *sd = + av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + + if (sd) { + AVMasteringDisplayMetadata *mdm = (AVMasteringDisplayMetadata *)sd->data; + + // SEI is needed when both the primaries and luminance are set + if (mdm->has_primaries && mdm->has_luminance) { + SEIRawMasteringDisplayColourVolume *mdcv = + &enc->sei_mastering_display; + const int mapping[3] = {1, 2, 0}; + const int chroma_den = 50000; + const int luma_den = 10000; + + for (int i = 0; i < 3; i++) { + const int j = mapping[i]; + mdcv->display_primaries_x[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][0])), + chroma_den); + mdcv->display_primaries_y[i] = + FFMIN(lrint(chroma_den * + av_q2d(mdm->display_primaries[j][1])), + chroma_den); + } + + mdcv->white_point_x = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), + chroma_den); + mdcv->white_point_y = + FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), + chroma_den); + + mdcv->max_display_mastering_luminance = + lrint(luma_den * av_q2d(mdm->max_luminance)); + mdcv->min_display_mastering_luminance = + FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), + mdcv->max_display_mastering_luminance); + + hp->units_needed |= UNIT_SEI_MASTERING_DISPLAY; + } + } + } + + if ((enc->unit_elems & UNIT_SEI_CONTENT_LIGHT_LEVEL) && + (pic->type == FF_HW_PICTURE_TYPE_I || pic->type == FF_HW_PICTURE_TYPE_IDR)) { + AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + + if (sd) { + AVContentLightMetadata *clm = (AVContentLightMetadata *)sd->data; + SEIRawContentLightLevelInfo *clli = &enc->sei_content_light_level; + + clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); + clli->max_pic_average_light_level = FFMIN(clm->MaxFALL, 65535); + + hp->units_needed |= UNIT_SEI_CONTENT_LIGHT_LEVEL; + } + } + + if (enc->unit_elems & UNIT_SEI_A53_CC) { + int err; + size_t sei_a53cc_len; + av_freep(&enc->sei_a53cc_data); + err = ff_alloc_a53_sei(pic->input_image, 0, &enc->sei_a53cc_data, &sei_a53cc_len); + if (err < 0) + return err; + if (enc->sei_a53cc_data != NULL) { + enc->sei_a53cc.itu_t_t35_country_code = 181; + enc->sei_a53cc.data = (uint8_t *)enc->sei_a53cc_data + 1; + enc->sei_a53cc.data_length = sei_a53cc_len - 1; + + hp->units_needed |= UNIT_SEI_A53_CC; + } + } + + return 0; +} + +static void setup_slices(AVCodecContext *avctx, + FFHWBaseEncodePicture *pic) +{ + VulkanEncodeH265Context *enc = avctx->priv_data; + VulkanEncodeH265Picture *hp = pic->codec_priv; + + hp->slice_wt = (StdVideoEncodeH265WeightTable) { + .flags = (StdVideoEncodeH265WeightTableFlags) { + .luma_weight_l0_flag = 0, + .chroma_weight_l0_flag = 0, + .luma_weight_l1_flag = 0, + .chroma_weight_l1_flag = 0, + }, + .luma_log2_weight_denom = 0, + .delta_chroma_log2_weight_denom = 0, + .delta_luma_weight_l0 = { 0 }, + .luma_offset_l0 = { 0 }, + .delta_chroma_weight_l0 = { { 0 } }, + .delta_chroma_offset_l0 = { { 0 } }, + .delta_luma_weight_l1 = { 0 }, + .luma_offset_l1 = { 0 }, + .delta_chroma_weight_l1 = { { 0 } }, + .delta_chroma_offset_l1 = { { 0 } }, + }; + + hp->slice_hdr = (StdVideoEncodeH265SliceSegmentHeader) { + .flags = (StdVideoEncodeH265SliceSegmentHeaderFlags) { + .first_slice_segment_in_pic_flag = 1, + .dependent_slice_segment_flag = 0, + .slice_sao_luma_flag = enc->units.raw_sps.sample_adaptive_offset_enabled_flag, + .slice_sao_chroma_flag = enc->units.raw_sps.sample_adaptive_offset_enabled_flag, + .num_ref_idx_active_override_flag = 0, + .mvd_l1_zero_flag = 0, + .cabac_init_flag = 0, + .cu_chroma_qp_offset_enabled_flag = 0, + .deblocking_filter_override_flag = 0, + .slice_deblocking_filter_disabled_flag = 0, + .collocated_from_l0_flag = 1, + .slice_loop_filter_across_slices_enabled_flag = 0, + /* Reserved */ + }, + .slice_type = hp->slice_type, + .slice_segment_address = 0, + .collocated_ref_idx = 0, + .MaxNumMergeCand = 5, + .slice_cb_qp_offset = 0, + .slice_cr_qp_offset = 0, + .slice_beta_offset_div2 = 0, + .slice_tc_offset_div2 = 0, + .slice_act_y_qp_offset = 0, + .slice_act_cb_qp_offset = 0, + .slice_act_cr_qp_offset = 0, + .slice_qp_delta = 0, /* Filled in below */ + /* Reserved */ + .pWeightTable = NULL, // &hp->slice_wt, + }; + + hp->vkslice = (VkVideoEncodeH265NaluSliceSegmentInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_NALU_SLICE_SEGMENT_INFO_KHR, + .pNext = NULL, + .constantQp = pic->type == FF_HW_PICTURE_TYPE_B ? enc->fixed_qp_b : + pic->type == FF_HW_PICTURE_TYPE_P ? enc->fixed_qp_p : + enc->fixed_qp_idr, + .pStdSliceSegmentHeader = &hp->slice_hdr, + }; + + hp->slice_hdr.slice_qp_delta = hp->vkslice.constantQp - + (enc->units.raw_pps.init_qp_minus26 + 26); + + if (enc->common.opts.rc_mode != VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR) + hp->vkslice.constantQp = 0; + + hp->vkh265pic_info.pNaluSliceSegmentEntries = &hp->vkslice; + hp->vkh265pic_info.naluSliceSegmentEntryCount = 1; +} + +static void setup_refs(AVCodecContext *avctx, + FFHWBaseEncodePicture *pic, + VkVideoEncodeInfoKHR *encode_info) +{ + int idx, n, i, j; + VulkanEncodeH265Context *enc = avctx->priv_data; + VulkanEncodeH265Picture *hp = pic->codec_priv; + FFHWBaseEncodePicture *prev = pic->prev; + FFHWBaseEncodePicture *def_l0[MAX_DPB_SIZE], *def_l1[MAX_DPB_SIZE]; + VulkanEncodeH265Picture *href; + + hp->ref_list_info = (StdVideoEncodeH265ReferenceListsInfo) { + .flags = (StdVideoEncodeH265ReferenceListsInfoFlags) { + .ref_pic_list_modification_flag_l0 = 0, + .ref_pic_list_modification_flag_l1 = 0, + /* Reserved */ + }, + /* May be overridden during setup_slices() */ + .num_ref_idx_l0_active_minus1 = pic->nb_refs[0] - 1, + .num_ref_idx_l1_active_minus1 = pic->nb_refs[1] - 1, + /* Reserved */ + .list_entry_l0 = { 0 }, + .list_entry_l1 = { 0 }, + }; + + for (i = 0; i < STD_VIDEO_H265_MAX_NUM_LIST_REF; i++) + hp->ref_list_info.RefPicList0[i] = hp->ref_list_info.RefPicList1[i] = -1; + + /* Note: really not sure */ + for (int i = 0; i < pic->nb_refs[0]; i++) { + FFHWBaseEncodePicture *ref = pic->refs[0][i]; + FFVulkanEncodePicture *rvp = ref->priv; + VkVideoReferenceSlotInfoKHR *slot_info; + slot_info = (VkVideoReferenceSlotInfoKHR *)&encode_info->pReferenceSlots[i]; + hp->ref_list_info.RefPicList0[i] = slot_info->slotIndex; + } + + /* Note: really not sure */ + for (int i = 0; i < pic->nb_refs[1]; i++) { + FFHWBaseEncodePicture *ref = pic->refs[1][i]; + FFVulkanEncodePicture *rvp = ref->priv; + VkVideoReferenceSlotInfoKHR *slot_info; + slot_info = (VkVideoReferenceSlotInfoKHR *)&encode_info->pReferenceSlots[pic->nb_refs[0] + i]; + hp->ref_list_info.RefPicList1[i] = slot_info->slotIndex; + } + + hp->h265pic_info.pRefLists = &hp->ref_list_info; + + if (pic->type != FF_HW_PICTURE_TYPE_IDR) { + StdVideoH265ShortTermRefPicSet *rps; + VulkanEncodeH265Picture *strp; + int rps_poc[MAX_DPB_SIZE]; + int rps_used[MAX_DPB_SIZE]; + int i, j, poc, rps_pics; + + hp->h265pic_info.flags.short_term_ref_pic_set_sps_flag = 0; + + rps = &hp->s_rps; + memset(rps, 0, sizeof(*rps)); + + rps_pics = 0; + for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { + for (j = 0; j < pic->nb_refs[i]; j++) { + strp = pic->refs[i][j]->codec_priv; + rps_poc[rps_pics] = strp->pic_order_cnt; + rps_used[rps_pics] = 1; + ++rps_pics; + } + } + + for (i = 0; i < pic->nb_dpb_pics; i++) { + if (pic->dpb[i] == pic) + continue; + + for (j = 0; j < pic->nb_refs[0]; j++) { + if (pic->dpb[i] == pic->refs[0][j]) + break; + } + if (j < pic->nb_refs[0]) + continue; + + for (j = 0; j < pic->nb_refs[1]; j++) { + if (pic->dpb[i] == pic->refs[1][j]) + break; + } + if (j < pic->nb_refs[1]) + continue; + + strp = pic->dpb[i]->codec_priv; + rps_poc[rps_pics] = strp->pic_order_cnt; + rps_used[rps_pics] = 0; + ++rps_pics; + } + + for (i = 1; i < rps_pics; i++) { + for (j = i; j > 0; j--) { + if (rps_poc[j] > rps_poc[j - 1]) + break; + av_assert0(rps_poc[j] != rps_poc[j - 1]); + FFSWAP(int, rps_poc[j], rps_poc[j - 1]); + FFSWAP(int, rps_used[j], rps_used[j - 1]); + } + } + + av_log(avctx, AV_LOG_DEBUG, "RPS for POC %d:", hp->pic_order_cnt); + for (i = 0; i < rps_pics; i++) + av_log(avctx, AV_LOG_DEBUG, " (%d,%d)", rps_poc[i], rps_used[i]); + + av_log(avctx, AV_LOG_DEBUG, "\n"); + + for (i = 0; i < rps_pics; i++) { + av_assert0(rps_poc[i] != hp->pic_order_cnt); + if (rps_poc[i] > hp->pic_order_cnt) + break; + } + + rps->num_negative_pics = i; + rps->used_by_curr_pic_s0_flag = 0x0; + poc = hp->pic_order_cnt; + for (j = i - 1; j >= 0; j--) { + rps->delta_poc_s0_minus1[i - 1 - j] = poc - rps_poc[j] - 1; + rps->used_by_curr_pic_s0_flag |= rps_used[j] << (i - 1 - j); + poc = rps_poc[j]; + } + + rps->num_positive_pics = rps_pics - i; + rps->used_by_curr_pic_s1_flag = 0x0; + poc = hp->pic_order_cnt; + for (j = i; j < rps_pics; j++) { + rps->delta_poc_s1_minus1[j - i] = rps_poc[j] - poc - 1; + rps->used_by_curr_pic_s1_flag |= rps_used[j] << (j - i); + poc = rps_poc[j]; + } + + hp->l_rps.num_long_term_sps = 0; + hp->l_rps.num_long_term_pics = 0; + + // when this flag is not present, it is inerred to 1. + hp->slice_hdr.flags.collocated_from_l0_flag = 1; + hp->h265pic_info.flags.slice_temporal_mvp_enabled_flag = + enc->units.raw_sps.sps_temporal_mvp_enabled_flag; + if (hp->h265pic_info.flags.slice_temporal_mvp_enabled_flag) { + if (hp->slice_hdr.slice_type == STD_VIDEO_H265_SLICE_TYPE_B) + hp->slice_hdr.flags.collocated_from_l0_flag = 1; + hp->slice_hdr.collocated_ref_idx = 0; + } + + hp->slice_hdr.flags.num_ref_idx_active_override_flag = 0; + hp->ref_list_info.num_ref_idx_l0_active_minus1 = enc->units.raw_pps.num_ref_idx_l0_default_active_minus1; + hp->ref_list_info.num_ref_idx_l1_active_minus1 = enc->units.raw_pps.num_ref_idx_l1_default_active_minus1; + +#if 0 + for (int i = 0; i < STD_VIDEO_H265_MAX_SHORT_TERM_REF_PIC_SETS; i++) { + hp->s_rps[i] = (StdVideoH265ShortTermRefPicSet) { + .flags = (StdVideoH265ShortTermRefPicSetFlags) { + .inter_ref_pic_set_prediction_flag = st_rps->inter_ref_pic_set_prediction_flag, + .delta_rps_sign = st_rps->delta_rps_sign, + }, + .delta_idx_minus1 = st_rps->delta_idx_minus1, + .use_delta_flag = 0x0, + .abs_delta_rps_minus1 = st_rps->abs_delta_rps_minus1, + .used_by_curr_pic_flag = 0x0, + .used_by_curr_pic_s0_flag = 0x0, + .used_by_curr_pic_s1_flag = 0x0, + /* Reserved */ + /* Reserved */ + /* Reserved */ + .num_negative_pics = st_rps->num_negative_pics, + .num_positive_pics = st_rps->num_positive_pics, + }; + } +#endif + } + + hp->h265pic_info.pShortTermRefPicSet = &hp->s_rps; + hp->h265pic_info.pLongTermRefPics = &hp->l_rps; +} + +static int init_pic_params(AVCodecContext *avctx, FFHWBaseEncodePicture *pic, + VkVideoEncodeInfoKHR *encode_info) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodePicture *vp = pic->priv; + VulkanEncodeH265Picture *hp = pic->codec_priv; + VkVideoReferenceSlotInfoKHR *ref_slot; + + err = vk_enc_h265_update_pic_info(avctx, pic); + if (err < 0) + return err; + + hp->vkh265pic_info = (VkVideoEncodeH265PictureInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_PICTURE_INFO_KHR, + .pNext = NULL, + .pNaluSliceSegmentEntries = NULL, // Filled in during setup_slices() + .naluSliceSegmentEntryCount = 0, // Filled in during setup_slices() + .pStdPictureInfo = &hp->h265pic_info, + }; + + hp->h265pic_info = (StdVideoEncodeH265PictureInfo) { + .flags = (StdVideoEncodeH265PictureInfoFlags) { + .is_reference = pic->is_reference, + .IrapPicFlag = pic->type == FF_HW_PICTURE_TYPE_IDR, + .used_for_long_term_reference = 0, + .discardable_flag = 0, + .cross_layer_bla_flag = 0, + .pic_output_flag = 1, + .no_output_of_prior_pics_flag = 0, + .short_term_ref_pic_set_sps_flag = 0, + .slice_temporal_mvp_enabled_flag = enc->units.raw_sps.sps_temporal_mvp_enabled_flag, + /* Reserved */ + }, + .pic_type = hp->pic_type, + .sps_video_parameter_set_id = 0, + .pps_seq_parameter_set_id = 0, + .pps_pic_parameter_set_id = 0, + .short_term_ref_pic_set_idx = 0, + .PicOrderCntVal = hp->pic_order_cnt, + .TemporalId = 0, + /* Reserved */ + .pRefLists = NULL, // Filled in during setup_refs + .pShortTermRefPicSet = NULL, + .pLongTermRefPics = NULL, + }; + encode_info->pNext = &hp->vkh265pic_info; + + hp->h265dpb_info = (StdVideoEncodeH265ReferenceInfo) { + .flags = (StdVideoEncodeH265ReferenceInfoFlags) { + .used_for_long_term_reference = 0, + .unused_for_reference = 0, + /* Reserved */ + }, + .pic_type = hp->h265pic_info.pic_type, + .PicOrderCntVal = hp->h265pic_info.PicOrderCntVal, + .TemporalId = hp->h265pic_info.TemporalId, + }; + hp->vkh265dpb_info = (VkVideoEncodeH265DpbSlotInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_DPB_SLOT_INFO_KHR, + .pStdReferenceInfo = &hp->h265dpb_info, + }; + + vp->dpb_slot.pNext = &hp->vkh265dpb_info; + + ref_slot = (VkVideoReferenceSlotInfoKHR *)encode_info->pSetupReferenceSlot; + ref_slot->pNext = &hp->vkh265dpb_info; + + setup_refs(avctx, pic, encode_info); + + setup_slices(avctx, pic); + + return 0; +} + +static int init_profile(AVCodecContext *avctx, + VkVideoProfileInfoKHR *profile, void *pnext) +{ + VkResult ret; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + FFVulkanContext *s = &ctx->s; + FFVulkanFunctions *vk = &ctx->s.vkfn; + VkVideoEncodeH265CapabilitiesKHR h265_caps = { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_CAPABILITIES_KHR, + }; + VkVideoEncodeCapabilitiesKHR enc_caps = { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_CAPABILITIES_KHR, + .pNext = &h265_caps, + }; + VkVideoCapabilitiesKHR caps = { + .sType = VK_STRUCTURE_TYPE_VIDEO_CAPABILITIES_KHR, + .pNext = &enc_caps, + }; + + /* In order of preference */ + int last_supported = AV_PROFILE_UNKNOWN; + static const int known_profiles[] = { + AV_PROFILE_HEVC_MAIN, + AV_PROFILE_HEVC_MAIN_10, + AV_PROFILE_HEVC_REXT, + }; + int nb_profiles = FF_ARRAY_ELEMS(known_profiles); + + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(s->frames->sw_format); + if (!desc) + return AVERROR(EINVAL); + + if (s->frames->sw_format == AV_PIX_FMT_NV12) + nb_profiles = 1; + else if (s->frames->sw_format == AV_PIX_FMT_P010) + nb_profiles = 2; + + enc->profile = (VkVideoEncodeH265ProfileInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_PROFILE_INFO_KHR, + .pNext = pnext, + .stdProfileIdc = ff_vk_h265_profile_to_vk(avctx->profile), + }; + profile->pNext = &enc->profile; + + /* User has explicitly specified a profile. */ + if (avctx->profile != AV_PROFILE_UNKNOWN) + return 0; + + av_log(avctx, AV_LOG_DEBUG, "Supported profiles:\n"); + for (int i = 0; i < nb_profiles; i++) { + enc->profile.stdProfileIdc = ff_vk_h265_profile_to_vk(known_profiles[i]); + ret = vk->GetPhysicalDeviceVideoCapabilitiesKHR(s->hwctx->phys_dev, + profile, + &caps); + if (ret == VK_SUCCESS) { + av_log(avctx, AV_LOG_DEBUG, " %s\n", + avcodec_profile_name(avctx->codec_id, known_profiles[i])); + last_supported = known_profiles[i]; + } + } + + if (last_supported == AV_PROFILE_UNKNOWN) { + av_log(avctx, AV_LOG_ERROR, "No supported profiles for given format\n"); + return AVERROR(ENOTSUP); + } + + enc->profile.stdProfileIdc = ff_vk_h265_profile_to_vk(last_supported); + av_log(avctx, AV_LOG_VERBOSE, "Using profile %s\n", + avcodec_profile_name(avctx->codec_id, last_supported)); + avctx->profile = last_supported; + + return 0; +} + +static int init_enc_options(AVCodecContext *avctx) +{ + VulkanEncodeH265Context *enc = avctx->priv_data; + FFHWBaseEncodeH265Opts *unit_opts = &enc->unit_opts; + + if (avctx->rc_buffer_size) + enc->hrd_buffer_size = avctx->rc_buffer_size; + else if (avctx->rc_max_rate > 0) + enc->hrd_buffer_size = avctx->rc_max_rate; + else + enc->hrd_buffer_size = avctx->bit_rate; + + if (avctx->rc_initial_buffer_occupancy) { + if (avctx->rc_initial_buffer_occupancy > enc->hrd_buffer_size) { + av_log(avctx, AV_LOG_ERROR, "Invalid RC buffer settings: " + "must have initial buffer size (%d) <= " + "buffer size (%"PRId64").\n", + avctx->rc_initial_buffer_occupancy, enc->hrd_buffer_size); + return AVERROR(EINVAL); + } + enc->initial_buffer_fullness = avctx->rc_initial_buffer_occupancy; + } else { + enc->initial_buffer_fullness = enc->hrd_buffer_size * 3 / 4; + } + + if (enc->common.opts.rc_mode == VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR) { + enc->fixed_qp_p = av_clip(enc->common.opts.qp, 0, 51); + if (avctx->i_quant_factor > 0.0) + enc->fixed_qp_idr = av_clip((avctx->i_quant_factor * enc->fixed_qp_p + + avctx->i_quant_offset) + 0.5, 0, 51); + else + enc->fixed_qp_idr = enc->fixed_qp_p; + + if (avctx->b_quant_factor > 0.0) + enc->fixed_qp_b = av_clip((avctx->b_quant_factor * enc->fixed_qp_p + + avctx->b_quant_offset) + 0.5, 0, 51); + else + enc->fixed_qp_b = enc->fixed_qp_p; + + av_log(avctx, AV_LOG_DEBUG, "Using fixed QP = " + "%d / %d / %d for IDR- / P- / B-frames.\n", + enc->fixed_qp_idr, enc->fixed_qp_p, enc->fixed_qp_b); + } else { + enc->fixed_qp_idr = 26; + enc->fixed_qp_p = 26; + enc->fixed_qp_b = 26; + } + + return 0; +} + +static av_cold int init_sequence_headers(AVCodecContext *avctx) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + FFHWBaseEncodeContext *base_ctx = &ctx->base; + + FFHWBaseEncodeH265 *units = &enc->units; + FFHWBaseEncodeH265Opts *unit_opts = &enc->unit_opts; + + unit_opts->tier = enc->tier; + unit_opts->fixed_qp_idr = enc->fixed_qp_idr; + unit_opts->cu_qp_delta_enabled_flag = enc->common.opts.rc_mode != VK_VIDEO_ENCODE_RATE_CONTROL_MODE_DISABLED_BIT_KHR; + + unit_opts->nb_slices = 1; + +// int tile_rows; +// int tile_cols; + + unit_opts->slice_block_rows = (avctx->height + base_ctx->slice_block_height - 1) / + base_ctx->slice_block_height; + unit_opts->slice_block_cols = (avctx->width + base_ctx->slice_block_width - 1) / + base_ctx->slice_block_width; + + // Tile width of the i-th column. + int col_width[22]; + // Tile height of i-th row. + int row_height[22]; + int max_ctb_size; + int min_ctb_size; + unsigned min_tb_size; + unsigned max_tb_size; + unsigned max_transform_hierarchy; + + /* cabac already set via an option */ + /* fixed_qp_idr initialized in init_enc_options() */ + /* hrd_buffer_size initialized in init_enc_options() */ + /* initial_buffer_fullness initialized in init_enc_options() */ + + err = ff_hw_base_encode_init_params_h265(&enc->common.base, avctx, + units, unit_opts); + if (err < 0) + return err; + + units->raw_sps.sample_adaptive_offset_enabled_flag = + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SAMPLE_ADAPTIVE_OFFSET_ENABLED_FLAG_SET_BIT_KHR); + units->raw_pps.transform_skip_enabled_flag = + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_TRANSFORM_SKIP_ENABLED_FLAG_SET_BIT_KHR); + + units->raw_sps.log2_diff_max_min_luma_coding_block_size = 3; + units->raw_sps.max_transform_hierarchy_depth_intra = 4; + units->raw_sps.max_transform_hierarchy_depth_intra = 4; + + max_ctb_size = 16; + min_ctb_size = 64; + + /* coding blocks from 8x8 to max CTB size. */ + if (enc->caps.ctbSizes & VK_VIDEO_ENCODE_H265_CTB_SIZE_16_BIT_KHR) + min_ctb_size = 16; + else if (enc->caps.ctbSizes & VK_VIDEO_ENCODE_H265_CTB_SIZE_32_BIT_KHR) + min_ctb_size = 32; + + if (enc->caps.ctbSizes & VK_VIDEO_ENCODE_H265_CTB_SIZE_64_BIT_KHR) + max_ctb_size = 64; + else if (enc->caps.ctbSizes & VK_VIDEO_ENCODE_H265_CTB_SIZE_32_BIT_KHR) + max_ctb_size = 32; + + min_tb_size = 0; + max_tb_size = 0; + if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_4_BIT_KHR) + min_tb_size = 4; + else if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_8_BIT_KHR) + min_tb_size = 8; + else if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_16_BIT_KHR) + min_tb_size = 16; + else if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_32_BIT_KHR) + min_tb_size = 32; + + if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_32_BIT_KHR) + max_tb_size = 32; + else if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_16_BIT_KHR) + max_tb_size = 16; + else if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_8_BIT_KHR) + max_tb_size = 8; + else if (enc->caps.transformBlockSizes & VK_VIDEO_ENCODE_H265_TRANSFORM_BLOCK_SIZE_4_BIT_KHR) + max_tb_size = 4; + + units->raw_sps.log2_min_luma_coding_block_size_minus3 = 0; + units->raw_sps.log2_diff_max_min_luma_coding_block_size = av_log2(max_ctb_size) - 3; + units->raw_sps.log2_min_luma_transform_block_size_minus2 = av_log2(min_tb_size) - 2; + units->raw_sps.log2_diff_max_min_luma_transform_block_size = av_log2(max_tb_size) - av_log2(min_tb_size); + + max_transform_hierarchy = av_log2(max_ctb_size) - av_log2(min_tb_size); + units->raw_sps.max_transform_hierarchy_depth_intra = max_transform_hierarchy; + units->raw_sps.max_transform_hierarchy_depth_intra = max_transform_hierarchy; + + units->raw_sps.vui.bitstream_restriction_flag = 0; + units->raw_sps.vui.max_bytes_per_pic_denom = 2; + units->raw_sps.vui.max_bits_per_min_cu_denom = 1; + + units->raw_sps.sps_temporal_mvp_enabled_flag = 0; + + return 0; +} + +typedef struct VulkanH265Units { + StdVideoH265SequenceParameterSet sps; + StdVideoH265ShortTermRefPicSet str[STD_VIDEO_H265_SUBLAYERS_LIST_SIZE]; + StdVideoH265LongTermRefPicsSps ltr; + StdVideoH265ProfileTierLevel ptl_sps; + StdVideoH265DecPicBufMgr dpbm_sps; + + StdVideoH265HrdParameters vui_header_sps; + StdVideoH265SequenceParameterSetVui vui_sps; + + StdVideoH265SubLayerHrdParameters slhdrnal[HEVC_MAX_SUB_LAYERS]; + StdVideoH265SubLayerHrdParameters slhdrvcl[HEVC_MAX_SUB_LAYERS]; + + StdVideoH265PictureParameterSet pps; + StdVideoH265ScalingLists pps_scaling; + + StdVideoH265VideoParameterSet vps; + StdVideoH265ProfileTierLevel ptl_vps; + StdVideoH265DecPicBufMgr dpbm_vps; + StdVideoH265HrdParameters vui_header_vps; +} VulkanH265Units; + +static av_cold int base_unit_to_vk(AVCodecContext *avctx, + VulkanH265Units *vk_units) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + FFHWBaseEncodeContext *base_ctx = &ctx->base; + + H265RawSPS *sps = &enc->units.raw_sps; + StdVideoH265SequenceParameterSet *vksps = &vk_units->sps; + StdVideoH265ShortTermRefPicSet *str = vk_units->str; + StdVideoH265LongTermRefPicsSps *ltr = &vk_units->ltr; + StdVideoH265ProfileTierLevel *ptl_sps = &vk_units->ptl_sps; + StdVideoH265DecPicBufMgr *dpbm_sps = &vk_units->dpbm_sps; + + StdVideoH265HrdParameters *vui_header_sps = &vk_units->vui_header_sps; + StdVideoH265SequenceParameterSetVui *vui_sps = &vk_units->vui_sps; + + StdVideoH265SubLayerHrdParameters *slhdrnal = vk_units->slhdrnal; + StdVideoH265SubLayerHrdParameters *slhdrvcl = vk_units->slhdrvcl; + + H265RawPPS *pps = &enc->units.raw_pps; + StdVideoH265PictureParameterSet *vkpps = &vk_units->pps; + + H265RawVPS *vps = &enc->units.raw_vps; + StdVideoH265VideoParameterSet *vkvps = &vk_units->vps; + StdVideoH265ProfileTierLevel *ptl_vps = &vk_units->ptl_vps; + StdVideoH265DecPicBufMgr *dpbm_vps = &vk_units->dpbm_vps; + StdVideoH265HrdParameters *vui_header_vps = &vk_units->vui_header_vps; + + /* SPS */ + for (int i = 0; i < HEVC_MAX_SUB_LAYERS; i++) { + memcpy(&slhdrnal[i], &sps->vui.hrd_parameters.nal_sub_layer_hrd_parameters[i], sizeof(*slhdrnal)); + memcpy(&slhdrvcl[i], &sps->vui.hrd_parameters.vcl_sub_layer_hrd_parameters[i], sizeof(*slhdrvcl)); + slhdrnal[i].cbr_flag = 0x0; + slhdrvcl[i].cbr_flag = 0x0; + for (int j = 0; j < HEVC_MAX_CPB_CNT; j++) { + slhdrnal[i].cbr_flag |= sps->vui.hrd_parameters.nal_sub_layer_hrd_parameters[i].cbr_flag[j] << i; + slhdrvcl[i].cbr_flag |= sps->vui.hrd_parameters.vcl_sub_layer_hrd_parameters[i].cbr_flag[j] << i; + } + } + + *vui_header_sps = (StdVideoH265HrdParameters) { + .flags = (StdVideoH265HrdFlags) { + .nal_hrd_parameters_present_flag = sps->vui.hrd_parameters.nal_hrd_parameters_present_flag, + .vcl_hrd_parameters_present_flag = sps->vui.hrd_parameters.vcl_hrd_parameters_present_flag, + .sub_pic_hrd_params_present_flag = sps->vui.hrd_parameters.sub_pic_hrd_params_present_flag, + .sub_pic_cpb_params_in_pic_timing_sei_flag = sps->vui.hrd_parameters.sub_pic_cpb_params_in_pic_timing_sei_flag, + .fixed_pic_rate_general_flag = 0x0, + .fixed_pic_rate_within_cvs_flag = 0x0, + .low_delay_hrd_flag = 0x0, + }, + .tick_divisor_minus2 = sps->vui.hrd_parameters.tick_divisor_minus2, + .du_cpb_removal_delay_increment_length_minus1 = sps->vui.hrd_parameters.du_cpb_removal_delay_increment_length_minus1, + .dpb_output_delay_du_length_minus1 = sps->vui.hrd_parameters.dpb_output_delay_du_length_minus1, + .bit_rate_scale = sps->vui.hrd_parameters.bit_rate_scale, + .cpb_size_scale = sps->vui.hrd_parameters.cpb_size_scale, + .cpb_size_du_scale = sps->vui.hrd_parameters.cpb_size_du_scale, + .initial_cpb_removal_delay_length_minus1 = sps->vui.hrd_parameters.initial_cpb_removal_delay_length_minus1, + .au_cpb_removal_delay_length_minus1 = sps->vui.hrd_parameters.au_cpb_removal_delay_length_minus1, + .dpb_output_delay_length_minus1 = sps->vui.hrd_parameters.dpb_output_delay_length_minus1, + /* Reserved - 3*16 bits */ + .pSubLayerHrdParametersNal = slhdrnal, + .pSubLayerHrdParametersVcl = slhdrvcl, + }; + + for (int i = 0; i < HEVC_MAX_SUB_LAYERS; i++) { + vui_header_sps->flags.fixed_pic_rate_general_flag |= sps->vui.hrd_parameters.fixed_pic_rate_general_flag[i] << i; + vui_header_sps->flags.fixed_pic_rate_within_cvs_flag |= sps->vui.hrd_parameters.fixed_pic_rate_within_cvs_flag[i] << i; + vui_header_sps->flags.low_delay_hrd_flag |= sps->vui.hrd_parameters.low_delay_hrd_flag[i] << i; + } + + for (int i = 0; i < STD_VIDEO_H265_SUBLAYERS_LIST_SIZE; i++) { + dpbm_sps->max_latency_increase_plus1[i] = sps->sps_max_latency_increase_plus1[i]; + dpbm_sps->max_dec_pic_buffering_minus1[i] = sps->sps_max_dec_pic_buffering_minus1[i]; + dpbm_sps->max_num_reorder_pics[i] = sps->sps_max_num_reorder_pics[i]; + } + + *ptl_sps = (StdVideoH265ProfileTierLevel) { + .flags = (StdVideoH265ProfileTierLevelFlags) { + .general_tier_flag = sps->profile_tier_level.general_tier_flag, + .general_progressive_source_flag = sps->profile_tier_level.general_progressive_source_flag, + .general_interlaced_source_flag = sps->profile_tier_level.general_interlaced_source_flag, + .general_non_packed_constraint_flag = sps->profile_tier_level.general_non_packed_constraint_flag, + .general_frame_only_constraint_flag = sps->profile_tier_level.general_frame_only_constraint_flag, + }, + .general_profile_idc = ff_vk_h265_profile_to_vk(sps->profile_tier_level.general_profile_idc), + .general_level_idc = ff_vk_h265_level_to_vk(sps->profile_tier_level.general_level_idc), + }; + + for (int i = 0; i < STD_VIDEO_H265_MAX_SHORT_TERM_REF_PIC_SETS; i++) { + const H265RawSTRefPicSet *st_rps = &sps->st_ref_pic_set[i]; + + str[i] = (StdVideoH265ShortTermRefPicSet) { + .flags = (StdVideoH265ShortTermRefPicSetFlags) { + .inter_ref_pic_set_prediction_flag = st_rps->inter_ref_pic_set_prediction_flag, + .delta_rps_sign = st_rps->delta_rps_sign, + }, + .delta_idx_minus1 = st_rps->delta_idx_minus1, + .use_delta_flag = 0x0, + .abs_delta_rps_minus1 = st_rps->abs_delta_rps_minus1, + .used_by_curr_pic_flag = 0x0, + .used_by_curr_pic_s0_flag = 0x0, + .used_by_curr_pic_s1_flag = 0x0, + /* Reserved */ + /* Reserved */ + /* Reserved */ + .num_negative_pics = st_rps->num_negative_pics, + .num_positive_pics = st_rps->num_positive_pics, + }; + + for (int j = 0; j < HEVC_MAX_REFS; j++) { + str[i].use_delta_flag |= st_rps->use_delta_flag[j] << i; + str[i].used_by_curr_pic_flag |= st_rps->used_by_curr_pic_flag[j] << i; + str[i].used_by_curr_pic_s0_flag |= st_rps->used_by_curr_pic_s0_flag[j] << i; + str[i].used_by_curr_pic_s1_flag |= st_rps->used_by_curr_pic_s1_flag[j] << i; + str[i].delta_poc_s0_minus1[j] = st_rps->delta_poc_s0_minus1[j]; + str[i].delta_poc_s1_minus1[j] = st_rps->delta_poc_s1_minus1[j]; + } + } + + ltr->used_by_curr_pic_lt_sps_flag = 0; + for (int i = 0; i < STD_VIDEO_H265_MAX_LONG_TERM_REF_PICS_SPS; i++) { + ltr->used_by_curr_pic_lt_sps_flag |= sps->lt_ref_pic_poc_lsb_sps[i] << i; + ltr->lt_ref_pic_poc_lsb_sps[i] = sps->lt_ref_pic_poc_lsb_sps[i]; + } + + *vksps = (StdVideoH265SequenceParameterSet) { + .flags = (StdVideoH265SpsFlags) { + .sps_temporal_id_nesting_flag = sps->sps_temporal_id_nesting_flag, + .separate_colour_plane_flag = sps->separate_colour_plane_flag, + .conformance_window_flag = sps->conformance_window_flag, + .sps_sub_layer_ordering_info_present_flag = sps->sps_sub_layer_ordering_info_present_flag, + .scaling_list_enabled_flag = sps->scaling_list_enabled_flag, + .sps_scaling_list_data_present_flag = sps->sps_scaling_list_data_present_flag, + .amp_enabled_flag = sps->amp_enabled_flag, + .sample_adaptive_offset_enabled_flag = sps->sample_adaptive_offset_enabled_flag, + .pcm_enabled_flag = sps->pcm_enabled_flag, + .pcm_loop_filter_disabled_flag = sps->pcm_loop_filter_disabled_flag, + .long_term_ref_pics_present_flag = sps->long_term_ref_pics_present_flag, + .sps_temporal_mvp_enabled_flag = sps->sps_temporal_mvp_enabled_flag, + .strong_intra_smoothing_enabled_flag = sps->strong_intra_smoothing_enabled_flag, + .vui_parameters_present_flag = sps->vui_parameters_present_flag, + .sps_extension_present_flag = sps->sps_extension_present_flag, + .sps_range_extension_flag = sps->sps_range_extension_flag, + .transform_skip_rotation_enabled_flag = sps->transform_skip_rotation_enabled_flag, + .transform_skip_context_enabled_flag = sps->transform_skip_context_enabled_flag, + .implicit_rdpcm_enabled_flag = sps->implicit_rdpcm_enabled_flag, + .explicit_rdpcm_enabled_flag = sps->explicit_rdpcm_enabled_flag, + .extended_precision_processing_flag = sps->extended_precision_processing_flag, + .intra_smoothing_disabled_flag = sps->intra_smoothing_disabled_flag, + .high_precision_offsets_enabled_flag = sps->high_precision_offsets_enabled_flag, + .persistent_rice_adaptation_enabled_flag = sps->persistent_rice_adaptation_enabled_flag, + .cabac_bypass_alignment_enabled_flag = sps->cabac_bypass_alignment_enabled_flag, + .sps_scc_extension_flag = sps->sps_scc_extension_flag, + .sps_curr_pic_ref_enabled_flag = sps->sps_curr_pic_ref_enabled_flag, + .palette_mode_enabled_flag = sps->palette_mode_enabled_flag, + .sps_palette_predictor_initializers_present_flag = sps->sps_palette_predictor_initializer_present_flag, + .intra_boundary_filtering_disabled_flag = sps->intra_boundary_filtering_disable_flag, + }, + .chroma_format_idc = sps->chroma_format_idc, + .pic_width_in_luma_samples = sps->pic_width_in_luma_samples, + .pic_height_in_luma_samples = sps->pic_height_in_luma_samples, + .sps_video_parameter_set_id = sps->sps_video_parameter_set_id, + .sps_max_sub_layers_minus1 = sps->sps_max_sub_layers_minus1, + .sps_seq_parameter_set_id = sps->sps_seq_parameter_set_id, + .bit_depth_luma_minus8 = sps->bit_depth_luma_minus8, + .bit_depth_chroma_minus8 = sps->bit_depth_chroma_minus8, + .log2_max_pic_order_cnt_lsb_minus4 = sps->log2_max_pic_order_cnt_lsb_minus4, + .log2_min_luma_coding_block_size_minus3 = sps->log2_min_luma_coding_block_size_minus3, + .log2_diff_max_min_luma_coding_block_size = sps->log2_diff_max_min_luma_coding_block_size, + .log2_min_luma_transform_block_size_minus2 = sps->log2_min_luma_transform_block_size_minus2, + .log2_diff_max_min_luma_transform_block_size = sps->log2_diff_max_min_luma_transform_block_size, + .max_transform_hierarchy_depth_inter = sps->max_transform_hierarchy_depth_inter, + .max_transform_hierarchy_depth_intra = sps->max_transform_hierarchy_depth_intra, + .num_short_term_ref_pic_sets = sps->num_short_term_ref_pic_sets, + .num_long_term_ref_pics_sps = sps->num_long_term_ref_pics_sps, + .pcm_sample_bit_depth_luma_minus1 = sps->pcm_sample_bit_depth_luma_minus1, + .pcm_sample_bit_depth_chroma_minus1 = sps->pcm_sample_bit_depth_chroma_minus1, + .log2_min_pcm_luma_coding_block_size_minus3 = sps->log2_min_pcm_luma_coding_block_size_minus3, + .log2_diff_max_min_pcm_luma_coding_block_size = sps->log2_diff_max_min_pcm_luma_coding_block_size, + /* Reserved */ + /* Reserved */ + .palette_max_size = sps->palette_max_size, + .delta_palette_max_predictor_size = sps->delta_palette_max_predictor_size, + .motion_vector_resolution_control_idc = sps->motion_vector_resolution_control_idc, + .sps_num_palette_predictor_initializers_minus1 = sps->sps_num_palette_predictor_initializer_minus1, + .conf_win_left_offset = sps->conf_win_left_offset, + .conf_win_right_offset = sps->conf_win_right_offset, + .conf_win_top_offset = sps->conf_win_top_offset, + .conf_win_bottom_offset = sps->conf_win_bottom_offset, + .pProfileTierLevel = ptl_sps, + .pDecPicBufMgr = dpbm_sps, + .pScalingLists = NULL, + .pShortTermRefPicSet = str, + .pLongTermRefPicsSps = ltr, + .pSequenceParameterSetVui = vui_sps, + .pPredictorPaletteEntries = NULL, + }; + + /* PPS */ + *vkpps = (StdVideoH265PictureParameterSet) { + .flags = (StdVideoH265PpsFlags) { + .dependent_slice_segments_enabled_flag = pps->dependent_slice_segments_enabled_flag, + .output_flag_present_flag = pps->output_flag_present_flag, + .sign_data_hiding_enabled_flag = pps->sign_data_hiding_enabled_flag, + .cabac_init_present_flag = pps->cabac_init_present_flag, + .constrained_intra_pred_flag = pps->constrained_intra_pred_flag, + .transform_skip_enabled_flag = pps->transform_skip_enabled_flag, + .cu_qp_delta_enabled_flag = pps->cu_qp_delta_enabled_flag, + .pps_slice_chroma_qp_offsets_present_flag = pps->pps_slice_chroma_qp_offsets_present_flag, + .weighted_pred_flag = pps->weighted_pred_flag, + .weighted_bipred_flag = pps->weighted_bipred_flag, + .transquant_bypass_enabled_flag = pps->transquant_bypass_enabled_flag, + .tiles_enabled_flag = pps->tiles_enabled_flag, + .entropy_coding_sync_enabled_flag = pps->entropy_coding_sync_enabled_flag, + .uniform_spacing_flag = pps->uniform_spacing_flag, + .loop_filter_across_tiles_enabled_flag = pps->loop_filter_across_tiles_enabled_flag, + .pps_loop_filter_across_slices_enabled_flag = pps->pps_loop_filter_across_slices_enabled_flag, + .deblocking_filter_control_present_flag = pps->deblocking_filter_control_present_flag, + .deblocking_filter_override_enabled_flag = pps->deblocking_filter_override_enabled_flag, + .pps_deblocking_filter_disabled_flag = pps->pps_deblocking_filter_disabled_flag, + .pps_scaling_list_data_present_flag = pps->pps_scaling_list_data_present_flag, + .lists_modification_present_flag = pps->lists_modification_present_flag, + .slice_segment_header_extension_present_flag = pps->slice_segment_header_extension_present_flag, + .pps_extension_present_flag = pps->pps_extension_present_flag, + .cross_component_prediction_enabled_flag = pps->cross_component_prediction_enabled_flag, + .chroma_qp_offset_list_enabled_flag = pps->chroma_qp_offset_list_enabled_flag, + .pps_curr_pic_ref_enabled_flag = pps->pps_curr_pic_ref_enabled_flag, + .residual_adaptive_colour_transform_enabled_flag = pps->residual_adaptive_colour_transform_enabled_flag, + .pps_slice_act_qp_offsets_present_flag = pps->pps_slice_act_qp_offsets_present_flag, + .pps_palette_predictor_initializers_present_flag = pps->pps_palette_predictor_initializer_present_flag, + .monochrome_palette_flag = pps->monochrome_palette_flag, + .pps_range_extension_flag = pps->pps_range_extension_flag, + }, + .pps_pic_parameter_set_id = pps->pps_pic_parameter_set_id, + .pps_seq_parameter_set_id = pps->pps_seq_parameter_set_id, + .sps_video_parameter_set_id = sps->sps_video_parameter_set_id, + .num_extra_slice_header_bits = pps->num_extra_slice_header_bits, + .num_ref_idx_l0_default_active_minus1 = pps->num_ref_idx_l0_default_active_minus1, + .num_ref_idx_l1_default_active_minus1 = pps->num_ref_idx_l1_default_active_minus1, + .init_qp_minus26 = pps->init_qp_minus26, + .diff_cu_qp_delta_depth = pps->diff_cu_qp_delta_depth, + .pps_cb_qp_offset = pps->pps_cb_qp_offset, + .pps_cr_qp_offset = pps->pps_cr_qp_offset, + .pps_beta_offset_div2 = pps->pps_beta_offset_div2, + .pps_tc_offset_div2 = pps->pps_tc_offset_div2, + .log2_parallel_merge_level_minus2 = pps->log2_parallel_merge_level_minus2, + .log2_max_transform_skip_block_size_minus2 = pps->log2_max_transform_skip_block_size_minus2, + .diff_cu_chroma_qp_offset_depth = pps->diff_cu_chroma_qp_offset_depth, + .chroma_qp_offset_list_len_minus1 = pps->chroma_qp_offset_list_len_minus1, + .log2_sao_offset_scale_luma = pps->log2_sao_offset_scale_luma, + .log2_sao_offset_scale_chroma = pps->log2_sao_offset_scale_chroma, + .pps_act_y_qp_offset_plus5 = pps->pps_act_y_qp_offset_plus5, + .pps_act_cb_qp_offset_plus5 = pps->pps_act_cb_qp_offset_plus5, + .pps_act_cr_qp_offset_plus3 = pps->pps_act_cr_qp_offset_plus3, + .pps_num_palette_predictor_initializers = pps->pps_num_palette_predictor_initializer, + .luma_bit_depth_entry_minus8 = pps->luma_bit_depth_entry_minus8, + .chroma_bit_depth_entry_minus8 = pps->chroma_bit_depth_entry_minus8, + .num_tile_columns_minus1 = pps->num_tile_columns_minus1, + .num_tile_rows_minus1 = pps->num_tile_rows_minus1, + .pScalingLists = NULL, + .pPredictorPaletteEntries = NULL, + }; + + for (int i = 0; i < pps->num_tile_columns_minus1; i++) + vkpps->column_width_minus1[i] = pps->column_width_minus1[i]; + + for (int i = 0; i < pps->num_tile_rows_minus1; i++) + vkpps->row_height_minus1[i] = pps->row_height_minus1[i]; + + for (int i = 0; i <= pps->chroma_qp_offset_list_len_minus1; i++) { + vkpps->cb_qp_offset_list[i] = pps->cb_qp_offset_list[i]; + vkpps->cr_qp_offset_list[i] = pps->cr_qp_offset_list[i]; + } + + /* VPS */ + for (int i = 0; i < STD_VIDEO_H265_SUBLAYERS_LIST_SIZE; i++) { + dpbm_vps->max_latency_increase_plus1[i] = vps->vps_max_latency_increase_plus1[i]; + dpbm_vps->max_dec_pic_buffering_minus1[i] = vps->vps_max_dec_pic_buffering_minus1[i]; + dpbm_vps->max_num_reorder_pics[i] = vps->vps_max_num_reorder_pics[i]; + } + + *ptl_vps = (StdVideoH265ProfileTierLevel) { + .flags = (StdVideoH265ProfileTierLevelFlags) { + .general_tier_flag = vps->profile_tier_level.general_tier_flag, + .general_progressive_source_flag = vps->profile_tier_level.general_progressive_source_flag, + .general_interlaced_source_flag = vps->profile_tier_level.general_interlaced_source_flag, + .general_non_packed_constraint_flag = vps->profile_tier_level.general_non_packed_constraint_flag, + .general_frame_only_constraint_flag = vps->profile_tier_level.general_frame_only_constraint_flag, + }, + .general_profile_idc = ff_vk_h265_profile_to_vk(vps->profile_tier_level.general_profile_idc), + .general_level_idc = ff_vk_h265_level_to_vk(vps->profile_tier_level.general_level_idc), + }; + + *vkvps = (StdVideoH265VideoParameterSet) { + .flags = (StdVideoH265VpsFlags) { + .vps_temporal_id_nesting_flag = vps->vps_temporal_id_nesting_flag, + .vps_sub_layer_ordering_info_present_flag = vps->vps_sub_layer_ordering_info_present_flag, + .vps_timing_info_present_flag = vps->vps_timing_info_present_flag, + .vps_poc_proportional_to_timing_flag = vps->vps_poc_proportional_to_timing_flag, + }, + .vps_video_parameter_set_id = vps->vps_video_parameter_set_id, + .vps_max_sub_layers_minus1 = vps->vps_max_sub_layers_minus1, + /* Reserved */ + /* Reserved */ + .vps_num_units_in_tick = vps->vps_num_units_in_tick, + .vps_time_scale = vps->vps_time_scale, + .vps_num_ticks_poc_diff_one_minus1 = vps->vps_num_ticks_poc_diff_one_minus1, + /* Reserved */ + .pDecPicBufMgr = dpbm_vps, +// .pHrdParameters = vui_header_vps, + .pProfileTierLevel = ptl_vps, + }; + + return 0; +} + +static int create_session_params(AVCodecContext *avctx) +{ + int err; + VkResult ret; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + FFVulkanContext *s = &ctx->s; + FFVulkanFunctions *vk = &ctx->s.vkfn; + + VulkanH265Units vk_units = { 0 }; + + VkVideoEncodeH265SessionParametersAddInfoKHR h265_params_info; + VkVideoEncodeH265SessionParametersCreateInfoKHR h265_params; + VkVideoSessionParametersCreateInfoKHR session_params_create; + + /* Convert it to Vulkan */ + err = base_unit_to_vk(avctx, &vk_units); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Unable to convert SPS/PPS units to Vulkan: %s\n", + av_err2str(err)); + return err; + } + + /* Destroy the session params */ + if (ctx->session_params) + vk->DestroyVideoSessionParametersKHR(s->hwctx->act_dev, + ctx->session_params, + s->hwctx->alloc); + + h265_params_info = (VkVideoEncodeH265SessionParametersAddInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_SESSION_PARAMETERS_ADD_INFO_KHR, + .pStdSPSs = &vk_units.sps, + .stdSPSCount = 1, + .pStdPPSs = &vk_units.pps, + .stdPPSCount = 1, + .pStdVPSs = &vk_units.vps, + .stdVPSCount = 1, + }; + h265_params = (VkVideoEncodeH265SessionParametersCreateInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_SESSION_PARAMETERS_CREATE_INFO_KHR, + .maxStdSPSCount = 1, + .maxStdPPSCount = 1, + .maxStdVPSCount = 1, + .pParametersAddInfo = &h265_params_info, + }; + session_params_create = (VkVideoSessionParametersCreateInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_SESSION_PARAMETERS_CREATE_INFO_KHR, + .pNext = &h265_params, + .videoSession = ctx->common.session, + .videoSessionParametersTemplate = NULL, + }; + + /* Create session parameters */ + ret = vk->CreateVideoSessionParametersKHR(s->hwctx->act_dev, &session_params_create, + s->hwctx->alloc, &ctx->session_params); + if (ret != VK_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Unable to create Vulkan video session parameters: %s!\n", + ff_vk_ret2str(ret)); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int parse_feedback_units(AVCodecContext *avctx, + const uint8_t *data, size_t size, + int sps_override, int pps_override) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + + CodedBitstreamContext *cbs; + CodedBitstreamFragment au = { 0 }; + + err = ff_cbs_init(&cbs, AV_CODEC_ID_HEVC, avctx); + if (err < 0) + return err; + + err = ff_cbs_read(cbs, &au, data, size); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Unable to parse feedback units, bad drivers: %s\n", + av_err2str(err)); + return err; + } + + if (sps_override) { + for (int i = 0; i < au.nb_units; i++) { + if (au.units[i].type == HEVC_NAL_SPS) { + H265RawSPS *sps = au.units[i].content; + enc->units.raw_sps.pic_width_in_luma_samples = sps->pic_width_in_luma_samples; + enc->units.raw_sps.pic_height_in_luma_samples = sps->pic_height_in_luma_samples; + enc->units.raw_sps.log2_diff_max_min_luma_coding_block_size = sps->log2_diff_max_min_luma_coding_block_size; + enc->units.raw_sps.max_transform_hierarchy_depth_inter = sps->max_transform_hierarchy_depth_inter; + enc->units.raw_sps.max_transform_hierarchy_depth_intra = sps->max_transform_hierarchy_depth_intra; + } + } + } + + /* If PPS has an override, just copy it entirely. */ + if (pps_override) { + for (int i = 0; i < au.nb_units; i++) { + if (au.units[i].type == HEVC_NAL_PPS) { + H265RawPPS *pps = au.units[i].content; + memcpy(&enc->units.raw_pps, pps, sizeof(*pps)); + enc->fixed_qp_idr = pps->init_qp_minus26 + 26; + break; + } + } + } + + ff_cbs_fragment_free(&au); + ff_cbs_close(&cbs); + + return 0; +} + +static int init_base_units(AVCodecContext *avctx) +{ + int err; + VkResult ret; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + FFVulkanContext *s = &ctx->s; + FFVulkanFunctions *vk = &ctx->s.vkfn; + + VkVideoEncodeH265SessionParametersGetInfoKHR h265_params_info; + VkVideoEncodeSessionParametersGetInfoKHR params_info; + VkVideoEncodeH265SessionParametersFeedbackInfoKHR h265_params_feedback; + VkVideoEncodeSessionParametersFeedbackInfoKHR params_feedback; + + void *data = NULL; + size_t data_size = 0; + + /* Generate SPS/PPS unit info */ + err = init_sequence_headers(avctx); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Unable to initialize SPS/PPS units: %s\n", + av_err2str(err)); + return err; + } + + /* Create session parameters from them */ + err = create_session_params(avctx); + if (err < 0) + return err; + + h265_params_info = (VkVideoEncodeH265SessionParametersGetInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_SESSION_PARAMETERS_GET_INFO_KHR, + .writeStdSPS = 1, + .writeStdPPS = 1, + .writeStdVPS = 1, + .stdSPSId = enc->units.raw_sps.sps_seq_parameter_set_id, + .stdPPSId = enc->units.raw_pps.pps_pic_parameter_set_id, + .stdVPSId = enc->units.raw_vps.vps_video_parameter_set_id, + }; + params_info = (VkVideoEncodeSessionParametersGetInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_SESSION_PARAMETERS_GET_INFO_KHR, + .pNext = &h265_params_info, + .videoSessionParameters = ctx->session_params, + }; + + h265_params_feedback = (VkVideoEncodeH265SessionParametersFeedbackInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_SESSION_PARAMETERS_FEEDBACK_INFO_KHR, + }; + params_feedback = (VkVideoEncodeSessionParametersFeedbackInfoKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_SESSION_PARAMETERS_FEEDBACK_INFO_KHR, + .pNext = &h265_params_feedback, + }; + + ret = vk->GetEncodedVideoSessionParametersKHR(s->hwctx->act_dev, ¶ms_info, + ¶ms_feedback, + &data_size, data); + if (ret == VK_INCOMPLETE || + (ret == VK_SUCCESS) && (data_size > 0)) { + data = av_mallocz(data_size); + if (!data) + return AVERROR(ENOMEM); + } else { + av_log(avctx, AV_LOG_ERROR, "Unable to get feedback for H.265 units = %lu\n", data_size); + return err; + } + + ret = vk->GetEncodedVideoSessionParametersKHR(s->hwctx->act_dev, ¶ms_info, + ¶ms_feedback, + &data_size, data); + if (ret != VK_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Error writing feedback units\n"); + return err; + } + + av_log(avctx, AV_LOG_VERBOSE, "Feedback units written, overrides: %i (SPS: %i PPS: %i VPS: %i)\n", + params_feedback.hasOverrides, + h265_params_feedback.hasStdSPSOverrides, + h265_params_feedback.hasStdPPSOverrides, + h265_params_feedback.hasStdVPSOverrides); + + params_feedback.hasOverrides = 1; + h265_params_feedback.hasStdSPSOverrides = 1; + h265_params_feedback.hasStdPPSOverrides = 1; + + /* No need to sync any overrides */ + if (!params_feedback.hasOverrides) + return 0; + + /* Parse back tne units and override */ + err = parse_feedback_units(avctx, data, data_size, + h265_params_feedback.hasStdSPSOverrides, + h265_params_feedback.hasStdPPSOverrides); + if (err < 0) + return err; + + /* Create final session parameters */ + err = create_session_params(avctx); + if (err < 0) + return err; + + return 0; +} + +static int vulkan_encode_h265_add_nal(AVCodecContext *avctx, + CodedBitstreamFragment *au, + void *nal_unit) +{ + H265RawNALUnitHeader *header = nal_unit; + + int err = ff_cbs_insert_unit_content(au, -1, + header->nal_unit_type, nal_unit, NULL); + if (err < 0) + av_log(avctx, AV_LOG_ERROR, "Failed to add NAL unit: " + "type = %d.\n", header->nal_unit_type); + + return err; +} + +static int write_access_unit(AVCodecContext *avctx, + uint8_t *data, size_t *data_len, + CodedBitstreamFragment *au) +{ + VulkanEncodeH265Context *enc = avctx->priv_data; + + int err = ff_cbs_write_fragment_data(enc->cbs, au); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write packed header.\n"); + return err; + } + + if (*data_len < au->data_size) { + av_log(avctx, AV_LOG_ERROR, "Access unit too large: %zu < %zu.\n", + *data_len, au->data_size); + return AVERROR(ENOSPC); + } + + memcpy(data, au->data, au->data_size); + *data_len = au->data_size; + + return 0; +} + +static int write_sequence_header(AVCodecContext *avctx, + FFHWBaseEncodePicture *base_pic, + uint8_t *data, size_t *data_len) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + VulkanEncodeH265Picture *hp = base_pic ? base_pic->codec_priv : NULL; + CodedBitstreamFragment *au = &enc->current_access_unit; + + if (hp && hp->units_needed & UNIT_AUD) { + err = vulkan_encode_h265_add_nal(avctx, au, &enc->raw_aud); + if (err < 0) + goto fail; + hp->units_needed &= ~UNIT_AUD; + } + + err = vulkan_encode_h265_add_nal(avctx, au, &enc->units.raw_vps); + if (err < 0) + goto fail; + + err = vulkan_encode_h265_add_nal(avctx, au, &enc->units.raw_sps); + if (err < 0) + goto fail; + + err = vulkan_encode_h265_add_nal(avctx, au, &enc->units.raw_pps); + if (err < 0) + goto fail; + + err = write_access_unit(avctx, data, data_len, au); +fail: + ff_cbs_fragment_reset(au); + return err; +} + +static int write_extra_headers(AVCodecContext *avctx, + FFHWBaseEncodePicture *base_pic, + uint8_t *data, size_t *data_len) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + VulkanEncodeH265Picture *hp = base_pic->codec_priv; + CodedBitstreamFragment *au = &enc->current_access_unit; + + if (hp->units_needed & UNIT_AUD) { + err = vulkan_encode_h265_add_nal(avctx, au, &enc->raw_aud); + if (err < 0) + goto fail; + } + + if (hp->units_needed & UNIT_SEI_MASTERING_DISPLAY) { + err = ff_cbs_sei_add_message(enc->cbs, au, 1, + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME, + &enc->sei_mastering_display, NULL); + if (err < 0) + goto fail; + } + + if (hp->units_needed & UNIT_SEI_CONTENT_LIGHT_LEVEL) { + err = ff_cbs_sei_add_message(enc->cbs, au, 1, + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, + &enc->sei_content_light_level, NULL); + if (err < 0) + goto fail; + } + if (hp->units_needed & UNIT_SEI_A53_CC) { + err = ff_cbs_sei_add_message(enc->cbs, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &enc->sei_a53cc, NULL); + if (err < 0) + goto fail; + } + + if (hp->units_needed) { + err = write_access_unit(avctx, data, data_len, au); + if (err < 0) + goto fail; + } else { + *data_len = 0; + } + +fail: + ff_cbs_fragment_reset(au); + return err; +} + +static int write_filler(AVCodecContext *avctx, uint32_t filler, + uint8_t *data, size_t *data_len) +{ + int err; + VulkanEncodeH265Context *enc = avctx->priv_data; + CodedBitstreamFragment *au = &enc->current_access_unit; + + H265RawFiller raw_filler = { + .nal_unit_header = + { + .nal_unit_type = HEVC_NAL_FD_NUT, + .nuh_temporal_id_plus1 = 1, + }, + .filler_size = filler, + }; + + err = vulkan_encode_h265_add_nal(avctx, au, &raw_filler); + if (err < 0) + goto fail; + + err = write_access_unit(avctx, data, data_len, au); +fail: + ff_cbs_fragment_reset(au); + return err; +} + +static const FFVulkanCodec enc_cb = { + .flags = FF_HW_FLAG_B_PICTURES | + FF_HW_FLAG_B_PICTURE_REFERENCES | + FF_HW_FLAG_NON_IDR_KEY_PICTURES | + FF_HW_FLAG_SLICE_CONTROL, + .picture_priv_data_size = sizeof(VulkanEncodeH265Picture), + .filler_header_size = 7, + .init_profile = init_profile, + .init_pic_rc = init_pic_rc, + .init_pic_params = init_pic_params, + .write_sequence_headers = write_sequence_header, + .write_extra_headers = write_extra_headers, + .write_filler = write_filler, +}; + +static av_cold int vulkan_encode_h265_init(AVCodecContext *avctx) +{ + int err, ref_l0, ref_l1; + VulkanEncodeH265Context *enc = avctx->priv_data; + FFVulkanEncodeContext *ctx = &enc->common; + FFVulkanContext *s = &ctx->s; + FFHWBaseEncodeContext *base_ctx = &ctx->base; + int flags; + + if (avctx->profile == AV_PROFILE_UNKNOWN) + avctx->profile = enc->common.opts.profile; + + enc->caps = (VkVideoEncodeH265CapabilitiesKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_CAPABILITIES_KHR, + }; + + enc->quality_props = (VkVideoEncodeH265QualityLevelPropertiesKHR) { + .sType = VK_STRUCTURE_TYPE_VIDEO_ENCODE_H265_QUALITY_LEVEL_PROPERTIES_KHR, + }; + + err = ff_vulkan_encode_init(avctx, &enc->common, + &ff_vk_enc_h265_desc, &enc_cb, + &enc->caps, &enc->quality_props); + if (err < 0) + return err; + + av_log(avctx, AV_LOG_VERBOSE, "H265 encoder capabilities:\n"); + av_log(avctx, AV_LOG_VERBOSE, " Standard capability flags:\n"); + av_log(avctx, AV_LOG_VERBOSE, " separate_color_plane: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SEPARATE_COLOR_PLANE_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " sample_adaptive_offset: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SAMPLE_ADAPTIVE_OFFSET_ENABLED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " scaling_lists: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SCALING_LIST_DATA_PRESENT_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " pcm: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_PCM_ENABLED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " temporal_mvp: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SPS_TEMPORAL_MVP_ENABLED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " init_qp: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_INIT_QP_MINUS26_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " weighted:%s%s\n", + enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_WEIGHTED_PRED_FLAG_SET_BIT_KHR ? + " pred" : "", + enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_WEIGHTED_BIPRED_FLAG_SET_BIT_KHR ? + " bipred" : ""); + av_log(avctx, AV_LOG_VERBOSE, " parallel_merge_level: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_LOG2_PARALLEL_MERGE_LEVEL_MINUS2_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " sign_data_hiding: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SIGN_DATA_HIDING_ENABLED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " transform_skip:%s%s\n", + enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_TRANSFORM_SKIP_ENABLED_FLAG_SET_BIT_KHR ? + " set" : "", + enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_TRANSFORM_SKIP_ENABLED_FLAG_UNSET_BIT_KHR ? + " unset" : ""); + av_log(avctx, AV_LOG_VERBOSE, " slice_chroma_qp_offsets: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " transquant_bypass: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_TRANSQUANT_BYPASS_ENABLED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " constrained_intra_pred: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_CONSTRAINED_INTRA_PRED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " entrypy_coding_sync: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_ENTROPY_CODING_SYNC_ENABLED_FLAG_SET_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " dependent_slice_segment:%s%s\n", + enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_DEPENDENT_SLICE_SEGMENTS_ENABLED_FLAG_SET_BIT_KHR ? + " enabled" : "", + enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_DEPENDENT_SLICE_SEGMENT_FLAG_SET_BIT_KHR ? + " set" : ""); + av_log(avctx, AV_LOG_VERBOSE, " slice_qp_delta: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_SLICE_QP_DELTA_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " different_slice_qp_delta: %i\n", + !!(enc->caps.stdSyntaxFlags & VK_VIDEO_ENCODE_H265_STD_DIFFERENT_SLICE_QP_DELTA_BIT_KHR)); + + av_log(avctx, AV_LOG_VERBOSE, " Capability flags:\n"); + av_log(avctx, AV_LOG_VERBOSE, " hdr_compliance: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_HRD_COMPLIANCE_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " pred_weight_table_generated: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_PREDICTION_WEIGHT_TABLE_GENERATED_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " row_unaligned_slice: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_ROW_UNALIGNED_SLICE_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " different_slice_type: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_DIFFERENT_SLICE_TYPE_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " b_frame_in_l0_list: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L0_LIST_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " b_frame_in_l1_list: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_B_FRAME_IN_L1_LIST_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " per_pict_type_min_max_qp: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_PER_PICTURE_TYPE_MIN_MAX_QP_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " per_slice_constant_qp: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_PER_SLICE_CONSTANT_QP_BIT_KHR)); + av_log(avctx, AV_LOG_VERBOSE, " generate_prefix_nalu: %i\n", + !!(enc->caps.flags & VK_VIDEO_ENCODE_H264_CAPABILITY_GENERATE_PREFIX_NALU_BIT_KHR)); + + av_log(avctx, AV_LOG_VERBOSE, " Capabilities:\n"); + av_log(avctx, AV_LOG_VERBOSE, " maxLevelIdc: %i\n", + enc->caps.maxLevelIdc); + av_log(avctx, AV_LOG_VERBOSE, " maxSliceCount: %i\n", + enc->caps.maxSliceSegmentCount); + av_log(avctx, AV_LOG_VERBOSE, " maxTiles: %ix%i\n", + enc->caps.maxTiles.width, enc->caps.maxTiles.height); + av_log(avctx, AV_LOG_VERBOSE, " cbtSizes: 0x%x\n", + enc->caps.ctbSizes); + av_log(avctx, AV_LOG_VERBOSE, " transformBlockSizes: 0x%x\n", + enc->caps.transformBlockSizes); + av_log(avctx, AV_LOG_VERBOSE, " max(P/B)PictureL0ReferenceCount: %i P's; %i B's\n", + enc->caps.maxPPictureL0ReferenceCount, + enc->caps.maxBPictureL0ReferenceCount); + av_log(avctx, AV_LOG_VERBOSE, " maxL1ReferenceCount: %i\n", + enc->caps.maxL1ReferenceCount); + av_log(avctx, AV_LOG_VERBOSE, " maxSubLayerCount: %i\n", + enc->caps.maxSubLayerCount); + av_log(avctx, AV_LOG_VERBOSE, " expectDyadicTemporalLayerPattern: %i\n", + enc->caps.expectDyadicTemporalSubLayerPattern); + av_log(avctx, AV_LOG_VERBOSE, " min/max Qp: [%i, %i]\n", + enc->caps.maxQp, enc->caps.minQp); + av_log(avctx, AV_LOG_VERBOSE, " prefersGopRemainingFrames: %i\n", + enc->caps.prefersGopRemainingFrames); + av_log(avctx, AV_LOG_VERBOSE, " requiresGopRemainingFrames: %i\n", + enc->caps.requiresGopRemainingFrames); + + err = init_enc_options(avctx); + if (err < 0) + return err; + + flags = ctx->codec->flags; + if (!enc->caps.maxPPictureL0ReferenceCount && + !enc->caps.maxBPictureL0ReferenceCount && + !enc->caps.maxL1ReferenceCount) { + /* Intra-only */ + flags |= FF_HW_FLAG_INTRA_ONLY; + ref_l0 = ref_l1 = 0; + } else if (!enc->caps.maxPPictureL0ReferenceCount) { + /* No P-frames? How. */ + base_ctx->p_to_gpb = 1; + ref_l0 = enc->caps.maxBPictureL0ReferenceCount; + ref_l1 = enc->caps.maxL1ReferenceCount; + } else if (!enc->caps.maxBPictureL0ReferenceCount && + !enc->caps.maxL1ReferenceCount) { + /* No B-frames */ + flags &= ~(FF_HW_FLAG_B_PICTURES | FF_HW_FLAG_B_PICTURE_REFERENCES); + ref_l0 = enc->caps.maxPPictureL0ReferenceCount; + ref_l1 = 0; + } else { + /* P and B frames */ + ref_l0 = FFMIN(enc->caps.maxPPictureL0ReferenceCount, + enc->caps.maxBPictureL0ReferenceCount); + ref_l1 = enc->caps.maxL1ReferenceCount; + } + + err = ff_hw_base_init_gop_structure(base_ctx, avctx, ref_l0, ref_l1, + flags, 0); + if (err < 0) + return err; + + base_ctx->output_delay = base_ctx->b_per_p; + base_ctx->decode_delay = base_ctx->max_b_depth; + + /* Create units and session parameters */ + err = init_base_units(avctx); + if (err < 0) + return err; + + /* Init CBS */ + err = ff_cbs_init(&enc->cbs, AV_CODEC_ID_HEVC, avctx); + if (err < 0) + return err; + + /* Write extradata if needed */ + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { + uint8_t data[4096]; + size_t data_len = sizeof(data); + + err = write_sequence_header(avctx, NULL, data, &data_len); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write sequence header " + "for extradata: %d.\n", err); + return err; + } else { + avctx->extradata_size = data_len; + avctx->extradata = av_mallocz(avctx->extradata_size + + AV_INPUT_BUFFER_PADDING_SIZE); + if (!avctx->extradata) { + err = AVERROR(ENOMEM); + return err; + } + memcpy(avctx->extradata, data, avctx->extradata_size); + } + } + + return 0; +} + +static av_cold int vulkan_encode_h265_close(AVCodecContext *avctx) +{ + VulkanEncodeH265Context *enc = avctx->priv_data; + ff_vulkan_encode_uninit(&enc->common); + return 0; +} + +#define OFFSET(x) offsetof(VulkanEncodeH265Context, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) +static const AVOption vulkan_encode_h265_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, + VULKAN_ENCODE_COMMON_OPTIONS, + + { "profile", "Set profile (profile_idc and constraint_set*_flag)", + OFFSET(common.opts.profile), AV_OPT_TYPE_INT, + { .i64 = AV_PROFILE_UNKNOWN }, AV_PROFILE_UNKNOWN, 0xffff, FLAGS, .unit = "profile" }, + +#define PROFILE(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, .unit = "profile" + { PROFILE("main", AV_PROFILE_HEVC_MAIN) }, + { PROFILE("main10", AV_PROFILE_HEVC_MAIN_10) }, + { PROFILE("rext", AV_PROFILE_HEVC_REXT) }, +#undef PROFILE + + { "tier", "Set tier (general_tier_flag)", OFFSET(tier), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, FLAGS, .unit = "tier" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, 0, 0, FLAGS, .unit = "tier" }, + { "high", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, 0, 0, FLAGS, .unit = "tier" }, + + { "level", "Set level (general_level_idc)", + OFFSET(common.opts.level), AV_OPT_TYPE_INT, + { .i64 = AV_LEVEL_UNKNOWN }, AV_LEVEL_UNKNOWN, 0xff, FLAGS, .unit = "level" }, + +#define LEVEL(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, .unit = "level" + { LEVEL("1", 30) }, + { LEVEL("2", 60) }, + { LEVEL("2.1", 63) }, + { LEVEL("3", 90) }, + { LEVEL("3.1", 93) }, + { LEVEL("4", 120) }, + { LEVEL("4.1", 123) }, + { LEVEL("5", 150) }, + { LEVEL("5.1", 153) }, + { LEVEL("5.2", 156) }, + { LEVEL("6", 180) }, + { LEVEL("6.1", 183) }, + { LEVEL("6.2", 186) }, +#undef LEVEL + + { "units", "Set units to include", OFFSET(unit_elems), AV_OPT_TYPE_FLAGS, { .i64 = UNIT_SEI_MASTERING_DISPLAY | UNIT_SEI_CONTENT_LIGHT_LEVEL | UNIT_SEI_A53_CC }, 0, INT_MAX, FLAGS, "units" }, + { "hdr", "Include HDR metadata for mastering display colour volume and content light level information", 0, AV_OPT_TYPE_CONST, { .i64 = UNIT_SEI_MASTERING_DISPLAY | UNIT_SEI_CONTENT_LIGHT_LEVEL }, INT_MIN, INT_MAX, FLAGS, "units" }, + { "a53_cc", "Include A/53 caption data", 0, AV_OPT_TYPE_CONST, { .i64 = UNIT_SEI_A53_CC }, INT_MIN, INT_MAX, FLAGS, "units" }, + + { NULL }, +}; + +static const FFCodecDefault vulkan_encode_h265_defaults[] = { + { "b", "0" }, + { "bf", "2" }, + { "g", "300" }, + { "i_qfactor", "1" }, + { "i_qoffset", "0" }, + { "b_qfactor", "6/5" }, + { "b_qoffset", "0" }, + { "qmin", "-1" }, + { "qmax", "-1" }, + { NULL }, +}; + +static const AVClass vulkan_encode_h265_class = { + .class_name = "hevc_vulkan", + .item_name = av_default_item_name, + .option = vulkan_encode_h265_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_hevc_vulkan_encoder = { + .p.name = "hevc_vulkan", + CODEC_LONG_NAME("H.265/HEVC (Vulkan)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_HEVC, + .priv_data_size = sizeof(VulkanEncodeH265Context), + .init = &vulkan_encode_h265_init, + FF_CODEC_RECEIVE_PACKET_CB(&ff_vulkan_encode_receive_packet), + .close = &vulkan_encode_h265_close, + .p.priv_class = &vulkan_encode_h265_class, + .p.capabilities = AV_CODEC_CAP_DELAY | + AV_CODEC_CAP_HARDWARE | + AV_CODEC_CAP_DR1 | + AV_CODEC_CAP_ENCODER_FLUSH | + AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, + .defaults = vulkan_encode_h265_defaults, + .p.pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_VULKAN, + AV_PIX_FMT_NONE, + }, + .hw_configs = ff_vulkan_encode_hw_configs, + .p.wrapper_name = "vulkan", +}; diff --git a/libavcodec/vulkan_hevc.c b/libavcodec/vulkan_hevc.c index 68819436a4..0b20005687 100644 --- a/libavcodec/vulkan_hevc.c +++ b/libavcodec/vulkan_hevc.c @@ -179,25 +179,6 @@ static int vk_hevc_fill_pict(AVCodecContext *avctx, HEVCFrame **ref_src, return 0; } -static StdVideoH265LevelIdc convert_to_vk_level_idc(int level_idc) -{ - switch (level_idc) { - case 10: return STD_VIDEO_H265_LEVEL_IDC_1_0; - case 20: return STD_VIDEO_H265_LEVEL_IDC_2_0; - case 21: return STD_VIDEO_H265_LEVEL_IDC_2_1; - case 30: return STD_VIDEO_H265_LEVEL_IDC_3_0; - case 31: return STD_VIDEO_H265_LEVEL_IDC_3_1; - case 40: return STD_VIDEO_H265_LEVEL_IDC_4_0; - case 41: return STD_VIDEO_H265_LEVEL_IDC_4_1; - case 50: return STD_VIDEO_H265_LEVEL_IDC_5_0; - case 51: return STD_VIDEO_H265_LEVEL_IDC_5_1; - case 60: return STD_VIDEO_H265_LEVEL_IDC_6_0; - case 61: return STD_VIDEO_H265_LEVEL_IDC_6_1; - default: - case 62: return STD_VIDEO_H265_LEVEL_IDC_6_2; - } -} - static void copy_scaling_list(const ScalingList *sl, StdVideoH265ScalingLists *vksl) { for (int i = 0; i < STD_VIDEO_H265_SCALING_LIST_4X4_NUM_LISTS; i++) { @@ -338,7 +319,7 @@ static void set_sps(const HEVCSPS *sps, int sps_idx, .general_frame_only_constraint_flag = sps->ptl.general_ptl.frame_only_constraint_flag, }, .general_profile_idc = sps->ptl.general_ptl.profile_idc, - .general_level_idc = convert_to_vk_level_idc(sps->ptl.general_ptl.level_idc), + .general_level_idc = ff_vk_h265_level_to_vk(sps->ptl.general_ptl.level_idc), }; for (int i = 0; i < sps->max_sub_layers; i++) { @@ -607,8 +588,8 @@ static void set_vps(const HEVCVPS *vps, .general_non_packed_constraint_flag = vps->ptl.general_ptl.non_packed_constraint_flag, .general_frame_only_constraint_flag = vps->ptl.general_ptl.frame_only_constraint_flag, }, - .general_profile_idc = vps->ptl.general_ptl.profile_idc, - .general_level_idc = convert_to_vk_level_idc(vps->ptl.general_ptl.level_idc), + .general_profile_idc = ff_vk_h265_profile_to_vk(vps->ptl.general_ptl.profile_idc), + .general_level_idc = ff_vk_h265_level_to_vk(vps->ptl.general_ptl.level_idc), }; for (int i = 0; i < vps->vps_max_sub_layers; i++) { diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c index c056737d24..fbaf9bfef8 100644 --- a/libavcodec/vulkan_video.c +++ b/libavcodec/vulkan_video.c @@ -203,6 +203,25 @@ int ff_vk_h265_level_to_av(StdVideoH265LevelIdc level) } } +StdVideoH265LevelIdc ff_vk_h265_level_to_vk(int level_idc) +{ + switch (level_idc) { + case 10: return STD_VIDEO_H265_LEVEL_IDC_1_0; + case 20: return STD_VIDEO_H265_LEVEL_IDC_2_0; + case 21: return STD_VIDEO_H265_LEVEL_IDC_2_1; + case 30: return STD_VIDEO_H265_LEVEL_IDC_3_0; + case 31: return STD_VIDEO_H265_LEVEL_IDC_3_1; + case 40: return STD_VIDEO_H265_LEVEL_IDC_4_0; + case 41: return STD_VIDEO_H265_LEVEL_IDC_4_1; + case 50: return STD_VIDEO_H265_LEVEL_IDC_5_0; + case 51: return STD_VIDEO_H265_LEVEL_IDC_5_1; + case 60: return STD_VIDEO_H265_LEVEL_IDC_6_0; + case 61: return STD_VIDEO_H265_LEVEL_IDC_6_1; + default: + case 62: return STD_VIDEO_H265_LEVEL_IDC_6_2; + } +} + StdVideoH264ProfileIdc ff_vk_h264_profile_to_vk(int profile) { switch (profile) { @@ -214,6 +233,16 @@ StdVideoH264ProfileIdc ff_vk_h264_profile_to_vk(int profile) } } +StdVideoH265ProfileIdc ff_vk_h265_profile_to_vk(int profile) +{ + switch (profile) { + case AV_PROFILE_HEVC_MAIN: return STD_VIDEO_H265_PROFILE_IDC_MAIN; + case AV_PROFILE_HEVC_MAIN_10: return STD_VIDEO_H265_PROFILE_IDC_MAIN_10; + case AV_PROFILE_HEVC_REXT: return STD_VIDEO_H265_PROFILE_IDC_FORMAT_RANGE_EXTENSIONS; + default: return STD_VIDEO_H265_PROFILE_IDC_INVALID; + } +} + int ff_vk_h264_profile_to_av(StdVideoH264ProfileIdc profile) { switch (profile) { diff --git a/libavcodec/vulkan_video.h b/libavcodec/vulkan_video.h index 01659f6501..0ef42c9a93 100644 --- a/libavcodec/vulkan_video.h +++ b/libavcodec/vulkan_video.h @@ -76,11 +76,13 @@ int ff_vk_h264_level_to_av(StdVideoH264LevelIdc level); int ff_vk_h265_level_to_av(StdVideoH265LevelIdc level); StdVideoH264LevelIdc ff_vk_h264_level_to_vk(int level_idc); +StdVideoH265LevelIdc ff_vk_h265_level_to_vk(int level_idc); /** * Convert profile from/to AV to Vulkan */ StdVideoH264ProfileIdc ff_vk_h264_profile_to_vk(int profile); +StdVideoH265ProfileIdc ff_vk_h265_profile_to_vk(int profile); int ff_vk_h264_profile_to_av(StdVideoH264ProfileIdc profile); /**