From patchwork Sun Sep 1 00:02:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51272 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1134257vqb; Sat, 31 Aug 2024 17:03:40 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUHixqImQNTUrupQ1u6XfCmao5MbPoRNLL65P9siHwmU4eqJgdlpqXQ2a4ETsJWGoS+tSryrKEaL6fNFife/s9i@gmail.com X-Google-Smtp-Source: AGHT+IGIBTvoLxQCmMvoNIBGclvvL5aTSY62jDVSB+V3urmoSinoYaT2eNPUEonR9wPPWs2ELNNf X-Received: by 2002:a05:6000:4402:b0:374:c429:b066 with SMTP id ffacd0b85a97d-374c429b877mr933008f8f.63.1725149019879; Sat, 31 Aug 2024 17:03:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725149019; cv=none; d=google.com; s=arc-20240605; b=CXYxVkCyOB9OFYjlu3rwnspYsnQePCUuUtorWC26yfLCVK+cAi/tT6wD6GtAStSGvj OyIZuD0/08PKzhHMluw8lsDYFTL9wKbNLo7ua3MD2eBfIjMrS/DqXiiOM79zhQywJL+a O4Z3hMsGq05ByYVYhXhv6B1b0qktFRbAp18erb4+q06TkFBRdd+SwDziH6KDMoVIo8Lj fhDA1LE2c4ExQarXfkmEVAPG3q8QJF4JOvwm/uFz8ZBjBiBxIpDqR/vtHzd9QS4q2NL1 UEEl3WDIp0vBZetZkVBolTc0oAnTyW16Z/p3WUtkx4UF6GWgw6101J/wdeShpJ6IZkDx SwJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to :delivered-to; bh=Aik5ldyaPPQ3Sv6JHA/mtO7RFGSodXJsCQAMf0unoaI=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=KFpiS4tEZocfQIw/4UBWKhFdIhX3nng9kwo2qpX/Co8wZvi/yv+/XihpO2NqXm5/Jh LoZ5QVRu7apTG3d9kL+EjoFQzJeGLkID+gt+nzQ+ClesG9X/tb/mGgTQa3oGgOCqD3mj Gjegexa+kkdxXPEaQ9k6AQfyq/4ubPfMNhguH7Hqr5Bw3F8en190S35dn8oHW5SNHn89 54Apqec+lPgcelX1bpJKsqI31N55zuDBu0r/XzkzEL1uWgGLx0eT36sI+s96an85Jjxe ZUTEwJgCVIdvo5vdjFggSU7ICAgLS4j25Zq5QQYJiba2p8SCQalEcMjfWR+H6t5lKcP+ B1EQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a8988fb4d1esi500786766b.272.2024.08.31.17.03.39; Sat, 31 Aug 2024 17:03:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 634C068DECA; Sun, 1 Sep 2024 03:03:35 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5BBE668DADC for ; Sun, 1 Sep 2024 03:03:28 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:02:59 +0200 Message-ID: <20240901000314.379276-1-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/10] hw_base_encode: move VAAPI SPS/PPS constructors to a shared file X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 07V9LrCTzhog --- libavcodec/Makefile | 2 +- libavcodec/hw_base_encode_h264.c | 265 +++++++++++++++++++++++++++++++ libavcodec/hw_base_encode_h264.h | 53 +++++++ libavcodec/vaapi_encode_h264.c | 262 +++--------------------------- 4 files changed, 341 insertions(+), 241 deletions(-) create mode 100644 libavcodec/hw_base_encode_h264.c create mode 100644 libavcodec/hw_base_encode_h264.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3b4b8681f5..2e53dd723a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -166,7 +166,7 @@ OBJS-$(CONFIG_STARTCODE) += startcode.o OBJS-$(CONFIG_TEXTUREDSP) += texturedsp.o OBJS-$(CONFIG_TEXTUREDSPENC) += texturedspenc.o OBJS-$(CONFIG_TPELDSP) += tpeldsp.o -OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o hw_base_encode.o +OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o hw_base_encode.o hw_base_encode_h264.o OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o OBJS-$(CONFIG_VC1DSP) += vc1dsp.o OBJS-$(CONFIG_VIDEODSP) += videodsp.o diff --git a/libavcodec/hw_base_encode_h264.c b/libavcodec/hw_base_encode_h264.c new file mode 100644 index 0000000000..5c3957cddb --- /dev/null +++ b/libavcodec/hw_base_encode_h264.c @@ -0,0 +1,265 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "hw_base_encode_h264.h" + +#include "h2645data.h" +#include "h264_levels.h" + +#include "libavutil/pixdesc.h" + +int ff_hw_base_encode_init_params_h264(FFHWBaseEncodeContext *base_ctx, + AVCodecContext *avctx, + FFHWBaseEncodeH264 *common, + FFHWBaseEncodeH264Opts *opts) +{ + H264RawSPS *sps = &common->raw_sps; + H264RawPPS *pps = &common->raw_pps; + const AVPixFmtDescriptor *desc; + int bit_depth; + + memset(sps, 0, sizeof(*sps)); + memset(pps, 0, sizeof(*pps)); + + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); + av_assert0(desc); + if (desc->nb_components == 1 || desc->log2_chroma_w != 1 || desc->log2_chroma_h != 1) { + av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " + "%s is not supported.\n", desc->name); + return AVERROR(EINVAL); + } + bit_depth = desc->comp[0].depth; + + sps->nal_unit_header.nal_ref_idc = 3; + sps->nal_unit_header.nal_unit_type = H264_NAL_SPS; + + sps->profile_idc = avctx->profile & 0xff; + + if (avctx->profile == AV_PROFILE_H264_CONSTRAINED_BASELINE || + avctx->profile == AV_PROFILE_H264_MAIN) + sps->constraint_set1_flag = 1; + + if (avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) + sps->constraint_set3_flag = base_ctx->gop_size == 1; + + if (avctx->profile == AV_PROFILE_H264_MAIN || + avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) { + sps->constraint_set4_flag = 1; + sps->constraint_set5_flag = base_ctx->b_per_p == 0; + } + + if (base_ctx->gop_size == 1) + common->dpb_frames = 0; + else + common->dpb_frames = 1 + base_ctx->max_b_depth; + + if (avctx->level != AV_LEVEL_UNKNOWN) { + sps->level_idc = avctx->level; + } else { + const H264LevelDescriptor *level; + int framerate; + + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + framerate = avctx->framerate.num / avctx->framerate.den; + else + framerate = 0; + + level = ff_h264_guess_level(sps->profile_idc, + opts->bit_rate, + framerate, + common->mb_width * 16, + common->mb_height * 16, + common->dpb_frames); + if (level) { + av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); + if (level->constraint_set3_flag) + sps->constraint_set3_flag = 1; + sps->level_idc = level->level_idc; + } else { + av_log(avctx, AV_LOG_WARNING, "Stream will not conform " + "to any level: using level 6.2.\n"); + sps->level_idc = 62; + } + } + + sps->seq_parameter_set_id = 0; + sps->chroma_format_idc = 1; + sps->bit_depth_luma_minus8 = bit_depth - 8; + sps->bit_depth_chroma_minus8 = bit_depth - 8; + + sps->log2_max_frame_num_minus4 = 4; + sps->pic_order_cnt_type = base_ctx->max_b_depth ? 0 : 2; + if (sps->pic_order_cnt_type == 0) { + sps->log2_max_pic_order_cnt_lsb_minus4 = 4; + } + + sps->max_num_ref_frames = common->dpb_frames; + + sps->pic_width_in_mbs_minus1 = common->mb_width - 1; + sps->pic_height_in_map_units_minus1 = common->mb_height - 1; + + sps->frame_mbs_only_flag = 1; + sps->direct_8x8_inference_flag = 1; + + if (avctx->width != 16 * common->mb_width || + avctx->height != 16 * common->mb_height) { + sps->frame_cropping_flag = 1; + + sps->frame_crop_left_offset = 0; + sps->frame_crop_right_offset = + (16 * common->mb_width - avctx->width) / 2; + sps->frame_crop_top_offset = 0; + sps->frame_crop_bottom_offset = + (16 * common->mb_height - avctx->height) / 2; + } else { + sps->frame_cropping_flag = 0; + } + + sps->vui_parameters_present_flag = 1; + + if (avctx->sample_aspect_ratio.num != 0 && + avctx->sample_aspect_ratio.den != 0) { + int num, den, i; + av_reduce(&num, &den, avctx->sample_aspect_ratio.num, + avctx->sample_aspect_ratio.den, 65535); + for (i = 0; i < FF_ARRAY_ELEMS(ff_h2645_pixel_aspect); i++) { + if (num == ff_h2645_pixel_aspect[i].num && + den == ff_h2645_pixel_aspect[i].den) { + sps->vui.aspect_ratio_idc = i; + break; + } + } + if (i >= FF_ARRAY_ELEMS(ff_h2645_pixel_aspect)) { + sps->vui.aspect_ratio_idc = 255; + sps->vui.sar_width = num; + sps->vui.sar_height = den; + } + sps->vui.aspect_ratio_info_present_flag = 1; + } + + // Unspecified video format, from table E-2. + sps->vui.video_format = 5; + sps->vui.video_full_range_flag = + avctx->color_range == AVCOL_RANGE_JPEG; + sps->vui.colour_primaries = avctx->color_primaries; + sps->vui.transfer_characteristics = avctx->color_trc; + sps->vui.matrix_coefficients = avctx->colorspace; + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED || + avctx->color_trc != AVCOL_TRC_UNSPECIFIED || + avctx->colorspace != AVCOL_SPC_UNSPECIFIED) + sps->vui.colour_description_present_flag = 1; + if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED || + sps->vui.colour_description_present_flag) + sps->vui.video_signal_type_present_flag = 1; + + if (avctx->chroma_sample_location != AVCHROMA_LOC_UNSPECIFIED) { + sps->vui.chroma_loc_info_present_flag = 1; + sps->vui.chroma_sample_loc_type_top_field = + sps->vui.chroma_sample_loc_type_bottom_field = + avctx->chroma_sample_location - 1; + } + + sps->vui.timing_info_present_flag = 1; + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { + sps->vui.num_units_in_tick = avctx->framerate.den; + sps->vui.time_scale = 2 * avctx->framerate.num; + sps->vui.fixed_frame_rate_flag = 1; + } else { + sps->vui.num_units_in_tick = avctx->time_base.num; + sps->vui.time_scale = 2 * avctx->time_base.den; + sps->vui.fixed_frame_rate_flag = 0; + } + + if (opts->flags & FF_HW_H264_SEI_TIMING) { + H264RawHRD *hrd = &sps->vui.nal_hrd_parameters; + H264RawSEIBufferingPeriod *bp = &common->sei_buffering_period; + + sps->vui.nal_hrd_parameters_present_flag = 1; + + hrd->cpb_cnt_minus1 = 0; + + // Try to scale these to a sensible range so that the + // golomb encode of the value is not overlong. + hrd->bit_rate_scale = + av_clip_uintp2(av_log2(opts->bit_rate) - 15 - 6, 4); + hrd->bit_rate_value_minus1[0] = + (opts->bit_rate >> hrd->bit_rate_scale + 6) - 1; + + hrd->cpb_size_scale = + av_clip_uintp2(av_log2(opts->hrd_buffer_size) - 15 - 4, 4); + hrd->cpb_size_value_minus1[0] = + (opts->hrd_buffer_size >> hrd->cpb_size_scale + 4) - 1; + + // CBR mode as defined for the HRD cannot be achieved without filler + // data, so this flag cannot be set even with VAAPI CBR modes. + hrd->cbr_flag[0] = 0; + + hrd->initial_cpb_removal_delay_length_minus1 = 23; + hrd->cpb_removal_delay_length_minus1 = 23; + hrd->dpb_output_delay_length_minus1 = 7; + hrd->time_offset_length = 0; + + bp->seq_parameter_set_id = sps->seq_parameter_set_id; + + // This calculation can easily overflow 32 bits. + bp->nal.initial_cpb_removal_delay[0] = 90000 * + (uint64_t)opts->initial_buffer_fullness / + opts->hrd_buffer_size; + bp->nal.initial_cpb_removal_delay_offset[0] = 0; + } else { + sps->vui.nal_hrd_parameters_present_flag = 0; + sps->vui.low_delay_hrd_flag = 1 - sps->vui.fixed_frame_rate_flag; + } + + sps->vui.bitstream_restriction_flag = 1; + sps->vui.motion_vectors_over_pic_boundaries_flag = 1; + sps->vui.log2_max_mv_length_horizontal = 15; + sps->vui.log2_max_mv_length_vertical = 15; + sps->vui.max_num_reorder_frames = base_ctx->max_b_depth; + sps->vui.max_dec_frame_buffering = base_ctx->max_b_depth + 1; + + pps->nal_unit_header.nal_ref_idc = 3; + pps->nal_unit_header.nal_unit_type = H264_NAL_PPS; + + pps->pic_parameter_set_id = 0; + pps->seq_parameter_set_id = 0; + + pps->entropy_coding_mode_flag = + !(sps->profile_idc == AV_PROFILE_H264_BASELINE || + sps->profile_idc == AV_PROFILE_H264_EXTENDED || + sps->profile_idc == AV_PROFILE_H264_CAVLC_444); + if (!opts->cabac && pps->entropy_coding_mode_flag) + pps->entropy_coding_mode_flag = 0; + + pps->num_ref_idx_l0_default_active_minus1 = 0; + pps->num_ref_idx_l1_default_active_minus1 = 0; + + pps->pic_init_qp_minus26 = opts->fixed_qp_idr - 26; + + if (sps->profile_idc == AV_PROFILE_H264_BASELINE || + sps->profile_idc == AV_PROFILE_H264_EXTENDED || + sps->profile_idc == AV_PROFILE_H264_MAIN) { + pps->more_rbsp_data = 0; + } else { + pps->more_rbsp_data = 1; + + pps->transform_8x8_mode_flag = 1; + } + + return 0; +} diff --git a/libavcodec/hw_base_encode_h264.h b/libavcodec/hw_base_encode_h264.h new file mode 100644 index 0000000000..d1bae8f36f --- /dev/null +++ b/libavcodec/hw_base_encode_h264.h @@ -0,0 +1,53 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_HW_BASE_ENCODE_H264_H +#define AVCODEC_HW_BASE_ENCODE_H264_H + +#include "hw_base_encode.h" +#include "cbs_h264.h" + +typedef struct FFHWBaseEncodeH264 { + H264RawSPS raw_sps; + H264RawPPS raw_pps; + H264RawAUD raw_aud; + + H264RawSEIBufferingPeriod sei_buffering_period; + + int dpb_frames; + int mb_width; + int mb_height; +} FFHWBaseEncodeH264; + +typedef struct FFHWBaseEncodeH264Opts { + int flags; +#define FF_HW_H264_SEI_TIMING (1 << 0) + + int64_t bit_rate; + int cabac; + int fixed_qp_idr; + uint64_t hrd_buffer_size; + uint64_t initial_buffer_fullness; +} FFHWBaseEncodeH264Opts; + +int ff_hw_base_encode_init_params_h264(FFHWBaseEncodeContext *base_ctx, + AVCodecContext *avctx, + FFHWBaseEncodeH264 *common, + FFHWBaseEncodeH264Opts *opts); + +#endif /* AVCODEC_HW_BASE_ENCODE_H264_H */ diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index 35d3967766..443a569104 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -33,6 +33,7 @@ #include "cbs_h264.h" #include "codec_internal.h" #include "h264.h" +#include "hw_base_encode_h264.h" #include "h264_levels.h" #include "h2645data.h" #include "vaapi_encode.h" @@ -67,6 +68,7 @@ typedef struct VAAPIEncodeH264Picture { typedef struct VAAPIEncodeH264Context { VAAPIEncodeContext common; + FFHWBaseEncodeH264 units; // User options. int qp; @@ -85,15 +87,11 @@ typedef struct VAAPIEncodeH264Context { int fixed_qp_p; int fixed_qp_b; - int dpb_frames; - // Writer structures. CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; H264RawAUD raw_aud; - H264RawSPS raw_sps; - H264RawPPS raw_pps; H264RawSlice raw_slice; H264RawSEIBufferingPeriod sei_buffering_period; @@ -168,11 +166,11 @@ static int vaapi_encode_h264_write_sequence_header(AVCodecContext *avctx, priv->aud_needed = 0; } - err = vaapi_encode_h264_add_nal(avctx, au, &priv->raw_sps); + err = vaapi_encode_h264_add_nal(avctx, au, &priv->units.raw_sps); if (err < 0) goto fail; - err = vaapi_encode_h264_add_nal(avctx, au, &priv->raw_pps); + err = vaapi_encode_h264_add_nal(avctx, au, &priv->units.raw_pps); if (err < 0) goto fail; @@ -298,240 +296,24 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) FFHWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH264Context *priv = avctx->priv_data; - H264RawSPS *sps = &priv->raw_sps; - H264RawPPS *pps = &priv->raw_pps; + H264RawSPS *sps = &priv->units.raw_sps; + H264RawPPS *pps = &priv->units.raw_pps; VAEncSequenceParameterBufferH264 *vseq = ctx->codec_sequence_params; VAEncPictureParameterBufferH264 *vpic = ctx->codec_picture_params; - const AVPixFmtDescriptor *desc; - int bit_depth; - - memset(sps, 0, sizeof(*sps)); - memset(pps, 0, sizeof(*pps)); - - desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); - av_assert0(desc); - if (desc->nb_components == 1 || desc->log2_chroma_w != 1 || desc->log2_chroma_h != 1) { - av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " - "%s is not supported.\n", desc->name); - return AVERROR(EINVAL); - } - bit_depth = desc->comp[0].depth; - - sps->nal_unit_header.nal_ref_idc = 3; - sps->nal_unit_header.nal_unit_type = H264_NAL_SPS; - - sps->profile_idc = avctx->profile & 0xff; - - if (avctx->profile == AV_PROFILE_H264_CONSTRAINED_BASELINE || - avctx->profile == AV_PROFILE_H264_MAIN) - sps->constraint_set1_flag = 1; - - if (avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) - sps->constraint_set3_flag = base_ctx->gop_size == 1; - - if (avctx->profile == AV_PROFILE_H264_MAIN || - avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) { - sps->constraint_set4_flag = 1; - sps->constraint_set5_flag = base_ctx->b_per_p == 0; - } - - if (base_ctx->gop_size == 1) - priv->dpb_frames = 0; - else - priv->dpb_frames = 1 + base_ctx->max_b_depth; - - if (avctx->level != AV_LEVEL_UNKNOWN) { - sps->level_idc = avctx->level; - } else { - const H264LevelDescriptor *level; - int framerate; - - if (avctx->framerate.num > 0 && avctx->framerate.den > 0) - framerate = avctx->framerate.num / avctx->framerate.den; - else - framerate = 0; - - level = ff_h264_guess_level(sps->profile_idc, - avctx->bit_rate, - framerate, - priv->mb_width * 16, - priv->mb_height * 16, - priv->dpb_frames); - if (level) { - av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); - if (level->constraint_set3_flag) - sps->constraint_set3_flag = 1; - sps->level_idc = level->level_idc; - } else { - av_log(avctx, AV_LOG_WARNING, "Stream will not conform " - "to any level: using level 6.2.\n"); - sps->level_idc = 62; - } - } - - sps->seq_parameter_set_id = 0; - sps->chroma_format_idc = 1; - sps->bit_depth_luma_minus8 = bit_depth - 8; - sps->bit_depth_chroma_minus8 = bit_depth - 8; - - sps->log2_max_frame_num_minus4 = 4; - sps->pic_order_cnt_type = base_ctx->max_b_depth ? 0 : 2; - if (sps->pic_order_cnt_type == 0) { - sps->log2_max_pic_order_cnt_lsb_minus4 = 4; - } - - sps->max_num_ref_frames = priv->dpb_frames; - - sps->pic_width_in_mbs_minus1 = priv->mb_width - 1; - sps->pic_height_in_map_units_minus1 = priv->mb_height - 1; - - sps->frame_mbs_only_flag = 1; - sps->direct_8x8_inference_flag = 1; - - if (avctx->width != 16 * priv->mb_width || - avctx->height != 16 * priv->mb_height) { - sps->frame_cropping_flag = 1; - - sps->frame_crop_left_offset = 0; - sps->frame_crop_right_offset = - (16 * priv->mb_width - avctx->width) / 2; - sps->frame_crop_top_offset = 0; - sps->frame_crop_bottom_offset = - (16 * priv->mb_height - avctx->height) / 2; - } else { - sps->frame_cropping_flag = 0; - } - - sps->vui_parameters_present_flag = 1; - - if (avctx->sample_aspect_ratio.num != 0 && - avctx->sample_aspect_ratio.den != 0) { - int num, den, i; - av_reduce(&num, &den, avctx->sample_aspect_ratio.num, - avctx->sample_aspect_ratio.den, 65535); - for (i = 0; i < FF_ARRAY_ELEMS(ff_h2645_pixel_aspect); i++) { - if (num == ff_h2645_pixel_aspect[i].num && - den == ff_h2645_pixel_aspect[i].den) { - sps->vui.aspect_ratio_idc = i; - break; - } - } - if (i >= FF_ARRAY_ELEMS(ff_h2645_pixel_aspect)) { - sps->vui.aspect_ratio_idc = 255; - sps->vui.sar_width = num; - sps->vui.sar_height = den; - } - sps->vui.aspect_ratio_info_present_flag = 1; - } - - // Unspecified video format, from table E-2. - sps->vui.video_format = 5; - sps->vui.video_full_range_flag = - avctx->color_range == AVCOL_RANGE_JPEG; - sps->vui.colour_primaries = avctx->color_primaries; - sps->vui.transfer_characteristics = avctx->color_trc; - sps->vui.matrix_coefficients = avctx->colorspace; - if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED || - avctx->color_trc != AVCOL_TRC_UNSPECIFIED || - avctx->colorspace != AVCOL_SPC_UNSPECIFIED) - sps->vui.colour_description_present_flag = 1; - if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED || - sps->vui.colour_description_present_flag) - sps->vui.video_signal_type_present_flag = 1; - - if (avctx->chroma_sample_location != AVCHROMA_LOC_UNSPECIFIED) { - sps->vui.chroma_loc_info_present_flag = 1; - sps->vui.chroma_sample_loc_type_top_field = - sps->vui.chroma_sample_loc_type_bottom_field = - avctx->chroma_sample_location - 1; - } - - sps->vui.timing_info_present_flag = 1; - if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { - sps->vui.num_units_in_tick = avctx->framerate.den; - sps->vui.time_scale = 2 * avctx->framerate.num; - sps->vui.fixed_frame_rate_flag = 1; - } else { - sps->vui.num_units_in_tick = avctx->time_base.num; - sps->vui.time_scale = 2 * avctx->time_base.den; - sps->vui.fixed_frame_rate_flag = 0; - } - - if (priv->sei & SEI_TIMING) { - H264RawHRD *hrd = &sps->vui.nal_hrd_parameters; - H264RawSEIBufferingPeriod *bp = &priv->sei_buffering_period; - - sps->vui.nal_hrd_parameters_present_flag = 1; - - hrd->cpb_cnt_minus1 = 0; - - // Try to scale these to a sensible range so that the - // golomb encode of the value is not overlong. - hrd->bit_rate_scale = - av_clip_uintp2(av_log2(ctx->va_bit_rate) - 15 - 6, 4); - hrd->bit_rate_value_minus1[0] = - (ctx->va_bit_rate >> hrd->bit_rate_scale + 6) - 1; - - hrd->cpb_size_scale = - av_clip_uintp2(av_log2(ctx->hrd_params.buffer_size) - 15 - 4, 4); - hrd->cpb_size_value_minus1[0] = - (ctx->hrd_params.buffer_size >> hrd->cpb_size_scale + 4) - 1; - - // CBR mode as defined for the HRD cannot be achieved without filler - // data, so this flag cannot be set even with VAAPI CBR modes. - hrd->cbr_flag[0] = 0; - hrd->initial_cpb_removal_delay_length_minus1 = 23; - hrd->cpb_removal_delay_length_minus1 = 23; - hrd->dpb_output_delay_length_minus1 = 7; - hrd->time_offset_length = 0; - - bp->seq_parameter_set_id = sps->seq_parameter_set_id; - - // This calculation can easily overflow 32 bits. - bp->nal.initial_cpb_removal_delay[0] = 90000 * - (uint64_t)ctx->hrd_params.initial_buffer_fullness / - ctx->hrd_params.buffer_size; - bp->nal.initial_cpb_removal_delay_offset[0] = 0; - } else { - sps->vui.nal_hrd_parameters_present_flag = 0; - sps->vui.low_delay_hrd_flag = 1 - sps->vui.fixed_frame_rate_flag; - } - - sps->vui.bitstream_restriction_flag = 1; - sps->vui.motion_vectors_over_pic_boundaries_flag = 1; - sps->vui.log2_max_mv_length_horizontal = 15; - sps->vui.log2_max_mv_length_vertical = 15; - sps->vui.max_num_reorder_frames = base_ctx->max_b_depth; - sps->vui.max_dec_frame_buffering = base_ctx->max_b_depth + 1; - - pps->nal_unit_header.nal_ref_idc = 3; - pps->nal_unit_header.nal_unit_type = H264_NAL_PPS; - - pps->pic_parameter_set_id = 0; - pps->seq_parameter_set_id = 0; - - pps->entropy_coding_mode_flag = - !(sps->profile_idc == AV_PROFILE_H264_BASELINE || - sps->profile_idc == AV_PROFILE_H264_EXTENDED || - sps->profile_idc == AV_PROFILE_H264_CAVLC_444); - if (!priv->coder && pps->entropy_coding_mode_flag) - pps->entropy_coding_mode_flag = 0; - - pps->num_ref_idx_l0_default_active_minus1 = 0; - pps->num_ref_idx_l1_default_active_minus1 = 0; - - pps->pic_init_qp_minus26 = priv->fixed_qp_idr - 26; - - if (sps->profile_idc == AV_PROFILE_H264_BASELINE || - sps->profile_idc == AV_PROFILE_H264_EXTENDED || - sps->profile_idc == AV_PROFILE_H264_MAIN) { - pps->more_rbsp_data = 0; - } else { - pps->more_rbsp_data = 1; + FFHWBaseEncodeH264Opts unit_opts = { + .flags = (priv->sei & SEI_TIMING) ? FF_HW_H264_SEI_TIMING : 0, + .cabac = priv->coder, + .hrd_buffer_size = ctx->hrd_params.buffer_size, + .fixed_qp_idr = priv->fixed_qp_idr, + .initial_buffer_fullness = ctx->hrd_params.initial_buffer_fullness, + .bit_rate = ctx->va_bit_rate, + }; - pps->transform_8x8_mode_flag = 1; - } + int err = ff_hw_base_encode_init_params_h264(base_ctx, avctx, + &priv->units, &unit_opts); + if (err < 0) + return err; *vseq = (VAEncSequenceParameterBufferH264) { .seq_parameter_set_id = sps->seq_parameter_set_id, @@ -660,7 +442,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, } } hpic->pic_order_cnt = pic->display_order - hpic->last_idr_frame; - if (priv->raw_sps.pic_order_cnt_type == 2) { + if (priv->units.raw_sps.pic_order_cnt_type == 2) { hpic->pic_order_cnt *= 2; } @@ -870,8 +652,8 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, VAAPIEncodePicture *vaapi_pic = pic->priv; VAAPIEncodeH264Picture *hpic = pic->codec_priv; FFHWBaseEncodePicture *prev = pic->prev; - H264RawSPS *sps = &priv->raw_sps; - H264RawPPS *pps = &priv->raw_pps; + H264RawSPS *sps = &priv->units.raw_sps; + H264RawPPS *pps = &priv->units.raw_pps; H264RawSliceHeader *sh = &priv->raw_slice.header; VAEncPictureParameterBufferH264 *vpic = vaapi_pic->codec_picture_params; VAEncSliceParameterBufferH264 *vslice = slice->codec_slice_params; @@ -923,7 +705,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, ++keep; } } - av_assert0(keep <= priv->dpb_frames); + av_assert0(keep <= priv->units.dpb_frames); if (discard == 0) { sh->adaptive_ref_pic_marking_mode_flag = 0; From patchwork Sun Sep 1 00:03:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51273 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1134315vqb; Sat, 31 Aug 2024 17:03:51 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUYZFVHgwBivM3MagPld1bPXg98j5y7JFzG4W1s9+riVBm51vvjW8Y7sZVZYS0Ksu97jB6O/Ssc1o8GbHlmjKem@gmail.com X-Google-Smtp-Source: AGHT+IFAANUFCJcIBkJBmPrQO085kNer1+A0YtAAO8S2bSohAI79pMM73CVkO1eupJ8Nv6WxHxSB X-Received: by 2002:a5d:4652:0:b0:374:bdd7:f848 with SMTP id ffacd0b85a97d-374bdd7f9b4mr1597482f8f.59.1725149031407; Sat, 31 Aug 2024 17:03:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725149031; cv=none; d=google.com; s=arc-20240605; b=DnR0Jkpkr0RaO5v4SYkLWyiD5CvgIQH8RZBcx/gkSNx9i8T7QaeXBJwYsAyN1rXd5s JEIJtHdD8tiwEAJLUhYaq0IhgD2t2K1DnyyrZ6b0CEuXypr1v+IR8Iv+qI3wQJN+zgCw 8vnMx/5FG/t+WrgADZKDl0ZYv3lbv6EDucZGbVlH/s5EPBprwHfq/DeYo1TA1hSTEZZd h/Y9byMaDfWlaRPHVWTzc13dYyBfUjsK77yHYtELxpXANvdfCT4HQUvICuTT7DiZL+hX oW6zW9OT9g29j/avQ3S5VJSEOZ/MnRYfzkszQX2qu7ICMkxrDtKj6+OeZek8HVRtYaaw LirQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=IBT1e3Rqdk5saKo4Hm/0Jfu7XQy8NGjHOvo8uZ1p4Hg=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=kA56HQbiFD4kIgxhGicrg1IiqkhiIwoqKnZtPU1IHbzb85Vj1ItWFmpmj+i4Zo01c2 cm7hM3HQUVuwJZVY6AgTqeKCF5xHtFRsFG4HoPuXsyZpXO+Q0q7MXk1YcbuuUsg8G1HD tQv0YIGUVkIu51M6tlJ+Ctd3mAY3KzoZOeCp7WX7By0YwxsjpSrB4gteUF9kizucTk4X dxuRbj38XKnD0I0+iaI7VmCKMFNu6ahYSGi5Bo20uCh9fOKuUzBpaqriJVtMBk6dJtTF L5OvVQ7vuQn5JQNxddDtgJGPWIPVwPc8b7ylrhASayiQk95EGowNLeFgElqUT/QG4okN GMCA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a898916b23fsi495344666b.545.2024.08.31.17.03.50; Sat, 31 Aug 2024 17:03:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7EF1768DF4B; Sun, 1 Sep 2024 03:03:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3E3A168DEC7 for ; Sun, 1 Sep 2024 03:03:29 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:00 +0200 Message-ID: <20240901000314.379276-2-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/10] hw_base_encode: allocate DPB image upfront X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: W+eQ1QLlIv/m Vulkan requires this, as it needs to initialize state upfront. --- libavcodec/d3d12va_encode.c | 6 ------ libavcodec/hw_base_encode.c | 6 ++++++ libavcodec/vaapi_encode.c | 5 ----- 3 files changed, 6 insertions(+), 11 deletions(-) diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c index d437750598..68fb9927a8 100644 --- a/libavcodec/d3d12va_encode.c +++ b/libavcodec/d3d12va_encode.c @@ -264,12 +264,6 @@ static int d3d12va_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Input surface is %p.\n", pic->input_surface->texture); - err = av_hwframe_get_buffer(base_ctx->recon_frames_ref, base_pic->recon_image, 0); - if (err < 0) { - err = AVERROR(ENOMEM); - goto fail; - } - pic->recon_surface = (AVD3D12VAFrame *)base_pic->recon_image->data[0]; av_log(avctx, AV_LOG_DEBUG, "Recon surface is %p.\n", pic->recon_surface->texture); diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index 35dac6e596..8411cc7582 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -446,6 +446,12 @@ static int hw_base_encode_send_frame(AVCodecContext *avctx, FFHWBaseEncodeContex goto fail; } + err = av_hwframe_get_buffer(ctx->recon_frames_ref, pic->recon_image, 0); + if (err < 0) { + err = AVERROR(ENOMEM); + goto fail; + } + pic->priv = av_mallocz(ctx->op->priv_size); if (!pic->priv) { err = AVERROR(ENOMEM); diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index b72e9035cb..0058e1e772 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -313,11 +313,6 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Input surface is %#x.\n", pic->input_surface); - err = av_hwframe_get_buffer(base_ctx->recon_frames_ref, base_pic->recon_image, 0); - if (err < 0) { - err = AVERROR(ENOMEM); - goto fail; - } pic->recon_surface = (VASurfaceID)(uintptr_t)base_pic->recon_image->data[3]; av_log(avctx, AV_LOG_DEBUG, "Recon surface is %#x.\n", pic->recon_surface); From patchwork Sun Sep 1 00:03:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51277 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1138628vqb; Sat, 31 Aug 2024 17:20:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW2tmdcy9WO22Sy2yfZpFWrhCkgNMq1or1+Uz9Tp03++UqX0ni45v5hGVl3NZ9vuHeaIe0Ld7cCHHdr3edP87FZ@gmail.com X-Google-Smtp-Source: AGHT+IFYlagYwHsWPrXrdR1UucKFH5SopBwqqe/Bjo5ZUw2blgh4C92ySFQIXgKtpsrAOqjFFie2 X-Received: by 2002:a2e:6109:0:b0:2f3:c82d:9e7d with SMTP id 38308e7fff4ca-2f61052c6b9mr57972321fa.47.1725150012094; Sat, 31 Aug 2024 17:20:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725150012; cv=none; d=google.com; s=arc-20240605; b=f6GTUnoTbrZOAM83MZXmzt7HCM9XXUPIuVtSkSn47Ii0C0REovqL/S2YuckYcmHcc3 PPRM0xnX49vdLkW6SmkNUGQaV+mbV3YBWqKyIokKeXYnxyKklhHJz2xbQhWd59Zc+m0c zWo3HRi/NFl20+QbHU9uTzRwpQWfqk+AIR4balq1NFDSeJ7VghV4++jCy9vSLiMAaDHV nMfB7/CmmzHjNhijerj/zALqz2SqleashIYaiwBOXKwynPVQLnx7Qmcu4aqHcTIdSE6x h6ARcnfnyz9rQEU4jzs/thiKw0lxDuV7VaGtWsFH7LzU16HD5S7WDNt3b7mITwTZ3mHB 8wnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=AadiAKOh7rAbveUK2a2j51cS8dz3Hyjb4uzoam34XA0=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=Z6aFgeDPTXZS3GbByfUNnp8zHQdKhzxDJlgshP2OI95wag4Jg5Z4Kumn9472TQ/C4E Nz3vuKUKu8kV5rTgVQn4tzNjtUZxa0Al8sqHh+HY49Sy1hd1wg1EiOfEMCR5Ydi7mki9 frjDxNQJXtfPhLIqokBHCIyIx/wVO9MI4Ji5QgH4qiKxLrpxnDi+vqGKZXmsZpiS3I9N qnXbWK3pYvR+sEZYBi1IHnIibn8pfaXjgKl5ZB1UFaA2kPIyWLURHnE6ihbBuEosn73m +DHLKXuz9AUcu5rqHlNUGqsbwKr1ZvR/fhG4ehXOWwELMkcORNxuOfQp73n0SzSBNnLW uohg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f615161f52si18625041fa.371.2024.08.31.17.20.11; Sat, 31 Aug 2024 17:20:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B189968DF57; Sun, 1 Sep 2024 03:03:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 96B4168DF0C for ; Sun, 1 Sep 2024 03:03:30 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:01 +0200 Message-ID: <20240901000314.379276-3-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/10] hw_base_encode: make recon_frames_ref optional X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RbatJmATOdaw Vulkan supports some stupidly odd hardware, that unfortunately, most modern GPUs happen to be. The DPB images for encoders may be required to be preallocated all at once, and rather than be individual frames, be layers of a single frame. As the hw_base_encode code is written with the thought that either the driver or the device itself supports sane image allocation, Vulkan does not leave us with this option. So, in the case that the hardware does not support individual frames to be used as DPBs, make the DBP frames context optional, and let the subsystem manage this. --- libavcodec/hw_base_encode.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index 8411cc7582..7b6ec97d3b 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -440,16 +440,18 @@ static int hw_base_encode_send_frame(AVCodecContext *avctx, FFHWBaseEncodeContex goto fail; } - pic->recon_image = av_frame_alloc(); - if (!pic->recon_image) { - err = AVERROR(ENOMEM); - goto fail; - } + if (ctx->recon_frames_ref) { + pic->recon_image = av_frame_alloc(); + if (!pic->recon_image) { + err = AVERROR(ENOMEM); + goto fail; + } - err = av_hwframe_get_buffer(ctx->recon_frames_ref, pic->recon_image, 0); - if (err < 0) { - err = AVERROR(ENOMEM); - goto fail; + err = av_hwframe_get_buffer(ctx->recon_frames_ref, pic->recon_image, 0); + if (err < 0) { + err = AVERROR(ENOMEM); + goto fail; + } } pic->priv = av_mallocz(ctx->op->priv_size); From patchwork Sun Sep 1 00:03:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51274 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1134399vqb; Sat, 31 Aug 2024 17:04:10 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXdeF9zNErjbu1AgY706bG3vdB5h47XNZtENq7SOeT5sEZ/6RRMVqhpWAoz6FAJpQjU1rslePh5HQxDibjhvaYV@gmail.com X-Google-Smtp-Source: AGHT+IFpVn3OyEzRtj/ATj7aPXzClHu4hwEao4ZfS/NKa0gnr8rFwIwZ98jIl14MMi+0foWESXNC X-Received: by 2002:a05:6512:2208:b0:530:dfab:9315 with SMTP id 2adb3069b0e04-53546b191f5mr6208619e87.10.1725149050363; Sat, 31 Aug 2024 17:04:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725149050; cv=none; d=google.com; s=arc-20240605; b=IzS97t0te1/URu5d95gvvmBXfJYsg8Ww7YvRm+DW9Z6ywQdnzKxvRo0R8ivOpXy0/O szyo31Smcfgkj8w6O/n/Jq5SdryLaSJYgxCwAyrOyCL6Nz/Mof/5zG+AJM8BK6UWXObY ps1seC4Z80zEpxu0KifOifojC6lOgjptElaNmHnqwz+nizQHVaaA5a6pdr63jZxi0k2d gFSizAhpSddzSjlO6L31jSsYGgIKUT2v64u2lAI+2NDF4LYh3ZL92BL2xeQg1Uu2TdO1 GeoJx//TU2VA/guew9HxTEqPh2vasO/9qqLRlcW+v4ScUr8mdi8qzHPSgXmy56g+Xd48 kdIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=vzAjYPtoXZ/9NzCGkMnBadk5SbzGYX9CAE4BQgpYRYw=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=Kb7NRHQ+Qg+VeBzda0xmze6qNA5BPzH5CScpqs3ZokBjOQ4hK+7TB9sSfibfwSTW3A izhWCekJDVPvQO7B1y+1DiaMfRnBBNDlhr50Ho136OIJ5ubVRVTxJgt04ic2qnMDKY9S 5EY/SlO+gBOuS/8DNG/svh5OUPPAeY0TugP0OLE/Ia8SpXFbO3LKjR/awkLPL0dmJJ1m 1nKDTAYu17TiWgSjNqZpGV3WhULhw3JuOpSpt35//0lxzi1YcoQADO0u4U6vj+ui7CUO SuNCVM1sj4Zo/M0yLuH+qIYTHQqyHXw6PEMs/eCaeHD/mningtmf314EflHgkFDKLums YwZA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a89891f5e1fsi518369166b.963.2024.08.31.17.04.09; Sat, 31 Aug 2024 17:04:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E669668DF3B; Sun, 1 Sep 2024 03:03:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2FA1F68DF51 for ; Sun, 1 Sep 2024 03:03:32 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:02 +0200 Message-ID: <20240901000314.379276-4-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/10] vulkan_video: add utilities for H264 level/profile mapping X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZHvbRnLvc7j3 --- libavcodec/vulkan_video.c | 49 +++++++++++++++++++++++++++++++++++++++ libavcodec/vulkan_video.h | 8 +++++++ 2 files changed, 57 insertions(+) diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c index b9a0ed5022..a676f0332a 100644 --- a/libavcodec/vulkan_video.c +++ b/libavcodec/vulkan_video.c @@ -16,6 +16,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "defs.h" #include "libavutil/mem.h" #include "vulkan_video.h" @@ -158,6 +159,32 @@ int ff_vk_h264_level_to_av(StdVideoH264LevelIdc level) } } +StdVideoH264LevelIdc ff_vk_h264_level_to_vk(int level_idc) +{ + switch (level_idc) { + case 10: return STD_VIDEO_H264_LEVEL_IDC_1_0; + case 11: return STD_VIDEO_H264_LEVEL_IDC_1_1; + case 12: return STD_VIDEO_H264_LEVEL_IDC_1_2; + case 13: return STD_VIDEO_H264_LEVEL_IDC_1_3; + case 20: return STD_VIDEO_H264_LEVEL_IDC_2_0; + case 21: return STD_VIDEO_H264_LEVEL_IDC_2_1; + case 22: return STD_VIDEO_H264_LEVEL_IDC_2_2; + case 30: return STD_VIDEO_H264_LEVEL_IDC_3_0; + case 31: return STD_VIDEO_H264_LEVEL_IDC_3_1; + case 32: return STD_VIDEO_H264_LEVEL_IDC_3_2; + case 40: return STD_VIDEO_H264_LEVEL_IDC_4_0; + case 41: return STD_VIDEO_H264_LEVEL_IDC_4_1; + case 42: return STD_VIDEO_H264_LEVEL_IDC_4_2; + case 50: return STD_VIDEO_H264_LEVEL_IDC_5_0; + case 51: return STD_VIDEO_H264_LEVEL_IDC_5_1; + case 52: return STD_VIDEO_H264_LEVEL_IDC_5_2; + case 60: return STD_VIDEO_H264_LEVEL_IDC_6_0; + case 61: return STD_VIDEO_H264_LEVEL_IDC_6_1; + default: + case 62: return STD_VIDEO_H264_LEVEL_IDC_6_2; + } +} + int ff_vk_h265_level_to_av(StdVideoH265LevelIdc level) { switch (level) { @@ -177,6 +204,28 @@ int ff_vk_h265_level_to_av(StdVideoH265LevelIdc level) } } +StdVideoH264ProfileIdc ff_vk_h264_profile_to_vk(int profile) +{ + switch (profile) { + case AV_PROFILE_H264_CONSTRAINED_BASELINE: return STD_VIDEO_H264_PROFILE_IDC_BASELINE; + case AV_PROFILE_H264_MAIN: return STD_VIDEO_H264_PROFILE_IDC_MAIN; + case AV_PROFILE_H264_HIGH: return STD_VIDEO_H264_PROFILE_IDC_HIGH; + case AV_PROFILE_H264_HIGH_444_PREDICTIVE: return STD_VIDEO_H264_PROFILE_IDC_HIGH_444_PREDICTIVE; + default: return STD_VIDEO_H264_PROFILE_IDC_INVALID; + } +} + +int ff_vk_h264_profile_to_av(StdVideoH264ProfileIdc profile) +{ + switch (profile) { + case STD_VIDEO_H264_PROFILE_IDC_BASELINE: return AV_PROFILE_H264_CONSTRAINED_BASELINE; + case STD_VIDEO_H264_PROFILE_IDC_MAIN: return AV_PROFILE_H264_MAIN; + case STD_VIDEO_H264_PROFILE_IDC_HIGH: return AV_PROFILE_H264_HIGH; + case STD_VIDEO_H264_PROFILE_IDC_HIGH_444_PREDICTIVE: return AV_PROFILE_H264_HIGH_444_PREDICTIVE; + default: return AV_PROFILE_UNKNOWN; + } +} + int ff_vk_video_qf_init(FFVulkanContext *s, FFVkQueueFamilyCtx *qf, VkQueueFlagBits family, VkVideoCodecOperationFlagBitsKHR caps) { diff --git a/libavcodec/vulkan_video.h b/libavcodec/vulkan_video.h index 2cb9419fd8..a5f69c576c 100644 --- a/libavcodec/vulkan_video.h +++ b/libavcodec/vulkan_video.h @@ -66,6 +66,14 @@ int ff_vk_video_qf_init(FFVulkanContext *s, FFVkQueueFamilyCtx *qf, int ff_vk_h264_level_to_av(StdVideoH264LevelIdc level); int ff_vk_h265_level_to_av(StdVideoH265LevelIdc level); +StdVideoH264LevelIdc ff_vk_h264_level_to_vk(int level_idc); + +/** + * Convert profile from/to AV to Vulkan + */ +StdVideoH264ProfileIdc ff_vk_h264_profile_to_vk(int profile); +int ff_vk_h264_profile_to_av(StdVideoH264ProfileIdc profile); + /** * Initialize video session, allocating and binding necessary memory. */ From patchwork Sun Sep 1 00:03:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51278 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1140476vqb; Sat, 31 Aug 2024 17:30:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX5DUm5EqEiCx38gjM4F700THm+Zcn1lUPimL7+x4pDejPoRgNA7JlK5CI2ESEU6HtlQ0skCVU+mgG1Jl+C0rub@gmail.com X-Google-Smtp-Source: AGHT+IFAAt7GI2qnsSQqAMMmrm/57Dt1IQ9nYXrNVuA5ES77Cfdxp5JVPdtR3nGPrYDb5Xf5FVxg X-Received: by 2002:a17:907:3e92:b0:a80:a37f:c303 with SMTP id a640c23a62f3a-a89a357825bmr228971166b.4.1725150612589; Sat, 31 Aug 2024 17:30:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725150612; cv=none; d=google.com; s=arc-20240605; b=Mkzj6XsM4ZONQRfvMfZ/4N3Wk7AOUIX32D30RORHa82eV7lEW4R/E3xYK3z71tMlQd GQ0liwkPoPjWjSnZPxEBLQ+c3rOegcTyWGjp1mVZ2l+ytoLzEzUhc411vea2Quckiv3B /LmFWMD9B1kZLn9eBXhHwqNIzawnMxVTkH3xCOMeZ/2Syf05JwgZZvsZx8vctUJeXca6 QbmXJbFZCDdqvAqR2Z8sM08M33+lPYUF4b7Z/AtD6VcIPZRLnGmtdlzkdAgopk2vExpv oR0ItEdG0QNwJt59z/z8f301os4ZaZGWG3FIawDIG5MG3w7KN+mClEQ3jfo/cKKiFItj TQSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=BrjDfFzaOuIOa+KVT44cBdvQDadLMuqK5AHoTiBKAbs=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=VImWTnHzi2hBgUEU+Zb7i2wz+sSkzRPJ71ow/ixJBivJBB+lJtVkMwU4jab0hAJdQj omI+KDCx8GsQE8q3FtiCA4p6GZakzLd1346uExYTA2mk4iuab28q1FkGsFlQ5hZo3MVo /LxgwVh7RaAtZSH35ZYvj/aHE3cqYF8NE7j4GJ0G9Vnn1wxlHqC8ncZPjV6Oi9rhFBcu IdHo+pF/6L9N434Hu5aDSOxZCN1j39quJSMYFXzXc2BbwmLgs3rQLJio6pQIMe/CVtRi 4cjqwZHa18vhmMYygY2yGKBs37KsHhHS15SvWwsERGqN0Zu+OtyB4cxAPYvBJGTELHaz wjkQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a8988fb610dsi535307266b.246.2024.08.31.17.30.12; Sat, 31 Aug 2024 17:30:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 18F2668DF70; Sun, 1 Sep 2024 03:03:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ED5B368DF5A for ; Sun, 1 Sep 2024 03:03:32 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:03 +0200 Message-ID: <20240901000314.379276-5-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/10] vulkan: use correct return codes for query errors X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3uTJZzI61rD2 --- libavutil/vulkan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavutil/vulkan.c b/libavutil/vulkan.c index bb8e7ae786..fe5affa72a 100644 --- a/libavutil/vulkan.c +++ b/libavutil/vulkan.c @@ -444,7 +444,7 @@ VkResult ff_vk_exec_get_query(FFVulkanContext *s, FFVkExecContext *e, VkQueryResultFlags qf = 0; if (!e->had_submission) - return VK_NOT_READY; + return VK_INCOMPLETE; qf |= pool->query_64bit ? VK_QUERY_RESULT_64_BIT : 0x0; From patchwork Sun Sep 1 00:03:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51275 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1134489vqb; Sat, 31 Aug 2024 17:04:29 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWYz5ibTgD3qptSWG+fZ7Hq4qDKU92faSLmEomZJ9+cA5mU7ucqCKgrj7Pvp0vyTj307ewnVgqLm09BzJyI2H4B@gmail.com X-Google-Smtp-Source: AGHT+IFiX9wCDjXvkzRigBTy/FsUY64wGP0J+NvWDG2/hcVBCi8rqIOe+Z0u9DzIccD0n9TrtWbQ X-Received: by 2002:a2e:4611:0:b0:2f5:ce9:14d3 with SMTP id 38308e7fff4ca-2f610551290mr64658091fa.47.1725149069404; Sat, 31 Aug 2024 17:04:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725149069; cv=none; d=google.com; s=arc-20240605; b=d+f7sGHnWEQwJJxVajQ8Ht1gVmhPO+sxgXU1PtBSq7p3qQ3+s5GvEl4FwrpF6xDDlH u5+Uumbkakc9Cwb/6Sd7KAS97GZINuXa9HKILbGIAgOZvAvOQpG6MZjtvMqPwkbsWX7M oL0BIYI6FkZQvxUBlGOBSmcD1zcfFvyPX30m/swENHo9pFUzBexo+/bcUZKPfITHt9Z4 ZgNAfvpfmGcjxlYYQGSQmY+K+JvzeYTpblxUCOZKpyQNx0lT2eRLSWOVdzEnpoZf2igY /LCWpkJFqhgr9m8QGtisFPouN8RbYqRWKSKyHcH5KHYsOr99hGWutpmSsFgf6ixNTLVJ zxhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=9T0+urXc+Qn/Yp+Q2RY+7rLxt0ZduiCZZw6wNhn9Gts=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=IV6ilCnVss9Zvt5AkbS+wMfRdYRv6yo/ZbkYaeVDjNfWvwtQIzWC52brTQsCWXtqbV fv5KedYkoTxl93JditD1BvmkUBjiDT3IujbuuEjRyL4s2Hze9gH4TzyC6TxlUtEjLwSJ VW1de+eA9vwNuAis4d/YyiTmh6ar2v9Odfm1LNcJa5gZBQ+6kIJh/AeiqDSAmtzi1x3K XRIea+M1tfvXLqzbHsqG0rpfKPTU8gmneVTX6DJIb5yyEnCU83bX6Gv9FZbrJZ5/3MlX 7MDy6Ag9idSPnruDJw8xlkDGN/P9GXAYiftQfS9aO5Liky+jBeVx0pSZ3cerM4gR3hvW 8qCA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f614ece084si21501991fa.167.2024.08.31.17.04.29; Sat, 31 Aug 2024 17:04:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6163D68DF7C; Sun, 1 Sep 2024 03:03:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0161468DF65 for ; Sun, 1 Sep 2024 03:03:33 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:04 +0200 Message-ID: <20240901000314.379276-6-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/10] vulkan: error out if query is called without being initialized X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: u/AB5TmI+Vek --- libavutil/vulkan.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/libavutil/vulkan.c b/libavutil/vulkan.c index fe5affa72a..c2a3b5cfa3 100644 --- a/libavutil/vulkan.c +++ b/libavutil/vulkan.c @@ -446,6 +446,11 @@ VkResult ff_vk_exec_get_query(FFVulkanContext *s, FFVkExecContext *e, if (!e->had_submission) return VK_INCOMPLETE; + if (!e->query_data) { + av_log(s, AV_LOG_ERROR, "Requested a query with a NULL query_data pointer!\n"); + return VK_INCOMPLETE; + } + qf |= pool->query_64bit ? VK_QUERY_RESULT_64_BIT : 0x0; qf |= pool->query_statuses ? From patchwork Sun Sep 1 00:03:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51276 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1137599vqb; Sat, 31 Aug 2024 17:15:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUvdyZG1sYvIKwgXtePrnzq3rOJThl1dTQDM0p4+yv+iIcZDLGcFNfIefgqz4fbORZESFau3o/wfemDTo5tOO+1@gmail.com X-Google-Smtp-Source: AGHT+IFRKse/0osYWyeDYMW8hN+b6PD1gfUZ1QNrZpC7AIzqp4yUuz9XtEk3hjWg3QFN2rqbfgL/ X-Received: by 2002:a2e:a585:0:b0:2f6:2b51:ee3e with SMTP id 38308e7fff4ca-2f62b51eec3mr14267861fa.41.1725149714106; Sat, 31 Aug 2024 17:15:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725149714; cv=none; d=google.com; s=arc-20240605; b=IPFmmX6f4y/CUFo6V+Kr4eKhQa/0AozEzJ2zckrNMstkdjoF08n/DTygHWOtTGGGcR TRaKcDH32Hnje2SkHzz2eyiiWQxjgCyycsFpPf1Lm8V4J9VwvK2RBzW6LwELIAtIGBzM Qt0yx873zsGE2YdwXAmRXx1aUKxJ6A6HCNGx4OQTC5T+MsdrukBzPKkPJIeyY7MxHXiB Ksrh+o8IZqkWtEe6PYBvAqnEhLcmxAU5V/EPM5BJ1B+ajEn4RWyxeuzRwEJi214HRSpm +Cae5cae253nJAgLYT66mHn7EAGVvF3O3+CtV9fRwxKtq/JerA/5MRKOGI0jw9BKjwf6 aNkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=gDWce2C/r93aoCDrdYMg8E5ys14biNvDB0vZ3AiRIZU=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=Dh4eJ49BBHovEOLLuCB1biAXocNNTU7YrwWQfNFvFAdjygi0PP807m++wcnow8FXnK 99EAwyItdSvnZd0jfyYoePlfobDtJgG4XuccmwU7dYNLRD4AKyJNXPq8aJRgLnbVnb0O wAeNcN8Xt4mU9DimNU2izqg1htuPAobduRhrYrLOml9WLhixwHnxIyyeTc+ATHezobyg otBYVpE0B7BNszP5GUI+/zL4F9RuVNbGoArX1wUIxXgBqHSZeQ5KomwiusiOtr0/V99b O/OhtZIf5QJHeiNRcz4dY+Q2OSkHq3Y1HShFnhd7h960FZGGliT1tcg8TptXISZkVKP7 GhDw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c2524b0f9esi322787a12.265.2024.08.31.17.15.13; Sat, 31 Aug 2024 17:15:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B5DE568DF6E; Sun, 1 Sep 2024 03:03:44 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 38DD768DF81 for ; Sun, 1 Sep 2024 03:03:42 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:05 +0200 Message-ID: <20240901000314.379276-7-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/10] vulkan_video: move imageview creation and DPB fields to common context X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: it7ab2XObCcp Shared between decoders and encoders. --- libavcodec/vulkan_av1.c | 3 +- libavcodec/vulkan_decode.c | 137 +++++++++---------------------------- libavcodec/vulkan_decode.h | 9 --- libavcodec/vulkan_h264.c | 3 +- libavcodec/vulkan_hevc.c | 3 +- libavcodec/vulkan_video.c | 80 +++++++++++++++++++++- libavcodec/vulkan_video.h | 18 ++++- 7 files changed, 133 insertions(+), 120 deletions(-) diff --git a/libavcodec/vulkan_av1.c b/libavcodec/vulkan_av1.c index a550215e32..a0befb9c4e 100644 --- a/libavcodec/vulkan_av1.c +++ b/libavcodec/vulkan_av1.c @@ -79,6 +79,7 @@ static int vk_av1_fill_pict(AVCodecContext *avctx, const AV1Frame **ref_src, const uint8_t *saved_order_hints) { FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; + FFVulkanDecodeShared *ctx = dec->shared_ctx; AV1VulkanDecodePicture *hp = pic->hwaccel_picture_private; FFVulkanDecodePicture *vkpic = &hp->vp; @@ -119,7 +120,7 @@ static int vk_av1_fill_pict(AVCodecContext *avctx, const AV1Frame **ref_src, .sType = VK_STRUCTURE_TYPE_VIDEO_PICTURE_RESOURCE_INFO_KHR, .codedOffset = (VkOffset2D){ 0, 0 }, .codedExtent = (VkExtent2D){ pic->f->width, pic->f->height }, - .baseArrayLayer = ((has_grain || dec->dedicated_dpb) && dec->layered_dpb) ? + .baseArrayLayer = ((has_grain || dec->dedicated_dpb) && ctx->common.layered_dpb) ? hp->frame_id : 0, .imageViewBinding = vkpic->img_view_ref, }; diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c index e73d4f93c2..35966cd60d 100644 --- a/libavcodec/vulkan_decode.c +++ b/libavcodec/vulkan_decode.c @@ -111,7 +111,6 @@ int ff_vk_update_thread_context(AVCodecContext *dst, const AVCodecContext *src) } dst_ctx->dedicated_dpb = src_ctx->dedicated_dpb; - dst_ctx->layered_dpb = src_ctx->layered_dpb; dst_ctx->external_fg = src_ctx->external_fg; dst_ctx->frame_id_alloc_mask = src_ctx->frame_id_alloc_mask; @@ -125,51 +124,6 @@ int ff_vk_params_invalidate(AVCodecContext *avctx, int t, const uint8_t *b, uint return 0; } -static int vk_decode_create_view(FFVulkanDecodeContext *dec, VkImageView *dst_view, - VkImageAspectFlags *aspect, AVVkFrame *src, - VkFormat vkf, int is_current) -{ - VkResult ret; - FFVulkanDecodeShared *ctx = dec->shared_ctx; - FFVulkanFunctions *vk = &ctx->s.vkfn; - VkImageAspectFlags aspect_mask = ff_vk_aspect_bits_from_vkfmt(vkf); - - VkSamplerYcbcrConversionInfo yuv_sampler_info = { - .sType = VK_STRUCTURE_TYPE_SAMPLER_YCBCR_CONVERSION_INFO, - .conversion = ctx->yuv_sampler, - }; - VkImageViewCreateInfo img_view_create_info = { - .sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO, - .pNext = &yuv_sampler_info, - .viewType = dec->layered_dpb && !is_current ? - VK_IMAGE_VIEW_TYPE_2D_ARRAY : VK_IMAGE_VIEW_TYPE_2D, - .format = vkf, - .image = src->img[0], - .components = (VkComponentMapping) { - .r = VK_COMPONENT_SWIZZLE_IDENTITY, - .g = VK_COMPONENT_SWIZZLE_IDENTITY, - .b = VK_COMPONENT_SWIZZLE_IDENTITY, - .a = VK_COMPONENT_SWIZZLE_IDENTITY, - }, - .subresourceRange = (VkImageSubresourceRange) { - .aspectMask = VK_IMAGE_ASPECT_COLOR_BIT, - .baseArrayLayer = 0, - .layerCount = dec->layered_dpb && !is_current ? - VK_REMAINING_ARRAY_LAYERS : 1, - .levelCount = 1, - }, - }; - - ret = vk->CreateImageView(ctx->s.hwctx->act_dev, &img_view_create_info, - ctx->s.hwctx->alloc, dst_view); - if (ret != VK_SUCCESS) - return AVERROR_EXTERNAL; - - *aspect = aspect_mask; - - return 0; -} - static AVFrame *vk_get_dpb_pool(FFVulkanDecodeShared *ctx) { int err; @@ -177,7 +131,7 @@ static AVFrame *vk_get_dpb_pool(FFVulkanDecodeShared *ctx) if (!avf) return NULL; - err = av_hwframe_get_buffer(ctx->dpb_hwfc_ref, avf, 0x0); + err = av_hwframe_get_buffer(ctx->common.dpb_hwfc_ref, avf, 0x0); if (err < 0) av_frame_free(&avf); @@ -207,21 +161,21 @@ int ff_vk_decode_prepare_frame(FFVulkanDecodeContext *dec, AVFrame *pic, vkpic->destroy_image_view = vk->DestroyImageView; vkpic->wait_semaphores = vk->WaitSemaphores; - if (dec->layered_dpb && alloc_dpb) { - vkpic->img_view_ref = ctx->layered_view; - vkpic->img_aspect_ref = ctx->layered_aspect; + if (ctx->common.layered_dpb && alloc_dpb) { + vkpic->img_view_ref = ctx->common.layered_view; + vkpic->img_aspect_ref = ctx->common.layered_aspect; } else if (alloc_dpb) { - AVHWFramesContext *dpb_frames = (AVHWFramesContext *)ctx->dpb_hwfc_ref->data; + AVHWFramesContext *dpb_frames = (AVHWFramesContext *)ctx->common.dpb_hwfc_ref->data; AVVulkanFramesContext *dpb_hwfc = dpb_frames->hwctx; vkpic->dpb_frame = vk_get_dpb_pool(ctx); if (!vkpic->dpb_frame) return AVERROR(ENOMEM); - err = vk_decode_create_view(dec, &vkpic->img_view_ref, - &vkpic->img_aspect_ref, - (AVVkFrame *)vkpic->dpb_frame->data[0], - dpb_hwfc->format[0], is_current); + err = ff_vk_create_view(&ctx->s, &ctx->common, + &vkpic->img_view_ref, &vkpic->img_aspect_ref, + (AVVkFrame *)vkpic->dpb_frame->data[0], + dpb_hwfc->format[0], !is_current); if (err < 0) return err; @@ -232,10 +186,10 @@ int ff_vk_decode_prepare_frame(FFVulkanDecodeContext *dec, AVFrame *pic, AVHWFramesContext *frames = (AVHWFramesContext *)pic->hw_frames_ctx->data; AVVulkanFramesContext *hwfc = frames->hwctx; - err = vk_decode_create_view(dec, &vkpic->img_view_out, - &vkpic->img_aspect, - (AVVkFrame *)pic->data[0], - hwfc->format[0], is_current); + err = ff_vk_create_view(&ctx->s, &ctx->common, + &vkpic->img_view_out, &vkpic->img_aspect, + (AVVkFrame *)pic->data[0], + hwfc->format[0], !is_current); if (err < 0) return err; @@ -366,7 +320,7 @@ int ff_vk_decode_frame(AVCodecContext *avctx, AVVkFrame *vkf = (AVVkFrame *)pic->buf[0]->data; /* Quirks */ - const int layered_dpb = dec->layered_dpb; + const int layered_dpb = ctx->common.layered_dpb; VkVideoSessionParametersKHR *par = (VkVideoSessionParametersKHR *)dec->session_params->data; VkVideoBeginCodingInfoKHR decode_start = { @@ -470,7 +424,7 @@ int ff_vk_decode_frame(AVCodecContext *avctx, .srcAccessMask = VK_ACCESS_2_NONE, .dstAccessMask = VK_ACCESS_2_VIDEO_DECODE_WRITE_BIT_KHR, .oldLayout = vkf->layout[0], - .newLayout = (dec->layered_dpb || vp->dpb_frame) ? + .newLayout = (layered_dpb || vp->dpb_frame) ? VK_IMAGE_LAYOUT_VIDEO_DECODE_DST_KHR : VK_IMAGE_LAYOUT_VIDEO_DECODE_DPB_KHR, /* Spec, 07252 utter madness */ .srcQueueFamilyIndex = vkf->queue_family[0], @@ -545,7 +499,7 @@ int ff_vk_decode_frame(AVCodecContext *avctx, } else if (vp->decode_info.referenceSlotCount || vp->img_view_out != vp->img_view_ref) { /* Single barrier for a single layered ref */ - err = ff_vk_exec_add_dep_frame(&ctx->s, exec, ctx->layered_frame, + err = ff_vk_exec_add_dep_frame(&ctx->s, exec, ctx->common.layered_frame, VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR, VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR); if (err < 0) @@ -616,13 +570,8 @@ static void free_common(FFRefStructOpaque unused, void *obj) FFVulkanContext *s = &ctx->s; FFVulkanFunctions *vk = &ctx->s.vkfn; - /* Destroy layered view */ - if (ctx->layered_view) - vk->DestroyImageView(s->hwctx->act_dev, ctx->layered_view, s->hwctx->alloc); - /* This also frees all references from this pool */ - av_frame_free(&ctx->layered_frame); - av_buffer_unref(&ctx->dpb_hwfc_ref); + av_frame_free(&ctx->common.layered_frame); /* Destroy parameters */ if (ctx->empty_session_params) @@ -634,10 +583,6 @@ static void free_common(FFRefStructOpaque unused, void *obj) ff_vk_video_common_uninit(s, &ctx->common); - if (ctx->yuv_sampler) - vk->DestroySamplerYcbcrConversion(s->hwctx->act_dev, ctx->yuv_sampler, - s->hwctx->alloc); - ff_vk_uninit(s); } @@ -924,8 +869,8 @@ static int vulkan_decode_get_profile(AVCodecContext *avctx, AVBufferRef *frames_ } dec->dedicated_dpb = !(dec_caps->flags & VK_VIDEO_DECODE_CAPABILITY_DPB_AND_OUTPUT_COINCIDE_BIT_KHR); - dec->layered_dpb = !dec->dedicated_dpb ? 0 : - !(caps->flags & VK_VIDEO_CAPABILITY_SEPARATE_REFERENCE_IMAGES_BIT_KHR); + ctx->common.layered_dpb = !dec->dedicated_dpb ? 0 : + !(caps->flags & VK_VIDEO_CAPABILITY_SEPARATE_REFERENCE_IMAGES_BIT_KHR); if (dec->dedicated_dpb) { fmt_info.imageUsage = VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR; @@ -1128,7 +1073,7 @@ int ff_vk_decode_uninit(AVCodecContext *avctx) int ff_vk_decode_init(AVCodecContext *avctx) { - int err, cxpos = 0, cypos = 0, nb_q = 0; + int err, nb_q = 0; VkResult ret; FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; FFVulkanDecodeShared *ctx; @@ -1159,12 +1104,6 @@ int ff_vk_decode_init(AVCodecContext *avctx) VkVideoSessionCreateInfoKHR session_create = { .sType = VK_STRUCTURE_TYPE_VIDEO_SESSION_CREATE_INFO_KHR, }; - VkSamplerYcbcrConversionCreateInfo yuv_sampler_info = { - .sType = VK_STRUCTURE_TYPE_SAMPLER_YCBCR_CONVERSION_CREATE_INFO, - .components = ff_comp_identity_map, - .ycbcrModel = VK_SAMPLER_YCBCR_MODEL_CONVERSION_RGB_IDENTITY, - .ycbcrRange = avctx->color_range == AVCOL_RANGE_MPEG, /* Ignored */ - }; err = ff_decode_get_hw_frames_ctx(avctx, AV_HWDEVICE_TYPE_VULKAN); if (err < 0) @@ -1230,30 +1169,18 @@ int ff_vk_decode_init(AVCodecContext *avctx) if (err < 0) goto fail; - /* Get sampler */ - av_chroma_location_enum_to_pos(&cxpos, &cypos, avctx->chroma_sample_location); - yuv_sampler_info.xChromaOffset = cxpos >> 7; - yuv_sampler_info.yChromaOffset = cypos >> 7; - yuv_sampler_info.format = s->hwfc->format[0]; - ret = vk->CreateSamplerYcbcrConversion(s->hwctx->act_dev, &yuv_sampler_info, - s->hwctx->alloc, &ctx->yuv_sampler); - if (ret != VK_SUCCESS) { - err = AVERROR_EXTERNAL; - goto fail; - } - /* If doing an out-of-place decoding, create a DPB pool */ if (dec->dedicated_dpb || avctx->codec_id == AV_CODEC_ID_AV1) { AVHWFramesContext *dpb_frames; AVVulkanFramesContext *dpb_hwfc; - ctx->dpb_hwfc_ref = av_hwframe_ctx_alloc(s->frames->device_ref); - if (!ctx->dpb_hwfc_ref) { + ctx->common.dpb_hwfc_ref = av_hwframe_ctx_alloc(s->frames->device_ref); + if (!ctx->common.dpb_hwfc_ref) { err = AVERROR(ENOMEM); goto fail; } - dpb_frames = (AVHWFramesContext *)ctx->dpb_hwfc_ref->data; + dpb_frames = (AVHWFramesContext *)ctx->common.dpb_hwfc_ref->data; dpb_frames->format = s->frames->format; dpb_frames->sw_format = s->frames->sw_format; dpb_frames->width = avctx->coded_width; @@ -1267,23 +1194,25 @@ int ff_vk_decode_init(AVCodecContext *avctx) dpb_hwfc->usage = VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR | VK_IMAGE_USAGE_SAMPLED_BIT; /* Shuts validator up. */ - if (dec->layered_dpb) + if (ctx->common.layered_dpb) dpb_hwfc->nb_layers = ctx->caps.maxDpbSlots; - err = av_hwframe_ctx_init(ctx->dpb_hwfc_ref); + err = av_hwframe_ctx_init(ctx->common.dpb_hwfc_ref); if (err < 0) goto fail; - if (dec->layered_dpb) { - ctx->layered_frame = vk_get_dpb_pool(ctx); - if (!ctx->layered_frame) { + if (ctx->common.layered_dpb) { + ctx->common.layered_frame = vk_get_dpb_pool(ctx); + if (!ctx->common.layered_frame) { err = AVERROR(ENOMEM); goto fail; } - err = vk_decode_create_view(dec, &ctx->layered_view, &ctx->layered_aspect, - (AVVkFrame *)ctx->layered_frame->data[0], - s->hwfc->format[0], 0); + err = ff_vk_create_view(&ctx->s, &ctx->common, + &ctx->common.layered_view, + &ctx->common.layered_aspect, + (AVVkFrame *)ctx->common.layered_frame->data[0], + s->hwfc->format[0], 1); if (err < 0) goto fail; } diff --git a/libavcodec/vulkan_decode.h b/libavcodec/vulkan_decode.h index c181277cdc..76e60836b5 100644 --- a/libavcodec/vulkan_decode.h +++ b/libavcodec/vulkan_decode.h @@ -53,15 +53,7 @@ typedef struct FFVulkanDecodeShared { VkVideoCapabilitiesKHR caps; VkVideoDecodeCapabilitiesKHR dec_caps; - AVBufferRef *dpb_hwfc_ref; /* Only used for dedicated_dpb */ - - AVFrame *layered_frame; /* Only used for layered_dpb */ - VkImageView layered_view; - VkImageAspectFlags layered_aspect; - VkVideoSessionParametersKHR empty_session_params; - - VkSamplerYcbcrConversion yuv_sampler; } FFVulkanDecodeShared; typedef struct FFVulkanDecodeContext { @@ -70,7 +62,6 @@ typedef struct FFVulkanDecodeContext { FFVkExecPool exec_pool; int dedicated_dpb; /* Oddity #1 - separate DPB images */ - int layered_dpb; /* Madness #1 - layered DPB images */ int external_fg; /* Oddity #2 - hardware can't apply film grain */ uint32_t frame_id_alloc_mask; /* For AV1 only */ diff --git a/libavcodec/vulkan_h264.c b/libavcodec/vulkan_h264.c index 0b296b3cc3..05ac884138 100644 --- a/libavcodec/vulkan_h264.c +++ b/libavcodec/vulkan_h264.c @@ -60,6 +60,7 @@ static int vk_h264_fill_pict(AVCodecContext *avctx, H264Picture **ref_src, int dpb_slot_index) { FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; + FFVulkanDecodeShared *ctx = dec->shared_ctx; H264VulkanDecodePicture *hp = pic->hwaccel_picture_private; FFVulkanDecodePicture *vkpic = &hp->vp; @@ -95,7 +96,7 @@ static int vk_h264_fill_pict(AVCodecContext *avctx, H264Picture **ref_src, .sType = VK_STRUCTURE_TYPE_VIDEO_PICTURE_RESOURCE_INFO_KHR, .codedOffset = (VkOffset2D){ 0, 0 }, .codedExtent = (VkExtent2D){ pic->f->width, pic->f->height }, - .baseArrayLayer = dec->layered_dpb ? dpb_slot_index : 0, + .baseArrayLayer = ctx->common.layered_dpb ? dpb_slot_index : 0, .imageViewBinding = vkpic->img_view_ref, }; diff --git a/libavcodec/vulkan_hevc.c b/libavcodec/vulkan_hevc.c index 5228e41ad5..f64c854a60 100644 --- a/libavcodec/vulkan_hevc.c +++ b/libavcodec/vulkan_hevc.c @@ -136,6 +136,7 @@ static int vk_hevc_fill_pict(AVCodecContext *avctx, HEVCFrame **ref_src, HEVCFrame *pic, int is_current, int pic_id) { FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; + FFVulkanDecodeShared *ctx = dec->shared_ctx; HEVCVulkanDecodePicture *hp = pic->hwaccel_picture_private; FFVulkanDecodePicture *vkpic = &hp->vp; @@ -161,7 +162,7 @@ static int vk_hevc_fill_pict(AVCodecContext *avctx, HEVCFrame **ref_src, .sType = VK_STRUCTURE_TYPE_VIDEO_PICTURE_RESOURCE_INFO_KHR, .codedOffset = (VkOffset2D){ 0, 0 }, .codedExtent = (VkExtent2D){ pic->f->width, pic->f->height }, - .baseArrayLayer = dec->layered_dpb ? pic_id : 0, + .baseArrayLayer = ctx->common.layered_dpb ? pic_id : 0, .imageViewBinding = vkpic->img_view_ref, }; diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c index a676f0332a..b7e42476bb 100644 --- a/libavcodec/vulkan_video.c +++ b/libavcodec/vulkan_video.c @@ -16,7 +16,6 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -#include "defs.h" #include "libavutil/mem.h" #include "vulkan_video.h" @@ -240,6 +239,50 @@ int ff_vk_video_qf_init(FFVulkanContext *s, FFVkQueueFamilyCtx *qf, return AVERROR(ENOTSUP); } +int ff_vk_create_view(FFVulkanContext *s, FFVkVideoCommon *common, + VkImageView *view, VkImageAspectFlags *aspect, + AVVkFrame *src, VkFormat vkf, int is_dpb) +{ + VkResult ret; + FFVulkanFunctions *vk = &s->vkfn; + VkImageAspectFlags aspect_mask = ff_vk_aspect_bits_from_vkfmt(vkf); + + VkSamplerYcbcrConversionInfo yuv_sampler_info = { + .sType = VK_STRUCTURE_TYPE_SAMPLER_YCBCR_CONVERSION_INFO, + .conversion = common->yuv_sampler, + }; + VkImageViewCreateInfo img_view_create_info = { + .sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO, + .pNext = &yuv_sampler_info, + .viewType = common->layered_dpb && is_dpb ? + VK_IMAGE_VIEW_TYPE_2D_ARRAY : VK_IMAGE_VIEW_TYPE_2D, + .format = vkf, + .image = src->img[0], + .components = (VkComponentMapping) { + .r = VK_COMPONENT_SWIZZLE_IDENTITY, + .g = VK_COMPONENT_SWIZZLE_IDENTITY, + .b = VK_COMPONENT_SWIZZLE_IDENTITY, + .a = VK_COMPONENT_SWIZZLE_IDENTITY, + }, + .subresourceRange = (VkImageSubresourceRange) { + .aspectMask = VK_IMAGE_ASPECT_COLOR_BIT, + .baseArrayLayer = 0, + .layerCount = common->layered_dpb && is_dpb ? + VK_REMAINING_ARRAY_LAYERS : 1, + .levelCount = 1, + }, + }; + + ret = vk->CreateImageView(s->hwctx->act_dev, &img_view_create_info, + s->hwctx->alloc, view); + if (ret != VK_SUCCESS) + return AVERROR_EXTERNAL; + + *aspect = aspect_mask; + + return 0; +} + av_cold void ff_vk_video_common_uninit(FFVulkanContext *s, FFVkVideoCommon *common) { @@ -256,9 +299,21 @@ av_cold void ff_vk_video_common_uninit(FFVulkanContext *s, vk->FreeMemory(s->hwctx->act_dev, common->mem[i], s->hwctx->alloc); av_freep(&common->mem); + + if (common->layered_view) + vk->DestroyImageView(s->hwctx->act_dev, common->layered_view, + s->hwctx->alloc); + + av_frame_free(&common->layered_frame); + + av_buffer_unref(&common->dpb_hwfc_ref); + + if (common->yuv_sampler) + vk->DestroySamplerYcbcrConversion(s->hwctx->act_dev, common->yuv_sampler, + s->hwctx->alloc); } -av_cold int ff_vk_video_common_init(void *log, FFVulkanContext *s, +av_cold int ff_vk_video_common_init(AVCodecContext *avctx, FFVulkanContext *s, FFVkVideoCommon *common, VkVideoSessionCreateInfoKHR *session_create) { @@ -268,6 +323,25 @@ av_cold int ff_vk_video_common_init(void *log, FFVulkanContext *s, VkVideoSessionMemoryRequirementsKHR *mem = NULL; VkBindVideoSessionMemoryInfoKHR *bind_mem = NULL; + int cxpos, cypos; + VkSamplerYcbcrConversionCreateInfo yuv_sampler_info = { + .sType = VK_STRUCTURE_TYPE_SAMPLER_YCBCR_CONVERSION_CREATE_INFO, + .components = ff_comp_identity_map, + .ycbcrModel = VK_SAMPLER_YCBCR_MODEL_CONVERSION_RGB_IDENTITY, + .ycbcrRange = avctx->color_range == AVCOL_RANGE_MPEG, /* Ignored */ + .format = session_create->pictureFormat, + }; + + /* Create identity YUV sampler + * (VkImageViews of YUV image formats require it, even if it does nothing) */ + av_chroma_location_enum_to_pos(&cxpos, &cypos, avctx->chroma_sample_location); + yuv_sampler_info.xChromaOffset = cxpos >> 7; + yuv_sampler_info.yChromaOffset = cypos >> 7; + ret = vk->CreateSamplerYcbcrConversion(s->hwctx->act_dev, &yuv_sampler_info, + s->hwctx->alloc, &common->yuv_sampler); + if (ret != VK_SUCCESS) + return AVERROR_EXTERNAL; + /* Create session */ ret = vk->CreateVideoSessionKHR(s->hwctx->act_dev, session_create, s->hwctx->alloc, &common->session); @@ -333,7 +407,7 @@ av_cold int ff_vk_video_common_init(void *log, FFVulkanContext *s, .memorySize = mem[i].memoryRequirements.size, }; - av_log(log, AV_LOG_VERBOSE, "Allocating %"PRIu64" bytes in bind index %i for video session\n", + av_log(avctx, AV_LOG_VERBOSE, "Allocating %"PRIu64" bytes in bind index %i for video session\n", bind_mem[i].memorySize, bind_mem[i].memoryBindIndex); } diff --git a/libavcodec/vulkan_video.h b/libavcodec/vulkan_video.h index a5f69c576c..01659f6501 100644 --- a/libavcodec/vulkan_video.h +++ b/libavcodec/vulkan_video.h @@ -19,6 +19,7 @@ #ifndef AVCODEC_VULKAN_VIDEO_H #define AVCODEC_VULKAN_VIDEO_H +#include "avcodec.h" #include "vulkan.h" #include @@ -32,6 +33,14 @@ typedef struct FFVkVideoSession { VkVideoSessionKHR session; VkDeviceMemory *mem; uint32_t nb_mem; + + VkSamplerYcbcrConversion yuv_sampler; + + AVBufferRef *dpb_hwfc_ref; + int layered_dpb; + AVFrame *layered_frame; + VkImageView layered_view; + VkImageAspectFlags layered_aspect; } FFVkVideoCommon; /** @@ -74,10 +83,17 @@ StdVideoH264LevelIdc ff_vk_h264_level_to_vk(int level_idc); StdVideoH264ProfileIdc ff_vk_h264_profile_to_vk(int profile); int ff_vk_h264_profile_to_av(StdVideoH264ProfileIdc profile); +/** + * Creates image views for video frames. + */ +int ff_vk_create_view(FFVulkanContext *s, FFVkVideoCommon *common, + VkImageView *view, VkImageAspectFlags *aspect, + AVVkFrame *src, VkFormat vkf, int is_dpb); + /** * Initialize video session, allocating and binding necessary memory. */ -int ff_vk_video_common_init(void *log, FFVulkanContext *s, +int ff_vk_video_common_init(AVCodecContext *avctx, FFVulkanContext *s, FFVkVideoCommon *common, VkVideoSessionCreateInfoKHR *session_create); From patchwork Sun Sep 1 00:03:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51279 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1140482vqb; Sat, 31 Aug 2024 17:30:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVy5ij+g1afk0F8qIBvGazt2/sQRxl7Be7AMg3nRFPIB/wRrfMn7IdUAr6Cug1RA8wgKHJG5LpQ1Mo3+rSFVsVD@gmail.com X-Google-Smtp-Source: AGHT+IFKUot+05xFCo8884xAbFqDoGcimuKGdTDLTCWKluQyRiPyORAouDXRMn+C/fwdGOisp/2H X-Received: by 2002:a05:651c:2226:b0:2f5:2e2:eafb with SMTP id 38308e7fff4ca-2f6105b5e21mr78327081fa.7.1725150613964; Sat, 31 Aug 2024 17:30:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725150613; cv=none; d=google.com; s=arc-20240605; b=hSGFkB+cD4SEzc3EZC9q1UzXwoaMNsZa3TcIrNkfxnPgs+7DFUpoXkIqhIhjRdc6sD tlwYrHZtARVZBcA1IpaE0paI1M7rh0gWwu6FTv9sVjYVULeIIgAKmYHTvAip6gQ3xojM OtbsDR5cFwVfqhF1APsbc6QulojcdCKGlb0a2kCFvdFvzUYWMihUwzXmRyS53eCiV5Fa BxrvwJOhVvoxsHRHGOC0WqbJ2eeNTwEyyCHT6FUoIbOXj7ZoP2RNZT62IwrFzd9Jz1N4 WkODuLWDRUGL8oDldmyW9t1NfvbgMQB6LsONmjRkNteCbs4ze/hJ0czAZzZHSS+ZCvrI Ridg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=XK1mPYAK1H+jLw4URI8uub7iPTXa1Tx9aV4XvNur9ss=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=ebdeNiH5dCZFfB4PDhoDdpKb8rv/7TZc8gwtvLVVbtJrkLyBmT3uw43hoJZZVSL+jD oL1XTTwzcNZBNvSw5LZTrAM0tup3p2DVZgV6sm/hyETEJRK9RFypPq3UMp2M9DwQ811r LK3m4m78zF01RIuUVNb6FSBQpWv2kgDBoTF9uDdmBcccAC7P/59kG5kWezRebXo4orYj 8qV7+YpD8Q4MnAxlielXcR4cghAbDlUsvx3D6TCCcTGKyuEojN4SVwxOHpT4KtUEDhMT xi0eV8MkGBSXCi6C2n3hEyWBN1Q5OA1/IAGf3m/Fo/lyzUjRl1tnU6VN3qH5M7wHMBUO e2wA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f615151ce2si20903901fa.266.2024.08.31.17.30.13; Sat, 31 Aug 2024 17:30:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D230C68DF9D; Sun, 1 Sep 2024 03:03:45 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B328168DF6B for ; Sun, 1 Sep 2024 03:03:43 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:06 +0200 Message-ID: <20240901000314.379276-8-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/10] hwcontext_vulkan: add PREP_MODE_GENERAL for non-transfer_dst images X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: f66tuRa5cgqZ Vulkan filters don't need images which can be transferred into. --- libavutil/hwcontext_vulkan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index e4f155db6e..383e75397e 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2187,6 +2187,7 @@ static int alloc_bind_mem(AVHWFramesContext *hwfc, AVVkFrame *f, } enum PrepMode { + PREP_MODE_GENERAL, PREP_MODE_WRITE, PREP_MODE_EXTERNAL_EXPORT, PREP_MODE_EXTERNAL_IMPORT, @@ -2232,6 +2233,10 @@ static int prepare_frame(AVHWFramesContext *hwfc, FFVkExecPool *ectx, return err; switch (pmode) { + case PREP_MODE_GENERAL: + new_layout = VK_IMAGE_LAYOUT_GENERAL; + new_access = VK_ACCESS_TRANSFER_WRITE_BIT; + break; case PREP_MODE_WRITE: new_layout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; new_access = VK_ACCESS_TRANSFER_WRITE_BIT; @@ -2520,8 +2525,10 @@ static AVBufferRef *vulkan_pool_alloc(void *opaque, size_t size) err = prepare_frame(hwfc, &fp->compute_exec, f, PREP_MODE_DECODING_DST); else if (hwctx->usage & VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR) err = prepare_frame(hwfc, &fp->compute_exec, f, PREP_MODE_ENCODING_DPB); - else + else if (hwctx->usage & VK_IMAGE_USAGE_TRANSFER_DST_BIT) err = prepare_frame(hwfc, &fp->compute_exec, f, PREP_MODE_WRITE); + else + err = prepare_frame(hwfc, &fp->compute_exec, f, PREP_MODE_GENERAL); if (err) goto fail; From patchwork Sun Sep 1 00:03:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lynne X-Patchwork-Id: 51281 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1146034vqb; Sat, 31 Aug 2024 18:00:11 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVru5U/Bg+rd6TbFE4MfgpGSMdneX2z9fbocRfsH7Iqm4JTBxBPJhU2s2rGfcAEutzosfn5sIa6+QZlat1Jua95@gmail.com X-Google-Smtp-Source: AGHT+IEP3pUcXOuFeIURhm93DoePj8Gsz2neppakgaLoweVUmdJy/ub9keLoPRu6ybu7+5Bxzhvp X-Received: by 2002:a17:907:72d3:b0:a88:a48d:2b9a with SMTP id a640c23a62f3a-a897fa74459mr735622566b.52.1725152411570; Sat, 31 Aug 2024 18:00:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725152411; cv=none; d=google.com; s=arc-20240605; b=VoNiqEYv2c/wc2w0XiNmeusynjfNrI/2+BYJY7z0sWdzwPdl22a+4yi76O8CIyUtYT q9gd0LTGCoi0snGQ3ueyNWL4KKRItCnd540wkwcGecShMDt+T9hR5OHQgKap4IIyURGT YJkKfo+LHpVEaLgEB0vylZsRyIOBsg4mw5xPVEvtphRTbjbPvDJ8eHRAs9JhFxv1h1Xh W0F4kbU3A8qulR9Wc6BWYJnxvS4MZQjavor+9Uv5NfCZIs+uRaYvhpsGfAxH/yiSXfrt 9LHWudAxup4HL8B+aazSs+/nH7XeG6rum0VrkyGYjpNw6bePydAZgBJcQbawdTnFHn7o SlZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:delivered-to; bh=l1LkrFStaGdjY1IpXEiJAJIckwmnM3sbNj0cl1cNR4s=; fh=nenT92/WZoU6unXd3J6UhGUdod4piddKfVtctNBOh6k=; b=XsyXjHRoLU9EFHukRCh9tNm0TYsDF5/CRG+btKEFHtiDCrmkG5a/4WU9+EoLPFPXvh vydKZsHpRIlR2vZSw8RXH6NokdnLdDUcQivISZchbNSXFBqUbAg5/3CK9B849a0NjIuu YUh2odVgddw9EK9W21lq24urKfO+L05UmnTt7kVV6RGabQNqeLa2SHQAXs6kwjYZHKkO /+zwbQPXQ5UmcIOl5amHCe9TZzETelAPBn9N/d7yCFtYdNCv/JwRANuTB/r0ryJVv7i2 wZjh206/f7Ga5YWJNp14ayy98MIJYYr8Tw8RXvSFqT0SLmveeLdPH6i9qESmuURkD7u3 6mKA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a898916b595si523285166b.614.2024.08.31.18.00.11; Sat, 31 Aug 2024 18:00:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 15A3D68DFA7; Sun, 1 Sep 2024 03:03:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from vidala.lynne.ee (vidala.pars.ee [116.203.72.101]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1B6D568DF8B for ; Sun, 1 Sep 2024 03:03:45 +0300 (EEST) To: ffmpeg-devel@ffmpeg.org Date: Sun, 1 Sep 2024 02:03:07 +0200 Message-ID: <20240901000314.379276-9-dev@lynne.ee> X-Mailer: git-send-email 2.45.2.753.g447d99e1c3b In-Reply-To: <20240901000314.379276-1-dev@lynne.ee> References: <20240901000314.379276-1-dev@lynne.ee> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/10] vulkan_filter: require storage images properly, set usage flags explicitly X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Lynne via ffmpeg-devel From: Lynne Reply-To: FFmpeg development discussions and patches Cc: Lynne Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dU1w8PHWfDVL This caused images to be created without a storage usage, which broke at least lavapipe. --- libavfilter/vulkan_filter.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/libavfilter/vulkan_filter.c b/libavfilter/vulkan_filter.c index 64e9b8768a..c31d42b91a 100644 --- a/libavfilter/vulkan_filter.c +++ b/libavfilter/vulkan_filter.c @@ -68,7 +68,8 @@ int ff_vk_filter_init_context(AVFilterContext *avctx, FFVulkanContext *s, vk = &s->vkfn; /* Usage mismatch */ - usage_req = VK_IMAGE_USAGE_SAMPLED_BIT; + usage_req = VK_IMAGE_USAGE_SAMPLED_BIT | + VK_IMAGE_USAGE_STORAGE_BIT; /* If format supports hardware encoding, make sure * the context includes it. */ @@ -106,9 +107,11 @@ int ff_vk_filter_init_context(AVFilterContext *avctx, FFVulkanContext *s, /* Check if it's usable */ if (no_storage) { skip: + av_log(avctx, AV_LOG_VERBOSE, "Cannot reuse context, creating a new one\n"); device_ref = frames_ctx->device_ref; frames_ref = NULL; } else { + av_log(avctx, AV_LOG_VERBOSE, "Reusing existing frames context\n"); frames_ref = av_buffer_ref(frames_ref); if (!frames_ref) return AVERROR(ENOMEM); @@ -130,6 +133,12 @@ skip: frames_ctx->width = width; frames_ctx->height = height; + vk_frames = frames_ctx->hwctx; + vk_frames->tiling = VK_IMAGE_TILING_OPTIMAL; + vk_frames->usage = VK_IMAGE_USAGE_SAMPLED_BIT | + VK_IMAGE_USAGE_STORAGE_BIT | + VK_IMAGE_USAGE_TRANSFER_SRC_BIT; + err = av_hwframe_ctx_init(frames_ref); if (err < 0) { av_buffer_unref(&frames_ref);