From patchwork Wed Sep 18 07:10:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 51644 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:d32e:0:b0:48e:c0f8:d0de with SMTP id cf14csp732234vqb; Wed, 18 Sep 2024 00:19:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWH6MDgpsoB+nCNyEh/fqKAxQxpHrfDHms9SmUmoLAHUxVbYoiOahkg2gYFxIOOCo2eMZxauaWaE+yKwpKPBlHP@gmail.com X-Google-Smtp-Source: AGHT+IEtwGbSyHAOXsqqzFV41T8RjBtmti5x3u2cSE3MmpZ+/GCaH0hVCbo0wpHxykdEubGo+VAV X-Received: by 2002:a17:907:e28c:b0:a8a:837c:ebd4 with SMTP id a640c23a62f3a-a9029491985mr2135482066b.27.1726643959060; Wed, 18 Sep 2024 00:19:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726643959; cv=none; d=google.com; s=arc-20240605; b=iNjKRzNh7FBKznoThtyi7mXB7xv607ICKy93OC/t6xWPLB7prTssb81r9tTmeHcEaZ +eOhQEpJk4lLLo92vYnvUpPc5NAsFd98CkBRpVzQX/b8KpvFjkTYDvT41cIfaQIQi3+B A3SIWbC5oA6gzhZItt05xSms6quT+Vuoi0C4l4kszL+RCRyqo/el4E03RBeA2v6USjr6 55lk+Q3hEuyKHUJWWZXYMtiPmyOPvTLb739vl0gSgskRKsRkEIzL1TI3exo0l1D3fZGF r4zw+uHrCzTeKiBbSlsPReBKF7zGeoo2xAAULgJYcL2iJHcqidTLXMQ6nCnnoq95rKeq skkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=CFBeE6/l6Xx9qQGkKcMj1d59LYTxBuaBs9UTdL7gYFk=; fh=i4ESP4ZRFDcfYfwXKWpOXjc2YhmIGuOsCfZUnwNO0gc=; b=Fjdt9VpaggXRBe59qai8A+RsEyr4EhoNTYkXs2fa4wDT1dEYavDQzL0nor88x1PRnF YEq4z57Bk4fKPsQ6Mz3Ht7XPkf01j1enqZT9CNfAsHZGzln1jBEJF+S6zP9dUuOBlb9u Z8PO4bML9CwLRzxJBPFRjS0V5gMvbaQTQo/j/tePx8O7oCAW0jK1tlrGxR4EjE1YQ9Nu VdkxknwDhjYgja9qn8BxVaV7nf45QRqdxA4RuOM1+/3oSvPavm8lmkzZOnfP03KbCXtu QB2JnCEoA6mBWyu6F+GrmK3uRLilZ/GkEDq9b0rNw2+AYMqfzZE8zPJXJN9QTpvEDeI/ 5cyg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=LOYXJrNF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a90612f00adsi634950066b.598.2024.09.18.00.19.18; Wed, 18 Sep 2024 00:19:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=LOYXJrNF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 45CF268DD1A; Wed, 18 Sep 2024 10:08:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7E79C68DCD2 for ; Wed, 18 Sep 2024 10:08:20 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726643301; x=1758179301; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+1hiclMckRpFnQetlyQv9FxR5WDfTbp5avUV75QPWFo=; b=LOYXJrNF78MPMPnYGfjddPG555p9K5vhOTwLCumHQzcS8HAKNIwfyAmX FCCtpEreqMIG4xLV1GqPzvGWGxxGXr370uVqzqGioMfH02zbP51M+YLD4 Na83i4JWOAzUco5qqyWHXn7yG5RDAFme7eiKczmM1IXEoaJFfp2QQRaKU mYZcz3uvqvKO7HQ/FIiiz7ItIR6ndzXxij8/yezXcB5DIqZ7XrevH8Egw CEJcdunmtFejK2A0Ld4t0pyBjw4KaQTcNFp/HctGzdxuP+vS9vATv4tiR vw+FpUsBSLortsjzweEC0pNS9uEo68MqghBhqHelOlg1ZL/7kmQAFd1Kn g==; X-CSE-ConnectionGUID: AxUMsjiSRiCtWd9lNLUlXg== X-CSE-MsgGUID: 6o6NMtubSf2Md6RLUp89IA== X-IronPort-AV: E=McAfee;i="6700,10204,11198"; a="25695729" X-IronPort-AV: E=Sophos;i="6.10,238,1719903600"; d="scan'208";a="25695729" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 00:08:13 -0700 X-CSE-ConnectionGUID: fauw/pbXRZSpHdGZNVPD7Q== X-CSE-MsgGUID: i0kv1zjPTKigsOlJ2DSL3g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,238,1719903600"; d="scan'208";a="69452282" Received: from feiwan1-desk3.sh.intel.com ([10.238.208.39]) by orviesa009.jf.intel.com with ESMTP; 18 Sep 2024 00:08:12 -0700 From: fei.w.wang-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 18 Sep 2024 15:10:30 +0800 Message-Id: <20240918071031.1377336-7-fei.w.wang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240918071031.1377336-1-fei.w.wang@intel.com> References: <20240918071031.1377336-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 7/8] lavc/vvc_dec: Add hardware decode API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: fei.w.wang@intel.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VjBDACl4nPRZ From: Fei Wang Signed-off-by: Fei Wang --- libavcodec/vvc/dec.c | 73 +++++++++++++++++++++++++++++++++++++------ libavcodec/vvc/dec.h | 4 +++ libavcodec/vvc/refs.c | 6 ++++ 3 files changed, 73 insertions(+), 10 deletions(-) diff --git a/libavcodec/vvc/dec.c b/libavcodec/vvc/dec.c index edf2607f50..c9f0e44889 100644 --- a/libavcodec/vvc/dec.c +++ b/libavcodec/vvc/dec.c @@ -22,6 +22,8 @@ */ #include "libavcodec/codec_internal.h" #include "libavcodec/decode.h" +#include "libavcodec/hwaccel_internal.h" +#include "libavcodec/hwconfig.h" #include "libavcodec/profiles.h" #include "libavcodec/refstruct.h" #include "libavutil/cpu.h" @@ -610,6 +612,8 @@ static int ref_frame(VVCFrame *dst, const VVCFrame *src) ff_refstruct_replace(&dst->rpl_tab, src->rpl_tab); ff_refstruct_replace(&dst->rpl, src->rpl); + ff_refstruct_replace(&dst->hwaccel_picture_private, + src->hwaccel_picture_private); dst->nb_rpl_elems = src->nb_rpl_elems; dst->poc = src->poc; @@ -770,17 +774,41 @@ static int slice_start(SliceContext *sc, VVCContext *s, VVCFrameContext *fc, return 0; } +static enum AVPixelFormat get_format(AVCodecContext *avctx, const VVCSPS *sps) +{ +#define HWACCEL_MAX 0 + + enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmt = pix_fmts; + + switch (sps->pix_fmt) { + case AV_PIX_FMT_YUV420P: + break; + case AV_PIX_FMT_YUV420P10: + break; + } + + *fmt++ = sps->pix_fmt; + *fmt = AV_PIX_FMT_NONE; + + return ff_get_format(avctx, pix_fmts); +} + static void export_frame_params(VVCContext *s, const VVCFrameContext *fc) { AVCodecContext *c = s->avctx; const VVCSPS *sps = fc->ps.sps; const VVCPPS *pps = fc->ps.pps; - c->pix_fmt = sps->pix_fmt; - c->coded_width = pps->width; - c->coded_height = pps->height; - c->width = pps->width - ((pps->r->pps_conf_win_left_offset + pps->r->pps_conf_win_right_offset) << sps->hshift[CHROMA]); - c->height = pps->height - ((pps->r->pps_conf_win_top_offset + pps->r->pps_conf_win_bottom_offset) << sps->vshift[CHROMA]); + // Reset HW config if pix_fmt/w/h change. + if (s->pix_fmt != sps->pix_fmt || c->coded_width != pps->width || c->coded_height != pps->height) { + c->coded_width = pps->width; + c->coded_height = pps->height; + c->pix_fmt = get_format(c, sps); + s->pix_fmt = sps->pix_fmt; + } + + c->width = pps->width - ((pps->r->pps_conf_win_left_offset + pps->r->pps_conf_win_right_offset) << sps->hshift[CHROMA]); + c->height = pps->height - ((pps->r->pps_conf_win_top_offset + pps->r->pps_conf_win_bottom_offset) << sps->vshift[CHROMA]); c->has_b_frames = sps->r->sps_dpb_params.dpb_max_num_reorder_pics[sps->r->sps_max_sublayers_minus1]; } @@ -824,6 +852,20 @@ static int decode_slice(VVCContext *s, VVCFrameContext *fc, const H2645NAL *nal, ret = slice_init_entry_points(sc, fc, nal, unit); if (ret < 0) return ret; + + if (s->avctx->hwaccel) { + if (is_first_slice) { + ret = FF_HW_CALL(s->avctx, start_frame, NULL, 0); + if (ret < 0) + return ret; + } + + ret = FF_HW_CALL(s->avctx, decode_slice, + nal->raw_data, nal->raw_size); + if (ret < 0) + return ret; + } + fc->nb_slices++; return 0; @@ -939,17 +981,26 @@ static int wait_delayed_frame(VVCContext *s, AVFrame *output, int *got_output) static int submit_frame(VVCContext *s, VVCFrameContext *fc, AVFrame *output, int *got_output) { - int ret = ff_vvc_frame_submit(s, fc); + int ret; - if (ret < 0) { - ff_vvc_report_frame_finished(fc->ref); - return ret; + if (s->avctx->hwaccel) { + if (ret = FF_HW_SIMPLE_CALL(s->avctx, end_frame) < 0) { + av_log(s->avctx, AV_LOG_ERROR, + "Hardware accelerator failed to decode picture\n"); + ff_vvc_unref_frame(fc, fc->ref, ~0); + return ret; + } + } else { + if (ret = ff_vvc_frame_submit(s, fc) < 0) { + ff_vvc_report_frame_finished(fc->ref); + return ret; + } } s->nb_frames++; s->nb_delayed++; - if (s->nb_delayed >= s->nb_fcs) { + if (s->nb_delayed >= s->nb_fcs || s->avctx->hwaccel) { if ((ret = wait_delayed_frame(s, output, got_output)) < 0) return ret; } @@ -1095,6 +1146,8 @@ static av_cold int vvc_decode_init(AVCodecContext *avctx) GDR_SET_RECOVERED(s); ff_thread_once(&init_static_once, init_default_scale_m); + s->pix_fmt = AV_PIX_FMT_NONE; + return 0; } diff --git a/libavcodec/vvc/dec.h b/libavcodec/vvc/dec.h index d27cf52ca2..776b38b20f 100644 --- a/libavcodec/vvc/dec.h +++ b/libavcodec/vvc/dec.h @@ -101,6 +101,8 @@ typedef struct VVCFrame { * A combination of VVC_FRAME_FLAG_* */ uint8_t flags; + + void *hwaccel_picture_private; ///< hardware accelerator private data } VVCFrame; typedef struct SliceContext { @@ -243,6 +245,8 @@ typedef struct VVCContext { uint64_t nb_frames; ///< processed frames int nb_delayed; ///< delayed frames + + enum AVPixelFormat pix_fmt; ///< pix format of current frame } VVCContext ; #endif /* AVCODEC_VVC_DEC_H */ diff --git a/libavcodec/vvc/refs.c b/libavcodec/vvc/refs.c index 3e5573df29..a41a83631a 100644 --- a/libavcodec/vvc/refs.c +++ b/libavcodec/vvc/refs.c @@ -26,6 +26,7 @@ #include "libavutil/thread.h" #include "libavcodec/refstruct.h" #include "libavcodec/thread.h" +#include "libavcodec/decode.h" #include "refs.h" @@ -59,6 +60,7 @@ void ff_vvc_unref_frame(VVCFrameContext *fc, VVCFrame *frame, int flags) ff_refstruct_unref(&frame->rpl_tab); frame->collocated_ref = NULL; + ff_refstruct_unref(&frame->hwaccel_picture_private); } } @@ -153,6 +155,10 @@ static VVCFrame *alloc_frame(VVCContext *s, VVCFrameContext *fc) if (!frame->progress) goto fail; + ret = ff_hwaccel_frame_priv_alloc(s->avctx, &frame->hwaccel_picture_private); + if (ret < 0) + goto fail; + return frame; fail: ff_vvc_unref_frame(fc, frame, ~0);