From patchwork Thu Mar 28 01:26:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Fei W" X-Patchwork-Id: 47562 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:9f96:b0:1a3:b6bb:3029 with SMTP id mm22csp1013707pzb; Wed, 27 Mar 2024 18:27:13 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU+60/h9bQ/JcORxJRp3xq9Kj6qxjhw2FAitq3iHa20dxDv2bBMO1Tl/JoInmdo7oxxkFtSOiM4IGjNfGpVW+jhZ5pqEFZe70TFEQ== X-Google-Smtp-Source: AGHT+IG1NKaguPpW6fmxmDKli+AmigGQDHv2Ucx1Fmd4iEPoYMf6iXG6uyGeNxPaEorx2Ot86Th/ X-Received: by 2002:a5d:5143:0:b0:33b:364c:e83d with SMTP id u3-20020a5d5143000000b0033b364ce83dmr940905wrt.1.1711589232995; Wed, 27 Mar 2024 18:27:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1711589232; cv=none; d=google.com; s=arc-20160816; b=lXShalYCeOL3re+j+UWK0LgVKIuFp7eBuIDO45eaumwLYXj7vEMjj+SWypfZtprUny AeTKEjBt9SR2RDUb1rELa4iCzHIEh3OeEgxXnAl0kM5qJ52s6jNYTMCjRAWJkVTkVccz 2t03n6o1nvXE6tt3Nz13n0btrB+2eAwQpg+AUUlNP1yDE1z9XQCmqndwExL0ICHLT1Nx TwyZ0ZETAUl8ph5OnQs5ME3NOkNYgUkhNFXhmKJ0/52GYXosWHMp1mOiS6yECSvgiuT0 efXybh6Hmy1d25qsUGQusoWiATFS8mLEDQtp7Q7iO9aDv+41ZTl17kxGdLJzbl0eniwz nMZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=/bdNFESLdrSCm1kJlq0L5YoS9egFKoYfoO8JFd9GLfA=; fh=i4ESP4ZRFDcfYfwXKWpOXjc2YhmIGuOsCfZUnwNO0gc=; b=y8rnYUA/nmXVq5I9YsrnxNAmELQ7tr6J3SXI97jWZWLX3opBy/oEvO2e3QpebslYBa iFyDW7kAVwhWNvtgSHr1QHa4o5lxRxX/djUYmKU+u6zVS+8Yav1nrq7wH1W36VdTQWc8 EF4TeU0XSAd+RBoFl7syjvxux2U9gp3vnT9NCLbwP6i/5JFVC6jIxeDbBGxWTUFoRp3A cZIxN0yJymxD/FCAsefKXPNjrfNEBKIziGm6V2ry/NJYELkOUGcJ/DrEQguMMuze7i24 swRv9oJR/URuOg/nwzQ1m1v5ngzeQlRLSCFyAuIJjkPwJ5iekMiUmyo5YRMU9wJgwta9 CJ3A==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=djDfVHDF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id um27-20020a170906cf9b00b00a46fbfbbeb5si151395ejb.779.2024.03.27.18.27.12; Wed, 27 Mar 2024 18:27:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=djDfVHDF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 062E268D6EE; Thu, 28 Mar 2024 03:26:33 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B542968D6B9 for ; Thu, 28 Mar 2024 03:26:24 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1711589190; x=1743125190; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x1riYjFhT1A6wTr/zSEBVMs+ObK19XbhJjOjgeTBYmc=; b=djDfVHDFvj1gyU+9ialxQBDxJtdrPL3WDpoFs073yjGw6Ko7hzPzcCtd ws+W+9jcrKV9Chd6VnbpyNPm844gqkOPnSHUX7M0UOMr9VY0TXvUZxI5E 2gdbrkIaP6gYja+a/nS+5VHHjFMTCgo3seim4hUmP2ukJE3bS+q0AQyGM QU7ULMwD6GgQN1T2eWxX53xngBMsgGYoHK9wxNQ4K9EIuCzY8FMYeZ2fb WedABQw1ib/R8iIfjG5uOzZAftTkXIbCO8blwobRplO5+13eCVGoAsL0F X8u+U4+1GgDwBsRB/MMN8sCjuQgEzloHfoHKjiJ1xyuGvL5aX6zSOIpNg A==; X-CSE-ConnectionGUID: UunS3TsHRp2JyE3vQAZ9Sg== X-CSE-MsgGUID: vF/j6s8VR9q4nQrMAju6pQ== X-IronPort-AV: E=McAfee;i="6600,9927,11026"; a="7324920" X-IronPort-AV: E=Sophos;i="6.07,160,1708416000"; d="scan'208";a="7324920" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Mar 2024 18:26:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,160,1708416000"; d="scan'208";a="20996882" Received: from t-dg2.sh.intel.com ([10.239.159.57]) by fmviesa003.fm.intel.com with ESMTP; 27 Mar 2024 18:26:17 -0700 From: fei.w.wang-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Thu, 28 Mar 2024 09:26:30 +0800 Message-Id: <20240328012631.777476-6-fei.w.wang@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240328012631.777476-1-fei.w.wang@intel.com> References: <20240328012631.777476-1-fei.w.wang@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v1 6/7] lavc/vvc_dec: Add hardware decode API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: fei.w.wang@intel.com Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HPCKLeVKOvgJ From: Fei Wang Signed-off-by: Fei Wang --- libavcodec/vvc/vvc_refs.c | 6 ++++ libavcodec/vvc/vvcdec.c | 67 +++++++++++++++++++++++++++++++++++---- libavcodec/vvc/vvcdec.h | 5 +++ 3 files changed, 72 insertions(+), 6 deletions(-) diff --git a/libavcodec/vvc/vvc_refs.c b/libavcodec/vvc/vvc_refs.c index bf70777550..c9f89a5a0a 100644 --- a/libavcodec/vvc/vvc_refs.c +++ b/libavcodec/vvc/vvc_refs.c @@ -25,6 +25,7 @@ #include "libavutil/thread.h" #include "libavcodec/refstruct.h" #include "libavcodec/thread.h" +#include "libavcodec/decode.h" #include "vvc_refs.h" @@ -56,6 +57,7 @@ void ff_vvc_unref_frame(VVCFrameContext *fc, VVCFrame *frame, int flags) ff_refstruct_unref(&frame->rpl_tab); frame->collocated_ref = NULL; + ff_refstruct_unref(&frame->hwaccel_picture_private); } } @@ -138,6 +140,10 @@ static VVCFrame *alloc_frame(VVCContext *s, VVCFrameContext *fc) if (!frame->progress) goto fail; + ret = ff_hwaccel_frame_priv_alloc(s->avctx, &frame->hwaccel_picture_private); + if (ret < 0) + goto fail; + return frame; fail: ff_vvc_unref_frame(fc, frame, ~0); diff --git a/libavcodec/vvc/vvcdec.c b/libavcodec/vvc/vvcdec.c index d5704aca25..f2e269ce76 100644 --- a/libavcodec/vvc/vvcdec.c +++ b/libavcodec/vvc/vvcdec.c @@ -24,6 +24,8 @@ #include "libavcodec/decode.h" #include "libavcodec/profiles.h" #include "libavcodec/refstruct.h" +#include "libavcodec/hwconfig.h" +#include "libavcodec/hwaccel_internal.h" #include "libavutil/cpu.h" #include "libavutil/thread.h" @@ -563,6 +565,8 @@ static int ref_frame(VVCFrame *dst, const VVCFrame *src) ff_refstruct_replace(&dst->rpl_tab, src->rpl_tab); ff_refstruct_replace(&dst->rpl, src->rpl); + ff_refstruct_replace(&dst->hwaccel_picture_private, + src->hwaccel_picture_private); dst->nb_rpl_elems = src->nb_rpl_elems; dst->poc = src->poc; @@ -718,17 +722,41 @@ static int slice_start(SliceContext *sc, VVCContext *s, VVCFrameContext *fc, return 0; } +static enum AVPixelFormat get_format(AVCodecContext *avctx, const VVCSPS *sps) +{ +#define HWACCEL_MAX 0 + + enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmt = pix_fmts; + + switch (sps->pix_fmt) { + case AV_PIX_FMT_YUV420P: + break; + case AV_PIX_FMT_YUV420P10: + break; + } + + *fmt++ = sps->pix_fmt; + *fmt = AV_PIX_FMT_NONE; + + return ff_get_format(avctx, pix_fmts); +} + static void export_frame_params(VVCContext *s, const VVCFrameContext *fc) { AVCodecContext *c = s->avctx; const VVCSPS *sps = fc->ps.sps; const VVCPPS *pps = fc->ps.pps; - c->pix_fmt = sps->pix_fmt; - c->coded_width = pps->width; - c->coded_height = pps->height; - c->width = pps->width - ((pps->r->pps_conf_win_left_offset + pps->r->pps_conf_win_right_offset) << sps->hshift[CHROMA]); - c->height = pps->height - ((pps->r->pps_conf_win_top_offset + pps->r->pps_conf_win_bottom_offset) << sps->vshift[CHROMA]); + // Reset HW config if pix_fmt/w/h change. + if (s->pix_fmt != sps->pix_fmt || c->coded_width != pps->width || c->coded_height != pps->height) { + c->coded_width = pps->width; + c->coded_height = pps->height; + c->pix_fmt = get_format(c, sps); + s->pix_fmt = sps->pix_fmt; + } + + c->width = pps->width - ((pps->r->pps_conf_win_left_offset + pps->r->pps_conf_win_right_offset) << sps->hshift[CHROMA]); + c->height = pps->height - ((pps->r->pps_conf_win_top_offset + pps->r->pps_conf_win_bottom_offset) << sps->vshift[CHROMA]); } static int frame_setup(VVCFrameContext *fc, VVCContext *s) @@ -771,6 +799,20 @@ static int decode_slice(VVCContext *s, VVCFrameContext *fc, const H2645NAL *nal, ret = slice_init_entry_points(sc, fc, nal, unit); if (ret < 0) return ret; + + if (s->avctx->hwaccel) { + if (is_first_slice) { + ret = FF_HW_CALL(s->avctx, start_frame, NULL, 0); + if (ret < 0) + return ret; + } + + ret = FF_HW_CALL(s->avctx, decode_slice, + nal->raw_data, nal->raw_size); + if (ret < 0) + return ret; + } + fc->nb_slices++; return 0; @@ -885,9 +927,20 @@ static int wait_delayed_frame(VVCContext *s, AVFrame *output, int *got_output) static int submit_frame(VVCContext *s, VVCFrameContext *fc, AVFrame *output, int *got_output) { int ret; + + if (s->avctx->hwaccel) { + if (ret = FF_HW_SIMPLE_CALL(s->avctx, end_frame) < 0) { + av_log(s->avctx, AV_LOG_ERROR, + "Hardware accelerator failed to decode picture\n"); + ff_vvc_unref_frame(fc, fc->ref, ~0); + return ret; + } + } else + ff_vvc_frame_submit(s, fc); + s->nb_frames++; s->nb_delayed++; - ff_vvc_frame_submit(s, fc); + if (s->nb_delayed >= s->nb_fcs) { if ((ret = wait_delayed_frame(s, output, got_output)) < 0) return ret; @@ -1027,6 +1080,8 @@ static av_cold int vvc_decode_init(AVCodecContext *avctx) GDR_SET_RECOVERED(s); ff_thread_once(&init_static_once, init_default_scale_m); + s->pix_fmt = AV_PIX_FMT_NONE; + return 0; } diff --git a/libavcodec/vvc/vvcdec.h b/libavcodec/vvc/vvcdec.h index aa3d715524..009d57424e 100644 --- a/libavcodec/vvc/vvcdec.h +++ b/libavcodec/vvc/vvcdec.h @@ -78,6 +78,9 @@ typedef struct VVCFrame { * A combination of VVC_FRAME_FLAG_* */ uint8_t flags; + + AVBufferRef *hwaccel_priv_buf; + void *hwaccel_picture_private; ///< hardware accelerator private data } VVCFrame; typedef struct SliceContext { @@ -224,6 +227,8 @@ typedef struct VVCContext { uint64_t nb_frames; ///< processed frames int nb_delayed; ///< delayed frames + + enum AVPixelFormat pix_fmt; ///< pix format of current frame } VVCContext ; #endif /* AVCODEC_VVC_VVCDEC_H */