From patchwork Mon May 20 14:52:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49052 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4513651pzb; Mon, 20 May 2024 07:54:09 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVGiK8mDG/YzPzRto/X53zPTqT9qsHuh5eI7Ogtnjpb9mOCA+jCZFwo4pugO/Jvwn/QtcajjZJC9p2L+uDNccLwDkpR6CxlJZB7SA== X-Google-Smtp-Source: AGHT+IGNzIJulgagW/hyDwcP0BVzPVUohvAeeHOC3W43EBi8tEU3N8cec/KyGI4j3QNPHi/4z9ov X-Received: by 2002:a50:9e8e:0:b0:56e:2daf:1edf with SMTP id 4fb4d7f45d1cf-5752b4d085emr4844914a12.21.1716216849411; Mon, 20 May 2024 07:54:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216849; cv=none; d=google.com; s=arc-20160816; b=KErvCuZfTdZmMZ7P3/z0aF4t8B+Lp221Vw4A7As5c5pAtjjvFXqGys1D39NfnY0N6A Qz9tMK7Q3334Yt0HxPp322EnOOAz+H3CJK3fOx+rA91Gw9/MPK+oWehNt4y/rNdOxbXx 9uchuF4jHjKL6QKZcawJreNf3A++LqjSAlFVkZFeieYE9ADhx9kV9dl7G2z5cN4m1ejy eI9ePsqvVJ9x5HW39JwssLoDvMeIe7eN9WV55W3KaAYcgihPT+XhL/4CVUlx1dG0AQBB Xs31mbDc9kiAWL8mjNX9x2nTNqHqo+2PALbP/q501lXVxjhlc/dU+u+l8ItWVzN1nUi0 5S3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=wsJ3TJuX1NfALfWGavJ3p9g3UvnVOe/26HSaQtWVUjs=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=SL3BrdRvc6jUSldB1GaWOTDaPsfaBnDSzX9yBd6fCwGAQg145sJbB188wIs4LRuNQp 0hS2SNwzqFYp3a4cdzx26HS1YokTh1Db+ugUOPyrETkyqaJuCQ/Zeb9F3r3+pPLzY/MR ICYjvAmEQnQkIMO6n68PD2pTI3sk0ByR+LbAtqFYXwahoF+I+fMkV+3hjNEfB4z/WuOX aoH2IPChUcxLig1YTF1CEEKUQ0U5T250yH0OR5wAGYRpMF1bfEIrxZVdRw0shLx77FIj SfG8wRb2An4/UCfx21/jFkh8tWPWYamv3oUedWpIVKoo7W3Ydl8zcSfd9KTctg/qwB9B zv3w==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Z2dBs9vz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-576932432dfsi2115072a12.399.2024.05.20.07.54.08; Mon, 20 May 2024 07:54:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Z2dBs9vz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2176F68CEBC; Mon, 20 May 2024 17:54:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C947F68CA16 for ; Mon, 20 May 2024 17:53:57 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216843; x=1747752843; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=x7XrFxP5LibwB3DN39MXwR2Z3n8fd79D2curPjW9Dv4=; b=Z2dBs9vz+7AowWSOKMDTn0hso0Iuf299ZtzUCuUNVFFCtIpmV8nKXSHu fiDaeIDlqJfpleCxfI4fNZRfZf5u3hRxQ5FMg93+J9I/E2Qe+/kR8be7X F++s97Qbv4QnHrE3uWnHQ+kGpgQUTU65ZcbY4uE6WERr5dRGTe353hqIN AsiD0/lsKEy0v8Edb1o0tt+JhcZ1Q8DVaFT75CKFc36D6GEVtSylWaVfi iQ1+pXxbbWI1GbXZoc3EyiMlVj8KYhUe87Ut12H9n+ezs02T+sWKqEb15 1AVLaOnBPcWtf2AcqsGf9wdV3naGs4AiwXxC16yRS/+afNvCzsaqGZA5S w==; X-CSE-ConnectionGUID: /K9arBA7S9OhApQwhzcXew== X-CSE-MsgGUID: qQNFOrPNR1m0c/Y8QqgA5Q== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197242" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197242" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:53:56 -0700 X-CSE-ConnectionGUID: kWy0HEJoTuau+HJZDaZ3Lg== X-CSE-MsgGUID: Vhw8AVXGQAOkANQn35zBPQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787856" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:53:54 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:10 +0800 Message-ID: <20240520145222.291-1-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 01/13] avcodec/vaapi_encode: introduce a base layer for vaapi encode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: riajmDC4WUfx From: Tong Wu Since VAAPI and future D3D12VA implementation may share some common parameters, a base layer encode context is introduced as vaapi context's base. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.h | 56 +++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 39 +++++--------------------- 2 files changed, 63 insertions(+), 32 deletions(-) create mode 100644 libavcodec/hw_base_encode.h diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h new file mode 100644 index 0000000000..1996179456 --- /dev/null +++ b/libavcodec/hw_base_encode.h @@ -0,0 +1,56 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_HW_BASE_ENCODE_H +#define AVCODEC_HW_BASE_ENCODE_H + +#define MAX_DPB_SIZE 16 +#define MAX_PICTURE_REFERENCES 2 +#define MAX_REORDER_DELAY 16 +#define MAX_ASYNC_DEPTH 64 +#define MAX_REFERENCE_LIST_NUM 2 + +enum { + PICTURE_TYPE_IDR = 0, + PICTURE_TYPE_I = 1, + PICTURE_TYPE_P = 2, + PICTURE_TYPE_B = 3, +}; + +enum { + // Codec supports controlling the subdivision of pictures into slices. + FLAG_SLICE_CONTROL = 1 << 0, + // Codec only supports constant quality (no rate control). + FLAG_CONSTANT_QUALITY_ONLY = 1 << 1, + // Codec is intra-only. + FLAG_INTRA_ONLY = 1 << 2, + // Codec supports B-pictures. + FLAG_B_PICTURES = 1 << 3, + // Codec supports referencing B-pictures. + FLAG_B_PICTURE_REFERENCES = 1 << 4, + // Codec supports non-IDR key pictures (that is, key pictures do + // not necessarily empty the DPB). + FLAG_NON_IDR_KEY_PICTURES = 1 << 5, +}; + +typedef struct HWBaseEncodeContext { + const AVClass *class; +} HWBaseEncodeContext; + +#endif /* AVCODEC_HW_BASE_ENCODE_H */ + diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 0eed9691ca..f5c9be8973 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -33,34 +33,27 @@ #include "avcodec.h" #include "hwconfig.h" +#include "hw_base_encode.h" struct VAAPIEncodeType; struct VAAPIEncodePicture; +// Codec output packet without timestamp delay, which means the +// output packet has same PTS and DTS. +#define FLAG_TIMESTAMP_NO_DELAY 1 << 6 + enum { MAX_CONFIG_ATTRIBUTES = 4, MAX_GLOBAL_PARAMS = 4, - MAX_DPB_SIZE = 16, - MAX_PICTURE_REFERENCES = 2, - MAX_REORDER_DELAY = 16, MAX_PARAM_BUFFER_SIZE = 1024, // A.4.1: table A.6 allows at most 22 tile rows for any level. MAX_TILE_ROWS = 22, // A.4.1: table A.6 allows at most 20 tile columns for any level. MAX_TILE_COLS = 20, - MAX_ASYNC_DEPTH = 64, - MAX_REFERENCE_LIST_NUM = 2, }; extern const AVCodecHWConfigInternal *const ff_vaapi_encode_hw_configs[]; -enum { - PICTURE_TYPE_IDR = 0, - PICTURE_TYPE_I = 1, - PICTURE_TYPE_P = 2, - PICTURE_TYPE_B = 3, -}; - typedef struct VAAPIEncodeSlice { int index; int row_start; @@ -193,7 +186,8 @@ typedef struct VAAPIEncodeRCMode { } VAAPIEncodeRCMode; typedef struct VAAPIEncodeContext { - const AVClass *class; + // Base context. + HWBaseEncodeContext base; // Codec-specific hooks. const struct VAAPIEncodeType *codec; @@ -397,25 +391,6 @@ typedef struct VAAPIEncodeContext { AVPacket *tail_pkt; } VAAPIEncodeContext; -enum { - // Codec supports controlling the subdivision of pictures into slices. - FLAG_SLICE_CONTROL = 1 << 0, - // Codec only supports constant quality (no rate control). - FLAG_CONSTANT_QUALITY_ONLY = 1 << 1, - // Codec is intra-only. - FLAG_INTRA_ONLY = 1 << 2, - // Codec supports B-pictures. - FLAG_B_PICTURES = 1 << 3, - // Codec supports referencing B-pictures. - FLAG_B_PICTURE_REFERENCES = 1 << 4, - // Codec supports non-IDR key pictures (that is, key pictures do - // not necessarily empty the DPB). - FLAG_NON_IDR_KEY_PICTURES = 1 << 5, - // Codec output packet without timestamp delay, which means the - // output packet has same PTS and DTS. - FLAG_TIMESTAMP_NO_DELAY = 1 << 6, -}; - typedef struct VAAPIEncodeType { // List of supported profiles and corresponding VAAPI profiles. // (Must end with AV_PROFILE_UNKNOWN.) From patchwork Mon May 20 14:52:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49053 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4513770pzb; Mon, 20 May 2024 07:54:20 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVyQLoxtY7UxhhdR3LPv9sKrWPEG8P6bc16/sXNmuBEHFVdj/7O6pV+DSEeDKdraj2+8Yo/aDiXjVjiPpSwcXqmfSdqHZcgcsJFrA== X-Google-Smtp-Source: AGHT+IHq3dwT+UjTtj/aulFyvdEN8El72jsk9ZNBf8EYnjVZfq4IVaHDWM7K3AnFFIuYzgxjcWTh X-Received: by 2002:a50:8d16:0:b0:565:dfac:a686 with SMTP id 4fb4d7f45d1cf-5734d6de9b6mr20931167a12.38.1716216860339; Mon, 20 May 2024 07:54:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216860; cv=none; d=google.com; s=arc-20160816; b=WMsqGshj/PUtULNmVCnrG0hJ++n30jsY9X1+IVql/SqOvxW+Hezt7xX3YeSp8ftMad FtVifDLNsZXAPJD+gqWFHUB2u0xihUfXB2gcyrYAMtB1Wq7oGDRKYfogZm2hm+VnvkjU N4IsWychs3UX3Uk0M9geEIHZ6dBNAbyh1qvkn6DoLbozHOsJZkU34O4j70vAUIzUBwLa TB7bOmj/msWm7AW7dNe5NOVBLr8OCAbf6fU7jZVGn4nJRvwITSNgHFv5hj6tr8kWBJ+F BGiVAkfH9k/Fog7OVeV/W+NpWkr4vo/cPGnV+BRfbDegZGv9e/FXYpn1k8zfzvjcST2c R/Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=HKgbMHJaVakAuq/2qpGaujnrPFpo0XBj6fSfCo3c7mM=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=gocr4VW7Phn9hQ16sOd45krK2wYHaPHAodatRxzqWQ1e7pfDP9FKmV7RhAtdZYuAJo Hnbt+2DE3ZaNAqTSwFy+och0klys+HvbrSv8ZLYPSyssPhHLfyoArAh1aIF63vLCgFZG NMaNbLya1VBZstxkChXnirQOQteGlnSoBCWjTGvhtUKVAI2SdHH84oIeq/rfkyu2Tmgw rhP5/cGBeb+6JfzXZOgtsRSpWCAbnIFYt/vyYbql+8q/MUNUFwZu/F3cNhBDQjfwrx4i nwR1UpUW2/7BEqtBUWBMnfP/qnR1hz6PGaDvgSyMITp1LB1gLIe9HyWDtJHyoKlkiiOz qRPw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="mFgl9/Vt"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5750fdedda6si4547256a12.170.2024.05.20.07.54.19; Mon, 20 May 2024 07:54:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="mFgl9/Vt"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5D8D368D1BC; Mon, 20 May 2024 17:54:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3A11168CEBC for ; Mon, 20 May 2024 17:53:59 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216845; x=1747752845; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L+pX56Ae6U7EXYvxYNkP42r/F/Bqry4GF5aUtsvc24w=; b=mFgl9/VtrVHQJXeVZ183CZfuJ4u2g+MT1leU1DTsABHaiXa+wMSNOX12 7emFmpn/pBdy6WYI/8o4tzGTnhKQHVjQ6R0Z9qmH9U21vG5se/A1mBzpO hq6jpR6/nhNVEnqFdZcOohpDfQcuF+4cmvOo2ExryBl/qqBZ+wEnPbtus LtdYvkGnfbGQKYPXbD2KN0fyjYobIDzAJJv3eZrG1OL5nS3BBd2g/qyje T5yXLOCTEAZbVushJcSeiRNxA1RqjgMGMbDXTZZ9/OpWjYhjM7reW8Q9R u82UpX3s8v7wm5u/GHSVodguIi+JobSPEZozuAmgC3P0oMpy9w4w88Y7j A==; X-CSE-ConnectionGUID: BdDng9DQQLqXC1/6qIzQ3g== X-CSE-MsgGUID: m0eL/CgXQJqI7QBv0odFXw== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197244" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197244" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:53:57 -0700 X-CSE-ConnectionGUID: QPxw6KzgTOOliuFHruIwWA== X-CSE-MsgGUID: FcMkr/UfRJGk6h6C6dWGnw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787860" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:53:55 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:11 +0800 Message-ID: <20240520145222.291-2-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 02/13] avcodec/vaapi_encode: add async_depth to common options X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sg3ken9zCbB6 From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.h | 10 +++++++++- libavcodec/vaapi_encode.c | 13 ++++++++----- libavcodec/vaapi_encode.h | 7 ------- libavcodec/vaapi_encode_av1.c | 1 + libavcodec/vaapi_encode_h264.c | 1 + libavcodec/vaapi_encode_h265.c | 1 + libavcodec/vaapi_encode_mjpeg.c | 1 + libavcodec/vaapi_encode_mpeg2.c | 1 + libavcodec/vaapi_encode_vp8.c | 1 + libavcodec/vaapi_encode_vp9.c | 1 + 10 files changed, 24 insertions(+), 13 deletions(-) diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 1996179456..5272f2836d 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -50,7 +50,15 @@ enum { typedef struct HWBaseEncodeContext { const AVClass *class; + + // Max number of frame buffered in encoder. + int async_depth; } HWBaseEncodeContext; -#endif /* AVCODEC_HW_BASE_ENCODE_H */ +#define HW_BASE_ENCODE_COMMON_OPTIONS \ + { "async_depth", "Maximum processing parallelism. " \ + "Increase this to improve single channel performance.", \ + OFFSET(common.base.async_depth), AV_OPT_TYPE_INT, \ + { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS } +#endif /* AVCODEC_HW_BASE_ENCODE_H */ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f54b2579ec..9373512417 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -669,7 +669,8 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, VAAPIEncodePicture *pic, AVPacket *pkt) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; if (pic->type == PICTURE_TYPE_IDR) pkt->flags |= AV_PKT_FLAG_KEY; @@ -699,7 +700,7 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; } else { pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % - (3 * ctx->output_delay + ctx->async_depth)]; + (3 * ctx->output_delay + base_ctx->async_depth)]; } return 0; @@ -1320,6 +1321,7 @@ static int vaapi_encode_check_frame(AVCodecContext *avctx, static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodePicture *pic; int err; @@ -1365,7 +1367,7 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) ctx->dts_pts_diff = pic->pts - ctx->first_pts; if (ctx->output_delay > 0) ctx->ts_ring[ctx->input_order % - (3 * ctx->output_delay + ctx->async_depth)] = pic->pts; + (3 * ctx->output_delay + base_ctx->async_depth)] = pic->pts; pic->display_order = ctx->input_order; ++ctx->input_order; @@ -2773,7 +2775,8 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; AVVAAPIFramesContext *recon_hwctx = NULL; VAStatus vas; int err; @@ -2966,7 +2969,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) vas = vaSyncBuffer(ctx->hwctx->display, VA_INVALID_ID, 0); if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) { ctx->has_sync_buffer_func = 1; - ctx->encode_fifo = av_fifo_alloc2(ctx->async_depth, + ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, sizeof(VAAPIEncodePicture *), 0); if (!ctx->encode_fifo) diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index f5c9be8973..02410c72ec 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -374,8 +374,6 @@ typedef struct VAAPIEncodeContext { int has_sync_buffer_func; // Store buffered pic AVFifo *encode_fifo; - // Max number of frame buffered in encoder. - int async_depth; /** Head data for current output pkt, used only for AV1. */ //void *header_data; @@ -491,11 +489,6 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "Maximum B-frame reference depth", \ OFFSET(common.desired_b_depth), AV_OPT_TYPE_INT, \ { .i64 = 1 }, 1, INT_MAX, FLAGS }, \ - { "async_depth", "Maximum processing parallelism. " \ - "Increase this to improve single channel performance. This option " \ - "doesn't work if driver doesn't implement vaSyncBuffer function.", \ - OFFSET(common.async_depth), AV_OPT_TYPE_INT, \ - { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS }, \ { "max_frame_size", \ "Maximum frame size (in bytes)",\ OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index b868f5b66a..a5a475dca6 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -965,6 +965,7 @@ static av_cold int vaapi_encode_av1_close(AVCodecContext *avctx) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_av1_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, { "profile", "Set profile (seq_profile)", diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index d656b1020f..ebb1760cd3 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -1276,6 +1276,7 @@ static av_cold int vaapi_encode_h264_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeH264Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_h264_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 2f59161346..77bd5e31af 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1394,6 +1394,7 @@ static av_cold int vaapi_encode_h265_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeH265Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_h265_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index c17747e3a9..fb5c0d34c6 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -540,6 +540,7 @@ static av_cold int vaapi_encode_mjpeg_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeMJPEGContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_mjpeg_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, { "jfif", "Include JFIF header", diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index c9b16fbcfc..d0980c52b0 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -639,6 +639,7 @@ static av_cold int vaapi_encode_mpeg2_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeMPEG2Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_mpeg2_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index 8a557b967e..4e284f86e2 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -216,6 +216,7 @@ static av_cold int vaapi_encode_vp8_init(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeVP8Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_vp8_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index c2a8dec71b..88f951652c 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -273,6 +273,7 @@ static av_cold int vaapi_encode_vp9_init(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeVP9Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_vp9_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, From patchwork Mon May 20 14:52:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49054 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4513868pzb; Mon, 20 May 2024 07:54:29 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWyZkEEsJe+5N3o5foyC88Y1XOn9oqLlqPJRPt8EBh85ro0PcG8uBluL6nHj7Tvpp0EarxJb2kAT3BlqUtkJYFenDHdYAlwQ/rcNA== X-Google-Smtp-Source: AGHT+IHP0NEzqFTomV3IvZKoTZu9SfdvGJKAEkhNBYhKZIbgvUTe82aliKN+nlNHisfEsvGcOr6G X-Received: by 2002:a17:907:7292:b0:a61:a581:d5c with SMTP id a640c23a62f3a-a61a5810e00mr199516666b.6.1716216869590; Mon, 20 May 2024 07:54:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216869; cv=none; d=google.com; s=arc-20160816; b=j88HtXw7x1tkPpUpidzKggNZgyATg2+N+RPVAHTo9+RppfPcEZRkYABZpIlfnK+zRg FzQrDbNDONgxdjkIOLPvhm4vKM2p6J55W4HzVimQQbtbWggAfGNN7IDRgdDGDDzIJnY+ uDA+0AJntmUpnekZm8jE4bbqCTUIozyAaHz5wJNEzxc+dLGrK6YDuz0b1tESVP0ebllg d12qLoWey9LB1NOH3x/61xWYgFvrXKRrf8kppafjvCx1QWANs1mc4IZ/lqXbfe2JxcaE LmkctMlQOzOVbihKvjHTP6SyTt+WadcI4gB50f1zjeRaxtGL0L+EGEStYYbfcKEIsdgE I3Ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=q1kdfZ2WVoEFjfviXQUkQo/qA20jDHG+TCQ8dQEuWeI=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=s3h8qXIb/6SCa0HmOZCJijhbMQMfLm48LE9Abf+J85lDs4wXxcDDbDScPnCK6G2vQU 918z0FTGkAoczjT5i9wy5AyofEDOXjbR2FAH2q/HRk3iCJ6nHUgXp1Q0Y6uxEmelMmMs jJqKOzGQO8AXpgEDVagFQoqQkVYWw1bMN3qLXA/rf590nkHkPN/Pd1lgmg0OXXcGiqVs Y7j9lpZnDXtDNYPrULgckrJPmgXBgiGK+um9uCHQchB/DrXcPv/ySUfDHqYglkzGumfs FckwI1mI9QhX7sGgijimCbUXYJQNzydmTaL0k7L7vdwIntuXX47Frb+3ZEsMhspt51my 2trA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=nRapLtq1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a1871e3ccsi1340463966b.185.2024.05.20.07.54.29; Mon, 20 May 2024 07:54:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=nRapLtq1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8E92368D120; Mon, 20 May 2024 17:54:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4689C68D180 for ; Mon, 20 May 2024 17:54:00 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216846; x=1747752846; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KAG/yivTgGEHchufiYiAVJxZSI/iQ/DiNn32BP+e50Y=; b=nRapLtq16G8eiahX+P/os6ZudPEg8Zf0GqdNj43UwkYyES0wCl4+MiiZ XfnI39CImsPXQK/eykTs6efrTCXU7jdKfu79frgHVrs/mKDTKfbHp/sKb j+rcuzVP4geR+rWdTPVCA9iK3677lpIYTQycG8E68wB9i2CWdM83EOIj3 /V93+R8CjoflABkIXxqu8D3N/t0+LxCnqekTuAAiCduPzKbWlzD8idJ5V ap/9QsP0kP3Vnwfd8TAW6GckyCrwX9akuJsVfdzQaUqvQ9zpWsu1C4+6N abScZNIsARPLxXPz01r/DfMy1uFO9gMUOfcKkjPRRhJ5wbCBNKaIYF2iR Q==; X-CSE-ConnectionGUID: 5KfrfeDGQQinxqyV09TGiQ== X-CSE-MsgGUID: 7plwfAiHSV2sXSoZBuJO/Q== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197245" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197245" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:53:58 -0700 X-CSE-ConnectionGUID: PKbylybwTFayM4Ps8AsOkg== X-CSE-MsgGUID: 0OUmmkexQ9qPYCLCHCijZw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787866" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:53:57 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:12 +0800 Message-ID: <20240520145222.291-3-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 03/13] avcodec/vaapi_encode: add picture type name to base X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PrI6+qVJdiuR From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.h | 5 +++++ libavcodec/vaapi_encode.c | 4 +--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 5272f2836d..a578db8c06 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -25,6 +25,11 @@ #define MAX_ASYNC_DEPTH 64 #define MAX_REFERENCE_LIST_NUM 2 +static inline const char *ff_hw_base_encode_get_pictype_name(const int type) { + const char * const picture_type_name[] = { "IDR", "I", "P", "B" }; + return picture_type_name[type]; +} + enum { PICTURE_TYPE_IDR = 0, PICTURE_TYPE_I = 1, diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 9373512417..2d22e4bd85 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -38,8 +38,6 @@ const AVCodecHWConfigInternal *const ff_vaapi_encode_hw_configs[] = { NULL, }; -static const char * const picture_type_name[] = { "IDR", "I", "P", "B" }; - static int vaapi_encode_make_packed_header(AVCodecContext *avctx, VAAPIEncodePicture *pic, int type, char *data, size_t bit_len) @@ -277,7 +275,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " "as type %s.\n", pic->display_order, pic->encode_order, - picture_type_name[pic->type]); + ff_hw_base_encode_get_pictype_name(pic->type)); if (pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0) { av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); } else { From patchwork Mon May 20 14:52:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49055 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514001pzb; Mon, 20 May 2024 07:54:38 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXoGYaArAB/LfwcQy3nLzYoMlOdVS8bMukGlj8LtqZ4Vx0EowZXUb2pyz1Yu7y5S2coKoHKNj9pm6lSS8o6bN+JQXr++Y6ojISqjg== X-Google-Smtp-Source: AGHT+IFqx1f8ZNd37kSPfRhRiKjZnxpKTpKBkw0ngdPl07c8j8OUlD4PQNk6Fu5pLxvnL6adSHRe X-Received: by 2002:a17:906:348a:b0:a59:ced4:25af with SMTP id a640c23a62f3a-a5a2d5ccfacmr1906784766b.41.1716216878337; Mon, 20 May 2024 07:54:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216878; cv=none; d=google.com; s=arc-20160816; b=Xvt2px6jNvb3JOQjENgdBsbdDPtqa++0sekxHiVqSRry6K5Gq8VhzKMhNi2RlyHfzP KET/jRkRakxgBSO805ZWhgbNuvgSUkFMtut3oE4A0Q+OseuKVBh9r9dwLvJ6+FNuvcTO mNNfpxQsHEYsrXQykmsAyAg5g+9aG3pGQdE0jIT0G5+17Naj+tD9ozICreJ/51x4epin 0H9VmvOzeFxsa6cReRWSDJ4rpdz/Jj71KwZSVV0cMxoIbaFuvg6UnBnTBF7sy/M8dVuK WjTuwyQhwDC0gglB5NRbFO8dZw00Bw9WaVDCw+UMvn229lmO2oi9fBSjn3OvXA3Kv1GZ trrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=DruZqgCM0Ll/K9SNdY0r0CexfVTk23BkLKZDcD+w+o4=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=XEJgcMdJedpf2keHj9uugA6nR3+WbrNoyM2zLhd7t4kKYX0tPvpLMZlDN72yXs0v1/ Syu2X032NX+Es823pBNtp2j6zkrMz/WjLZ6vE+SCnIKkrdfOLeRyLcdCNkmCGaysyA6e JI9pwFxxRpGhMM6WzZXOg2vFgh9wW/djn39Hl8BH68iKLaLpmnaet7bAHyw3omO0e8Uv E9sgpmqvPHfOSKL7Eq7baMlHsEr8SYO/UOgK9jm4RvhOYPibL7oTUvgMYM2jf2dTjv/U AutX8PI1CzT6KEpOgtS30eK89qXChnnNu/53gnCzhIsEpNDqklmY1BLyJ21pjRo6Qz77 eWbw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ngafrVbc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17ba3a71si1345781366b.603.2024.05.20.07.54.38; Mon, 20 May 2024 07:54:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ngafrVbc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0197368D222; Mon, 20 May 2024 17:54:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B685768D120 for ; Mon, 20 May 2024 17:54:03 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216849; x=1747752849; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k+Cd3SbMsZI2AOBFYmvXlX99kdjYu6SRZFiVO28Uq+c=; b=ngafrVbcSvJWAL9VtUfuGwon2jt7ShQykJSTxk2Ec7yTC5yFlIWuT5Dt egEQQFQKoL/Nx9hHg/bPY3u15j46roJZkV3zPsz4GFf02qMyrPoXWsamy rBAg4KCqUm2y8t68Y/D0Pbf3OiUnkHhL7NqGPQFyS/+I7zzwbkj1KVSnq 7sQnOx3A0ukaCri+PqLdbQuiK4QGEGoR22LhHdIChPU7op0lUEmgEPja8 pS+jFMo7YScCZIdXq2L2TOi88d8x7CQHD6iRQXyRV5wRgOz6N8Bxwuz/7 dr3y0vlNpJ8UOIp6OhpFFKGkN1wRwZdyAQ7zHmdoHBqH/VuR0lBcRXGgj A==; X-CSE-ConnectionGUID: BIoCS/EzTfWUweQJqm0fAw== X-CSE-MsgGUID: +Y+7OsuqT8myBPrTmm9IxA== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197246" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197246" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:53:59 -0700 X-CSE-ConnectionGUID: XgwDwVBCRl6j+kkSrGTpaA== X-CSE-MsgGUID: nI5YP5JnQXygyAGGKU1SFA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787870" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:53:58 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:13 +0800 Message-ID: <20240520145222.291-4-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 04/13] avcodec/vaapi_encode: move pic->input_surface initialization to encode_alloc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9UWVt755kO1+ From: Tong Wu When allocating the VAAPIEncodePicture, pic->input_surface can be initialized right in the place. This movement simplifies the send_frame logic and is the preparation for moving vaapi_encode_send_frame to the base layer. Signed-off-by: Tong Wu --- libavcodec/vaapi_encode.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 2d22e4bd85..227cccae64 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -878,7 +878,8 @@ static int vaapi_encode_discard(AVCodecContext *avctx, return 0; } -static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx) +static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, + const AVFrame *frame) { VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodePicture *pic; @@ -895,7 +896,7 @@ static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx) } } - pic->input_surface = VA_INVALID_ID; + pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3]; pic->recon_surface = VA_INVALID_ID; pic->output_buffer = VA_INVALID_ID; @@ -1332,7 +1333,7 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) if (err < 0) return err; - pic = vaapi_encode_alloc(avctx); + pic = vaapi_encode_alloc(avctx, frame); if (!pic) return AVERROR(ENOMEM); @@ -1345,7 +1346,6 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I) pic->force_idr = 1; - pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3]; pic->pts = frame->pts; pic->duration = frame->duration; From patchwork Mon May 20 14:52:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49056 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514128pzb; Mon, 20 May 2024 07:54:49 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX82oQvUV/ydtuVENlKgsYSd/mvK1Q23J0rFMyfzk0dS0JWsxXWHD0TNl5mPTpz4ho4HDLYbptvQXCgEn+U+nchks1km8SmdQ1qOw== X-Google-Smtp-Source: AGHT+IGcnwKJoq1bCPbeaIH1fx0PncxiKkFrNWxAohIHvJF3+f1+R4PCsKqaIyPuVpGCDk9R7Hyw X-Received: by 2002:a05:651c:149:b0:2e2:9842:a9d3 with SMTP id 38308e7fff4ca-2e52039dc3emr238535381fa.46.1716216889268; Mon, 20 May 2024 07:54:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216889; cv=none; d=google.com; s=arc-20160816; b=sWEZMoHp+2KiMIn6cr7PhM4IjUbJkRkUwXQUCb9+uP76TnAX5eph7tkyD6SDiqH6vK NrxoFk36PJd0yC2TFhx5T4fFUYyWGqhEsSIavEYI985jHDNbfe6CHRIXimvZgYcii5OV hMGu1gmDJDvFEH59pvVqhYkKbovq2hyLS3SlOU76emm8IbALxmBhO5iQVn95PUBrKLDZ SGFs+yRWXjTF1UCwI6sML3fTdZbaNZoG2fzEIHR7cUeFq+ilYY3fChB4rlUqrJyGkM9F 408agOO21yr9k6BKTQfq9HHEc5Q5b26C65MyUngQ7bQJOJRvagZ/bJqtkRev9uZfJ+tF OGXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=CfVRKteViZZY2At9Yw1ZhjxmQ6NuXTS6IdNSAuUSibc=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=HZb6AvaSazN+/RuXmq8WnbXDgT/ZnOTgpDfS1gW+bMm+vwEalCbV9huKoxy6LH5kX1 mUdTBxdIR0BMzTAq/pdxBmElO9GogNoUGKjrH5a/gJVfL/dcj4sqJ3ma3NEgUrOLW1iZ Myr3qhLvM9KUCC0B50OlMEeEJ8wx0WdWUO05bmNwPf0Wm8Of4B+fF/w7DpMt/ZyXr6M6 OWwU4sMbQfTG/lm4dVdw6SrUgJyxvjKl1BLA8qA9PPotdwPo+bXmLrzWFZMKa7LJGvyV HYDjlOw6SyDsQhjPPVoQu6YmcqA2j+9q4N5UcGD/WnSQwMP6QPMU5Z7OH9xOewbOyzve solg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="LS6ZLcN/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5cfb4be933si517192766b.1013.2024.05.20.07.54.48; Mon, 20 May 2024 07:54:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="LS6ZLcN/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4DC2468D1AF; Mon, 20 May 2024 17:54:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 259B068D208 for ; Mon, 20 May 2024 17:54:04 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216851; x=1747752851; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZDiQ9vIROS3oVAcDgdhanC41aRj2Ab8ZKq7BlYsercA=; b=LS6ZLcN/WO/KrZPdA/eZiDcRBbMsI+asnnCbFErIWExRwcDzzRV2Ffuk aNG+GWL+KvmDcxdnAQ+YSVB1bl9wah28kfSvIm2UN3r/Pb5lFYHw/T9Jr SVHaFDSvayyP1sINtXRVM+YpC5oVR625RE72mWkzVZ4q0ouIw7rTvRwxH JuCDHcmZjSZ5bzxGk3I3arIhw/tuE3gCG8CcRGINfIg9EUE7V/20HnYoL sO5zwF8gw63WUDYTVlzXeibihuyIyyjU66OqtwlAKnNeIiTaO/KRRIgqp FCOw6wHbu0J8Xa7Yncic4njGKrpZre3pWij74dla38aGlWayKjH9ngns7 A==; X-CSE-ConnectionGUID: v7Vd5V6tSwW6/FNfL77Bfw== X-CSE-MsgGUID: 7OEZrhsmTtybk90C1xSbGg== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197247" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197247" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:00 -0700 X-CSE-ConnectionGUID: EroT772RSD6GdXPzeRoNeA== X-CSE-MsgGUID: YqmerUe4QFOeRhTQQobKhQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787878" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:53:59 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:14 +0800 Message-ID: <20240520145222.291-5-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 05/13] avcodec/vaapi_encode: move the dpb logic from VAAPI to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RIb7s+QgrrYC From: Tong Wu Move receive_packet function to base. This requires adding *alloc, *issue, *output, *free as hardware callbacks. HWBaseEncodePicture is introduced as the base layer structure. The related parameters in VAAPIEncodeContext are also extracted to HWBaseEncodeContext. Then DPB management logic can be fully extracted to base layer as-is. Signed-off-by: Tong Wu --- libavcodec/Makefile | 2 +- libavcodec/hw_base_encode.c | 597 ++++++++++++++++++++++++ libavcodec/hw_base_encode.h | 124 +++++ libavcodec/vaapi_encode.c | 793 +++++--------------------------- libavcodec/vaapi_encode.h | 102 +--- libavcodec/vaapi_encode_av1.c | 35 +- libavcodec/vaapi_encode_h264.c | 84 ++-- libavcodec/vaapi_encode_h265.c | 53 ++- libavcodec/vaapi_encode_mjpeg.c | 13 +- libavcodec/vaapi_encode_mpeg2.c | 33 +- libavcodec/vaapi_encode_vp8.c | 18 +- libavcodec/vaapi_encode_vp9.c | 24 +- 12 files changed, 988 insertions(+), 890 deletions(-) create mode 100644 libavcodec/hw_base_encode.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 2443d2c6fd..998f6b7e12 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -165,7 +165,7 @@ OBJS-$(CONFIG_STARTCODE) += startcode.o OBJS-$(CONFIG_TEXTUREDSP) += texturedsp.o OBJS-$(CONFIG_TEXTUREDSPENC) += texturedspenc.o OBJS-$(CONFIG_TPELDSP) += tpeldsp.o -OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o +OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o hw_base_encode.o OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o OBJS-$(CONFIG_VC1DSP) += vc1dsp.o OBJS-$(CONFIG_VIDEODSP) += videodsp.o diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c new file mode 100644 index 0000000000..ec7178d2dc --- /dev/null +++ b/libavcodec/hw_base_encode.c @@ -0,0 +1,597 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/internal.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" + +#include "encode.h" +#include "avcodec.h" +#include "hw_base_encode.h" + +static void hw_base_encode_add_ref(AVCodecContext *avctx, + HWBaseEncodePicture *pic, + HWBaseEncodePicture *target, + int is_ref, int in_dpb, int prev) +{ + int refs = 0; + + if (is_ref) { + av_assert0(pic != target); + av_assert0(pic->nb_refs[0] < MAX_PICTURE_REFERENCES && + pic->nb_refs[1] < MAX_PICTURE_REFERENCES); + if (target->display_order < pic->display_order) + pic->refs[0][pic->nb_refs[0]++] = target; + else + pic->refs[1][pic->nb_refs[1]++] = target; + ++refs; + } + + if (in_dpb) { + av_assert0(pic->nb_dpb_pics < MAX_DPB_SIZE); + pic->dpb[pic->nb_dpb_pics++] = target; + ++refs; + } + + if (prev) { + av_assert0(!pic->prev); + pic->prev = target; + ++refs; + } + + target->ref_count[0] += refs; + target->ref_count[1] += refs; +} + +static void hw_base_encode_remove_refs(AVCodecContext *avctx, + HWBaseEncodePicture *pic, + int level) +{ + int i; + + if (pic->ref_removed[level]) + return; + + for (i = 0; i < pic->nb_refs[0]; i++) { + av_assert0(pic->refs[0][i]); + --pic->refs[0][i]->ref_count[level]; + av_assert0(pic->refs[0][i]->ref_count[level] >= 0); + } + + for (i = 0; i < pic->nb_refs[1]; i++) { + av_assert0(pic->refs[1][i]); + --pic->refs[1][i]->ref_count[level]; + av_assert0(pic->refs[1][i]->ref_count[level] >= 0); + } + + for (i = 0; i < pic->nb_dpb_pics; i++) { + av_assert0(pic->dpb[i]); + --pic->dpb[i]->ref_count[level]; + av_assert0(pic->dpb[i]->ref_count[level] >= 0); + } + + av_assert0(pic->prev || pic->type == PICTURE_TYPE_IDR); + if (pic->prev) { + --pic->prev->ref_count[level]; + av_assert0(pic->prev->ref_count[level] >= 0); + } + + pic->ref_removed[level] = 1; +} + +static void hw_base_encode_set_b_pictures(AVCodecContext *avctx, + HWBaseEncodePicture *start, + HWBaseEncodePicture *end, + HWBaseEncodePicture *prev, + int current_depth, + HWBaseEncodePicture **last) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *next, *ref; + int i, len; + + av_assert0(start && end && start != end && start->next != end); + + // If we are at the maximum depth then encode all pictures as + // non-referenced B-pictures. Also do this if there is exactly one + // picture left, since there will be nothing to reference it. + if (current_depth == ctx->max_b_depth || start->next->next == end) { + for (pic = start->next; pic; pic = pic->next) { + if (pic == end) + break; + pic->type = PICTURE_TYPE_B; + pic->b_depth = current_depth; + + hw_base_encode_add_ref(avctx, pic, start, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, end, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, prev, 0, 0, 1); + + for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) + hw_base_encode_add_ref(avctx, pic, ref, 0, 1, 0); + } + *last = prev; + + } else { + // Split the current list at the midpoint with a referenced + // B-picture, then descend into each side separately. + len = 0; + for (pic = start->next; pic != end; pic = pic->next) + ++len; + for (pic = start->next, i = 1; 2 * i < len; pic = pic->next, i++); + + pic->type = PICTURE_TYPE_B; + pic->b_depth = current_depth; + + pic->is_reference = 1; + + hw_base_encode_add_ref(avctx, pic, pic, 0, 1, 0); + hw_base_encode_add_ref(avctx, pic, start, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, end, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, prev, 0, 0, 1); + + for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) + hw_base_encode_add_ref(avctx, pic, ref, 0, 1, 0); + + if (i > 1) + hw_base_encode_set_b_pictures(avctx, start, pic, pic, + current_depth + 1, &next); + else + next = pic; + + hw_base_encode_set_b_pictures(avctx, pic, end, next, + current_depth + 1, last); + } +} + +static void hw_base_encode_add_next_prev(AVCodecContext *avctx, + HWBaseEncodePicture *pic) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + int i; + + if (!pic) + return; + + if (pic->type == PICTURE_TYPE_IDR) { + for (i = 0; i < ctx->nb_next_prev; i++) { + --ctx->next_prev[i]->ref_count[0]; + ctx->next_prev[i] = NULL; + } + ctx->next_prev[0] = pic; + ++pic->ref_count[0]; + ctx->nb_next_prev = 1; + + return; + } + + if (ctx->nb_next_prev < MAX_PICTURE_REFERENCES) { + ctx->next_prev[ctx->nb_next_prev++] = pic; + ++pic->ref_count[0]; + } else { + --ctx->next_prev[0]->ref_count[0]; + for (i = 0; i < MAX_PICTURE_REFERENCES - 1; i++) + ctx->next_prev[i] = ctx->next_prev[i + 1]; + ctx->next_prev[i] = pic; + ++pic->ref_count[0]; + } +} + +static int hw_base_encode_pick_next(AVCodecContext *avctx, + HWBaseEncodePicture **pic_out) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic = NULL, *prev = NULL, *next, *start; + int i, b_counter, closed_gop_end; + + // If there are any B-frames already queued, the next one to encode + // is the earliest not-yet-issued frame for which all references are + // available. + for (pic = ctx->pic_start; pic; pic = pic->next) { + if (pic->encode_issued) + continue; + if (pic->type != PICTURE_TYPE_B) + continue; + for (i = 0; i < pic->nb_refs[0]; i++) { + if (!pic->refs[0][i]->encode_issued) + break; + } + if (i != pic->nb_refs[0]) + continue; + + for (i = 0; i < pic->nb_refs[1]; i++) { + if (!pic->refs[1][i]->encode_issued) + break; + } + if (i == pic->nb_refs[1]) + break; + } + + if (pic) { + av_log(avctx, AV_LOG_DEBUG, "Pick B-picture at depth %d to " + "encode next.\n", pic->b_depth); + *pic_out = pic; + return 0; + } + + // Find the B-per-Pth available picture to become the next picture + // on the top layer. + start = NULL; + b_counter = 0; + closed_gop_end = ctx->closed_gop || + ctx->idr_counter == ctx->gop_per_idr; + for (pic = ctx->pic_start; pic; pic = next) { + next = pic->next; + if (pic->encode_issued) { + start = pic; + continue; + } + // If the next available picture is force-IDR, encode it to start + // a new GOP immediately. + if (pic->force_idr) + break; + if (b_counter == ctx->b_per_p) + break; + // If this picture ends a closed GOP or starts a new GOP then it + // needs to be in the top layer. + if (ctx->gop_counter + b_counter + closed_gop_end >= ctx->gop_size) + break; + // If the picture after this one is force-IDR, we need to encode + // this one in the top layer. + if (next && next->force_idr) + break; + ++b_counter; + } + + // At the end of the stream the last picture must be in the top layer. + if (!pic && ctx->end_of_stream) { + --b_counter; + pic = ctx->pic_end; + if (pic->encode_complete) + return AVERROR_EOF; + else if (pic->encode_issued) + return AVERROR(EAGAIN); + } + + if (!pic) { + av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " + "need more input for reference pictures.\n"); + return AVERROR(EAGAIN); + } + if (ctx->input_order <= ctx->decode_delay && !ctx->end_of_stream) { + av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " + "need more input for timestamps.\n"); + return AVERROR(EAGAIN); + } + + if (pic->force_idr) { + av_log(avctx, AV_LOG_DEBUG, "Pick forced IDR-picture to " + "encode next.\n"); + pic->type = PICTURE_TYPE_IDR; + ctx->idr_counter = 1; + ctx->gop_counter = 1; + + } else if (ctx->gop_counter + b_counter >= ctx->gop_size) { + if (ctx->idr_counter == ctx->gop_per_idr) { + av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP IDR-picture to " + "encode next.\n"); + pic->type = PICTURE_TYPE_IDR; + ctx->idr_counter = 1; + } else { + av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP I-picture to " + "encode next.\n"); + pic->type = PICTURE_TYPE_I; + ++ctx->idr_counter; + } + ctx->gop_counter = 1; + + } else { + if (ctx->gop_counter + b_counter + closed_gop_end == ctx->gop_size) { + av_log(avctx, AV_LOG_DEBUG, "Pick group-end P-picture to " + "encode next.\n"); + } else { + av_log(avctx, AV_LOG_DEBUG, "Pick normal P-picture to " + "encode next.\n"); + } + pic->type = PICTURE_TYPE_P; + av_assert0(start); + ctx->gop_counter += 1 + b_counter; + } + pic->is_reference = 1; + *pic_out = pic; + + hw_base_encode_add_ref(avctx, pic, pic, 0, 1, 0); + if (pic->type != PICTURE_TYPE_IDR) { + // TODO: apply both previous and forward multi reference for all vaapi encoders. + // And L0/L1 reference frame number can be set dynamically through query + // VAConfigAttribEncMaxRefFrames attribute. + if (avctx->codec_id == AV_CODEC_ID_AV1) { + for (i = 0; i < ctx->nb_next_prev; i++) + hw_base_encode_add_ref(avctx, pic, ctx->next_prev[i], + pic->type == PICTURE_TYPE_P, + b_counter > 0, 0); + } else + hw_base_encode_add_ref(avctx, pic, start, + pic->type == PICTURE_TYPE_P, + b_counter > 0, 0); + + hw_base_encode_add_ref(avctx, pic, ctx->next_prev[ctx->nb_next_prev - 1], 0, 0, 1); + } + + if (b_counter > 0) { + hw_base_encode_set_b_pictures(avctx, start, pic, pic, 1, + &prev); + } else { + prev = pic; + } + hw_base_encode_add_next_prev(avctx, prev); + + return 0; +} + +static int hw_base_encode_clear_old(AVCodecContext *avctx) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *prev, *next; + + av_assert0(ctx->pic_start); + + // Remove direct references once each picture is complete. + for (pic = ctx->pic_start; pic; pic = pic->next) { + if (pic->encode_complete && pic->next) + hw_base_encode_remove_refs(avctx, pic, 0); + } + + // Remove indirect references once a picture has no direct references. + for (pic = ctx->pic_start; pic; pic = pic->next) { + if (pic->encode_complete && pic->ref_count[0] == 0) + hw_base_encode_remove_refs(avctx, pic, 1); + } + + // Clear out all complete pictures with no remaining references. + prev = NULL; + for (pic = ctx->pic_start; pic; pic = next) { + next = pic->next; + if (pic->encode_complete && pic->ref_count[1] == 0) { + av_assert0(pic->ref_removed[0] && pic->ref_removed[1]); + if (prev) + prev->next = next; + else + ctx->pic_start = next; + ctx->op->free(avctx, pic); + } else { + prev = pic; + } + } + + return 0; +} + +static int hw_base_encode_check_frame(AVCodecContext *avctx, + const AVFrame *frame) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + if ((frame->crop_top || frame->crop_bottom || + frame->crop_left || frame->crop_right) && !ctx->crop_warned) { + av_log(avctx, AV_LOG_WARNING, "Cropping information on input " + "frames ignored due to lack of API support.\n"); + ctx->crop_warned = 1; + } + + if (!ctx->roi_allowed) { + AVFrameSideData *sd = + av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); + + if (sd && !ctx->roi_warned) { + av_log(avctx, AV_LOG_WARNING, "ROI side data on input " + "frames ignored due to lack of driver support.\n"); + ctx->roi_warned = 1; + } + } + + return 0; +} + +static int hw_base_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic; + int err; + + if (frame) { + av_log(avctx, AV_LOG_DEBUG, "Input frame: %ux%u (%"PRId64").\n", + frame->width, frame->height, frame->pts); + + err = hw_base_encode_check_frame(avctx, frame); + if (err < 0) + return err; + + pic = ctx->op->alloc(avctx, frame); + if (!pic) + return AVERROR(ENOMEM); + + pic->input_image = av_frame_alloc(); + if (!pic->input_image) { + err = AVERROR(ENOMEM); + goto fail; + } + + pic->recon_image = av_frame_alloc(); + if (!pic->recon_image) { + err = AVERROR(ENOMEM); + goto fail; + } + + if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I) + pic->force_idr = 1; + + pic->pts = frame->pts; + pic->duration = frame->duration; + + if (avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { + err = av_buffer_replace(&pic->opaque_ref, frame->opaque_ref); + if (err < 0) + goto fail; + + pic->opaque = frame->opaque; + } + + av_frame_move_ref(pic->input_image, frame); + + if (ctx->input_order == 0) + ctx->first_pts = pic->pts; + if (ctx->input_order == ctx->decode_delay) + ctx->dts_pts_diff = pic->pts - ctx->first_pts; + if (ctx->output_delay > 0) + ctx->ts_ring[ctx->input_order % + (3 * ctx->output_delay + ctx->async_depth)] = pic->pts; + + pic->display_order = ctx->input_order; + ++ctx->input_order; + + if (ctx->pic_start) { + ctx->pic_end->next = pic; + ctx->pic_end = pic; + } else { + ctx->pic_start = pic; + ctx->pic_end = pic; + } + + } else { + ctx->end_of_stream = 1; + + // Fix timestamps if we hit end-of-stream before the initial decode + // delay has elapsed. + if (ctx->input_order < ctx->decode_delay) + ctx->dts_pts_diff = ctx->pic_end->pts - ctx->first_pts; + } + + return 0; + +fail: + ctx->op->free(avctx, pic); + return err; +} + +int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic = NULL; + AVFrame *frame = ctx->frame; + int err; + + av_assert0(ctx->op && ctx->op->alloc && ctx->op->issue && + ctx->op->output && ctx->op->free); + +start: + /** if no B frame before repeat P frame, sent repeat P frame out. */ + if (ctx->tail_pkt->size) { + for (HWBaseEncodePicture *tmp = ctx->pic_start; tmp; tmp = tmp->next) { + if (tmp->type == PICTURE_TYPE_B && tmp->pts < ctx->tail_pkt->pts) + break; + else if (!tmp->next) { + av_packet_move_ref(pkt, ctx->tail_pkt); + goto end; + } + } + } + + err = ff_encode_get_frame(avctx, frame); + if (err < 0 && err != AVERROR_EOF) + return err; + + if (err == AVERROR_EOF) + frame = NULL; + + err = hw_base_encode_send_frame(avctx, frame); + if (err < 0) + return err; + + if (!ctx->pic_start) { + if (ctx->end_of_stream) + return AVERROR_EOF; + else + return AVERROR(EAGAIN); + } + + if (ctx->async_encode) { + if (av_fifo_can_write(ctx->encode_fifo)) { + err = hw_base_encode_pick_next(avctx, &pic); + if (!err) { + av_assert0(pic); + pic->encode_order = ctx->encode_order + + av_fifo_can_read(ctx->encode_fifo); + err = ctx->op->issue(avctx, pic); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); + return err; + } + pic->encode_issued = 1; + av_fifo_write(ctx->encode_fifo, &pic, 1); + } + } + + if (!av_fifo_can_read(ctx->encode_fifo)) + return err; + + // More frames can be buffered + if (av_fifo_can_write(ctx->encode_fifo) && !ctx->end_of_stream) + return AVERROR(EAGAIN); + + av_fifo_read(ctx->encode_fifo, &pic, 1); + ctx->encode_order = pic->encode_order + 1; + } else { + err = hw_base_encode_pick_next(avctx, &pic); + if (err < 0) + return err; + av_assert0(pic); + + pic->encode_order = ctx->encode_order++; + + err = ctx->op->issue(avctx, pic); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); + return err; + } + + pic->encode_issued = 1; + } + + err = ctx->op->output(avctx, pic, pkt); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Output failed: %d.\n", err); + return err; + } + + ctx->output_order = pic->encode_order; + hw_base_encode_clear_old(avctx); + + /** loop to get an available pkt in encoder flushing. */ + if (ctx->end_of_stream && !pkt->size) + goto start; + +end: + if (pkt->size) + av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " + "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); + + return 0; +} diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index a578db8c06..2667ae61cd 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -19,6 +19,8 @@ #ifndef AVCODEC_HW_BASE_ENCODE_H #define AVCODEC_HW_BASE_ENCODE_H +#include "libavutil/fifo.h" + #define MAX_DPB_SIZE 16 #define MAX_PICTURE_REFERENCES 2 #define MAX_REORDER_DELAY 16 @@ -53,13 +55,135 @@ enum { FLAG_NON_IDR_KEY_PICTURES = 1 << 5, }; +typedef struct HWBaseEncodePicture { + struct HWBaseEncodePicture *next; + + int64_t display_order; + int64_t encode_order; + int64_t pts; + int64_t duration; + int force_idr; + + void *opaque; + AVBufferRef *opaque_ref; + + int type; + int b_depth; + int encode_issued; + int encode_complete; + + AVFrame *input_image; + AVFrame *recon_image; + + void *priv_data; + + // Whether this picture is a reference picture. + int is_reference; + + // The contents of the DPB after this picture has been decoded. + // This will contain the picture itself if it is a reference picture, + // but not if it isn't. + int nb_dpb_pics; + struct HWBaseEncodePicture *dpb[MAX_DPB_SIZE]; + // The reference pictures used in decoding this picture. If they are + // used by later pictures they will also appear in the DPB. ref[0][] for + // previous reference frames. ref[1][] for future reference frames. + int nb_refs[MAX_REFERENCE_LIST_NUM]; + struct HWBaseEncodePicture *refs[MAX_REFERENCE_LIST_NUM][MAX_PICTURE_REFERENCES]; + // The previous reference picture in encode order. Must be in at least + // one of the reference list and DPB list. + struct HWBaseEncodePicture *prev; + // Reference count for other pictures referring to this one through + // the above pointers, directly from incomplete pictures and indirectly + // through completed pictures. + int ref_count[2]; + int ref_removed[2]; +} HWBaseEncodePicture; + +typedef struct HWEncodePictureOperation { + // Alloc memory for the picture structure and initialize the API-specific internals + // based of the given frame. + HWBaseEncodePicture * (*alloc)(AVCodecContext *avctx, const AVFrame *frame); + // Issue the picture structure, which will send the frame surface to HW Encode API. + int (*issue)(AVCodecContext *avctx, const HWBaseEncodePicture *base_pic); + // Get the output AVPacket. + int (*output)(AVCodecContext *avctx, const HWBaseEncodePicture *base_pic, AVPacket *pkt); + // Free the picture structure. + int (*free)(AVCodecContext *avctx, HWBaseEncodePicture *base_pic); +} HWEncodePictureOperation; + typedef struct HWBaseEncodeContext { const AVClass *class; + // Hardware-specific hooks. + const struct HWEncodePictureOperation *op; + + // Current encoding window, in display (input) order. + HWBaseEncodePicture *pic_start, *pic_end; + // The next picture to use as the previous reference picture in + // encoding order. Order from small to large in encoding order. + HWBaseEncodePicture *next_prev[MAX_PICTURE_REFERENCES]; + int nb_next_prev; + + // Next input order index (display order). + int64_t input_order; + // Number of frames that output is behind input. + int64_t output_delay; + // Next encode order index. + int64_t encode_order; + // Number of frames decode output will need to be delayed. + int64_t decode_delay; + // Next output order index (in encode order). + int64_t output_order; + + // Timestamp handling. + int64_t first_pts; + int64_t dts_pts_diff; + int64_t ts_ring[MAX_REORDER_DELAY * 3 + + MAX_ASYNC_DEPTH]; + + // Frame type decision. + int gop_size; + int closed_gop; + int gop_per_idr; + int p_per_i; + int max_b_depth; + int b_per_p; + int force_idr; + int idr_counter; + int gop_counter; + int end_of_stream; + int p_to_gpb; + + // Whether the driver supports ROI at all. + int roi_allowed; + + // The encoder does not support cropping information, so warn about + // it the first time we encounter any nonzero crop fields. + int crop_warned; + // If the driver does not support ROI then warn the first time we + // encounter a frame with ROI side data. + int roi_warned; + + // The frame to be filled with data. + AVFrame *frame; + + // Whether the HW supports sync buffer function. + // If supported, encode_fifo/async_depth will be used together. + // Used for output buffer synchronization. + int async_encode; + + // Store buffered pic. + AVFifo *encode_fifo; // Max number of frame buffered in encoder. int async_depth; + + /** Tail data of a pic, now only used for av1 repeat frame header. */ + AVPacket *tail_pkt; } HWBaseEncodeContext; +int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); + #define HW_BASE_ENCODE_COMMON_OPTIONS \ { "async_depth", "Maximum processing parallelism. " \ "Increase this to improve single channel performance.", \ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 227cccae64..c4bb93c520 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -138,22 +138,26 @@ static int vaapi_encode_make_misc_param_buffer(AVCodecContext *avctx, static int vaapi_encode_wait(AVCodecContext *avctx, VAAPIEncodePicture *pic) { +#if VA_CHECK_VERSION(1, 9, 0) + HWBaseEncodeContext *base_ctx = avctx->priv_data; +#endif VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *base_pic = &pic->base; VAStatus vas; - av_assert0(pic->encode_issued); + av_assert0(base_pic->encode_issued); - if (pic->encode_complete) { + if (base_pic->encode_complete) { // Already waited for this picture. return 0; } av_log(avctx, AV_LOG_DEBUG, "Sync to pic %"PRId64"/%"PRId64" " - "(input surface %#x).\n", pic->display_order, - pic->encode_order, pic->input_surface); + "(input surface %#x).\n", base_pic->display_order, + base_pic->encode_order, pic->input_surface); #if VA_CHECK_VERSION(1, 9, 0) - if (ctx->has_sync_buffer_func) { + if (base_ctx->async_encode) { vas = vaSyncBuffer(ctx->hwctx->display, pic->output_buffer, VA_TIMEOUT_INFINITE); @@ -174,9 +178,9 @@ static int vaapi_encode_wait(AVCodecContext *avctx, } // Input is definitely finished with now. - av_frame_free(&pic->input_image); + av_frame_free(&base_pic->input_image); - pic->encode_complete = 1; + base_pic->encode_complete = 1; return 0; } @@ -263,9 +267,11 @@ static int vaapi_encode_make_tile_slice(AVCodecContext *avctx, } static int vaapi_encode_issue(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + const HWBaseEncodePicture *base_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodePicture *pic = (VAAPIEncodePicture*)base_pic; VAAPIEncodeSlice *slice; VAStatus vas; int err, i; @@ -274,52 +280,46 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_unused AVFrameSideData *sd; av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " - "as type %s.\n", pic->display_order, pic->encode_order, - ff_hw_base_encode_get_pictype_name(pic->type)); - if (pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0) { + "as type %s.\n", base_pic->display_order, base_pic->encode_order, + ff_hw_base_encode_get_pictype_name(base_pic->type)); + if (base_pic->nb_refs[0] == 0 && base_pic->nb_refs[1] == 0) { av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); } else { av_log(avctx, AV_LOG_DEBUG, "L0 refers to"); - for (i = 0; i < pic->nb_refs[0]; i++) { + for (i = 0; i < base_pic->nb_refs[0]; i++) { av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, - pic->refs[0][i]->display_order, pic->refs[0][i]->encode_order); + base_pic->refs[0][i]->display_order, base_pic->refs[0][i]->encode_order); } av_log(avctx, AV_LOG_DEBUG, ".\n"); - if (pic->nb_refs[1]) { + if (base_pic->nb_refs[1]) { av_log(avctx, AV_LOG_DEBUG, "L1 refers to"); - for (i = 0; i < pic->nb_refs[1]; i++) { + for (i = 0; i < base_pic->nb_refs[1]; i++) { av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, - pic->refs[1][i]->display_order, pic->refs[1][i]->encode_order); + base_pic->refs[1][i]->display_order, base_pic->refs[1][i]->encode_order); } av_log(avctx, AV_LOG_DEBUG, ".\n"); } } - av_assert0(!pic->encode_issued); - for (i = 0; i < pic->nb_refs[0]; i++) { - av_assert0(pic->refs[0][i]); - av_assert0(pic->refs[0][i]->encode_issued); + av_assert0(!base_pic->encode_issued); + for (i = 0; i < base_pic->nb_refs[0]; i++) { + av_assert0(base_pic->refs[0][i]); + av_assert0(base_pic->refs[0][i]->encode_issued); } - for (i = 0; i < pic->nb_refs[1]; i++) { - av_assert0(pic->refs[1][i]); - av_assert0(pic->refs[1][i]->encode_issued); + for (i = 0; i < base_pic->nb_refs[1]; i++) { + av_assert0(base_pic->refs[1][i]); + av_assert0(base_pic->refs[1][i]->encode_issued); } av_log(avctx, AV_LOG_DEBUG, "Input surface is %#x.\n", pic->input_surface); - pic->recon_image = av_frame_alloc(); - if (!pic->recon_image) { - err = AVERROR(ENOMEM); - goto fail; - } - - err = av_hwframe_get_buffer(ctx->recon_frames_ref, pic->recon_image, 0); + err = av_hwframe_get_buffer(ctx->recon_frames_ref, base_pic->recon_image, 0); if (err < 0) { err = AVERROR(ENOMEM); goto fail; } - pic->recon_surface = (VASurfaceID)(uintptr_t)pic->recon_image->data[3]; + pic->recon_surface = (VASurfaceID)(uintptr_t)base_pic->recon_image->data[3]; av_log(avctx, AV_LOG_DEBUG, "Recon surface is %#x.\n", pic->recon_surface); pic->output_buffer_ref = ff_refstruct_pool_get(ctx->output_buffer_pool); @@ -343,7 +343,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, pic->nb_param_buffers = 0; - if (pic->type == PICTURE_TYPE_IDR && ctx->codec->init_sequence_params) { + if (base_pic->type == PICTURE_TYPE_IDR && ctx->codec->init_sequence_params) { err = vaapi_encode_make_param_buffer(avctx, pic, VAEncSequenceParameterBufferType, ctx->codec_sequence_params, @@ -352,7 +352,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } - if (pic->type == PICTURE_TYPE_IDR) { + if (base_pic->type == PICTURE_TYPE_IDR) { for (i = 0; i < ctx->nb_global_params; i++) { err = vaapi_encode_make_misc_param_buffer(avctx, pic, ctx->global_params_type[i], @@ -389,7 +389,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } #endif - if (pic->type == PICTURE_TYPE_IDR) { + if (base_pic->type == PICTURE_TYPE_IDR) { if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && ctx->codec->write_sequence_header) { bit_len = 8 * sizeof(data); @@ -529,9 +529,9 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } #if VA_CHECK_VERSION(1, 0, 0) - sd = av_frame_get_side_data(pic->input_image, + sd = av_frame_get_side_data(base_pic->input_image, AV_FRAME_DATA_REGIONS_OF_INTEREST); - if (sd && ctx->roi_allowed) { + if (sd && base_ctx->roi_allowed) { const AVRegionOfInterest *roi; uint32_t roi_size; VAEncMiscParameterBufferROI param_roi; @@ -542,11 +542,11 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_assert0(roi_size && sd->size % roi_size == 0); nb_roi = sd->size / roi_size; if (nb_roi > ctx->roi_max_regions) { - if (!ctx->roi_warned) { + if (!base_ctx->roi_warned) { av_log(avctx, AV_LOG_WARNING, "More ROIs set than " "supported by driver (%d > %d).\n", nb_roi, ctx->roi_max_regions); - ctx->roi_warned = 1; + base_ctx->roi_warned = 1; } nb_roi = ctx->roi_max_regions; } @@ -639,8 +639,6 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } } - pic->encode_issued = 1; - return 0; fail_with_picture: @@ -657,14 +655,13 @@ fail_at_end: av_freep(&pic->param_buffers); av_freep(&pic->slices); av_freep(&pic->roi); - av_frame_free(&pic->recon_image); ff_refstruct_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; return err; } static int vaapi_encode_set_output_property(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + HWBaseEncodePicture *pic, AVPacket *pkt) { HWBaseEncodeContext *base_ctx = avctx->priv_data; @@ -689,16 +686,16 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, return 0; } - if (ctx->output_delay == 0) { + if (base_ctx->output_delay == 0) { pkt->dts = pkt->pts; - } else if (pic->encode_order < ctx->decode_delay) { - if (ctx->ts_ring[pic->encode_order] < INT64_MIN + ctx->dts_pts_diff) + } else if (pic->encode_order < base_ctx->decode_delay) { + if (base_ctx->ts_ring[pic->encode_order] < INT64_MIN + base_ctx->dts_pts_diff) pkt->dts = INT64_MIN; else - pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; + pkt->dts = base_ctx->ts_ring[pic->encode_order] - base_ctx->dts_pts_diff; } else { - pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % - (3 * ctx->output_delay + base_ctx->async_depth)]; + pkt->dts = base_ctx->ts_ring[(pic->encode_order - base_ctx->decode_delay) % + (3 * base_ctx->output_delay + base_ctx->async_depth)]; } return 0; @@ -817,9 +814,11 @@ end: } static int vaapi_encode_output(AVCodecContext *avctx, - VAAPIEncodePicture *pic, AVPacket *pkt) + const HWBaseEncodePicture *base_pic, AVPacket *pkt) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodePicture *pic = (VAAPIEncodePicture*)base_pic; AVPacket *pkt_ptr = pkt; int err; @@ -832,17 +831,17 @@ static int vaapi_encode_output(AVCodecContext *avctx, ctx->coded_buffer_ref = ff_refstruct_ref(pic->output_buffer_ref); if (pic->tail_size) { - if (ctx->tail_pkt->size) { + if (base_ctx->tail_pkt->size) { err = AVERROR_BUG; goto end; } - err = ff_get_encode_buffer(avctx, ctx->tail_pkt, pic->tail_size, 0); + err = ff_get_encode_buffer(avctx, base_ctx->tail_pkt, pic->tail_size, 0); if (err < 0) goto end; - memcpy(ctx->tail_pkt->data, pic->tail_data, pic->tail_size); - pkt_ptr = ctx->tail_pkt; + memcpy(base_ctx->tail_pkt->data, pic->tail_data, pic->tail_size); + pkt_ptr = base_ctx->tail_pkt; } } else { err = vaapi_encode_get_coded_data(avctx, pic, pkt); @@ -851,9 +850,9 @@ static int vaapi_encode_output(AVCodecContext *avctx, } av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", - pic->display_order, pic->encode_order); + base_pic->display_order, base_pic->encode_order); - vaapi_encode_set_output_property(avctx, pic, pkt_ptr); + vaapi_encode_set_output_property(avctx, (HWBaseEncodePicture*)pic, pkt_ptr); end: ff_refstruct_unref(&pic->output_buffer_ref); @@ -864,12 +863,14 @@ end: static int vaapi_encode_discard(AVCodecContext *avctx, VAAPIEncodePicture *pic) { + HWBaseEncodePicture *base_pic = &pic->base; + vaapi_encode_wait(avctx, pic); if (pic->output_buffer_ref) { av_log(avctx, AV_LOG_DEBUG, "Discard output for pic " "%"PRId64"/%"PRId64".\n", - pic->display_order, pic->encode_order); + base_pic->display_order, base_pic->encode_order); ff_refstruct_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; @@ -878,8 +879,8 @@ static int vaapi_encode_discard(AVCodecContext *avctx, return 0; } -static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, - const AVFrame *frame) +static HWBaseEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, + const AVFrame *frame) { VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodePicture *pic; @@ -889,8 +890,8 @@ static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, return NULL; if (ctx->codec->picture_priv_data_size > 0) { - pic->priv_data = av_mallocz(ctx->codec->picture_priv_data_size); - if (!pic->priv_data) { + pic->base.priv_data = av_mallocz(ctx->codec->picture_priv_data_size); + if (!pic->base.priv_data) { av_freep(&pic); return NULL; } @@ -900,15 +901,16 @@ static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, pic->recon_surface = VA_INVALID_ID; pic->output_buffer = VA_INVALID_ID; - return pic; + return &pic->base; } static int vaapi_encode_free(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + HWBaseEncodePicture *base_pic) { + VAAPIEncodePicture *pic = (VAAPIEncodePicture*)base_pic; int i; - if (pic->encode_issued) + if (base_pic->encode_issued) vaapi_encode_discard(avctx, pic); if (pic->slices) { @@ -916,17 +918,17 @@ static int vaapi_encode_free(AVCodecContext *avctx, av_freep(&pic->slices[i].codec_slice_params); } - av_frame_free(&pic->input_image); - av_frame_free(&pic->recon_image); + av_frame_free(&base_pic->input_image); + av_frame_free(&base_pic->recon_image); - av_buffer_unref(&pic->opaque_ref); + av_buffer_unref(&base_pic->opaque_ref); av_freep(&pic->param_buffers); av_freep(&pic->slices); // Output buffer should already be destroyed. av_assert0(pic->output_buffer == VA_INVALID_ID); - av_freep(&pic->priv_data); + av_freep(&base_pic->priv_data); av_freep(&pic->codec_picture_params); av_freep(&pic->roi); @@ -935,564 +937,6 @@ static int vaapi_encode_free(AVCodecContext *avctx, return 0; } -static void vaapi_encode_add_ref(AVCodecContext *avctx, - VAAPIEncodePicture *pic, - VAAPIEncodePicture *target, - int is_ref, int in_dpb, int prev) -{ - int refs = 0; - - if (is_ref) { - av_assert0(pic != target); - av_assert0(pic->nb_refs[0] < MAX_PICTURE_REFERENCES && - pic->nb_refs[1] < MAX_PICTURE_REFERENCES); - if (target->display_order < pic->display_order) - pic->refs[0][pic->nb_refs[0]++] = target; - else - pic->refs[1][pic->nb_refs[1]++] = target; - ++refs; - } - - if (in_dpb) { - av_assert0(pic->nb_dpb_pics < MAX_DPB_SIZE); - pic->dpb[pic->nb_dpb_pics++] = target; - ++refs; - } - - if (prev) { - av_assert0(!pic->prev); - pic->prev = target; - ++refs; - } - - target->ref_count[0] += refs; - target->ref_count[1] += refs; -} - -static void vaapi_encode_remove_refs(AVCodecContext *avctx, - VAAPIEncodePicture *pic, - int level) -{ - int i; - - if (pic->ref_removed[level]) - return; - - for (i = 0; i < pic->nb_refs[0]; i++) { - av_assert0(pic->refs[0][i]); - --pic->refs[0][i]->ref_count[level]; - av_assert0(pic->refs[0][i]->ref_count[level] >= 0); - } - - for (i = 0; i < pic->nb_refs[1]; i++) { - av_assert0(pic->refs[1][i]); - --pic->refs[1][i]->ref_count[level]; - av_assert0(pic->refs[1][i]->ref_count[level] >= 0); - } - - for (i = 0; i < pic->nb_dpb_pics; i++) { - av_assert0(pic->dpb[i]); - --pic->dpb[i]->ref_count[level]; - av_assert0(pic->dpb[i]->ref_count[level] >= 0); - } - - av_assert0(pic->prev || pic->type == PICTURE_TYPE_IDR); - if (pic->prev) { - --pic->prev->ref_count[level]; - av_assert0(pic->prev->ref_count[level] >= 0); - } - - pic->ref_removed[level] = 1; -} - -static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, - VAAPIEncodePicture *start, - VAAPIEncodePicture *end, - VAAPIEncodePicture *prev, - int current_depth, - VAAPIEncodePicture **last) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic, *next, *ref; - int i, len; - - av_assert0(start && end && start != end && start->next != end); - - // If we are at the maximum depth then encode all pictures as - // non-referenced B-pictures. Also do this if there is exactly one - // picture left, since there will be nothing to reference it. - if (current_depth == ctx->max_b_depth || start->next->next == end) { - for (pic = start->next; pic; pic = pic->next) { - if (pic == end) - break; - pic->type = PICTURE_TYPE_B; - pic->b_depth = current_depth; - - vaapi_encode_add_ref(avctx, pic, start, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); - - for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) - vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); - } - *last = prev; - - } else { - // Split the current list at the midpoint with a referenced - // B-picture, then descend into each side separately. - len = 0; - for (pic = start->next; pic != end; pic = pic->next) - ++len; - for (pic = start->next, i = 1; 2 * i < len; pic = pic->next, i++); - - pic->type = PICTURE_TYPE_B; - pic->b_depth = current_depth; - - pic->is_reference = 1; - - vaapi_encode_add_ref(avctx, pic, pic, 0, 1, 0); - vaapi_encode_add_ref(avctx, pic, start, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); - - for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) - vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); - - if (i > 1) - vaapi_encode_set_b_pictures(avctx, start, pic, pic, - current_depth + 1, &next); - else - next = pic; - - vaapi_encode_set_b_pictures(avctx, pic, end, next, - current_depth + 1, last); - } -} - -static void vaapi_encode_add_next_prev(AVCodecContext *avctx, - VAAPIEncodePicture *pic) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - int i; - - if (!pic) - return; - - if (pic->type == PICTURE_TYPE_IDR) { - for (i = 0; i < ctx->nb_next_prev; i++) { - --ctx->next_prev[i]->ref_count[0]; - ctx->next_prev[i] = NULL; - } - ctx->next_prev[0] = pic; - ++pic->ref_count[0]; - ctx->nb_next_prev = 1; - - return; - } - - if (ctx->nb_next_prev < MAX_PICTURE_REFERENCES) { - ctx->next_prev[ctx->nb_next_prev++] = pic; - ++pic->ref_count[0]; - } else { - --ctx->next_prev[0]->ref_count[0]; - for (i = 0; i < MAX_PICTURE_REFERENCES - 1; i++) - ctx->next_prev[i] = ctx->next_prev[i + 1]; - ctx->next_prev[i] = pic; - ++pic->ref_count[0]; - } -} - -static int vaapi_encode_pick_next(AVCodecContext *avctx, - VAAPIEncodePicture **pic_out) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic = NULL, *prev = NULL, *next, *start; - int i, b_counter, closed_gop_end; - - // If there are any B-frames already queued, the next one to encode - // is the earliest not-yet-issued frame for which all references are - // available. - for (pic = ctx->pic_start; pic; pic = pic->next) { - if (pic->encode_issued) - continue; - if (pic->type != PICTURE_TYPE_B) - continue; - for (i = 0; i < pic->nb_refs[0]; i++) { - if (!pic->refs[0][i]->encode_issued) - break; - } - if (i != pic->nb_refs[0]) - continue; - - for (i = 0; i < pic->nb_refs[1]; i++) { - if (!pic->refs[1][i]->encode_issued) - break; - } - if (i == pic->nb_refs[1]) - break; - } - - if (pic) { - av_log(avctx, AV_LOG_DEBUG, "Pick B-picture at depth %d to " - "encode next.\n", pic->b_depth); - *pic_out = pic; - return 0; - } - - // Find the B-per-Pth available picture to become the next picture - // on the top layer. - start = NULL; - b_counter = 0; - closed_gop_end = ctx->closed_gop || - ctx->idr_counter == ctx->gop_per_idr; - for (pic = ctx->pic_start; pic; pic = next) { - next = pic->next; - if (pic->encode_issued) { - start = pic; - continue; - } - // If the next available picture is force-IDR, encode it to start - // a new GOP immediately. - if (pic->force_idr) - break; - if (b_counter == ctx->b_per_p) - break; - // If this picture ends a closed GOP or starts a new GOP then it - // needs to be in the top layer. - if (ctx->gop_counter + b_counter + closed_gop_end >= ctx->gop_size) - break; - // If the picture after this one is force-IDR, we need to encode - // this one in the top layer. - if (next && next->force_idr) - break; - ++b_counter; - } - - // At the end of the stream the last picture must be in the top layer. - if (!pic && ctx->end_of_stream) { - --b_counter; - pic = ctx->pic_end; - if (pic->encode_complete) - return AVERROR_EOF; - else if (pic->encode_issued) - return AVERROR(EAGAIN); - } - - if (!pic) { - av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " - "need more input for reference pictures.\n"); - return AVERROR(EAGAIN); - } - if (ctx->input_order <= ctx->decode_delay && !ctx->end_of_stream) { - av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " - "need more input for timestamps.\n"); - return AVERROR(EAGAIN); - } - - if (pic->force_idr) { - av_log(avctx, AV_LOG_DEBUG, "Pick forced IDR-picture to " - "encode next.\n"); - pic->type = PICTURE_TYPE_IDR; - ctx->idr_counter = 1; - ctx->gop_counter = 1; - - } else if (ctx->gop_counter + b_counter >= ctx->gop_size) { - if (ctx->idr_counter == ctx->gop_per_idr) { - av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP IDR-picture to " - "encode next.\n"); - pic->type = PICTURE_TYPE_IDR; - ctx->idr_counter = 1; - } else { - av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP I-picture to " - "encode next.\n"); - pic->type = PICTURE_TYPE_I; - ++ctx->idr_counter; - } - ctx->gop_counter = 1; - - } else { - if (ctx->gop_counter + b_counter + closed_gop_end == ctx->gop_size) { - av_log(avctx, AV_LOG_DEBUG, "Pick group-end P-picture to " - "encode next.\n"); - } else { - av_log(avctx, AV_LOG_DEBUG, "Pick normal P-picture to " - "encode next.\n"); - } - pic->type = PICTURE_TYPE_P; - av_assert0(start); - ctx->gop_counter += 1 + b_counter; - } - pic->is_reference = 1; - *pic_out = pic; - - vaapi_encode_add_ref(avctx, pic, pic, 0, 1, 0); - if (pic->type != PICTURE_TYPE_IDR) { - // TODO: apply both previous and forward multi reference for all vaapi encoders. - // And L0/L1 reference frame number can be set dynamically through query - // VAConfigAttribEncMaxRefFrames attribute. - if (avctx->codec_id == AV_CODEC_ID_AV1) { - for (i = 0; i < ctx->nb_next_prev; i++) - vaapi_encode_add_ref(avctx, pic, ctx->next_prev[i], - pic->type == PICTURE_TYPE_P, - b_counter > 0, 0); - } else - vaapi_encode_add_ref(avctx, pic, start, - pic->type == PICTURE_TYPE_P, - b_counter > 0, 0); - - vaapi_encode_add_ref(avctx, pic, ctx->next_prev[ctx->nb_next_prev - 1], 0, 0, 1); - } - - if (b_counter > 0) { - vaapi_encode_set_b_pictures(avctx, start, pic, pic, 1, - &prev); - } else { - prev = pic; - } - vaapi_encode_add_next_prev(avctx, prev); - - return 0; -} - -static int vaapi_encode_clear_old(AVCodecContext *avctx) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic, *prev, *next; - - av_assert0(ctx->pic_start); - - // Remove direct references once each picture is complete. - for (pic = ctx->pic_start; pic; pic = pic->next) { - if (pic->encode_complete && pic->next) - vaapi_encode_remove_refs(avctx, pic, 0); - } - - // Remove indirect references once a picture has no direct references. - for (pic = ctx->pic_start; pic; pic = pic->next) { - if (pic->encode_complete && pic->ref_count[0] == 0) - vaapi_encode_remove_refs(avctx, pic, 1); - } - - // Clear out all complete pictures with no remaining references. - prev = NULL; - for (pic = ctx->pic_start; pic; pic = next) { - next = pic->next; - if (pic->encode_complete && pic->ref_count[1] == 0) { - av_assert0(pic->ref_removed[0] && pic->ref_removed[1]); - if (prev) - prev->next = next; - else - ctx->pic_start = next; - vaapi_encode_free(avctx, pic); - } else { - prev = pic; - } - } - - return 0; -} - -static int vaapi_encode_check_frame(AVCodecContext *avctx, - const AVFrame *frame) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - - if ((frame->crop_top || frame->crop_bottom || - frame->crop_left || frame->crop_right) && !ctx->crop_warned) { - av_log(avctx, AV_LOG_WARNING, "Cropping information on input " - "frames ignored due to lack of API support.\n"); - ctx->crop_warned = 1; - } - - if (!ctx->roi_allowed) { - AVFrameSideData *sd = - av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); - - if (sd && !ctx->roi_warned) { - av_log(avctx, AV_LOG_WARNING, "ROI side data on input " - "frames ignored due to lack of driver support.\n"); - ctx->roi_warned = 1; - } - } - - return 0; -} - -static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) -{ - HWBaseEncodeContext *base_ctx = avctx->priv_data; - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic; - int err; - - if (frame) { - av_log(avctx, AV_LOG_DEBUG, "Input frame: %ux%u (%"PRId64").\n", - frame->width, frame->height, frame->pts); - - err = vaapi_encode_check_frame(avctx, frame); - if (err < 0) - return err; - - pic = vaapi_encode_alloc(avctx, frame); - if (!pic) - return AVERROR(ENOMEM); - - pic->input_image = av_frame_alloc(); - if (!pic->input_image) { - err = AVERROR(ENOMEM); - goto fail; - } - - if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I) - pic->force_idr = 1; - - pic->pts = frame->pts; - pic->duration = frame->duration; - - if (avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { - err = av_buffer_replace(&pic->opaque_ref, frame->opaque_ref); - if (err < 0) - goto fail; - - pic->opaque = frame->opaque; - } - - av_frame_move_ref(pic->input_image, frame); - - if (ctx->input_order == 0) - ctx->first_pts = pic->pts; - if (ctx->input_order == ctx->decode_delay) - ctx->dts_pts_diff = pic->pts - ctx->first_pts; - if (ctx->output_delay > 0) - ctx->ts_ring[ctx->input_order % - (3 * ctx->output_delay + base_ctx->async_depth)] = pic->pts; - - pic->display_order = ctx->input_order; - ++ctx->input_order; - - if (ctx->pic_start) { - ctx->pic_end->next = pic; - ctx->pic_end = pic; - } else { - ctx->pic_start = pic; - ctx->pic_end = pic; - } - - } else { - ctx->end_of_stream = 1; - - // Fix timestamps if we hit end-of-stream before the initial decode - // delay has elapsed. - if (ctx->input_order < ctx->decode_delay) - ctx->dts_pts_diff = ctx->pic_end->pts - ctx->first_pts; - } - - return 0; - -fail: - vaapi_encode_free(avctx, pic); - return err; -} - -int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic = NULL; - AVFrame *frame = ctx->frame; - int err; - -start: - /** if no B frame before repeat P frame, sent repeat P frame out. */ - if (ctx->tail_pkt->size) { - for (VAAPIEncodePicture *tmp = ctx->pic_start; tmp; tmp = tmp->next) { - if (tmp->type == PICTURE_TYPE_B && tmp->pts < ctx->tail_pkt->pts) - break; - else if (!tmp->next) { - av_packet_move_ref(pkt, ctx->tail_pkt); - goto end; - } - } - } - - err = ff_encode_get_frame(avctx, frame); - if (err < 0 && err != AVERROR_EOF) - return err; - - if (err == AVERROR_EOF) - frame = NULL; - - err = vaapi_encode_send_frame(avctx, frame); - if (err < 0) - return err; - - if (!ctx->pic_start) { - if (ctx->end_of_stream) - return AVERROR_EOF; - else - return AVERROR(EAGAIN); - } - - if (ctx->has_sync_buffer_func) { - if (av_fifo_can_write(ctx->encode_fifo)) { - err = vaapi_encode_pick_next(avctx, &pic); - if (!err) { - av_assert0(pic); - pic->encode_order = ctx->encode_order + - av_fifo_can_read(ctx->encode_fifo); - err = vaapi_encode_issue(avctx, pic); - if (err < 0) { - av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); - return err; - } - av_fifo_write(ctx->encode_fifo, &pic, 1); - } - } - - if (!av_fifo_can_read(ctx->encode_fifo)) - return err; - - // More frames can be buffered - if (av_fifo_can_write(ctx->encode_fifo) && !ctx->end_of_stream) - return AVERROR(EAGAIN); - - av_fifo_read(ctx->encode_fifo, &pic, 1); - ctx->encode_order = pic->encode_order + 1; - } else { - err = vaapi_encode_pick_next(avctx, &pic); - if (err < 0) - return err; - av_assert0(pic); - - pic->encode_order = ctx->encode_order++; - - err = vaapi_encode_issue(avctx, pic); - if (err < 0) { - av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); - return err; - } - } - - err = vaapi_encode_output(avctx, pic, pkt); - if (err < 0) { - av_log(avctx, AV_LOG_ERROR, "Output failed: %d.\n", err); - return err; - } - - ctx->output_order = pic->encode_order; - vaapi_encode_clear_old(avctx); - - /** loop to get an available pkt in encoder flushing. */ - if (ctx->end_of_stream && !pkt->size) - goto start; - -end: - if (pkt->size) - av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " - "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); - - return 0; -} - static av_cold void vaapi_encode_add_global_param(AVCodecContext *avctx, int type, void *buffer, size_t size) { @@ -2188,7 +1632,8 @@ static av_cold int vaapi_encode_init_max_frame_size(AVCodecContext *avctx) static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; VAStatus vas; VAConfigAttrib attr = { VAConfigAttribEncMaxRefFrames }; uint32_t ref_l0, ref_l1; @@ -2211,7 +1656,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) ref_l1 = attr.value >> 16 & 0xffff; } - ctx->p_to_gpb = 0; + base_ctx->p_to_gpb = 0; prediction_pre_only = 0; #if VA_CHECK_VERSION(1, 9, 0) @@ -2247,7 +1692,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) if (attr.value & VA_PREDICTION_DIRECTION_BI_NOT_EMPTY) { if (ref_l0 > 0 && ref_l1 > 0) { - ctx->p_to_gpb = 1; + base_ctx->p_to_gpb = 1; av_log(avctx, AV_LOG_VERBOSE, "Driver does not support P-frames, " "replacing them with B-frames.\n"); } @@ -2259,7 +1704,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) if (ctx->codec->flags & FLAG_INTRA_ONLY || avctx->gop_size <= 1) { av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); - ctx->gop_size = 1; + base_ctx->gop_size = 1; } else if (ref_l0 < 1) { av_log(avctx, AV_LOG_ERROR, "Driver does not support any " "reference frames.\n"); @@ -2267,41 +1712,41 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || ref_l1 < 1 || avctx->max_b_frames < 1 || prediction_pre_only) { - if (ctx->p_to_gpb) + if (base_ctx->p_to_gpb) av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); else av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); - ctx->gop_size = avctx->gop_size; - ctx->p_per_i = INT_MAX; - ctx->b_per_p = 0; + base_ctx->gop_size = avctx->gop_size; + base_ctx->p_per_i = INT_MAX; + base_ctx->b_per_p = 0; } else { - if (ctx->p_to_gpb) + if (base_ctx->p_to_gpb) av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); else av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); - ctx->gop_size = avctx->gop_size; - ctx->p_per_i = INT_MAX; - ctx->b_per_p = avctx->max_b_frames; + base_ctx->gop_size = avctx->gop_size; + base_ctx->p_per_i = INT_MAX; + base_ctx->b_per_p = avctx->max_b_frames; if (ctx->codec->flags & FLAG_B_PICTURE_REFERENCES) { - ctx->max_b_depth = FFMIN(ctx->desired_b_depth, - av_log2(ctx->b_per_p) + 1); + base_ctx->max_b_depth = FFMIN(ctx->desired_b_depth, + av_log2(base_ctx->b_per_p) + 1); } else { - ctx->max_b_depth = 1; + base_ctx->max_b_depth = 1; } } if (ctx->codec->flags & FLAG_NON_IDR_KEY_PICTURES) { - ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); - ctx->gop_per_idr = ctx->idr_interval + 1; + base_ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); + base_ctx->gop_per_idr = ctx->idr_interval + 1; } else { - ctx->closed_gop = 1; - ctx->gop_per_idr = 1; + base_ctx->closed_gop = 1; + base_ctx->gop_per_idr = 1; } return 0; @@ -2614,7 +2059,8 @@ static av_cold int vaapi_encode_init_quality(AVCodecContext *avctx) static av_cold int vaapi_encode_init_roi(AVCodecContext *avctx) { #if VA_CHECK_VERSION(1, 0, 0) - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; VAStatus vas; VAConfigAttrib attr = { VAConfigAttribEncROI }; @@ -2629,14 +2075,14 @@ static av_cold int vaapi_encode_init_roi(AVCodecContext *avctx) } if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { - ctx->roi_allowed = 0; + base_ctx->roi_allowed = 0; } else { VAConfigAttribValEncROI roi = { .value = attr.value, }; ctx->roi_max_regions = roi.bits.num_roi_regions; - ctx->roi_allowed = ctx->roi_max_regions > 0 && + base_ctx->roi_allowed = ctx->roi_max_regions > 0 && (ctx->va_rc_mode == VA_RC_CQP || roi.bits.roi_rc_qp_delta_support); } @@ -2771,6 +2217,16 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) return err; } +static const HWEncodePictureOperation vaapi_op = { + .alloc = &vaapi_encode_alloc, + + .issue = &vaapi_encode_issue, + + .output = &vaapi_encode_output, + + .free = &vaapi_encode_free, +}; + av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *base_ctx = avctx->priv_data; @@ -2782,10 +2238,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) ctx->va_config = VA_INVALID_ID; ctx->va_context = VA_INVALID_ID; + base_ctx->op = &vaapi_op; + /* If you add something that can fail above this av_frame_alloc(), * modify ff_vaapi_encode_close() accordingly. */ - ctx->frame = av_frame_alloc(); - if (!ctx->frame) { + base_ctx->frame = av_frame_alloc(); + if (!base_ctx->frame) { return AVERROR(ENOMEM); } @@ -2810,8 +2268,8 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) ctx->device = (AVHWDeviceContext*)ctx->device_ref->data; ctx->hwctx = ctx->device->hwctx; - ctx->tail_pkt = av_packet_alloc(); - if (!ctx->tail_pkt) { + base_ctx->tail_pkt = av_packet_alloc(); + if (!base_ctx->tail_pkt) { err = AVERROR(ENOMEM); goto fail; } @@ -2910,8 +2368,8 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } - ctx->output_delay = ctx->b_per_p; - ctx->decode_delay = ctx->max_b_depth; + base_ctx->output_delay = base_ctx->b_per_p; + base_ctx->decode_delay = base_ctx->max_b_depth; if (ctx->codec->sequence_params_size > 0) { ctx->codec_sequence_params = @@ -2966,11 +2424,11 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) // check vaSyncBuffer function vas = vaSyncBuffer(ctx->hwctx->display, VA_INVALID_ID, 0); if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) { - ctx->has_sync_buffer_func = 1; - ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, - sizeof(VAAPIEncodePicture *), - 0); - if (!ctx->encode_fifo) + base_ctx->async_encode = 1; + base_ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, + sizeof(VAAPIEncodePicture*), + 0); + if (!base_ctx->encode_fifo) return AVERROR(ENOMEM); } #endif @@ -2983,15 +2441,16 @@ fail: av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic, *next; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *next; /* We check ctx->frame to know whether ff_vaapi_encode_init() * has been called and va_config/va_context initialized. */ - if (!ctx->frame) + if (!base_ctx->frame) return 0; - for (pic = ctx->pic_start; pic; pic = next) { + for (pic = base_ctx->pic_start; pic; pic = next) { next = pic->next; vaapi_encode_free(avctx, pic); } @@ -3008,12 +2467,12 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) ctx->va_config = VA_INVALID_ID; } - av_frame_free(&ctx->frame); - av_packet_free(&ctx->tail_pkt); + av_frame_free(&base_ctx->frame); + av_packet_free(&base_ctx->tail_pkt); av_freep(&ctx->codec_sequence_params); av_freep(&ctx->codec_picture_params); - av_fifo_freep2(&ctx->encode_fifo); + av_fifo_freep2(&base_ctx->encode_fifo); av_buffer_unref(&ctx->recon_frames_ref); av_buffer_unref(&ctx->input_frames_ref); diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 02410c72ec..13ccad8e47 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -29,7 +29,6 @@ #include "libavutil/hwcontext.h" #include "libavutil/hwcontext_vaapi.h" -#include "libavutil/fifo.h" #include "avcodec.h" #include "hwconfig.h" @@ -64,16 +63,7 @@ typedef struct VAAPIEncodeSlice { } VAAPIEncodeSlice; typedef struct VAAPIEncodePicture { - struct VAAPIEncodePicture *next; - - int64_t display_order; - int64_t encode_order; - int64_t pts; - int64_t duration; - int force_idr; - - void *opaque; - AVBufferRef *opaque_ref; + HWBaseEncodePicture base; #if VA_CHECK_VERSION(1, 0, 0) // ROI regions. @@ -82,15 +72,7 @@ typedef struct VAAPIEncodePicture { void *roi; #endif - int type; - int b_depth; - int encode_issued; - int encode_complete; - - AVFrame *input_image; VASurfaceID input_surface; - - AVFrame *recon_image; VASurfaceID recon_surface; int nb_param_buffers; @@ -100,31 +82,8 @@ typedef struct VAAPIEncodePicture { VABufferID *output_buffer_ref; VABufferID output_buffer; - void *priv_data; void *codec_picture_params; - // Whether this picture is a reference picture. - int is_reference; - - // The contents of the DPB after this picture has been decoded. - // This will contain the picture itself if it is a reference picture, - // but not if it isn't. - int nb_dpb_pics; - struct VAAPIEncodePicture *dpb[MAX_DPB_SIZE]; - // The reference pictures used in decoding this picture. If they are - // used by later pictures they will also appear in the DPB. ref[0][] for - // previous reference frames. ref[1][] for future reference frames. - int nb_refs[MAX_REFERENCE_LIST_NUM]; - struct VAAPIEncodePicture *refs[MAX_REFERENCE_LIST_NUM][MAX_PICTURE_REFERENCES]; - // The previous reference picture in encode order. Must be in at least - // one of the reference list and DPB list. - struct VAAPIEncodePicture *prev; - // Reference count for other pictures referring to this one through - // the above pointers, directly from incomplete pictures and indirectly - // through completed pictures. - int ref_count[2]; - int ref_removed[2]; - int nb_slices; VAAPIEncodeSlice *slices; @@ -298,30 +257,6 @@ typedef struct VAAPIEncodeContext { // structure (VAEncPictureParameterBuffer*). void *codec_picture_params; - // Current encoding window, in display (input) order. - VAAPIEncodePicture *pic_start, *pic_end; - // The next picture to use as the previous reference picture in - // encoding order. Order from small to large in encoding order. - VAAPIEncodePicture *next_prev[MAX_PICTURE_REFERENCES]; - int nb_next_prev; - - // Next input order index (display order). - int64_t input_order; - // Number of frames that output is behind input. - int64_t output_delay; - // Next encode order index. - int64_t encode_order; - // Number of frames decode output will need to be delayed. - int64_t decode_delay; - // Next output order index (in encode order). - int64_t output_order; - - // Timestamp handling. - int64_t first_pts; - int64_t dts_pts_diff; - int64_t ts_ring[MAX_REORDER_DELAY * 3 + - MAX_ASYNC_DEPTH]; - // Slice structure. int slice_block_rows; int slice_block_cols; @@ -340,41 +275,12 @@ typedef struct VAAPIEncodeContext { // Location of the i-th tile row boundary. int row_bd[MAX_TILE_ROWS + 1]; - // Frame type decision. - int gop_size; - int closed_gop; - int gop_per_idr; - int p_per_i; - int max_b_depth; - int b_per_p; - int force_idr; - int idr_counter; - int gop_counter; - int end_of_stream; - int p_to_gpb; - - // Whether the driver supports ROI at all. - int roi_allowed; // Maximum number of regions supported by the driver. int roi_max_regions; // Quantisation range for offset calculations. Set by codec-specific // code, as it may change based on parameters. int roi_quant_range; - // The encoder does not support cropping information, so warn about - // it the first time we encounter any nonzero crop fields. - int crop_warned; - // If the driver does not support ROI then warn the first time we - // encounter a frame with ROI side data. - int roi_warned; - - AVFrame *frame; - - // Whether the driver support vaSyncBuffer - int has_sync_buffer_func; - // Store buffered pic - AVFifo *encode_fifo; - /** Head data for current output pkt, used only for AV1. */ //void *header_data; //size_t header_data_size; @@ -384,9 +290,6 @@ typedef struct VAAPIEncodeContext { * This is a RefStruct reference. */ VABufferID *coded_buffer_ref; - - /** Tail data of a pic, now only used for av1 repeat frame header. */ - AVPacket *tail_pkt; } VAAPIEncodeContext; typedef struct VAAPIEncodeType { @@ -468,9 +371,6 @@ typedef struct VAAPIEncodeType { char *data, size_t *data_len); } VAAPIEncodeType; - -int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); - int ff_vaapi_encode_init(AVCodecContext *avctx); int ff_vaapi_encode_close(AVCodecContext *avctx); diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index a5a475dca6..f12fe54cff 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -360,6 +360,7 @@ static int vaapi_encode_av1_write_sequence_header(AVCodecContext *avctx, static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeAV1Context *priv = avctx->priv_data; AV1RawOBU *sh_obu = &priv->sh; @@ -441,8 +442,8 @@ static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) vseq->seq_level_idx = sh->seq_level_idx[0]; vseq->seq_tier = sh->seq_tier[0]; vseq->order_hint_bits_minus_1 = sh->order_hint_bits_minus_1; - vseq->intra_period = ctx->gop_size; - vseq->ip_period = ctx->b_per_p + 1; + vseq->intra_period = base_ctx->gop_size; + vseq->ip_period = base_ctx->b_per_p + 1; vseq->seq_fields.bits.enable_order_hint = sh->enable_order_hint; @@ -465,16 +466,17 @@ end: } static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeAV1Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeAV1Picture *hpic = pic->priv_data; AV1RawOBU *fh_obu = &priv->fh; AV1RawFrameHeader *fh = &fh_obu->obu.frame.header; - VAEncPictureParameterBufferAV1 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferAV1 *vpic = vaapi_pic->codec_picture_params; CodedBitstreamFragment *obu = &priv->current_obu; - VAAPIEncodePicture *ref; + HWBaseEncodePicture *ref; VAAPIEncodeAV1Picture *href; int slot, i; int ret; @@ -482,8 +484,8 @@ static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, { 1, 0, 0, 0, -1, 0, -1, -1 }; memset(fh_obu, 0, sizeof(*fh_obu)); - pic->nb_slices = priv->tile_groups; - pic->non_independent_frame = pic->encode_order < pic->display_order; + vaapi_pic->nb_slices = priv->tile_groups; + vaapi_pic->non_independent_frame = pic->encode_order < pic->display_order; fh_obu->header.obu_type = AV1_OBU_FRAME_HEADER; fh_obu->header.obu_has_size_field = 1; @@ -601,8 +603,8 @@ static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, vpic->frame_width_minus_1 = fh->frame_width_minus_1; vpic->frame_height_minus_1 = fh->frame_height_minus_1; vpic->primary_ref_frame = fh->primary_ref_frame; - vpic->reconstructed_frame = pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->reconstructed_frame = vaapi_pic->recon_surface; + vpic->coded_buf = vaapi_pic->output_buffer; vpic->tile_cols = fh->tile_cols; vpic->tile_rows = fh->tile_rows; vpic->order_hint = fh->order_hint; @@ -630,12 +632,12 @@ static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { for (int j = 0; j < pic->nb_refs[i]; j++) { - VAAPIEncodePicture *ref_pic = pic->refs[i][j]; + HWBaseEncodePicture *ref_pic = pic->refs[i][j]; slot = ((VAAPIEncodeAV1Picture*)ref_pic->priv_data)->slot; av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); - vpic->reference_frames[slot] = ref_pic->recon_surface; + vpic->reference_frames[slot] = ((VAAPIEncodePicture *)ref_pic)->recon_surface; } } @@ -752,7 +754,7 @@ static int vaapi_encode_av1_init_slice_params(AVCodecContext *avctx, } static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, char *data, size_t *data_len) { VAAPIEncodeAV1Context *priv = avctx->priv_data; @@ -760,10 +762,11 @@ static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, CodedBitstreamAV1Context *cbctx = priv->cbc->priv_data; AV1RawOBU *fh_obu = &priv->fh; AV1RawFrameHeader *rep_fh = &fh_obu->obu.frame_header; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeAV1Picture *href; int ret = 0; - pic->tail_size = 0; + vaapi_pic->tail_size = 0; /** Pack repeat frame header. */ if (pic->display_order > pic->encode_order) { memset(fh_obu, 0, sizeof(*fh_obu)); @@ -785,11 +788,11 @@ static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, if (ret < 0) goto end; - ret = vaapi_encode_av1_write_obu(avctx, pic->tail_data, &pic->tail_size, obu); + ret = vaapi_encode_av1_write_obu(avctx, vaapi_pic->tail_data, &vaapi_pic->tail_size, obu); if (ret < 0) goto end; - pic->tail_size /= 8; + vaapi_pic->tail_size /= 8; } memcpy(data, &priv->fh_data, MAX_PARAM_BUFFER_SIZE * sizeof(char)); @@ -1038,7 +1041,7 @@ const FFCodec ff_av1_vaapi_encoder = { .p.id = AV_CODEC_ID_AV1, .priv_data_size = sizeof(VAAPIEncodeAV1Context), .init = &vaapi_encode_av1_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_av1_close, .p.priv_class = &vaapi_encode_av1_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index ebb1760cd3..9182483762 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -233,7 +233,7 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, goto fail; } if (priv->sei_needed & SEI_TIMING) { - if (pic->type == PICTURE_TYPE_IDR) { + if (pic->base.type == PICTURE_TYPE_IDR) { err = ff_cbs_sei_add_message(priv->cbc, au, 1, SEI_TYPE_BUFFERING_PERIOD, &priv->sei_buffering_period, NULL); @@ -295,6 +295,7 @@ fail: static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH264Context *priv = avctx->priv_data; H264RawSPS *sps = &priv->raw_sps; @@ -326,18 +327,18 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) sps->constraint_set1_flag = 1; if (avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) - sps->constraint_set3_flag = ctx->gop_size == 1; + sps->constraint_set3_flag = base_ctx->gop_size == 1; if (avctx->profile == AV_PROFILE_H264_MAIN || avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) { sps->constraint_set4_flag = 1; - sps->constraint_set5_flag = ctx->b_per_p == 0; + sps->constraint_set5_flag = base_ctx->b_per_p == 0; } - if (ctx->gop_size == 1) + if (base_ctx->gop_size == 1) priv->dpb_frames = 0; else - priv->dpb_frames = 1 + ctx->max_b_depth; + priv->dpb_frames = 1 + base_ctx->max_b_depth; if (avctx->level != AV_LEVEL_UNKNOWN) { sps->level_idc = avctx->level; @@ -374,7 +375,7 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) sps->bit_depth_chroma_minus8 = bit_depth - 8; sps->log2_max_frame_num_minus4 = 4; - sps->pic_order_cnt_type = ctx->max_b_depth ? 0 : 2; + sps->pic_order_cnt_type = base_ctx->max_b_depth ? 0 : 2; if (sps->pic_order_cnt_type == 0) { sps->log2_max_pic_order_cnt_lsb_minus4 = 4; } @@ -501,8 +502,8 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) sps->vui.motion_vectors_over_pic_boundaries_flag = 1; sps->vui.log2_max_mv_length_horizontal = 15; sps->vui.log2_max_mv_length_vertical = 15; - sps->vui.max_num_reorder_frames = ctx->max_b_depth; - sps->vui.max_dec_frame_buffering = ctx->max_b_depth + 1; + sps->vui.max_num_reorder_frames = base_ctx->max_b_depth; + sps->vui.max_dec_frame_buffering = base_ctx->max_b_depth + 1; pps->nal_unit_header.nal_ref_idc = 3; pps->nal_unit_header.nal_unit_type = H264_NAL_PPS; @@ -535,9 +536,9 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) *vseq = (VAEncSequenceParameterBufferH264) { .seq_parameter_set_id = sps->seq_parameter_set_id, .level_idc = sps->level_idc, - .intra_period = ctx->gop_size, - .intra_idr_period = ctx->gop_size, - .ip_period = ctx->b_per_p + 1, + .intra_period = base_ctx->gop_size, + .intra_idr_period = base_ctx->gop_size, + .ip_period = base_ctx->b_per_p + 1, .bits_per_second = ctx->va_bit_rate, .max_num_ref_frames = sps->max_num_ref_frames, @@ -619,14 +620,15 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeH264Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH264Picture *hpic = pic->priv_data; - VAAPIEncodePicture *prev = pic->prev; + HWBaseEncodePicture *prev = pic->prev; VAAPIEncodeH264Picture *hprev = prev ? prev->priv_data : NULL; - VAEncPictureParameterBufferH264 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferH264 *vpic = vaapi_pic->codec_picture_params; int i, j = 0; if (pic->type == PICTURE_TYPE_IDR) { @@ -662,7 +664,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, hpic->pic_order_cnt *= 2; } - hpic->dpb_delay = pic->display_order - pic->encode_order + ctx->max_b_depth; + hpic->dpb_delay = pic->display_order - pic->encode_order + base_ctx->max_b_depth; hpic->cpb_delay = pic->encode_order - hpic->last_idr_frame; if (priv->aud) { @@ -699,7 +701,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_recovery_point = (H264RawSEIRecoveryPoint) { .recovery_frame_cnt = 0, .exact_match_flag = 1, - .broken_link_flag = ctx->b_per_p > 0, + .broken_link_flag = base_ctx->b_per_p > 0, }; priv->sei_needed |= SEI_RECOVERY_POINT; @@ -722,7 +724,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, } vpic->CurrPic = (VAPictureH264) { - .picture_id = pic->recon_surface, + .picture_id = vaapi_pic->recon_surface, .frame_idx = hpic->frame_num, .flags = 0, .TopFieldOrderCnt = hpic->pic_order_cnt, @@ -730,14 +732,14 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, }; for (int k = 0; k < MAX_REFERENCE_LIST_NUM; k++) { for (i = 0; i < pic->nb_refs[k]; i++) { - VAAPIEncodePicture *ref = pic->refs[k][i]; + HWBaseEncodePicture *ref = pic->refs[k][i]; VAAPIEncodeH264Picture *href; av_assert0(ref && ref->encode_order < pic->encode_order); href = ref->priv_data; vpic->ReferenceFrames[j++] = (VAPictureH264) { - .picture_id = ref->recon_surface, + .picture_id = ((VAAPIEncodePicture *)ref)->recon_surface, .frame_idx = href->frame_num, .flags = VA_PICTURE_H264_SHORT_TERM_REFERENCE, .TopFieldOrderCnt = href->pic_order_cnt, @@ -753,7 +755,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, }; } - vpic->coded_buf = pic->output_buffer; + vpic->coded_buf = vaapi_pic->output_buffer; vpic->frame_num = hpic->frame_num; @@ -764,12 +766,13 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, } static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, VAAPIEncodePicture **rpl0, VAAPIEncodePicture **rpl1, int *rpl_size) { - VAAPIEncodePicture *prev; + HWBaseEncodePicture *pic = &vaapi_pic->base; + HWBaseEncodePicture *prev; VAAPIEncodeH264Picture *hp, *hn, *hc; int i, j, n = 0; @@ -783,17 +786,17 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, if (pic->type == PICTURE_TYPE_P) { for (j = n; j > 0; j--) { - hc = rpl0[j - 1]->priv_data; + hc = rpl0[j - 1]->base.priv_data; av_assert0(hc->frame_num != hn->frame_num); if (hc->frame_num > hn->frame_num) break; rpl0[j] = rpl0[j - 1]; } - rpl0[j] = prev->dpb[i]; + rpl0[j] = (VAAPIEncodePicture *)prev->dpb[i]; } else if (pic->type == PICTURE_TYPE_B) { for (j = n; j > 0; j--) { - hc = rpl0[j - 1]->priv_data; + hc = rpl0[j - 1]->base.priv_data; av_assert0(hc->pic_order_cnt != hp->pic_order_cnt); if (hc->pic_order_cnt < hp->pic_order_cnt) { if (hn->pic_order_cnt > hp->pic_order_cnt || @@ -805,10 +808,10 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, } rpl0[j] = rpl0[j - 1]; } - rpl0[j] = prev->dpb[i]; + rpl0[j] = (VAAPIEncodePicture *)prev->dpb[i]; for (j = n; j > 0; j--) { - hc = rpl1[j - 1]->priv_data; + hc = rpl1[j - 1]->base.priv_data; av_assert0(hc->pic_order_cnt != hp->pic_order_cnt); if (hc->pic_order_cnt > hp->pic_order_cnt) { if (hn->pic_order_cnt < hp->pic_order_cnt || @@ -820,7 +823,7 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, } rpl1[j] = rpl1[j - 1]; } - rpl1[j] = prev->dpb[i]; + rpl1[j] = (VAAPIEncodePicture *)prev->dpb[i]; } ++n; @@ -840,7 +843,7 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Default RefPicList0 for fn=%d/poc=%d:", hp->frame_num, hp->pic_order_cnt); for (i = 0; i < n; i++) { - hn = rpl0[i]->priv_data; + hn = rpl0[i]->base.priv_data; av_log(avctx, AV_LOG_DEBUG, " fn=%d/poc=%d", hn->frame_num, hn->pic_order_cnt); } @@ -850,7 +853,7 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Default RefPicList1 for fn=%d/poc=%d:", hp->frame_num, hp->pic_order_cnt); for (i = 0; i < n; i++) { - hn = rpl1[i]->priv_data; + hn = rpl1[i]->base.priv_data; av_log(avctx, AV_LOG_DEBUG, " fn=%d/poc=%d", hn->frame_num, hn->pic_order_cnt); } @@ -861,16 +864,17 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, } static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, VAAPIEncodeSlice *slice) { VAAPIEncodeH264Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH264Picture *hpic = pic->priv_data; - VAAPIEncodePicture *prev = pic->prev; + HWBaseEncodePicture *prev = pic->prev; H264RawSPS *sps = &priv->raw_sps; H264RawPPS *pps = &priv->raw_pps; H264RawSliceHeader *sh = &priv->raw_slice.header; - VAEncPictureParameterBufferH264 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferH264 *vpic = vaapi_pic->codec_picture_params; VAEncSliceParameterBufferH264 *vslice = slice->codec_slice_params; int i, j; @@ -903,7 +907,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, sh->slice_qp_delta = priv->fixed_qp_idr - (pps->pic_init_qp_minus26 + 26); if (pic->is_reference && pic->type != PICTURE_TYPE_IDR) { - VAAPIEncodePicture *discard_list[MAX_DPB_SIZE]; + HWBaseEncodePicture *discard_list[MAX_DPB_SIZE]; int discard = 0, keep = 0; // Discard everything which is in the DPB of the previous frame but @@ -944,14 +948,14 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, VAAPIEncodeH264Picture *href; int n; - vaapi_encode_h264_default_ref_pic_list(avctx, pic, + vaapi_encode_h264_default_ref_pic_list(avctx, vaapi_pic, def_l0, def_l1, &n); if (pic->type == PICTURE_TYPE_P) { int need_rplm = 0; for (i = 0; i < pic->nb_refs[0]; i++) { av_assert0(pic->refs[0][i]); - if (pic->refs[0][i] != def_l0[i]) + if (pic->refs[0][i] != (HWBaseEncodePicture *)def_l0[i]) need_rplm = 1; } @@ -982,7 +986,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, av_assert0(pic->refs[0][i]); href = pic->refs[0][i]->priv_data; av_assert0(href->pic_order_cnt < hpic->pic_order_cnt); - if (pic->refs[0][i] != def_l0[n0]) + if (pic->refs[0][i] != (HWBaseEncodePicture *)def_l0[n0]) need_rplm_l0 = 1; ++n0; } @@ -991,7 +995,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, av_assert0(pic->refs[1][i]); href = pic->refs[1][i]->priv_data; av_assert0(href->pic_order_cnt > hpic->pic_order_cnt); - if (pic->refs[1][i] != def_l1[n1]) + if (pic->refs[1][i] != (HWBaseEncodePicture *)def_l1[n1]) need_rplm_l1 = 1; ++n1; } @@ -1380,7 +1384,7 @@ const FFCodec ff_h264_vaapi_encoder = { .p.id = AV_CODEC_ID_H264, .priv_data_size = sizeof(VAAPIEncodeH264Context), .init = &vaapi_encode_h264_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_h264_close, .p.priv_class = &vaapi_encode_h264_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 77bd5e31af..1f9d9c56a4 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -259,6 +259,7 @@ fail: static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; H265RawVPS *vps = &priv->raw_vps; @@ -340,7 +341,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) ptl->general_max_420chroma_constraint_flag = chroma_format <= 1; ptl->general_max_monochrome_constraint_flag = chroma_format == 0; - ptl->general_intra_constraint_flag = ctx->gop_size == 1; + ptl->general_intra_constraint_flag = base_ctx->gop_size == 1; ptl->general_one_picture_only_constraint_flag = 0; ptl->general_lower_bit_rate_constraint_flag = 1; @@ -353,7 +354,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) level = ff_h265_guess_level(ptl, avctx->bit_rate, ctx->surface_width, ctx->surface_height, ctx->nb_slices, ctx->tile_rows, ctx->tile_cols, - (ctx->b_per_p > 0) + 1); + (base_ctx->b_per_p > 0) + 1); if (level) { av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); ptl->general_level_idc = level->level_idc; @@ -367,8 +368,8 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) } vps->vps_sub_layer_ordering_info_present_flag = 0; - vps->vps_max_dec_pic_buffering_minus1[0] = ctx->max_b_depth + 1; - vps->vps_max_num_reorder_pics[0] = ctx->max_b_depth; + vps->vps_max_dec_pic_buffering_minus1[0] = base_ctx->max_b_depth + 1; + vps->vps_max_num_reorder_pics[0] = base_ctx->max_b_depth; vps->vps_max_latency_increase_plus1[0] = 0; vps->vps_max_layer_id = 0; @@ -642,9 +643,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) .general_level_idc = vps->profile_tier_level.general_level_idc, .general_tier_flag = vps->profile_tier_level.general_tier_flag, - .intra_period = ctx->gop_size, - .intra_idr_period = ctx->gop_size, - .ip_period = ctx->b_per_p + 1, + .intra_period = base_ctx->gop_size, + .intra_idr_period = base_ctx->gop_size, + .ip_period = base_ctx->b_per_p + 1, .bits_per_second = ctx->va_bit_rate, .pic_width_in_luma_samples = sps->pic_width_in_luma_samples, @@ -755,14 +756,15 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; + HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH265Picture *hpic = pic->priv_data; - VAAPIEncodePicture *prev = pic->prev; + HWBaseEncodePicture *prev = pic->prev; VAAPIEncodeH265Picture *hprev = prev ? prev->priv_data : NULL; - VAEncPictureParameterBufferHEVC *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferHEVC *vpic = vaapi_pic->codec_picture_params; int i, j = 0; if (pic->type == PICTURE_TYPE_IDR) { @@ -787,13 +789,13 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, hpic->slice_type = HEVC_SLICE_P; hpic->pic_type = 1; } else { - VAAPIEncodePicture *irap_ref; + HWBaseEncodePicture *irap_ref; av_assert0(pic->refs[0][0] && pic->refs[1][0]); for (irap_ref = pic; irap_ref; irap_ref = irap_ref->refs[1][0]) { if (irap_ref->type == PICTURE_TYPE_I) break; } - if (pic->b_depth == ctx->max_b_depth) { + if (pic->b_depth == base_ctx->max_b_depth) { hpic->slice_nal_unit = irap_ref ? HEVC_NAL_RASL_N : HEVC_NAL_TRAIL_N; } else { @@ -909,21 +911,21 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, } vpic->decoded_curr_pic = (VAPictureHEVC) { - .picture_id = pic->recon_surface, + .picture_id = vaapi_pic->recon_surface, .pic_order_cnt = hpic->pic_order_cnt, .flags = 0, }; for (int k = 0; k < MAX_REFERENCE_LIST_NUM; k++) { for (i = 0; i < pic->nb_refs[k]; i++) { - VAAPIEncodePicture *ref = pic->refs[k][i]; + HWBaseEncodePicture *ref = pic->refs[k][i]; VAAPIEncodeH265Picture *href; av_assert0(ref && ref->encode_order < pic->encode_order); href = ref->priv_data; vpic->reference_frames[j++] = (VAPictureHEVC) { - .picture_id = ref->recon_surface, + .picture_id = ((VAAPIEncodePicture *)ref)->recon_surface, .pic_order_cnt = href->pic_order_cnt, .flags = (ref->display_order < pic->display_order ? VA_PICTURE_HEVC_RPS_ST_CURR_BEFORE : 0) | @@ -940,7 +942,7 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, }; } - vpic->coded_buf = pic->output_buffer; + vpic->coded_buf = vaapi_pic->output_buffer; vpic->nal_unit_type = hpic->slice_nal_unit; @@ -970,16 +972,17 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, } static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, VAAPIEncodeSlice *slice) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH265Picture *hpic = pic->priv_data; const H265RawSPS *sps = &priv->raw_sps; const H265RawPPS *pps = &priv->raw_pps; H265RawSliceHeader *sh = &priv->raw_slice.header; - VAEncPictureParameterBufferHEVC *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferHEVC *vpic = vaapi_pic->codec_picture_params; VAEncSliceParameterBufferHEVC *vslice = slice->codec_slice_params; int i; @@ -996,7 +999,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, sh->slice_type = hpic->slice_type; - if (sh->slice_type == HEVC_SLICE_P && ctx->p_to_gpb) + if (sh->slice_type == HEVC_SLICE_P && base_ctx->p_to_gpb) sh->slice_type = HEVC_SLICE_B; sh->slice_pic_order_cnt_lsb = hpic->pic_order_cnt & @@ -1140,7 +1143,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, .slice_tc_offset_div2 = sh->slice_tc_offset_div2, .slice_fields.bits = { - .last_slice_of_pic_flag = slice->index == pic->nb_slices - 1, + .last_slice_of_pic_flag = slice->index == vaapi_pic->nb_slices - 1, .dependent_slice_segment_flag = sh->dependent_slice_segment_flag, .colour_plane_id = sh->colour_plane_id, .slice_temporal_mvp_enabled_flag = @@ -1171,7 +1174,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); vslice->ref_pic_list0[0] = vpic->reference_frames[0]; - if (ctx->p_to_gpb && pic->type == PICTURE_TYPE_P) + if (base_ctx->p_to_gpb && pic->type == PICTURE_TYPE_P) // Reference for GPB B-frame, L0 == L1 vslice->ref_pic_list1[0] = vpic->reference_frames[0]; } @@ -1181,7 +1184,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, vslice->ref_pic_list1[0] = vpic->reference_frames[1]; } - if (pic->type == PICTURE_TYPE_P && ctx->p_to_gpb) { + if (pic->type == PICTURE_TYPE_P && base_ctx->p_to_gpb) { vslice->slice_type = HEVC_SLICE_B; for (i = 0; i < FF_ARRAY_ELEMS(vslice->ref_pic_list0); i++) { vslice->ref_pic_list1[i].picture_id = vslice->ref_pic_list0[i].picture_id; @@ -1494,7 +1497,7 @@ const FFCodec ff_hevc_vaapi_encoder = { .p.id = AV_CODEC_ID_HEVC, .priv_data_size = sizeof(VAAPIEncodeH265Context), .init = &vaapi_encode_h265_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_h265_close, .p.priv_class = &vaapi_encode_h265_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index fb5c0d34c6..6726006774 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -220,12 +220,13 @@ static int vaapi_encode_mjpeg_write_extra_buffer(AVCodecContext *avctx, } static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { VAAPIEncodeMJPEGContext *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; JPEGRawFrameHeader *fh = &priv->frame_header; JPEGRawScanHeader *sh = &priv->scan.header; - VAEncPictureParameterBufferJPEG *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferJPEG *vpic = vaapi_pic->codec_picture_params; const AVPixFmtDescriptor *desc; const uint8_t components_rgb[3] = { 'R', 'G', 'B' }; const uint8_t components_yuv[3] = { 1, 2, 3 }; @@ -377,8 +378,8 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, *vpic = (VAEncPictureParameterBufferJPEG) { - .reconstructed_picture = pic->recon_surface, - .coded_buf = pic->output_buffer, + .reconstructed_picture = vaapi_pic->recon_surface, + .coded_buf = vaapi_pic->output_buffer, .picture_width = fh->X, .picture_height = fh->Y, @@ -406,7 +407,7 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, vpic->quantiser_table_selector[i] = fh->Tq[i]; } - pic->nb_slices = 1; + vaapi_pic->nb_slices = 1; return 0; } @@ -572,7 +573,7 @@ const FFCodec ff_mjpeg_vaapi_encoder = { .p.id = AV_CODEC_ID_MJPEG, .priv_data_size = sizeof(VAAPIEncodeMJPEGContext), .init = &vaapi_encode_mjpeg_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_mjpeg_close, .p.priv_class = &vaapi_encode_mjpeg_class, .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DR1 | diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index d0980c52b0..c9d3b60762 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -166,6 +166,7 @@ fail: static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeMPEG2Context *priv = avctx->priv_data; MPEG2RawSequenceHeader *sh = &priv->sequence_header; @@ -281,7 +282,7 @@ static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) se->bit_rate_extension = priv->bit_rate >> 18; se->vbv_buffer_size_extension = priv->vbv_buffer_size >> 10; - se->low_delay = ctx->b_per_p == 0; + se->low_delay = base_ctx->b_per_p == 0; se->frame_rate_extension_n = ext_n; se->frame_rate_extension_d = ext_d; @@ -353,8 +354,8 @@ static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) *vseq = (VAEncSequenceParameterBufferMPEG2) { - .intra_period = ctx->gop_size, - .ip_period = ctx->b_per_p + 1, + .intra_period = base_ctx->gop_size, + .ip_period = base_ctx->b_per_p + 1, .picture_width = avctx->width, .picture_height = avctx->height, @@ -417,12 +418,13 @@ static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { VAAPIEncodeMPEG2Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; MPEG2RawPictureHeader *ph = &priv->picture_header; MPEG2RawPictureCodingExtension *pce = &priv->picture_coding_extension.data.picture_coding; - VAEncPictureParameterBufferMPEG2 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferMPEG2 *vpic = vaapi_pic->codec_picture_params; if (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I) { ph->temporal_reference = 0; @@ -448,8 +450,8 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, pce->f_code[1][1] = 15; } - vpic->reconstructed_picture = pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->reconstructed_picture = vaapi_pic->recon_surface; + vpic->coded_buf = vaapi_pic->output_buffer; switch (pic->type) { case PICTURE_TYPE_IDR: @@ -458,12 +460,12 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, break; case PICTURE_TYPE_P: vpic->picture_type = VAEncPictureTypePredictive; - vpic->forward_reference_picture = pic->refs[0][0]->recon_surface; + vpic->forward_reference_picture = ((VAAPIEncodePicture *)pic->refs[0][0])->recon_surface; break; case PICTURE_TYPE_B: vpic->picture_type = VAEncPictureTypeBidirectional; - vpic->forward_reference_picture = pic->refs[0][0]->recon_surface; - vpic->backward_reference_picture = pic->refs[1][0]->recon_surface; + vpic->forward_reference_picture = ((VAAPIEncodePicture *)pic->refs[0][0])->recon_surface; + vpic->backward_reference_picture = ((VAAPIEncodePicture *)pic->refs[1][0])->recon_surface; break; default: av_assert0(0 && "invalid picture type"); @@ -479,11 +481,12 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, } static int vaapi_encode_mpeg2_init_slice_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic, - VAAPIEncodeSlice *slice) + VAAPIEncodePicture *vaapi_pic, + VAAPIEncodeSlice *slice) { - VAAPIEncodeMPEG2Context *priv = avctx->priv_data; - VAEncSliceParameterBufferMPEG2 *vslice = slice->codec_slice_params; + const HWBaseEncodePicture *pic = &vaapi_pic->base; + VAAPIEncodeMPEG2Context *priv = avctx->priv_data; + VAEncSliceParameterBufferMPEG2 *vslice = slice->codec_slice_params; int qp; vslice->macroblock_address = slice->block_start; @@ -695,7 +698,7 @@ const FFCodec ff_mpeg2_vaapi_encoder = { .p.id = AV_CODEC_ID_MPEG2VIDEO, .priv_data_size = sizeof(VAAPIEncodeMPEG2Context), .init = &vaapi_encode_mpeg2_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_mpeg2_close, .p.priv_class = &vaapi_encode_mpeg2_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index 4e284f86e2..c533536ceb 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -52,6 +52,7 @@ typedef struct VAAPIEncodeVP8Context { static int vaapi_encode_vp8_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAEncSequenceParameterBufferVP8 *vseq = ctx->codec_sequence_params; @@ -66,22 +67,23 @@ static int vaapi_encode_vp8_init_sequence_params(AVCodecContext *avctx) if (!(ctx->va_rc_mode & VA_RC_CQP)) { vseq->bits_per_second = ctx->va_bit_rate; - vseq->intra_period = ctx->gop_size; + vseq->intra_period = base_ctx->gop_size; } return 0; } static int vaapi_encode_vp8_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeVP8Context *priv = avctx->priv_data; - VAEncPictureParameterBufferVP8 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferVP8 *vpic = vaapi_pic->codec_picture_params; int i; - vpic->reconstructed_frame = pic->recon_surface; + vpic->reconstructed_frame = vaapi_pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->coded_buf = vaapi_pic->output_buffer; switch (pic->type) { case PICTURE_TYPE_IDR: @@ -101,7 +103,7 @@ static int vaapi_encode_vp8_init_picture_params(AVCodecContext *avctx, vpic->ref_last_frame = vpic->ref_gf_frame = vpic->ref_arf_frame = - pic->refs[0][0]->recon_surface; + ((VAAPIEncodePicture *)pic->refs[0][0])->recon_surface; break; default: av_assert0(0 && "invalid picture type"); @@ -145,7 +147,7 @@ static int vaapi_encode_vp8_write_quant_table(AVCodecContext *avctx, memset(&quant, 0, sizeof(quant)); - if (pic->type == PICTURE_TYPE_P) + if (pic->base.type == PICTURE_TYPE_P) q = priv->q_index_p; else q = priv->q_index_i; @@ -250,7 +252,7 @@ const FFCodec ff_vp8_vaapi_encoder = { .p.id = AV_CODEC_ID_VP8, .priv_data_size = sizeof(VAAPIEncodeVP8Context), .init = &vaapi_encode_vp8_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &ff_vaapi_encode_close, .p.priv_class = &vaapi_encode_vp8_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 88f951652c..36dafea2c0 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -53,6 +53,7 @@ typedef struct VAAPIEncodeVP9Context { static int vaapi_encode_vp9_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAEncSequenceParameterBufferVP9 *vseq = ctx->codec_sequence_params; VAEncPictureParameterBufferVP9 *vpic = ctx->codec_picture_params; @@ -64,7 +65,7 @@ static int vaapi_encode_vp9_init_sequence_params(AVCodecContext *avctx) if (!(ctx->va_rc_mode & VA_RC_CQP)) { vseq->bits_per_second = ctx->va_bit_rate; - vseq->intra_period = ctx->gop_size; + vseq->intra_period = base_ctx->gop_size; } vpic->frame_width_src = avctx->width; @@ -76,17 +77,18 @@ static int vaapi_encode_vp9_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeVP9Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeVP9Picture *hpic = pic->priv_data; - VAEncPictureParameterBufferVP9 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferVP9 *vpic = vaapi_pic->codec_picture_params; int i; int num_tile_columns; - vpic->reconstructed_frame = pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->reconstructed_frame = vaapi_pic->recon_surface; + vpic->coded_buf = vaapi_pic->output_buffer; // Maximum width of a tile in units of superblocks is MAX_TILE_WIDTH_B64(64) // So the number of tile columns is related to the width of the picture. @@ -107,7 +109,7 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, VAAPIEncodeVP9Picture *href = pic->refs[0][0]->priv_data; av_assert0(href->slot == 0 || href->slot == 1); - if (ctx->max_b_depth > 0) { + if (base_ctx->max_b_depth > 0) { hpic->slot = !href->slot; vpic->refresh_frame_flags = 1 << hpic->slot | 0xfc; } else { @@ -127,7 +129,7 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, av_assert0(href0->slot < pic->b_depth + 1 && href1->slot < pic->b_depth + 1); - if (pic->b_depth == ctx->max_b_depth) { + if (pic->b_depth == base_ctx->max_b_depth) { // Unreferenced frame. vpic->refresh_frame_flags = 0x00; hpic->slot = 8; @@ -159,11 +161,11 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { for (int j = 0; j < pic->nb_refs[i]; j++) { - VAAPIEncodePicture *ref_pic = pic->refs[i][j]; + HWBaseEncodePicture *ref_pic = pic->refs[i][j]; int slot; slot = ((VAAPIEncodeVP9Picture*)ref_pic->priv_data)->slot; av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); - vpic->reference_frames[slot] = ref_pic->recon_surface; + vpic->reference_frames[slot] = ((VAAPIEncodePicture *)ref_pic)->recon_surface; } } @@ -307,7 +309,7 @@ const FFCodec ff_vp9_vaapi_encoder = { .p.id = AV_CODEC_ID_VP9, .priv_data_size = sizeof(VAAPIEncodeVP9Context), .init = &vaapi_encode_vp9_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &ff_vaapi_encode_close, .p.priv_class = &vaapi_encode_vp9_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | From patchwork Mon May 20 14:52:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49064 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4523880pzb; Mon, 20 May 2024 08:05:06 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXVtf+4rtF486y4tbu9Fhe3ForjswAGaml3LlxaB/WSDWPlngrnBmJ2q1+ETeNSz5Ba13h9wugs8BweiXY07wigEYMU3fdkuRHD5g== X-Google-Smtp-Source: AGHT+IE6QL2jGpMrH9fvD68rCF/NxIL9hQmCbMf2244I0Tgnt/q6gdMmJCapn2Ib+hcsUodUt4sS X-Received: by 2002:a50:bb05:0:b0:572:8aab:4420 with SMTP id 4fb4d7f45d1cf-5734d6de8c1mr20540753a12.39.1716217505970; Mon, 20 May 2024 08:05:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716217505; cv=none; d=google.com; s=arc-20160816; b=ENdSyFLPoJQvLV19useSbQ/DwwGmUKXFXVdrB9Tc5XhWxFUf/VEZMwrtU/AEkHXtFC EP3zwNrii2qbQhini2sTjq9xVXN+df8f+5vSSTjVNhopgrSKxrI5vj7v0Mukcood5y8e NqLoAQdpLHUIKs27gWYmaMhk02bvjvS1lsUw8bzRV8gJIiQls+LNhroOG7DhqdwrHmyh jVWOPpGlutfSmU34CdYjhkCK/PnB361m0joL0SzclXN3mUnamnPYGWuIxjHJ0t5Qr4C9 WDq7niAliT3Mz13WjH2WoUCkLIppS35S0WTiJmXwIpbU2DJ4AZ/yZMwZrrbiJrCRSoyT B+pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=uUhs56DzUW8z9FwL2dpuphlluFhwDv1pdQLIhD4+66Q=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=x4B+5kloakudRTeM26s/sg3qhBbxfmud/fTEUTwubGTJURAJ/u8nwgUouqyGwpb9z3 2Rz5f4N8x/HyljvLlN1/34gLoLrtM4/3f+BgQNLa7HQtQA90dXIJ4pq6r39BSxRRbLtT ZQ7dTYkeaIl/JIp95JbLB/Aij02EgbTxUcidzV1Pi/7xp0Va1rzdlLx9QVJn+rp9HcQn DVgQVv4RgTAvcsKSYuausQp7k8lCQjCQ20t52SGAkDKH1rUPKIc8KaWjI6cKjYQLRSKR 2jSiN69DPcOGx77LKca8DHlrMO0n8jF6gWPTNCp8DrQhaOaHEUQpYoWWL+CNxMsV4M3u cI7w==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TB1OBZ+W; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5750c0c649bsi4926537a12.625.2024.05.20.08.05.03; Mon, 20 May 2024 08:05:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TB1OBZ+W; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A6D1068D243; Mon, 20 May 2024 17:54:14 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 41D4E68D22E for ; Mon, 20 May 2024 17:54:06 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216852; x=1747752852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TInwZ7xeuVyQT0K5b+SylT+s11hTZniuj6SiPJ2pxhc=; b=TB1OBZ+W39QxY6tnvlB+4nZRM3Ud3uBL949bs4+tY7gNG1lrjw+nZyCz sHZdQigS7HycYV94jRnAW6QZ8ladxbO30TgxMchomoDwKhdzjiazyFqtO v2+E7bJA4JXceiehKY52wxZa0HDXWn1O0SYG8G7NPuUU5GgRAxVUClZCy LV0j4Bay2yb3U2Uowz5L1FfAMItTJiqy7pN4NdLa7YTucEDSK4EQRGw1g oVvqn/wSQ+kYulagkg0w/RRV89JyDy+Xn1dkjeV1VnrONFzdDCYxqH7d2 0oO5Y8QJm5SfnvoX0qbHgwmyc8/OIPS9lcimk6gig7wxYW83fIjIRkFYA g==; X-CSE-ConnectionGUID: TfyaxN3vTN+WAkQvVjD+kg== X-CSE-MsgGUID: P82aIXW3ScOQVOlvROLStQ== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197248" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197248" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:01 -0700 X-CSE-ConnectionGUID: pw7+UbrRRnapwf8qNa0PaQ== X-CSE-MsgGUID: hXlEatGpS/S/YwcOxZFoaA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787883" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:00 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:15 +0800 Message-ID: <20240520145222.291-6-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 06/13] avcodec/vaapi_encode: extract the init and close function to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: st197x63aoyA From: Tong Wu Related parameters such as device context, frame context are also moved to base layer. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 49 ++++++++++++++++++ libavcodec/hw_base_encode.h | 17 +++++++ libavcodec/vaapi_encode.c | 90 +++++++++++---------------------- libavcodec/vaapi_encode.h | 10 ---- libavcodec/vaapi_encode_av1.c | 2 +- libavcodec/vaapi_encode_h264.c | 2 +- libavcodec/vaapi_encode_h265.c | 2 +- libavcodec/vaapi_encode_mjpeg.c | 6 ++- 8 files changed, 102 insertions(+), 76 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index ec7178d2dc..f2b6ef4a3a 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -595,3 +595,52 @@ end: return 0; } + +int ff_hw_base_encode_init(AVCodecContext *avctx) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + ctx->frame = av_frame_alloc(); + if (!ctx->frame) + return AVERROR(ENOMEM); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is " + "required to associate the encoding device.\n"); + return AVERROR(EINVAL); + } + + ctx->input_frames_ref = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->input_frames_ref) + return AVERROR(ENOMEM); + + ctx->input_frames = (AVHWFramesContext *)ctx->input_frames_ref->data; + + ctx->device_ref = av_buffer_ref(ctx->input_frames->device_ref); + if (!ctx->device_ref) + return AVERROR(ENOMEM); + + ctx->device = (AVHWDeviceContext *)ctx->device_ref->data; + + ctx->tail_pkt = av_packet_alloc(); + if (!ctx->tail_pkt) + return AVERROR(ENOMEM); + + return 0; +} + +int ff_hw_base_encode_close(AVCodecContext *avctx) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + av_fifo_freep2(&ctx->encode_fifo); + + av_frame_free(&ctx->frame); + av_packet_free(&ctx->tail_pkt); + + av_buffer_unref(&ctx->device_ref); + av_buffer_unref(&ctx->input_frames_ref); + av_buffer_unref(&ctx->recon_frames_ref); + + return 0; +} diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 2667ae61cd..7e20f57b9c 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -19,6 +19,7 @@ #ifndef AVCODEC_HW_BASE_ENCODE_H #define AVCODEC_HW_BASE_ENCODE_H +#include "libavutil/hwcontext.h" #include "libavutil/fifo.h" #define MAX_DPB_SIZE 16 @@ -118,6 +119,18 @@ typedef struct HWBaseEncodeContext { // Hardware-specific hooks. const struct HWEncodePictureOperation *op; + // The hardware device context. + AVBufferRef *device_ref; + AVHWDeviceContext *device; + + // The hardware frame context containing the input frames. + AVBufferRef *input_frames_ref; + AVHWFramesContext *input_frames; + + // The hardware frame context containing the reconstructed frames. + AVBufferRef *recon_frames_ref; + AVHWFramesContext *recon_frames; + // Current encoding window, in display (input) order. HWBaseEncodePicture *pic_start, *pic_end; // The next picture to use as the previous reference picture in @@ -184,6 +197,10 @@ typedef struct HWBaseEncodeContext { int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); +int ff_hw_base_encode_init(AVCodecContext *avctx); + +int ff_hw_base_encode_close(AVCodecContext *avctx); + #define HW_BASE_ENCODE_COMMON_OPTIONS \ { "async_depth", "Maximum processing parallelism. " \ "Increase this to improve single channel performance.", \ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index c4bb93c520..c3ab0fc192 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -314,7 +314,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Input surface is %#x.\n", pic->input_surface); - err = av_hwframe_get_buffer(ctx->recon_frames_ref, base_pic->recon_image, 0); + err = av_hwframe_get_buffer(base_ctx->recon_frames_ref, base_pic->recon_image, 0); if (err < 0) { err = AVERROR(ENOMEM); goto fail; @@ -996,9 +996,10 @@ static const VAEntrypoint vaapi_encode_entrypoints_low_power[] = { static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAProfile *va_profiles = NULL; - VAEntrypoint *va_entrypoints = NULL; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAProfile *va_profiles = NULL; + VAEntrypoint *va_entrypoints = NULL; VAStatus vas; const VAEntrypoint *usable_entrypoints; const VAAPIEncodeProfile *profile; @@ -1021,10 +1022,10 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) usable_entrypoints = vaapi_encode_entrypoints_normal; } - desc = av_pix_fmt_desc_get(ctx->input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); if (!desc) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%d).\n", - ctx->input_frames->sw_format); + base_ctx->input_frames->sw_format); return AVERROR(EINVAL); } depth = desc->comp[0].depth; @@ -2131,20 +2132,21 @@ static int vaapi_encode_alloc_output_buffer(FFRefStructOpaque opaque, void *obj) static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; AVVAAPIHWConfig *hwconfig = NULL; AVHWFramesConstraints *constraints = NULL; enum AVPixelFormat recon_format; int err, i; - hwconfig = av_hwdevice_hwconfig_alloc(ctx->device_ref); + hwconfig = av_hwdevice_hwconfig_alloc(base_ctx->device_ref); if (!hwconfig) { err = AVERROR(ENOMEM); goto fail; } hwconfig->config_id = ctx->va_config; - constraints = av_hwdevice_get_hwframe_constraints(ctx->device_ref, + constraints = av_hwdevice_get_hwframe_constraints(base_ctx->device_ref, hwconfig); if (!constraints) { err = AVERROR(ENOMEM); @@ -2157,9 +2159,9 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) recon_format = AV_PIX_FMT_NONE; if (constraints->valid_sw_formats) { for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; i++) { - if (ctx->input_frames->sw_format == + if (base_ctx->input_frames->sw_format == constraints->valid_sw_formats[i]) { - recon_format = ctx->input_frames->sw_format; + recon_format = base_ctx->input_frames->sw_format; break; } } @@ -2170,7 +2172,7 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) } } else { // No idea what to use; copy input format. - recon_format = ctx->input_frames->sw_format; + recon_format = base_ctx->input_frames->sw_format; } av_log(avctx, AV_LOG_DEBUG, "Using %s as format of " "reconstructed frames.\n", av_get_pix_fmt_name(recon_format)); @@ -2191,19 +2193,19 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) av_freep(&hwconfig); av_hwframe_constraints_free(&constraints); - ctx->recon_frames_ref = av_hwframe_ctx_alloc(ctx->device_ref); - if (!ctx->recon_frames_ref) { + base_ctx->recon_frames_ref = av_hwframe_ctx_alloc(base_ctx->device_ref); + if (!base_ctx->recon_frames_ref) { err = AVERROR(ENOMEM); goto fail; } - ctx->recon_frames = (AVHWFramesContext*)ctx->recon_frames_ref->data; + base_ctx->recon_frames = (AVHWFramesContext*)base_ctx->recon_frames_ref->data; - ctx->recon_frames->format = AV_PIX_FMT_VAAPI; - ctx->recon_frames->sw_format = recon_format; - ctx->recon_frames->width = ctx->surface_width; - ctx->recon_frames->height = ctx->surface_height; + base_ctx->recon_frames->format = AV_PIX_FMT_VAAPI; + base_ctx->recon_frames->sw_format = recon_format; + base_ctx->recon_frames->width = ctx->surface_width; + base_ctx->recon_frames->height = ctx->surface_height; - err = av_hwframe_ctx_init(ctx->recon_frames_ref); + err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); if (err < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to initialise reconstructed " "frame context: %d.\n", err); @@ -2235,44 +2237,16 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) VAStatus vas; int err; + err = ff_hw_base_encode_init(avctx); + if (err < 0) + goto fail; + ctx->va_config = VA_INVALID_ID; ctx->va_context = VA_INVALID_ID; base_ctx->op = &vaapi_op; - /* If you add something that can fail above this av_frame_alloc(), - * modify ff_vaapi_encode_close() accordingly. */ - base_ctx->frame = av_frame_alloc(); - if (!base_ctx->frame) { - return AVERROR(ENOMEM); - } - - if (!avctx->hw_frames_ctx) { - av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is " - "required to associate the encoding device.\n"); - return AVERROR(EINVAL); - } - - ctx->input_frames_ref = av_buffer_ref(avctx->hw_frames_ctx); - if (!ctx->input_frames_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - ctx->input_frames = (AVHWFramesContext*)ctx->input_frames_ref->data; - - ctx->device_ref = av_buffer_ref(ctx->input_frames->device_ref); - if (!ctx->device_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - ctx->device = (AVHWDeviceContext*)ctx->device_ref->data; - ctx->hwctx = ctx->device->hwctx; - - base_ctx->tail_pkt = av_packet_alloc(); - if (!base_ctx->tail_pkt) { - err = AVERROR(ENOMEM); - goto fail; - } + ctx->hwctx = base_ctx->device->hwctx; err = vaapi_encode_profile_entrypoint(avctx); if (err < 0) @@ -2339,7 +2313,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; - recon_hwctx = ctx->recon_frames->hwctx; + recon_hwctx = base_ctx->recon_frames->hwctx; vas = vaCreateContext(ctx->hwctx->display, ctx->va_config, ctx->surface_width, ctx->surface_height, VA_PROGRESSIVE, @@ -2467,16 +2441,10 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) ctx->va_config = VA_INVALID_ID; } - av_frame_free(&base_ctx->frame); - av_packet_free(&base_ctx->tail_pkt); - av_freep(&ctx->codec_sequence_params); av_freep(&ctx->codec_picture_params); - av_fifo_freep2(&base_ctx->encode_fifo); - av_buffer_unref(&ctx->recon_frames_ref); - av_buffer_unref(&ctx->input_frames_ref); - av_buffer_unref(&ctx->device_ref); + ff_hw_base_encode_close(avctx); return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 13ccad8e47..c25eab1cb7 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -219,18 +219,8 @@ typedef struct VAAPIEncodeContext { VAConfigID va_config; VAContextID va_context; - AVBufferRef *device_ref; - AVHWDeviceContext *device; AVVAAPIDeviceContext *hwctx; - // The hardware frame context containing the input frames. - AVBufferRef *input_frames_ref; - AVHWFramesContext *input_frames; - - // The hardware frame context containing the reconstructed frames. - AVBufferRef *recon_frames_ref; - AVHWFramesContext *recon_frames; - // Pool of (reusable) bitstream output buffers. struct FFRefStructPool *output_buffer_pool; diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index f12fe54cff..a15b74f409 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -373,7 +373,7 @@ static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) memset(sh_obu, 0, sizeof(*sh_obu)); sh_obu->header.obu_type = AV1_OBU_SEQUENCE_HEADER; - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); sh->seq_profile = avctx->profile; diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index 9182483762..d41b9649e8 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -308,7 +308,7 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) memset(sps, 0, sizeof(*sps)); memset(pps, 0, sizeof(*pps)); - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); if (desc->nb_components == 1 || desc->log2_chroma_w != 1 || desc->log2_chroma_h != 1) { av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1f9d9c56a4..ec48773d95 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -278,7 +278,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) memset(pps, 0, sizeof(*pps)); - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); if (desc->nb_components == 1) { chroma_format = 0; diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index 6726006774..24acc88a21 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -222,6 +222,7 @@ static int vaapi_encode_mjpeg_write_extra_buffer(AVCodecContext *avctx, static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, VAAPIEncodePicture *vaapi_pic) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeMJPEGContext *priv = avctx->priv_data; const HWBaseEncodePicture *pic = &vaapi_pic->base; JPEGRawFrameHeader *fh = &priv->frame_header; @@ -235,7 +236,7 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_IDR); - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); if (desc->flags & AV_PIX_FMT_FLAG_RGB) components = components_rgb; @@ -437,10 +438,11 @@ static int vaapi_encode_mjpeg_init_slice_params(AVCodecContext *avctx, static av_cold int vaapi_encode_mjpeg_get_encoder_caps(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; const AVPixFmtDescriptor *desc; - desc = av_pix_fmt_desc_get(ctx->input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); From patchwork Mon May 20 14:52:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49062 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4515024pzb; Mon, 20 May 2024 07:56:03 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVoMT6xdpHy+EKmjDuxadKbnUIuR/NUndnJdQNEU6/4OkpwDau0IJWfxSvsI/eIyrzoyWbsA9DlsFdVnrH+7jXwZ3g9O/c7XyuqjQ== X-Google-Smtp-Source: AGHT+IEyX7cFCsAHOQo0W0uxXM4iZcPXk3JvyzxtBRSoYdqSP+VHRn+g9aMEarGJfWei2KCFaTB4 X-Received: by 2002:a2e:9094:0:b0:2df:4bad:cb7f with SMTP id 38308e7fff4ca-2e51fd4b333mr181486491fa.2.1716216963284; Mon, 20 May 2024 07:56:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216963; cv=none; d=google.com; s=arc-20160816; b=C3iwPr1bpsKjGG3wpCXwrve9/r8paf87vYNn2wiQhXCQ/D9nzASgw0o3bbJnzOgotR 6yu7Lh5vcvDKiUV9+U5W/EfwmvkR3gLHi2FrA9eF32JwIJABjRK63n54BFvpue1USAjP 3i0nrphD2+deS2Ud/REVe6gugGcF/fENcC7FLGN7Hr62ivTgXap8EBGfGcbBib+SwsRG +4OXciuL96s0wnm/RqBEGv+q65dYWX/IxQ8Cz5wd2D/jr0E/TYvCptfFM0oSiIfW7D7K b+Yj4VsBxxxCCZPeFuXKdNOboZ2WJhpPTMlMqlYJ4Bh6gi1IbL2ri2sLAO/hPPrNT+3l yfTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=SLiWgWqqTs0LVUIa/xKlR1BLjfUmEq8sOhkqZaBjFfA=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=TCUjnUlU15a7NR8yjTh+mpwMPdVYTceNp3k6fyoHeX8ZbRVw8YK4no6hp0MVz3AR4K PKBBAFaGjqVt8icaxx8kFeugdX/JFX+Thru7QOmAcj9AM5YodQVPTmXCfqNU1CqxNxmZ TDDV54K4o2Paokii35DqqceFawuHqI+OFw5mvmkPorWPGfawJKrp5C31su7VScoJER4P KMLRSoHS4Exfm+X9keqWKwQCcMq3ViWWRcWlnWYoNjT85soXLo1zu5k3EXlcJUFC+maM R2vwMlD1HC2FCviIJFg+1gxnqisecYW9/w+/AXLRkXrm+nfImFqc1o4KgW19ZY42pjni 6MMA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=OKDcHxaF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-57521ceab32si3345247a12.420.2024.05.20.07.56.01; Mon, 20 May 2024 07:56:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=OKDcHxaF; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 746CB68D30D; Mon, 20 May 2024 17:54:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A31B968D212 for ; Mon, 20 May 2024 17:54:09 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216855; x=1747752855; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2HxOvL7oTDqLNpaxVXEDZz1WorudUaVdNSw2U2Jbk8E=; b=OKDcHxaFmGJq2KtlWmBy9sM5VwRHG+ow1Saav4+tFpjQX/cbLbYxKpVn DkA5PICCFydEBosWch3BDSS8GUOm8zmbe7nbCNrqa2ofpqaw0Mbr+T/6Z y9a+mVMvBLYteaZxQqQNH14282Hq9UML29CX2LycWTKWu2C8F5OpAx3aY 5kHqmTYb58ahA0f9CdZuyJs0C2Ax097ENn/cjaeYp0Jo0jXXc4bjKQRmg OnBsAmV+KegK/wo5BoHskNCXgX9SwCwbvoS1w3qeXsBU2mtdsz0fEVng3 iea6/K3iTRyXCGScW58s+K9nOCENcXoOYgBedKSwTCOqj7GdrU8rtLu2U w==; X-CSE-ConnectionGUID: w6kjtGOeQwuJt9RvdVuzYA== X-CSE-MsgGUID: 53csfS0aSyOMETkFrnPW+A== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197250" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197250" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:02 -0700 X-CSE-ConnectionGUID: 1MKsfcKoT12cTrD8RUWLLQ== X-CSE-MsgGUID: UVXL4KBUQ8GUMb2W/fXkxw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787888" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:01 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:16 +0800 Message-ID: <20240520145222.291-7-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 07/13] avcodec/vaapi_encode: extract gop configuration and two options to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: i+hnFHWnwudB From: Tong Wu idr_interval and desired_b_depth are moved to HW_BASE_ENCODE_COMMON_OPTIONS. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 54 +++++++++++++++++++++++++++++++++++++ libavcodec/hw_base_encode.h | 19 +++++++++++++ libavcodec/vaapi_encode.c | 52 +++-------------------------------- libavcodec/vaapi_encode.h | 16 ----------- 4 files changed, 77 insertions(+), 64 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index f2b6ef4a3a..dfe20c4e67 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -596,6 +596,60 @@ end: return 0; } +int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, + int flags, int prediction_pre_only) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + if (flags & FLAG_INTRA_ONLY || avctx->gop_size <= 1) { + av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); + ctx->gop_size = 1; + } else if (ref_l0 < 1) { + av_log(avctx, AV_LOG_ERROR, "Driver does not support any " + "reference frames.\n"); + return AVERROR(EINVAL); + } else if (!(flags & FLAG_B_PICTURES) || ref_l1 < 1 || + avctx->max_b_frames < 1 || prediction_pre_only) { + if (ctx->p_to_gpb) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " + "(supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); + ctx->gop_size = avctx->gop_size; + ctx->p_per_i = INT_MAX; + ctx->b_per_p = 0; + } else { + if (ctx->p_to_gpb) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " + "(supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); + ctx->gop_size = avctx->gop_size; + ctx->p_per_i = INT_MAX; + ctx->b_per_p = avctx->max_b_frames; + if (flags & FLAG_B_PICTURE_REFERENCES) { + ctx->max_b_depth = FFMIN(ctx->desired_b_depth, + av_log2(ctx->b_per_p) + 1); + } else { + ctx->max_b_depth = 1; + } + } + + if (flags & FLAG_NON_IDR_KEY_PICTURES) { + ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); + ctx->gop_per_idr = ctx->idr_interval + 1; + } else { + ctx->closed_gop = 1; + ctx->gop_per_idr = 1; + } + + return 0; +} + int ff_hw_base_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 7e20f57b9c..07936bf9bb 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -119,6 +119,14 @@ typedef struct HWBaseEncodeContext { // Hardware-specific hooks. const struct HWEncodePictureOperation *op; + // Global options. + + // Number of I frames between IDR frames. + int idr_interval; + + // Desired B frame reference depth. + int desired_b_depth; + // The hardware device context. AVBufferRef *device_ref; AVHWDeviceContext *device; @@ -197,11 +205,22 @@ typedef struct HWBaseEncodeContext { int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); +int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, + int flags, int prediction_pre_only); + int ff_hw_base_encode_init(AVCodecContext *avctx); int ff_hw_base_encode_close(AVCodecContext *avctx); #define HW_BASE_ENCODE_COMMON_OPTIONS \ + { "idr_interval", \ + "Distance (in I-frames) between key frames", \ + OFFSET(common.base.idr_interval), AV_OPT_TYPE_INT, \ + { .i64 = 0 }, 0, INT_MAX, FLAGS }, \ + { "b_depth", \ + "Maximum B-frame reference depth", \ + OFFSET(common.base.desired_b_depth), AV_OPT_TYPE_INT, \ + { .i64 = 1 }, 1, INT_MAX, FLAGS }, \ { "async_depth", "Maximum processing parallelism. " \ "Increase this to improve single channel performance.", \ OFFSET(common.base.async_depth), AV_OPT_TYPE_INT, \ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index c3ab0fc192..4db64b686b 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1638,7 +1638,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) VAStatus vas; VAConfigAttrib attr = { VAConfigAttribEncMaxRefFrames }; uint32_t ref_l0, ref_l1; - int prediction_pre_only; + int prediction_pre_only, err; vas = vaGetConfigAttributes(ctx->hwctx->display, ctx->va_profile, @@ -1702,53 +1702,9 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) } #endif - if (ctx->codec->flags & FLAG_INTRA_ONLY || - avctx->gop_size <= 1) { - av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); - base_ctx->gop_size = 1; - } else if (ref_l0 < 1) { - av_log(avctx, AV_LOG_ERROR, "Driver does not support any " - "reference frames.\n"); - return AVERROR(EINVAL); - } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || - ref_l1 < 1 || avctx->max_b_frames < 1 || - prediction_pre_only) { - if (base_ctx->p_to_gpb) - av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " - "(supported references: %d / %d).\n", - ref_l0, ref_l1); - else - av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); - base_ctx->gop_size = avctx->gop_size; - base_ctx->p_per_i = INT_MAX; - base_ctx->b_per_p = 0; - } else { - if (base_ctx->p_to_gpb) - av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " - "(supported references: %d / %d).\n", - ref_l0, ref_l1); - else - av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); - base_ctx->gop_size = avctx->gop_size; - base_ctx->p_per_i = INT_MAX; - base_ctx->b_per_p = avctx->max_b_frames; - if (ctx->codec->flags & FLAG_B_PICTURE_REFERENCES) { - base_ctx->max_b_depth = FFMIN(ctx->desired_b_depth, - av_log2(base_ctx->b_per_p) + 1); - } else { - base_ctx->max_b_depth = 1; - } - } - - if (ctx->codec->flags & FLAG_NON_IDR_KEY_PICTURES) { - base_ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); - base_ctx->gop_per_idr = ctx->idr_interval + 1; - } else { - base_ctx->closed_gop = 1; - base_ctx->gop_per_idr = 1; - } + err = ff_hw_base_init_gop_structure(avctx, ref_l0, ref_l1, ctx->codec->flags, prediction_pre_only); + if (err < 0) + return err; return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index c25eab1cb7..76fb645d71 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -151,17 +151,9 @@ typedef struct VAAPIEncodeContext { // Codec-specific hooks. const struct VAAPIEncodeType *codec; - // Global options. - // Use low power encoding mode. int low_power; - // Number of I frames between IDR frames. - int idr_interval; - - // Desired B frame reference depth. - int desired_b_depth; - // Max Frame Size int max_frame_size; @@ -371,14 +363,6 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "may not support all encoding features)", \ OFFSET(common.low_power), AV_OPT_TYPE_BOOL, \ { .i64 = 0 }, 0, 1, FLAGS }, \ - { "idr_interval", \ - "Distance (in I-frames) between IDR frames", \ - OFFSET(common.idr_interval), AV_OPT_TYPE_INT, \ - { .i64 = 0 }, 0, INT_MAX, FLAGS }, \ - { "b_depth", \ - "Maximum B-frame reference depth", \ - OFFSET(common.desired_b_depth), AV_OPT_TYPE_INT, \ - { .i64 = 1 }, 1, INT_MAX, FLAGS }, \ { "max_frame_size", \ "Maximum frame size (in bytes)",\ OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ From patchwork Mon May 20 14:52:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49057 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514408pzb; Mon, 20 May 2024 07:55:10 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX5ctkGWg4u0x+eEMptHPyqtidwnAUK8F1O1FZyy/WbsXHFsQ6dcKLNtNkfEHRXwKqbLCbjA+2PnJ14K+uDQnntVBzf+ewdw3MHYA== X-Google-Smtp-Source: AGHT+IG2nzbSKUJ43jCpkSoWV2QLuVF+yCdrlTyWbPbiyj52bq0Zutakf/h75CiO6NO4lQZAZSbc X-Received: by 2002:a17:907:3208:b0:a59:c367:560c with SMTP id a640c23a62f3a-a5a2d665e1emr2121266166b.60.1716216910670; Mon, 20 May 2024 07:55:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216910; cv=none; d=google.com; s=arc-20160816; b=O5Bv/+D6TPv0gb44I8qMJ+KLgrO3ISuEYCDq5iV4Dz2zgW1ldRHwSYA1eSCE3n1tm4 9c3qZrGjuLn+ZtzT/N88k1FBFy++q3O2pWvqpe+smhauclsy8yjwMKMOJzI80W4elEQ6 KN/mpt1YZWf3I0/MPfaaeCok11gDJGZ9bbNwrOv/eIhGlHXWaeI+ezkSYnjbH/sXTdM7 b9ksWkByekFBgmllx7+yXPLVQAaypEF77kxkNNAuwFXptemKB0JgMwuaX2KY72BoM/vv GdF01S+X878G4+EBogUwk8QOiyl26r5Jx8bAhHJ184+5Q5FUbqL+qtbnz7+Nn6MwDc5p Q9aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=1QmX/9xo4/hc8pm9VjpzI+I+H3vm2NTPpDWCyb9mwzA=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=ZTtjsf0+l1TRONwS5+yx7VMsxp4Q6Iv4aYGJzt/e4P7uj5m3Gf8KqlUsSEkKt6pZX5 M2e/4w44b6hbwJz1i5jyajVAc0GOAspboozaZ47afH0FXngCdeWdfpUKNqUT/eEqeU40 u/yQEcKc6b4YuE2ztK5jruNIylxEWNgI8JxVmKYcashxNOVSKWBhmeb4AAbksfzx/wvP pTm9jz9gVG4EZTpHxKrll5M0RSO6LaE33opBsOzb1K28YSOkrS730i/yWzs2UNAQh4oA YhjCMp+F8q7ibT0g4cvzefA/i57/8jAoSU0KUg0sHRV6i/XtmEzyXNobERQqn/Pa6yuL eIOw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="HaaCKr+/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5cdc6fd9f2si555993166b.546.2024.05.20.07.55.10; Mon, 20 May 2024 07:55:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b="HaaCKr+/"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A85E068D2AB; Mon, 20 May 2024 17:54:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id F305168D1A3 for ; Mon, 20 May 2024 17:54:11 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216852; x=1747752852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ILKhRkw+7e2FkdCBOJoN+eocfS2wScMEDAqCDKDZeAc=; b=HaaCKr+/VHkGh4p6emY960tn0t0U5Aj7a/a/m+Auq5lh0afBsUGpKzmp i28E4439Qr8ShlwG4YxkBFeAiQTUhrDP2XzyWBzt6Einh14B1QoG0Wjze qJMLjJN8zTnztxRZv159YPEO6oCSv/v9JS4FBX3ufflhSveKSSZ+JibC4 OMvSh8mhTzcLFxQdWhuICzstFzi/kJiyowVUH398gZwvP0/mqEJwIi0AN cccs0y0uwwY/9IRlCCPAUjNHs1e9gxbjdRFWktjeV5tYx0+flo8JTclUR imHfwdfNIXzHXCsv321gkM1yA3FnK7rtlh04GVLtfZcT6DvjvigO2TaNA g==; X-CSE-ConnectionGUID: mLSa18u/QGKlK0QtnaKB1w== X-CSE-MsgGUID: HHwAJiVlS2Ow5cG6IVZsNQ== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197252" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197252" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:03 -0700 X-CSE-ConnectionGUID: Nb60mRYiT42CEIqgh9jjDw== X-CSE-MsgGUID: IODEUqY3SZajV9iLrjf61Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787895" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:02 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:17 +0800 Message-ID: <20240520145222.291-8-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 08/13] avcodec/vaapi_encode: extract set_output_property to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: M0TcIQEQy3PP From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 40 +++++++++++++++++++++++++++++++++ libavcodec/hw_base_encode.h | 3 +++ libavcodec/vaapi_encode.c | 44 ++----------------------------------- 3 files changed, 45 insertions(+), 42 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index dfe20c4e67..df820c2f83 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -491,6 +491,46 @@ fail: return err; } +int ff_hw_base_encode_set_output_property(AVCodecContext *avctx, + HWBaseEncodePicture *pic, + AVPacket *pkt, int flag_no_delay) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + if (pic->type == PICTURE_TYPE_IDR) + pkt->flags |= AV_PKT_FLAG_KEY; + + pkt->pts = pic->pts; + pkt->duration = pic->duration; + + // for no-delay encoders this is handled in generic codec + if (avctx->codec->capabilities & AV_CODEC_CAP_DELAY && + avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { + pkt->opaque = pic->opaque; + pkt->opaque_ref = pic->opaque_ref; + pic->opaque_ref = NULL; + } + + if (flag_no_delay) { + pkt->dts = pkt->pts; + return 0; + } + + if (ctx->output_delay == 0) { + pkt->dts = pkt->pts; + } else if (pic->encode_order < ctx->decode_delay) { + if (ctx->ts_ring[pic->encode_order] < INT64_MIN + ctx->dts_pts_diff) + pkt->dts = INT64_MIN; + else + pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; + } else { + pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % + (3 * ctx->output_delay + ctx->async_depth)]; + } + + return 0; +} + int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 07936bf9bb..d363819329 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -203,6 +203,9 @@ typedef struct HWBaseEncodeContext { AVPacket *tail_pkt; } HWBaseEncodeContext; +int ff_hw_base_encode_set_output_property(AVCodecContext *avctx, HWBaseEncodePicture *pic, + AVPacket *pkt, int flag_no_delay); + int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 4db64b686b..6ab1b633ed 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -660,47 +660,6 @@ fail_at_end: return err; } -static int vaapi_encode_set_output_property(AVCodecContext *avctx, - HWBaseEncodePicture *pic, - AVPacket *pkt) -{ - HWBaseEncodeContext *base_ctx = avctx->priv_data; - VAAPIEncodeContext *ctx = avctx->priv_data; - - if (pic->type == PICTURE_TYPE_IDR) - pkt->flags |= AV_PKT_FLAG_KEY; - - pkt->pts = pic->pts; - pkt->duration = pic->duration; - - // for no-delay encoders this is handled in generic codec - if (avctx->codec->capabilities & AV_CODEC_CAP_DELAY && - avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { - pkt->opaque = pic->opaque; - pkt->opaque_ref = pic->opaque_ref; - pic->opaque_ref = NULL; - } - - if (ctx->codec->flags & FLAG_TIMESTAMP_NO_DELAY) { - pkt->dts = pkt->pts; - return 0; - } - - if (base_ctx->output_delay == 0) { - pkt->dts = pkt->pts; - } else if (pic->encode_order < base_ctx->decode_delay) { - if (base_ctx->ts_ring[pic->encode_order] < INT64_MIN + base_ctx->dts_pts_diff) - pkt->dts = INT64_MIN; - else - pkt->dts = base_ctx->ts_ring[pic->encode_order] - base_ctx->dts_pts_diff; - } else { - pkt->dts = base_ctx->ts_ring[(pic->encode_order - base_ctx->decode_delay) % - (3 * base_ctx->output_delay + base_ctx->async_depth)]; - } - - return 0; -} - static int vaapi_encode_get_coded_buffer_size(AVCodecContext *avctx, VABufferID buf_id) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -852,7 +811,8 @@ static int vaapi_encode_output(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", base_pic->display_order, base_pic->encode_order); - vaapi_encode_set_output_property(avctx, (HWBaseEncodePicture*)pic, pkt_ptr); + ff_hw_base_encode_set_output_property(avctx, (HWBaseEncodePicture*)base_pic, pkt_ptr, + ctx->codec->flags & FLAG_TIMESTAMP_NO_DELAY); end: ff_refstruct_unref(&pic->output_buffer_ref); From patchwork Mon May 20 14:52:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49058 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514526pzb; Mon, 20 May 2024 07:55:21 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUAwRcuswiAb4dlafO0Q+mB5cm82LzB4tn/KG8W6C1Xr1TY8+sNv77lLRgdl+A1Yz9zstd/vUHav4E1SJ+NhDuzpvtIJJa7a/rNXQ== X-Google-Smtp-Source: AGHT+IHQY0bNnR9NFn78OIy5jfshtyw0z40DGX9LnLkk3XdOL8xSmYGqsO9ZD6ougGC0BXZmcLyA X-Received: by 2002:a50:d714:0:b0:574:eb80:3308 with SMTP id 4fb4d7f45d1cf-574eb803478mr12121073a12.9.1716216920816; Mon, 20 May 2024 07:55:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216920; cv=none; d=google.com; s=arc-20160816; b=s/z5LavdGn8xI79sjpiSoE7TnDBQHQR9Gt2O8GpVz/kuHT0bwGiCo2Y5lO5mJ6Ljj8 QQl98n+Wk/Vj3GZpRRpNyLrS1f/NyHwwi7X+/Z3pFJaZu5tSRUrJ1p4D2cBu9+5oROp9 cdtJfBgTxanCK0QoAmaG0aVg7iV5bhMFr5kF49mMwXE1vYBYMhBZw6CXHQ+4iz9WknJk sjiIGdqCAhs6WWlNMoO8o4No4UmzPPSlquB+ITOm3c3pCQ8nzk82XYfDQOwzu6WB0Pbl hbLAeiYGpbJJjLx5SPUwLVwYUrPMJTZLjUy+NwpI/6ZnD4OszCeUN84f/pxr3v/9PeLA tgMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=MvrM3lM7/mlEwlvPMcGxITsWo5q2/lZ1TEP4NTSdJyI=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=mbW9R66UjHvlfkISGKACSZL9+KtLLX+XEFXmOg3ssi2ZomTZDxohAfJGdE/K/RIdTZ mwCYEVmPwqOCDTezeoNeicGaK8vctCE3M7RTVWelkK8YZyq32Xl8gxWqEej0xUSNyIPR bgyFlTvV8D9+ehZm7rN9q+Q+ObDq/ruvsIU0Eqcvc8OcMXSz09kl91SQxPOsCQ/MReRU Jf8/A9XXAYrZiL2SJeLN4Q1Uoa7PgaL+EUkeQ8WM1zgS232XObEfELQRWG9COyN/7APS jS6Me9tUprZ4JNjlUcNCUCiha+N/vHMNMehZekeTeFywX0A2VnkaoSCeIqle/G83Ea9m p+1w==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HCkVQyQz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5733beac891si13716046a12.73.2024.05.20.07.55.20; Mon, 20 May 2024 07:55:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=HCkVQyQz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D1C0868D20C; Mon, 20 May 2024 17:54:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 83C6968D1AF for ; Mon, 20 May 2024 17:54:12 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216853; x=1747752853; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o6JQCmdbK+Pobn9rQYnsXpsrrgpYHkAr8T3wtER+ckw=; b=HCkVQyQzvx0e2Rkg51+73DyFded3vHK0MlXxk853C4tSyVPFLiH6iBqZ 1Of8SzmbHbsWvTRRyHtZaCiXQbP9dx1874vCfUsPotscJr/QazPhFfd2l gpeSP/jNtF1pvVj8EPQsgcmfSgfMuOtC7yIun9s6m5YewvMtBAw96/5t9 dzTTwmCyXfrD0odxk+Afwco0qxS4OFBT13/cPxmC0qzUXRr9yq+h2pMBF T8qOtgojY9CA6OL00jPT4+UVvXxyXumdT7moqkAzi6Ni5pngV8Q/oAz1c AcaT3v6fkWStA1gjQEHxT8PnMMPMp5oyffudOkvSozIUY04eahLdRMRDN A==; X-CSE-ConnectionGUID: MpwiSoNHRRCGAGOaqyaBMg== X-CSE-MsgGUID: bySHcGBxROqLLZ1LqxE5vA== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197254" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197254" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:04 -0700 X-CSE-ConnectionGUID: ktvjj63GS+SCZGCygiHSlg== X-CSE-MsgGUID: cKsGvpUTTTCLujDtUR8ysQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787901" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:03 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:18 +0800 Message-ID: <20240520145222.291-9-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 09/13] avcodec/vaapi_encode: extract a get_recon_format function to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mHCwYumZzvCe From: Tong Wu Surface size and block size parameters are also moved to base layer. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 58 +++++++++++++++++++++++ libavcodec/hw_base_encode.h | 12 +++++ libavcodec/vaapi_encode.c | 81 ++++++++------------------------- libavcodec/vaapi_encode.h | 10 ---- libavcodec/vaapi_encode_av1.c | 10 ++-- libavcodec/vaapi_encode_h264.c | 11 +++-- libavcodec/vaapi_encode_h265.c | 25 +++++----- libavcodec/vaapi_encode_mjpeg.c | 5 +- libavcodec/vaapi_encode_vp9.c | 6 +-- 9 files changed, 118 insertions(+), 100 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index df820c2f83..42c40cb48f 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -690,6 +690,64 @@ int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32 return 0; } +int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void *hwconfig, enum AVPixelFormat *fmt) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + AVHWFramesConstraints *constraints = NULL; + enum AVPixelFormat recon_format; + int err, i; + + constraints = av_hwdevice_get_hwframe_constraints(ctx->device_ref, + hwconfig); + if (!constraints) { + err = AVERROR(ENOMEM); + goto fail; + } + + // Probably we can use the input surface format as the surface format + // of the reconstructed frames. If not, we just pick the first (only?) + // format in the valid list and hope that it all works. + recon_format = AV_PIX_FMT_NONE; + if (constraints->valid_sw_formats) { + for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; i++) { + if (ctx->input_frames->sw_format == + constraints->valid_sw_formats[i]) { + recon_format = ctx->input_frames->sw_format; + break; + } + } + if (recon_format == AV_PIX_FMT_NONE) { + // No match. Just use the first in the supported list and + // hope for the best. + recon_format = constraints->valid_sw_formats[0]; + } + } else { + // No idea what to use; copy input format. + recon_format = ctx->input_frames->sw_format; + } + av_log(avctx, AV_LOG_DEBUG, "Using %s as format of " + "reconstructed frames.\n", av_get_pix_fmt_name(recon_format)); + + if (ctx->surface_width < constraints->min_width || + ctx->surface_height < constraints->min_height || + ctx->surface_width > constraints->max_width || + ctx->surface_height > constraints->max_height) { + av_log(avctx, AV_LOG_ERROR, "Hardware does not support encoding at " + "size %dx%d (constraints: width %d-%d height %d-%d).\n", + ctx->surface_width, ctx->surface_height, + constraints->min_width, constraints->max_width, + constraints->min_height, constraints->max_height); + err = AVERROR(EINVAL); + goto fail; + } + + *fmt = recon_format; + err = 0; +fail: + av_hwframe_constraints_free(&constraints); + return err; +} + int ff_hw_base_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index d363819329..7d40da039c 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -127,6 +127,16 @@ typedef struct HWBaseEncodeContext { // Desired B frame reference depth. int desired_b_depth; + // The required size of surfaces. This is probably the input + // size (AVCodecContext.width|height) aligned up to whatever + // block size is required by the codec. + int surface_width; + int surface_height; + + // The block size for slice calculations. + int slice_block_width; + int slice_block_height; + // The hardware device context. AVBufferRef *device_ref; AVHWDeviceContext *device; @@ -211,6 +221,8 @@ int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, int flags, int prediction_pre_only); +int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void *hwconfig, enum AVPixelFormat *fmt); + int ff_hw_base_encode_init(AVCodecContext *avctx); int ff_hw_base_encode_close(AVCodecContext *avctx); diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 6ab1b633ed..98b8c82da3 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1777,6 +1777,7 @@ static av_cold int vaapi_encode_init_tile_slice_structure(AVCodecContext *avctx, static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAConfigAttrib attr[3] = { { VAConfigAttribEncMaxSlices }, { VAConfigAttribEncSliceStructure }, @@ -1796,12 +1797,12 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return 0; } - av_assert0(ctx->slice_block_height > 0 && ctx->slice_block_width > 0); + av_assert0(base_ctx->slice_block_height > 0 && base_ctx->slice_block_width > 0); - ctx->slice_block_rows = (avctx->height + ctx->slice_block_height - 1) / - ctx->slice_block_height; - ctx->slice_block_cols = (avctx->width + ctx->slice_block_width - 1) / - ctx->slice_block_width; + ctx->slice_block_rows = (avctx->height + base_ctx->slice_block_height - 1) / + base_ctx->slice_block_height; + ctx->slice_block_cols = (avctx->width + base_ctx->slice_block_width - 1) / + base_ctx->slice_block_width; if (avctx->slices <= 1 && !ctx->tile_rows && !ctx->tile_cols) { ctx->nb_slices = 1; @@ -2023,7 +2024,8 @@ static void vaapi_encode_free_output_buffer(FFRefStructOpaque opaque, static int vaapi_encode_alloc_output_buffer(FFRefStructOpaque opaque, void *obj) { AVCodecContext *avctx = opaque.nc; - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; VABufferID *buffer_id = obj; VAStatus vas; @@ -2033,7 +2035,7 @@ static int vaapi_encode_alloc_output_buffer(FFRefStructOpaque opaque, void *obj) // bound on that. vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, VAEncCodedBufferType, - 3 * ctx->surface_width * ctx->surface_height + + 3 * base_ctx->surface_width * base_ctx->surface_height + (1 << 16), 1, 0, buffer_id); if (vas != VA_STATUS_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to create bitstream " @@ -2051,9 +2053,8 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; AVVAAPIHWConfig *hwconfig = NULL; - AVHWFramesConstraints *constraints = NULL; enum AVPixelFormat recon_format; - int err, i; + int err; hwconfig = av_hwdevice_hwconfig_alloc(base_ctx->device_ref); if (!hwconfig) { @@ -2062,52 +2063,9 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) } hwconfig->config_id = ctx->va_config; - constraints = av_hwdevice_get_hwframe_constraints(base_ctx->device_ref, - hwconfig); - if (!constraints) { - err = AVERROR(ENOMEM); - goto fail; - } - - // Probably we can use the input surface format as the surface format - // of the reconstructed frames. If not, we just pick the first (only?) - // format in the valid list and hope that it all works. - recon_format = AV_PIX_FMT_NONE; - if (constraints->valid_sw_formats) { - for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; i++) { - if (base_ctx->input_frames->sw_format == - constraints->valid_sw_formats[i]) { - recon_format = base_ctx->input_frames->sw_format; - break; - } - } - if (recon_format == AV_PIX_FMT_NONE) { - // No match. Just use the first in the supported list and - // hope for the best. - recon_format = constraints->valid_sw_formats[0]; - } - } else { - // No idea what to use; copy input format. - recon_format = base_ctx->input_frames->sw_format; - } - av_log(avctx, AV_LOG_DEBUG, "Using %s as format of " - "reconstructed frames.\n", av_get_pix_fmt_name(recon_format)); - - if (ctx->surface_width < constraints->min_width || - ctx->surface_height < constraints->min_height || - ctx->surface_width > constraints->max_width || - ctx->surface_height > constraints->max_height) { - av_log(avctx, AV_LOG_ERROR, "Hardware does not support encoding at " - "size %dx%d (constraints: width %d-%d height %d-%d).\n", - ctx->surface_width, ctx->surface_height, - constraints->min_width, constraints->max_width, - constraints->min_height, constraints->max_height); - err = AVERROR(EINVAL); + err = ff_hw_base_get_recon_format(avctx, (const void*)hwconfig, &recon_format); + if (err < 0) goto fail; - } - - av_freep(&hwconfig); - av_hwframe_constraints_free(&constraints); base_ctx->recon_frames_ref = av_hwframe_ctx_alloc(base_ctx->device_ref); if (!base_ctx->recon_frames_ref) { @@ -2118,8 +2076,8 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) base_ctx->recon_frames->format = AV_PIX_FMT_VAAPI; base_ctx->recon_frames->sw_format = recon_format; - base_ctx->recon_frames->width = ctx->surface_width; - base_ctx->recon_frames->height = ctx->surface_height; + base_ctx->recon_frames->width = base_ctx->surface_width; + base_ctx->recon_frames->height = base_ctx->surface_height; err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); if (err < 0) { @@ -2131,7 +2089,6 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) err = 0; fail: av_freep(&hwconfig); - av_hwframe_constraints_free(&constraints); return err; } @@ -2174,11 +2131,11 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } else { // Assume 16x16 blocks. - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); + base_ctx->surface_width = FFALIGN(avctx->width, 16); + base_ctx->surface_height = FFALIGN(avctx->height, 16); if (ctx->codec->flags & FLAG_SLICE_CONTROL) { - ctx->slice_block_width = 16; - ctx->slice_block_height = 16; + base_ctx->slice_block_width = 16; + base_ctx->slice_block_height = 16; } } @@ -2231,7 +2188,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) recon_hwctx = base_ctx->recon_frames->hwctx; vas = vaCreateContext(ctx->hwctx->display, ctx->va_config, - ctx->surface_width, ctx->surface_height, + base_ctx->surface_width, base_ctx->surface_height, VA_PROGRESSIVE, recon_hwctx->surface_ids, recon_hwctx->nb_surfaces, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 76fb645d71..8c7568e9ae 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -171,16 +171,6 @@ typedef struct VAAPIEncodeContext { // Desired packed headers. unsigned int desired_packed_headers; - // The required size of surfaces. This is probably the input - // size (AVCodecContext.width|height) aligned up to whatever - // block size is required by the codec. - int surface_width; - int surface_height; - - // The block size for slice calculations. - int slice_block_width; - int slice_block_height; - // Everything above this point must be set before calling // ff_vaapi_encode_init(). diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index a15b74f409..a767d22d37 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -112,12 +112,12 @@ static void vaapi_encode_av1_trace_write_log(void *ctx, static av_cold int vaapi_encode_av1_get_encoder_caps(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodeAV1Context *priv = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; // Surfaces must be aligned to superblock boundaries. - ctx->surface_width = FFALIGN(avctx->width, priv->use_128x128_superblock ? 128 : 64); - ctx->surface_height = FFALIGN(avctx->height, priv->use_128x128_superblock ? 128 : 64); + base_ctx->surface_width = FFALIGN(avctx->width, priv->use_128x128_superblock ? 128 : 64); + base_ctx->surface_height = FFALIGN(avctx->height, priv->use_128x128_superblock ? 128 : 64); return 0; } @@ -425,7 +425,7 @@ static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) framerate = 0; level = ff_av1_guess_level(avctx->bit_rate, priv->tier, - ctx->surface_width, ctx->surface_height, + base_ctx->surface_width, base_ctx->surface_height, priv->tile_rows * priv->tile_cols, priv->tile_cols, framerate); if (level) { diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index d41b9649e8..f69bbeebc1 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -1205,8 +1205,9 @@ static const VAAPIEncodeType vaapi_encode_type_h264 = { static av_cold int vaapi_encode_h264_init(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodeH264Context *priv = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeH264Context *priv = avctx->priv_data; ctx->codec = &vaapi_encode_type_h264; @@ -1254,10 +1255,10 @@ static av_cold int vaapi_encode_h264_init(AVCodecContext *avctx) VA_ENC_PACKED_HEADER_SLICE | // Slice headers. VA_ENC_PACKED_HEADER_MISC; // SEI. - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); + base_ctx->surface_width = FFALIGN(avctx->width, 16); + base_ctx->surface_height = FFALIGN(avctx->height, 16); - ctx->slice_block_height = ctx->slice_block_width = 16; + base_ctx->slice_block_height = base_ctx->slice_block_width = 16; if (priv->qp > 0) ctx->explicit_qp = priv->qp; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index ec48773d95..ff153f778a 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -352,7 +352,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) const H265LevelDescriptor *level; level = ff_h265_guess_level(ptl, avctx->bit_rate, - ctx->surface_width, ctx->surface_height, + base_ctx->surface_width, base_ctx->surface_height, ctx->nb_slices, ctx->tile_rows, ctx->tile_cols, (base_ctx->b_per_p > 0) + 1); if (level) { @@ -410,18 +410,18 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) sps->chroma_format_idc = chroma_format; sps->separate_colour_plane_flag = 0; - sps->pic_width_in_luma_samples = ctx->surface_width; - sps->pic_height_in_luma_samples = ctx->surface_height; + sps->pic_width_in_luma_samples = base_ctx->surface_width; + sps->pic_height_in_luma_samples = base_ctx->surface_height; - if (avctx->width != ctx->surface_width || - avctx->height != ctx->surface_height) { + if (avctx->width != base_ctx->surface_width || + avctx->height != base_ctx->surface_height) { sps->conformance_window_flag = 1; sps->conf_win_left_offset = 0; sps->conf_win_right_offset = - (ctx->surface_width - avctx->width) >> desc->log2_chroma_w; + (base_ctx->surface_width - avctx->width) >> desc->log2_chroma_w; sps->conf_win_top_offset = 0; sps->conf_win_bottom_offset = - (ctx->surface_height - avctx->height) >> desc->log2_chroma_h; + (base_ctx->surface_height - avctx->height) >> desc->log2_chroma_h; } else { sps->conformance_window_flag = 0; } @@ -1197,11 +1197,12 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodeH265Context *priv = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeH265Context *priv = avctx->priv_data; #if VA_CHECK_VERSION(1, 13, 0) { + VAAPIEncodeContext *ctx = avctx->priv_data; VAConfigAttribValEncHEVCBlockSizes block_size; VAConfigAttrib attr; VAStatus vas; @@ -1249,10 +1250,10 @@ static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) "min CB size %dx%d.\n", priv->ctu_size, priv->ctu_size, priv->min_cb_size, priv->min_cb_size); - ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); - ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); + base_ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); + base_ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); - ctx->slice_block_width = ctx->slice_block_height = priv->ctu_size; + base_ctx->slice_block_width = base_ctx->slice_block_height = priv->ctu_size; return 0; } diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index 24acc88a21..6ed58d7118 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -439,14 +439,13 @@ static int vaapi_encode_mjpeg_init_slice_params(AVCodecContext *avctx, static av_cold int vaapi_encode_mjpeg_get_encoder_caps(AVCodecContext *avctx) { HWBaseEncodeContext *base_ctx = avctx->priv_data; - VAAPIEncodeContext *ctx = avctx->priv_data; const AVPixFmtDescriptor *desc; desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); - ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); - ctx->surface_height = FFALIGN(avctx->height, 8 << desc->log2_chroma_h); + base_ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); + base_ctx->surface_height = FFALIGN(avctx->height, 8 << desc->log2_chroma_h); return 0; } diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 36dafea2c0..5e1bf3a7c4 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -190,11 +190,11 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, static av_cold int vaapi_encode_vp9_get_encoder_caps(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; // Surfaces must be aligned to 64x64 superblock boundaries. - ctx->surface_width = FFALIGN(avctx->width, 64); - ctx->surface_height = FFALIGN(avctx->height, 64); + base_ctx->surface_width = FFALIGN(avctx->width, 64); + base_ctx->surface_height = FFALIGN(avctx->height, 64); return 0; } From patchwork Mon May 20 14:52:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49059 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514651pzb; Mon, 20 May 2024 07:55:31 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVcdqJuhQQFAaURQvZKxoYwQWG+DI7Euw4Drzkz0ULe7h9YzMpwhjhVmSJU3zZHtComUpJiNcDoynRmEkk6q1j35hqUJgTUgS92mg== X-Google-Smtp-Source: AGHT+IHGBn9DR5EVlTHtHXRXuPKILBqHsJVcvTTpZinmdLss/HjWV5V5QrXa0F+MXTFa4GbMc7f3 X-Received: by 2002:a17:907:2685:b0:a5c:daf2:1cfb with SMTP id a640c23a62f3a-a5cdaf29605mr1845679366b.63.1716216931155; Mon, 20 May 2024 07:55:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216931; cv=none; d=google.com; s=arc-20160816; b=gOg11xsoannWdO1ZfQcnu01YZAeaUvjiRl3xHgLdFVMvVZANI5Bwaixp4fGO+XcKaT gjJUVBslCe/3gf7IbHM4Al27QMAL0hphABOWTunN783sUGwUNND9/+HHNwOrv5MXT0BY 5qxJfiIXlkrir37ZcuyAAhdkh1GhY1Evhp02tBMjINfXnuFpxduqelQRETD+4niaM/73 7hvvEze9jeb4QWOlI3B1NmD0fvb0C1XDlC6NkD1pbDfJ0ATDU1yNXUVYrcFJd1A33TQJ /Jp0r49LGqQIqDPCH8/qFA+Y9CPd40VGjgLWhmCrQq0/WK9Fd/UUIpSpWfJJQubJOpNL +HLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=wfyUQ2gNra6aUKJRctOZaKmOEBfF+ZhD29YktJwxr7Q=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=UXuKDN9rVsLfuU4wl/SZDWCa0puGjSDfeJVPZOBO6bdx1iPaFkQCjlIqXOLH/WaA9O bqvVx+rj47p0xvWH5ScuZzzhqEMulb38gXRgbz/1cR+vs5zXypgDnuJEoOEDE9Msb/wF egEbStnSANcX9nLW4rxYeH0oKskjZMMvo/vOiKN65AmzwA+GD0UMOW6CeQQhazZ/yxDT 2Gupj/gJxFprJ79p5MVxct+4hqbBI7XfliU4KkgE2KJMa9UEeUuMWLT5vhxyi9dJ5kGk Rkmsi8vsY9NTlOH2fQzqJuOB8p9gqQTc/lqNOt1UpmDgV0n+t6ujqbHz3VZfqcbju9It Sjbw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TMXXLQ5M; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a5b1ae877si1041171966b.998.2024.05.20.07.55.30; Mon, 20 May 2024 07:55:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=TMXXLQ5M; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B299168D2C7; Mon, 20 May 2024 17:54:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DF73468D255 for ; Mon, 20 May 2024 17:54:12 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216853; x=1747752853; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cQWovVr5I/lD/RbrUYvuZefltPgzt8wh96+kMJzho40=; b=TMXXLQ5M7nxdj0sBgtBRbRYCinee/kXuFdirKyp/NNeMT/bG4yW+BkL/ gdQE9DOo+IuyHVC1lb7rkEAHozwfShkyYBLXJJH38ew22RmE1oHAbKcdX FBXenwZm2RyjDb/RTa0xnB50v/cKa/rvJRS4PjndVbyHFQFiJytooNhnR 2DeVyOIkAHTVqYoNF8oK4TYd5w2/nD2WcI9/vz0DSOwHpe+jcG/eLj0Qb nPZsUPNGblb0Arg/CseRRNi/67lDEr5ZA4WYh3lJCrT8UWOEs+mB3Ezx2 Km6pBRWMFaRCDdkNxIZ8pUk+usZyIRno3t8gWEAMggkn00MDjePbnAnXQ A==; X-CSE-ConnectionGUID: V3G1Fk7cRpKWEXXkA98Ang== X-CSE-MsgGUID: yUMkEYayQWafBBdKyGm2HA== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197255" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197255" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:05 -0700 X-CSE-ConnectionGUID: iW2JQ96TTqSdVRvDey83Tg== X-CSE-MsgGUID: otDhmvNtQvKw9Tvq8dAQvg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787910" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:04 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:19 +0800 Message-ID: <20240520145222.291-10-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 10/13] avcodec/vaapi_encode: extract a free funtion to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bbeAdRzz8v9B From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 11 +++++++++++ libavcodec/hw_base_encode.h | 2 ++ libavcodec/vaapi_encode.c | 6 +----- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index 42c40cb48f..f743d119cd 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -748,6 +748,17 @@ fail: return err; } +int ff_hw_base_encode_free(AVCodecContext *avctx, HWBaseEncodePicture *pic) +{ + av_frame_free(&pic->input_image); + av_frame_free(&pic->recon_image); + + av_buffer_unref(&pic->opaque_ref); + av_freep(&pic->priv_data); + + return 0; +} + int ff_hw_base_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 7d40da039c..76a39e2d97 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -223,6 +223,8 @@ int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32 int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void *hwconfig, enum AVPixelFormat *fmt); +int ff_hw_base_encode_free(AVCodecContext *avctx, HWBaseEncodePicture *pic); + int ff_hw_base_encode_init(AVCodecContext *avctx); int ff_hw_base_encode_close(AVCodecContext *avctx); diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 98b8c82da3..e89d6e01af 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -878,17 +878,13 @@ static int vaapi_encode_free(AVCodecContext *avctx, av_freep(&pic->slices[i].codec_slice_params); } - av_frame_free(&base_pic->input_image); - av_frame_free(&base_pic->recon_image); - - av_buffer_unref(&base_pic->opaque_ref); + ff_hw_base_encode_free(avctx, base_pic); av_freep(&pic->param_buffers); av_freep(&pic->slices); // Output buffer should already be destroyed. av_assert0(pic->output_buffer == VA_INVALID_ID); - av_freep(&base_pic->priv_data); av_freep(&pic->codec_picture_params); av_freep(&pic->roi); From patchwork Mon May 20 14:52:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49060 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514764pzb; Mon, 20 May 2024 07:55:41 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVCFO0dAuW/7YfweVuuA7uGpIGdeUVDuz0PTFmBwJzQp5VCVYXmbnJuhHy0dXXrTs5gii6fBxZtpuZjNoDX0S+z/JMittdu6m9tTg== X-Google-Smtp-Source: AGHT+IGOa3Yma8lTJjS3BuCfZ/qgeqo61EKFV4K33QLIlYXZLPiAb1S+5sCYGH0RHRKpmuG7XwdP X-Received: by 2002:a17:906:da8d:b0:a58:ea85:9114 with SMTP id a640c23a62f3a-a5a2d672fb6mr2411691866b.50.1716216941439; Mon, 20 May 2024 07:55:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216941; cv=none; d=google.com; s=arc-20160816; b=qxHgGIEXO24oeYIURQNIqdfSr/t27Bve6JDSJqeHrulFLMeV/FKVQ9X4Sa2hWyX2hT NmADav0kLxgB3UUT7Wjcyn6f23F1tfnKDiPlBiZ2p3hbLZKY5SlZF3qlDQEg/diWnAOV eXFlxfn4FmJksUfNJXPNHAXIoBW+pk9YZsdM+TQor8IeJTDBTjgBW1TvqPa2FM/rPSjO HpSXUAQFwTWDNrDShAYnhcLJrz7Z+Gbt+CI/XkhOWAddrC0DW8Lwb7fwyACaPsnJVwmr 3a7uu18Ku9yiyyMU3s79PXK6oPxIaDhTpjWsyuw5jevWiTy7W8EqaJlGp+LlsPelLbDd Lb8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=79tqFLIMODwhIuznLIT2Ppcghi2g4eTsncAXJwJJEyQ=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=x4Tm3J7Hj1/DmXoArvYt0Mb/0FNY55g0j3UXlkqJuLD68tAJKjNq7QujRb789448GG 1cDS1jqolXuceHRE8oGoNvf0MkrUhw0EKWM4Q/OLN+QcuG3pUCXyqbvcb7qiIoRg5gFE J/bwXEkqsJFYgeQNvjGEJFxtWNtEB/GEmNKZUP8K2Nr3PdVYXpyz7KrcPM3JJDrhfKjS YAgM7qD/0GZXAMPhOVtV175hmTM6omI/K/BlCVbLfxG5ln/Ewj2DA6JaR5GwVe0M4Nlv cAECNJV5kcz+3CCcnsq1Vhvuk6mM70nZA1ifYFFrLA2oZro/eD1DhjqjdMvcLlMSNOsE +VLw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=CZj+5tLv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17bffec9si1271917566b.880.2024.05.20.07.55.40; Mon, 20 May 2024 07:55:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=CZj+5tLv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9D14B68D212; Mon, 20 May 2024 17:54:18 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 968A268D255 for ; Mon, 20 May 2024 17:54:13 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216854; x=1747752854; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fT+oUvN6amkXLtGkG8lLLmKv+i7G2kWQBqW1fywKgKA=; b=CZj+5tLv3eiiYYZz81WPFKHX/O4dj60V3ZPyHwu9oQNdHosHA9WOkuR8 HPbUxrVrf0wbKLvLmdIHI79m2I3nD8WtSAW0zqLvl8cVVY18Y8UCwYIS9 JrgztR4wphMq4Qwgss9cikE0lALD7XYl99BA7lKOSqSqY/olTjspNhzwt AcXXyW8SHF3lVH80Oh/9s7r5Peu/BjcmMUVBXO5qbhTPI6IU6nNJqUmYs 2A60CK5bT22WxhfxlBD/CC8W9GO8cuvcUMCkH5A0HIJjXKan9oqCFm2ew cIfp2jFxYfEnMrf7gtz0ZNRnF1aABqlA/s6N+ZqFUnfC0pXIxDGkaxfWc Q==; X-CSE-ConnectionGUID: vjbltCPtQQeCdC/+5ZPx0A== X-CSE-MsgGUID: Y7ntNyWERE2PlUJcgepTqg== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197256" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197256" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:06 -0700 X-CSE-ConnectionGUID: 9wvWlkk7Qai9Myk5h8a1sA== X-CSE-MsgGUID: FU8fhiNBQtCVRxrcPK1FAQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787916" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:05 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:20 +0800 Message-ID: <20240520145222.291-11-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 11/13] avutil/hwcontext_d3d12va: add Flags for resource creation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Td3WXKK1s9MB From: Tong Wu Flags field is added to support diffferent resource creation. Signed-off-by: Tong Wu --- doc/APIchanges | 3 +++ libavutil/hwcontext_d3d12va.c | 2 +- libavutil/hwcontext_d3d12va.h | 8 ++++++++ libavutil/version.h | 2 +- 4 files changed, 13 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 269fd36559..808ba02f2d 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-01-xx - xxxxxxxxxx - lavu 59.20.100 - hwcontext_d3d12va.h + Add AVD3D12VAFramesContext.flags + 2024-05-xx - xxxxxxxxxx - lavu 59.19.100 - hwcontext_qsv.h Add AVQSVFramesContext.info diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c index cfc016315d..6507cf69c1 100644 --- a/libavutil/hwcontext_d3d12va.c +++ b/libavutil/hwcontext_d3d12va.c @@ -247,7 +247,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size) .Format = hwctx->format, .SampleDesc = {.Count = 1, .Quality = 0 }, .Layout = D3D12_TEXTURE_LAYOUT_UNKNOWN, - .Flags = D3D12_RESOURCE_FLAG_NONE, + .Flags = hwctx->flags, }; frame = av_mallocz(sizeof(AVD3D12VAFrame)); diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h index ff06e6f2ef..608dbac97f 100644 --- a/libavutil/hwcontext_d3d12va.h +++ b/libavutil/hwcontext_d3d12va.h @@ -129,6 +129,14 @@ typedef struct AVD3D12VAFramesContext { * If unset, will be automatically set. */ DXGI_FORMAT format; + + /** + * This field is used to specify options for working with resources. + * If unset, this will be D3D12_RESOURCE_FLAG_NONE. + * + * @see: https://learn.microsoft.com/en-us/windows/win32/api/d3d12/ne-d3d12-d3d12_resource_flags. + */ + D3D12_RESOURCE_FLAGS flags; } AVD3D12VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D12VA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 3221c4c592..9c7146c228 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 19 +#define LIBAVUTIL_VERSION_MINOR 20 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Mon May 20 14:52:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49061 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4514901pzb; Mon, 20 May 2024 07:55:52 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVUcA/JYyXJ/LHKWcciDcClT2nowzTCh8U1wKqTGMZX6QDG8JTH6L1hIkSFPjB17jEWpQwWVBonoL+yHvRvxx9moxf97QWeKBTCAg== X-Google-Smtp-Source: AGHT+IFICgsOxZcSO6SFCNz6+/C/MsdcKl7hiQQolLZLU30/0x/h0DHVcyFocXi/eRZbXIy+85/X X-Received: by 2002:a17:907:1b08:b0:a5a:62c9:bdf3 with SMTP id a640c23a62f3a-a5a62c9be77mr1766413866b.1.1716216952546; Mon, 20 May 2024 07:55:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216952; cv=none; d=google.com; s=arc-20160816; b=V6nG23rz3wHcq6lcpUSRR9YZ5Jh+99i1nUWUdKMj8ygLKN5YAtkZ8Dt74cLLLnNWZj jj4vcxiGnLwEdg+V3PWBkZBu1YjVZgn9d+zMUMw4wOho65QC7pEMYjAxsTCALoWzhjFa ZsxHOokMDWLocw2p18V696VYoLbMX9NzlQmtQPQRtwvByVex4bJHJR5KDavGTZ9iOH3d F93rYK6DYAzUCutdq1lkkGlNFGiCcYPI+AsGsXMnI51ejP4iWbX0W+NPO0M/Me21+XWk 5UWhBoiD8UlH8sN+UhisTs1PprPCJ9p+sbMU3f7if2kfk1dzlcEQA/VME0JymztSKwsF rfQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=8rCW+mWUToVM/ZVam6C/TnBWqRalwgiyBK1hJHZUe00=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=xdM2H5tEOFqFUENdPlO9NJLsfVG+44K3OSQxzZa5f/hj0IqsOOQrW3fUvRlQ6qvZO2 ZELoXeGRGKC6vRMHRMwPcS/tNLgN2CG960idSusANX6yadSehLwJh3CVXJi2qcANPON1 zzFpzlJWiW6qcDdPU1L1hRogipztvQAURxFL/ZX2LcqttMuCvIJuwYhbuO6sjbbueGaI W3qTbSFjMf0HkeRYnOhSUNhEo+j8mbqhp7WdU/ozrT5lEaUGqMhjbG9unGJQSo3G++oT RV2UEfNY6sDdDuLvUc+rS5Ehd9+wDlUWR5NtVP4TGiY7PBzm/skwL5D3Iw3wtAncvegq 1Tcg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Fe08foUp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5ce194bf3bsi529685066b.701.2024.05.20.07.55.51; Mon, 20 May 2024 07:55:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Fe08foUp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3915A68D2FE; Mon, 20 May 2024 17:54:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CAE1B68D22E for ; Mon, 20 May 2024 17:54:13 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216854; x=1747752854; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9hJwgHP4vPl9C5DBCzmOUtLDISkDKKFLyu89+pdbtNg=; b=Fe08foUpUozq5iWns790CAnb1tbPGaLz8QYP8+hwj4x07K4QX2Cm1/Hk FWKHPPT71dinzVcY5HyeRxMV+agwPFXJ/nO/ooEbGkUrY8i8B7P1xqx4c taXy1/2lKRcUP5QH63WNJ1tMBqsiUz85JE80E6CAoeUhuEerTSBWSHRH5 D4wcx/lnrzG/WD6+fkEhWtzit6NvZUMUkI/jcNIL6xDSkmMQ3IvwYhLSd OsJeBwnB7OmhxVgTVWc68ifL06SWGW7sQGeMWd4sJUq3vhjPfw7ibcqmA kA5dGLqpgICPDQE1sFfH9TDEbiwourZmBndrlzghDmlyzerzLtaRd0D27 w==; X-CSE-ConnectionGUID: AbT8MZ8JSPWY594Nqbn0Sg== X-CSE-MsgGUID: CTnXj9smTmG8qUCqdEXnAw== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197257" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197257" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:07 -0700 X-CSE-ConnectionGUID: dc3uNe/oR4KAJEUyWUbfmw== X-CSE-MsgGUID: tYGsA6zXQf2utEdvurGkeQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787921" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:06 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:21 +0800 Message-ID: <20240520145222.291-12-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 12/13] avcodec: add D3D12VA hardware HEVC encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sFP3ZTCdi3BL From: Tong Wu This implementation is based on D3D12 Video Encoding Spec: https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html Sample command line for transcoding: ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4 -c:v hevc_d3d12va output.mp4 Signed-off-by: Tong Wu --- configure | 6 + libavcodec/Makefile | 5 +- libavcodec/allcodecs.c | 1 + libavcodec/d3d12va_encode.c | 1558 ++++++++++++++++++++++++++++++ libavcodec/d3d12va_encode.h | 334 +++++++ libavcodec/d3d12va_encode_hevc.c | 1007 +++++++++++++++++++ 6 files changed, 2910 insertions(+), 1 deletion(-) create mode 100644 libavcodec/d3d12va_encode.c create mode 100644 libavcodec/d3d12va_encode.h create mode 100644 libavcodec/d3d12va_encode_hevc.c diff --git a/configure b/configure index b16722d83d..127d68e60c 100755 --- a/configure +++ b/configure @@ -2551,6 +2551,7 @@ CONFIG_EXTRA=" cbs_mpeg2 cbs_vp8 cbs_vp9 + d3d12va_encode deflate_wrapper dirac_parse dnn @@ -3287,6 +3288,7 @@ wmv3_vaapi_hwaccel_select="vc1_vaapi_hwaccel" wmv3_vdpau_hwaccel_select="vc1_vdpau_hwaccel" # hardware-accelerated codecs +d3d12va_encode_deps="d3d12va ID3D12VideoEncoder d3d12_encoder_feature" mediafoundation_deps="mftransform_h MFCreateAlignedMemoryBuffer" omx_deps="libdl pthreads" omx_rpi_select="omx" @@ -3354,6 +3356,7 @@ h264_v4l2m2m_encoder_deps="v4l2_m2m h264_v4l2_m2m" hevc_amf_encoder_deps="amf" hevc_cuvid_decoder_deps="cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" +hevc_d3d12va_encoder_select="cbs_h265 d3d12va_encode" hevc_mediacodec_decoder_deps="mediacodec" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" hevc_mediacodec_encoder_deps="mediacodec" @@ -6725,6 +6728,9 @@ check_type "windows.h d3d11.h" "ID3D11VideoDecoder" check_type "windows.h d3d11.h" "ID3D11VideoContext" check_type "windows.h d3d12.h" "ID3D12Device" check_type "windows.h d3d12video.h" "ID3D12VideoDecoder" +check_type "windows.h d3d12video.h" "ID3D12VideoEncoder" +test_code cc "windows.h d3d12video.h" "D3D12_FEATURE_VIDEO feature = D3D12_FEATURE_VIDEO_ENCODER_CODEC" && \ +test_code cc "windows.h d3d12video.h" "D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS req" && enable d3d12_encoder_feature check_type "windows.h" "DPI_AWARENESS_CONTEXT" -D_WIN32_WINNT=0x0A00 check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -D_WIN32_WINNT=0x0602 check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 998f6b7e12..6c4500ce6d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -86,6 +86,7 @@ OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP8) += cbs_vp8.o vp8data.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o +OBJS-$(CONFIG_D3D12VA_ENCODE) += d3d12va_encode.o hw_base_encode.o OBJS-$(CONFIG_DEFLATE_WRAPPER) += zlib_wrapper.o OBJS-$(CONFIG_DOVI_RPUDEC) += dovi_rpu.o dovi_rpudec.o OBJS-$(CONFIG_DOVI_RPUENC) += dovi_rpu.o dovi_rpuenc.o @@ -436,6 +437,8 @@ OBJS-$(CONFIG_HEVC_DECODER) += hevcdec.o hevc_mvs.o \ h274.o aom_film_grain.o OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o +OBJS-$(CONFIG_HEVC_D3D12VA_ENCODER) += d3d12va_encode_hevc.o h265_profile_level.o \ + h2645data.o OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_HEVC_MF_ENCODER) += mfenc.o mf_utils.o @@ -1265,7 +1268,7 @@ SKIPHEADERS += %_tablegen.h \ SKIPHEADERS-$(CONFIG_AMF) += amfenc.h SKIPHEADERS-$(CONFIG_D3D11VA) += d3d11va.h dxva2_internal.h -SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h +SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h d3d12va_encode.h SKIPHEADERS-$(CONFIG_DXVA2) += dxva2.h dxva2_internal.h SKIPHEADERS-$(CONFIG_JNI) += ffjni.h SKIPHEADERS-$(CONFIG_LCMS2) += fflcms2.h diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index b102a8069e..463ffbbd08 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -848,6 +848,7 @@ extern const FFCodec ff_h264_vaapi_encoder; extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; extern const FFCodec ff_hevc_cuvid_decoder; +extern const FFCodec ff_hevc_d3d12va_encoder; extern const FFCodec ff_hevc_mediacodec_decoder; extern const FFCodec ff_hevc_mediacodec_encoder; extern const FFCodec ff_hevc_mf_encoder; diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c new file mode 100644 index 0000000000..a20e6cc961 --- /dev/null +++ b/libavcodec/d3d12va_encode.c @@ -0,0 +1,1558 @@ +/* + * Direct3D 12 HW acceleration video encoder + * + * Copyright (c) 2024 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/internal.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" +#include "libavutil/hwcontext_d3d12va.h" + +#include "avcodec.h" +#include "d3d12va_encode.h" +#include "encode.h" + +const AVCodecHWConfigInternal *const ff_d3d12va_encode_hw_configs[] = { + HW_CONFIG_ENCODER_FRAMES(D3D12, D3D12VA), + NULL, +}; + +static int d3d12va_fence_completion(AVD3D12VASyncContext *psync_ctx) +{ + uint64_t completion = ID3D12Fence_GetCompletedValue(psync_ctx->fence); + if (completion < psync_ctx->fence_value) { + if (FAILED(ID3D12Fence_SetEventOnCompletion(psync_ctx->fence, psync_ctx->fence_value, psync_ctx->event))) + return AVERROR(EINVAL); + + WaitForSingleObjectEx(psync_ctx->event, INFINITE, FALSE); + } + + return 0; +} + +static int d3d12va_sync_with_gpu(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + + DX_CHECK(ID3D12CommandQueue_Signal(ctx->command_queue, ctx->sync_ctx.fence, ++ctx->sync_ctx.fence_value)); + return d3d12va_fence_completion(&ctx->sync_ctx); + +fail: + return AVERROR(EINVAL); +} + +typedef struct CommandAllocator { + ID3D12CommandAllocator *command_allocator; + uint64_t fence_value; +} CommandAllocator; + +static int d3d12va_get_valid_command_allocator(AVCodecContext *avctx, ID3D12CommandAllocator **ppAllocator) +{ + HRESULT hr; + D3D12VAEncodeContext *ctx = avctx->priv_data; + CommandAllocator allocator; + + if (av_fifo_peek(ctx->allocator_queue, &allocator, 1, 0) >= 0) { + uint64_t completion = ID3D12Fence_GetCompletedValue(ctx->sync_ctx.fence); + if (completion >= allocator.fence_value) { + *ppAllocator = allocator.command_allocator; + av_fifo_read(ctx->allocator_queue, &allocator, 1); + return 0; + } + } + + hr = ID3D12Device_CreateCommandAllocator(ctx->hwctx->device, D3D12_COMMAND_LIST_TYPE_VIDEO_ENCODE, + &IID_ID3D12CommandAllocator, (void **)ppAllocator); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create a new command allocator!\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int d3d12va_discard_command_allocator(AVCodecContext *avctx, ID3D12CommandAllocator *pAllocator, uint64_t fence_value) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + + CommandAllocator allocator = { + .command_allocator = pAllocator, + .fence_value = fence_value, + }; + + av_fifo_write(ctx->allocator_queue, &allocator, 1); + + return 0; +} + +static int d3d12va_encode_wait(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *base_pic = &pic->base; + uint64_t completion; + + av_assert0(base_pic->encode_issued); + + if (base_pic->encode_complete) { + // Already waited for this picture. + return 0; + } + + completion = ID3D12Fence_GetCompletedValue(ctx->sync_ctx.fence); + if (completion < pic->fence_value) { + if (FAILED(ID3D12Fence_SetEventOnCompletion(ctx->sync_ctx.fence, pic->fence_value, + ctx->sync_ctx.event))) + return AVERROR(EINVAL); + + WaitForSingleObjectEx(ctx->sync_ctx.event, INFINITE, FALSE); + } + + av_log(avctx, AV_LOG_DEBUG, "Sync to pic %"PRId64"/%"PRId64" " + "(input surface %p).\n", base_pic->display_order, + base_pic->encode_order, pic->input_surface->texture); + + av_frame_free(&base_pic->input_image); + + base_pic->encode_complete = 1; + return 0; +} + +static int d3d12va_encode_create_metadata_buffers(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + int width = sizeof(D3D12_VIDEO_ENCODER_OUTPUT_METADATA) + sizeof(D3D12_VIDEO_ENCODER_FRAME_SUBREGION_METADATA); + D3D12_HEAP_PROPERTIES encoded_meta_props = { .Type = D3D12_HEAP_TYPE_DEFAULT }, resolved_meta_props; + D3D12_HEAP_TYPE resolved_heap_type = D3D12_HEAP_TYPE_READBACK; + HRESULT hr; + + D3D12_RESOURCE_DESC meta_desc = { + .Dimension = D3D12_RESOURCE_DIMENSION_BUFFER, + .Alignment = 0, + .Width = ctx->req.MaxEncoderOutputMetadataBufferSize, + .Height = 1, + .DepthOrArraySize = 1, + .MipLevels = 1, + .Format = DXGI_FORMAT_UNKNOWN, + .SampleDesc = { .Count = 1, .Quality = 0 }, + .Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR, + .Flags = D3D12_RESOURCE_FLAG_NONE, + }; + + hr = ID3D12Device_CreateCommittedResource(ctx->hwctx->device, &encoded_meta_props, D3D12_HEAP_FLAG_NONE, + &meta_desc, D3D12_RESOURCE_STATE_COMMON, NULL, + &IID_ID3D12Resource, (void **)&pic->encoded_metadata); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create metadata buffer.\n"); + return AVERROR_UNKNOWN; + } + + ctx->hwctx->device->lpVtbl->GetCustomHeapProperties(ctx->hwctx->device, &resolved_meta_props, 0, resolved_heap_type); + + meta_desc.Width = width; + + hr = ID3D12Device_CreateCommittedResource(ctx->hwctx->device, &resolved_meta_props, D3D12_HEAP_FLAG_NONE, + &meta_desc, D3D12_RESOURCE_STATE_COMMON, NULL, + &IID_ID3D12Resource, (void **)&pic->resolved_metadata); + + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create output metadata buffer.\n"); + return AVERROR_UNKNOWN; + } + + return 0; +} + +static int d3d12va_encode_issue(AVCodecContext *avctx, + const HWBaseEncodePicture *base_pic) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + AVD3D12VAFramesContext *frames_hwctx = base_ctx->input_frames->hwctx; + D3D12VAEncodePicture *pic = (D3D12VAEncodePicture *)base_pic; + int err, i, j; + HRESULT hr; + char data[MAX_PARAM_BUFFER_SIZE]; + void *ptr; + size_t bit_len; + ID3D12CommandAllocator *command_allocator = NULL; + ID3D12VideoEncodeCommandList2 *cmd_list = ctx->command_list; + D3D12_RESOURCE_BARRIER barriers[32] = { 0 }; + D3D12_VIDEO_ENCODE_REFERENCE_FRAMES d3d12_refs = { 0 }; + + D3D12_VIDEO_ENCODER_ENCODEFRAME_INPUT_ARGUMENTS input_args = { + .SequenceControlDesc = { + .Flags = D3D12_VIDEO_ENCODER_SEQUENCE_CONTROL_FLAG_NONE, + .IntraRefreshConfig = { 0 }, + .RateControl = ctx->rc, + .PictureTargetResolution = ctx->resolution, + .SelectedLayoutMode = D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME, + .FrameSubregionsLayoutData = { 0 }, + .CodecGopSequence = ctx->gop, + }, + .pInputFrame = pic->input_surface->texture, + .InputFrameSubresource = 0, + }; + + D3D12_VIDEO_ENCODER_ENCODEFRAME_OUTPUT_ARGUMENTS output_args = { 0 }; + + D3D12_VIDEO_ENCODER_RESOLVE_METADATA_INPUT_ARGUMENTS input_metadata = { + .EncoderCodec = ctx->codec->d3d12_codec, + .EncoderProfile = ctx->profile->d3d12_profile, + .EncoderInputFormat = frames_hwctx->format, + .EncodedPictureEffectiveResolution = ctx->resolution, + }; + + D3D12_VIDEO_ENCODER_RESOLVE_METADATA_OUTPUT_ARGUMENTS output_metadata = { 0 }; + + memset(data, 0, sizeof(data)); + + av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " + "as type %s.\n", base_pic->display_order, base_pic->encode_order, + ff_hw_base_encode_get_pictype_name(base_pic->type)); + if (base_pic->nb_refs[0] == 0 && base_pic->nb_refs[1] == 0) { + av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); + } else { + av_log(avctx, AV_LOG_DEBUG, "L0 refers to"); + for (i = 0; i < base_pic->nb_refs[0]; i++) { + av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, + base_pic->refs[0][i]->display_order, base_pic->refs[0][i]->encode_order); + } + av_log(avctx, AV_LOG_DEBUG, ".\n"); + + if (base_pic->nb_refs[1]) { + av_log(avctx, AV_LOG_DEBUG, "L1 refers to"); + for (i = 0; i < base_pic->nb_refs[1]; i++) { + av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, + base_pic->refs[1][i]->display_order, base_pic->refs[1][i]->encode_order); + } + av_log(avctx, AV_LOG_DEBUG, ".\n"); + } + } + + av_assert0(!base_pic->encode_issued); + for (i = 0; i < base_pic->nb_refs[0]; i++) { + av_assert0(base_pic->refs[0][i]); + av_assert0(base_pic->refs[0][i]->encode_issued); + } + for (i = 0; i < base_pic->nb_refs[1]; i++) { + av_assert0(base_pic->refs[1][i]); + av_assert0(base_pic->refs[1][i]->encode_issued); + } + + av_log(avctx, AV_LOG_DEBUG, "Input surface is %p.\n", pic->input_surface->texture); + + err = av_hwframe_get_buffer(base_ctx->recon_frames_ref, base_pic->recon_image, 0); + if (err < 0) { + err = AVERROR(ENOMEM); + goto fail; + } + + pic->recon_surface = (AVD3D12VAFrame *)base_pic->recon_image->data[0]; + av_log(avctx, AV_LOG_DEBUG, "Recon surface is %p.\n", + pic->recon_surface->texture); + + pic->output_buffer_ref = av_buffer_pool_get(ctx->output_buffer_pool); + if (!pic->output_buffer_ref) { + err = AVERROR(ENOMEM); + goto fail; + } + pic->output_buffer = (ID3D12Resource *)pic->output_buffer_ref->data; + av_log(avctx, AV_LOG_DEBUG, "Output buffer is %p.\n", + pic->output_buffer); + + err = d3d12va_encode_create_metadata_buffers(avctx, pic); + if (err < 0) + goto fail; + + if (ctx->codec->init_picture_params) { + err = ctx->codec->init_picture_params(avctx, pic); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialise picture " + "parameters: %d.\n", err); + goto fail; + } + } + + if (base_pic->type == PICTURE_TYPE_IDR) { + if (ctx->codec->write_sequence_header) { + bit_len = 8 * sizeof(data); + err = ctx->codec->write_sequence_header(avctx, data, &bit_len); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write per-sequence " + "header: %d.\n", err); + goto fail; + } + } + + pic->header_size = (int)bit_len / 8; + pic->header_size = pic->header_size % ctx->req.CompressedBitstreamBufferAccessAlignment ? + FFALIGN(pic->header_size, ctx->req.CompressedBitstreamBufferAccessAlignment) : + pic->header_size; + + hr = ID3D12Resource_Map(pic->output_buffer, 0, NULL, (void **)&ptr); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + memcpy(ptr, data, pic->header_size); + ID3D12Resource_Unmap(pic->output_buffer, 0, NULL); + } + + d3d12_refs.NumTexture2Ds = base_pic->nb_refs[0] + base_pic->nb_refs[1]; + if (d3d12_refs.NumTexture2Ds) { + d3d12_refs.ppTexture2Ds = av_calloc(d3d12_refs.NumTexture2Ds, + sizeof(*d3d12_refs.ppTexture2Ds)); + if (!d3d12_refs.ppTexture2Ds) { + err = AVERROR(ENOMEM); + goto fail; + } + + i = 0; + for (j = 0; j < base_pic->nb_refs[0]; j++) + d3d12_refs.ppTexture2Ds[i++] = ((D3D12VAEncodePicture *)base_pic->refs[0][j])->recon_surface->texture; + for (j = 0; j < base_pic->nb_refs[1]; j++) + d3d12_refs.ppTexture2Ds[i++] = ((D3D12VAEncodePicture *)base_pic->refs[1][j])->recon_surface->texture; + } + + input_args.PictureControlDesc.IntraRefreshFrameIndex = 0; + if (base_pic->is_reference) + input_args.PictureControlDesc.Flags |= D3D12_VIDEO_ENCODER_PICTURE_CONTROL_FLAG_USED_AS_REFERENCE_PICTURE; + + input_args.PictureControlDesc.PictureControlCodecData = pic->pic_ctl; + input_args.PictureControlDesc.ReferenceFrames = d3d12_refs; + input_args.CurrentFrameBitstreamMetadataSize = pic->header_size; + + output_args.Bitstream.pBuffer = pic->output_buffer; + output_args.Bitstream.FrameStartOffset = pic->header_size; + output_args.ReconstructedPicture.pReconstructedPicture = pic->recon_surface->texture; + output_args.ReconstructedPicture.ReconstructedPictureSubresource = 0; + output_args.EncoderOutputMetadata.pBuffer = pic->encoded_metadata; + output_args.EncoderOutputMetadata.Offset = 0; + + input_metadata.HWLayoutMetadata.pBuffer = pic->encoded_metadata; + input_metadata.HWLayoutMetadata.Offset = 0; + + output_metadata.ResolvedLayoutMetadata.pBuffer = pic->resolved_metadata; + output_metadata.ResolvedLayoutMetadata.Offset = 0; + + err = d3d12va_get_valid_command_allocator(avctx, &command_allocator); + if (err < 0) + goto fail; + + hr = ID3D12CommandAllocator_Reset(command_allocator); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12VideoEncodeCommandList2_Reset(cmd_list, command_allocator); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + +#define TRANSITION_BARRIER(res, before, after) \ + (D3D12_RESOURCE_BARRIER) { \ + .Type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION, \ + .Flags = D3D12_RESOURCE_BARRIER_FLAG_NONE, \ + .Transition = { \ + .pResource = res, \ + .Subresource = D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, \ + .StateBefore = before, \ + .StateAfter = after, \ + }, \ + } + + barriers[0] = TRANSITION_BARRIER(pic->input_surface->texture, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + barriers[1] = TRANSITION_BARRIER(pic->output_buffer, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + barriers[2] = TRANSITION_BARRIER(pic->recon_surface->texture, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + barriers[4] = TRANSITION_BARRIER(pic->resolved_metadata, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 5, barriers); + + if (d3d12_refs.NumTexture2Ds) { + D3D12_RESOURCE_BARRIER refs_barriers[3]; + + for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) + refs_barriers[i] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, d3d12_refs.NumTexture2Ds, + refs_barriers); + } + + ID3D12VideoEncodeCommandList2_EncodeFrame(cmd_list, ctx->encoder, ctx->encoder_heap, + &input_args, &output_args); + + barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 1, &barriers[3]); + + ID3D12VideoEncodeCommandList2_ResolveEncoderOutputMetadata(cmd_list, &input_metadata, &output_metadata); + + if (d3d12_refs.NumTexture2Ds) { + D3D12_RESOURCE_BARRIER refs_barriers[3]; + + for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) + refs_barriers[i] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, + D3D12_RESOURCE_STATE_COMMON); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, d3d12_refs.NumTexture2Ds, + refs_barriers); + } + + barriers[0] = TRANSITION_BARRIER(pic->input_surface->texture, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, + D3D12_RESOURCE_STATE_COMMON); + barriers[1] = TRANSITION_BARRIER(pic->output_buffer, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_COMMON); + barriers[2] = TRANSITION_BARRIER(pic->recon_surface->texture, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_COMMON); + barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, + D3D12_RESOURCE_STATE_COMMON); + barriers[4] = TRANSITION_BARRIER(pic->resolved_metadata, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_COMMON); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 5, barriers); + + hr = ID3D12VideoEncodeCommandList2_Close(cmd_list); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12CommandQueue_Wait(ctx->command_queue, pic->input_surface->sync_ctx.fence, + pic->input_surface->sync_ctx.fence_value); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + ID3D12CommandQueue_ExecuteCommandLists(ctx->command_queue, 1, (ID3D12CommandList **)&ctx->command_list); + + hr = ID3D12CommandQueue_Signal(ctx->command_queue, pic->input_surface->sync_ctx.fence, + ++pic->input_surface->sync_ctx.fence_value); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12CommandQueue_Signal(ctx->command_queue, ctx->sync_ctx.fence, ++ctx->sync_ctx.fence_value); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + err = d3d12va_discard_command_allocator(avctx, command_allocator, ctx->sync_ctx.fence_value); + if (err < 0) + goto fail; + + pic->fence_value = ctx->sync_ctx.fence_value; + + if (d3d12_refs.ppTexture2Ds) + av_freep(&d3d12_refs.ppTexture2Ds); + + return 0; + +fail: + if (command_allocator) + d3d12va_discard_command_allocator(avctx, command_allocator, ctx->sync_ctx.fence_value); + + if (d3d12_refs.ppTexture2Ds) + av_freep(&d3d12_refs.ppTexture2Ds); + + if (ctx->codec->free_picture_params) + ctx->codec->free_picture_params(pic); + + av_buffer_unref(&pic->output_buffer_ref); + pic->output_buffer = NULL; + D3D12_OBJECT_RELEASE(pic->encoded_metadata); + D3D12_OBJECT_RELEASE(pic->resolved_metadata); + return err; +} + +static int d3d12va_encode_discard(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + HWBaseEncodePicture *base_pic = &pic->base; + d3d12va_encode_wait(avctx, pic); + + if (pic->output_buffer_ref) { + av_log(avctx, AV_LOG_DEBUG, "Discard output for pic " + "%"PRId64"/%"PRId64".\n", + base_pic->display_order, base_pic->encode_order); + + av_buffer_unref(&pic->output_buffer_ref); + pic->output_buffer = NULL; + } + + D3D12_OBJECT_RELEASE(pic->encoded_metadata); + D3D12_OBJECT_RELEASE(pic->resolved_metadata); + + return 0; +} + +static int d3d12va_encode_free_rc_params(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + + switch (ctx->rc.Mode) + { + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CQP: + av_freep(&ctx->rc.ConfigParams.pConfiguration_CQP); + break; + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CBR: + av_freep(&ctx->rc.ConfigParams.pConfiguration_CBR); + break; + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_VBR: + av_freep(&ctx->rc.ConfigParams.pConfiguration_VBR); + break; + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_QVBR: + av_freep(&ctx->rc.ConfigParams.pConfiguration_QVBR); + break; + default: + break; + } + + return 0; +} + +static HWBaseEncodePicture *d3d12va_encode_alloc(AVCodecContext *avctx, + const AVFrame *frame) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodePicture *pic; + + pic = av_mallocz(sizeof(*pic)); + if (!pic) + return NULL; + + if (ctx->codec->picture_priv_data_size > 0) { + pic->base.priv_data = av_mallocz(ctx->codec->picture_priv_data_size); + if (!pic->base.priv_data) { + av_freep(&pic); + return NULL; + } + } + + pic->input_surface = (AVD3D12VAFrame *)frame->data[0]; + + return &pic->base; +} + +static int d3d12va_encode_free(AVCodecContext *avctx, + HWBaseEncodePicture *base_pic) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodePicture *pic = (D3D12VAEncodePicture *)base_pic; + + if (base_pic->encode_issued) + d3d12va_encode_discard(avctx, pic); + + if (ctx->codec->free_picture_params) + ctx->codec->free_picture_params(pic); + + ff_hw_base_encode_free(avctx, base_pic); + + av_free(pic); + + return 0; +} + +static int d3d12va_encode_get_buffer_size(AVCodecContext *avctx, + D3D12VAEncodePicture *pic, size_t *size) +{ + D3D12_VIDEO_ENCODER_OUTPUT_METADATA *meta = NULL; + uint8_t *data; + HRESULT hr; + int err; + + hr = ID3D12Resource_Map(pic->resolved_metadata, 0, NULL, (void **)&data); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + return err; + } + + meta = (D3D12_VIDEO_ENCODER_OUTPUT_METADATA *)data; + + if (meta->EncodeErrorFlags != D3D12_VIDEO_ENCODER_ENCODE_ERROR_FLAG_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Encode failed %"PRIu64"\n", meta->EncodeErrorFlags); + err = AVERROR(EINVAL); + return err; + } + + if (meta->EncodedBitstreamWrittenBytesCount == 0) { + av_log(avctx, AV_LOG_ERROR, "No bytes were written to encoded bitstream\n"); + err = AVERROR(EINVAL); + return err; + } + + *size = meta->EncodedBitstreamWrittenBytesCount; + + ID3D12Resource_Unmap(pic->resolved_metadata, 0, NULL); + + return 0; +} + +static int d3d12va_encode_get_coded_data(AVCodecContext *avctx, + D3D12VAEncodePicture *pic, AVPacket *pkt) +{ + int err; + uint8_t *ptr, *mapped_data; + size_t total_size = 0; + HRESULT hr; + + err = d3d12va_encode_get_buffer_size(avctx, pic, &total_size); + if (err < 0) + goto end; + + total_size += pic->header_size; + av_log(avctx, AV_LOG_DEBUG, "Output buffer size %"PRId64"\n", total_size); + + hr = ID3D12Resource_Map(pic->output_buffer, 0, NULL, (void **)&mapped_data); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto end; + } + + err = ff_get_encode_buffer(avctx, pkt, total_size, 0); + if (err < 0) + goto end; + ptr = pkt->data; + + memcpy(ptr, mapped_data, total_size); + + ID3D12Resource_Unmap(pic->output_buffer, 0, NULL); + +end: + av_buffer_unref(&pic->output_buffer_ref); + pic->output_buffer = NULL; + return err; +} + +static int d3d12va_encode_output(AVCodecContext *avctx, + const HWBaseEncodePicture *base_pic, AVPacket *pkt) +{ + D3D12VAEncodePicture *pic = (D3D12VAEncodePicture *)base_pic; + AVPacket *pkt_ptr = pkt; + int err; + + err = d3d12va_encode_wait(avctx, pic); + if (err < 0) + return err; + + err = d3d12va_encode_get_coded_data(avctx, pic, pkt); + if (err < 0) + return err; + + av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", + base_pic->display_order, base_pic->encode_order); + + ff_hw_base_encode_set_output_property(avctx, (HWBaseEncodePicture *)base_pic, pkt_ptr, 0); + + return 0; +} + +static int d3d12va_encode_set_profile(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + const D3D12VAEncodeProfile *profile; + const AVPixFmtDescriptor *desc; + int i, depth; + + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); + if (!desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%d).\n", + base_ctx->input_frames->sw_format); + return AVERROR(EINVAL); + } + + depth = desc->comp[0].depth; + for (i = 1; i < desc->nb_components; i++) { + if (desc->comp[i].depth != depth) { + av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", + desc->name); + return AVERROR(EINVAL); + } + } + av_log(avctx, AV_LOG_VERBOSE, "Input surface format is %s.\n", + desc->name); + + av_assert0(ctx->codec->profiles); + for (i = 0; (ctx->codec->profiles[i].av_profile != + AV_PROFILE_UNKNOWN); i++) { + profile = &ctx->codec->profiles[i]; + if (depth != profile->depth || + desc->nb_components != profile->nb_components) + continue; + if (desc->nb_components > 1 && + (desc->log2_chroma_w != profile->log2_chroma_w || + desc->log2_chroma_h != profile->log2_chroma_h)) + continue; + if (avctx->profile != profile->av_profile && + avctx->profile != AV_PROFILE_UNKNOWN) + continue; + + ctx->profile = profile; + break; + } + if (!ctx->profile) { + av_log(avctx, AV_LOG_ERROR, "No usable encoding profile found.\n"); + return AVERROR(ENOSYS); + } + + avctx->profile = profile->av_profile; + return 0; +} + +static const D3D12VAEncodeRCMode d3d12va_encode_rc_modes[] = { + // Bitrate Quality + // | Maxrate | HRD/VBV + { 0 }, // | | | | + { RC_MODE_CQP, "CQP", 0, 0, 1, 0, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CQP }, + { RC_MODE_CBR, "CBR", 1, 0, 0, 1, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CBR }, + { RC_MODE_VBR, "VBR", 1, 1, 0, 1, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_VBR }, + { RC_MODE_QVBR, "QVBR", 1, 1, 1, 1, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_QVBR }, +}; + +static int check_rate_control_support(AVCodecContext *avctx, const D3D12VAEncodeRCMode *rc_mode) +{ + HRESULT hr; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12_FEATURE_DATA_VIDEO_ENCODER_RATE_CONTROL_MODE d3d12_rc_mode = { + .Codec = ctx->codec->d3d12_codec, + }; + + if (!rc_mode->d3d12_mode) + return 0; + + d3d12_rc_mode.IsSupported = 0; + d3d12_rc_mode.RateControlMode = rc_mode->d3d12_mode; + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, + D3D12_FEATURE_VIDEO_ENCODER_RATE_CONTROL_MODE, + &d3d12_rc_mode, sizeof(d3d12_rc_mode)); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to check rate control support.\n"); + return 0; + } + + return d3d12_rc_mode.IsSupported; +} + +static int d3d12va_encode_init_rate_control(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + int64_t rc_target_bitrate; + int64_t rc_peak_bitrate; + int rc_quality; + int64_t hrd_buffer_size; + int64_t hrd_initial_buffer_fullness; + int fr_num, fr_den; + const D3D12VAEncodeRCMode *rc_mode; + + // Rate control mode selection: + // * If the user has set a mode explicitly with the rc_mode option, + // use it and fail if it is not available. + // * If an explicit QP option has been set, use CQP. + // * If the codec is CQ-only, use CQP. + // * If the QSCALE avcodec option is set, use CQP. + // * If bitrate and quality are both set, try QVBR. + // * If quality is set, try CQP. + // * If bitrate and maxrate are set and have the same value, try CBR. + // * If a bitrate is set, try VBR, then CBR. + // * If no bitrate is set, try CQP. + +#define TRY_RC_MODE(mode, fail) do { \ + rc_mode = &d3d12va_encode_rc_modes[mode]; \ + if (!(rc_mode->d3d12_mode && check_rate_control_support(avctx, rc_mode))) { \ + if (fail) { \ + av_log(avctx, AV_LOG_ERROR, "Driver does not support %s " \ + "RC mode.\n", rc_mode->name); \ + return AVERROR(EINVAL); \ + } \ + av_log(avctx, AV_LOG_DEBUG, "Driver does not support %s " \ + "RC mode.\n", rc_mode->name); \ + rc_mode = NULL; \ + } else { \ + goto rc_mode_found; \ + } \ + } while (0) + + if (ctx->explicit_rc_mode) + TRY_RC_MODE(ctx->explicit_rc_mode, 1); + + if (ctx->explicit_qp) + TRY_RC_MODE(RC_MODE_CQP, 1); + + if (ctx->codec->flags & FLAG_CONSTANT_QUALITY_ONLY) + TRY_RC_MODE(RC_MODE_CQP, 1); + + if (avctx->flags & AV_CODEC_FLAG_QSCALE) + TRY_RC_MODE(RC_MODE_CQP, 1); + + if (avctx->bit_rate > 0 && avctx->global_quality > 0) + TRY_RC_MODE(RC_MODE_QVBR, 0); + + if (avctx->global_quality > 0) { + TRY_RC_MODE(RC_MODE_CQP, 0); + } + + if (avctx->bit_rate > 0 && avctx->rc_max_rate == avctx->bit_rate) + TRY_RC_MODE(RC_MODE_CBR, 0); + + if (avctx->bit_rate > 0) { + TRY_RC_MODE(RC_MODE_VBR, 0); + TRY_RC_MODE(RC_MODE_CBR, 0); + } else { + TRY_RC_MODE(RC_MODE_CQP, 0); + } + + av_log(avctx, AV_LOG_ERROR, "Driver does not support any " + "RC mode compatible with selected options.\n"); + return AVERROR(EINVAL); + +rc_mode_found: + if (rc_mode->bitrate) { + if (avctx->bit_rate <= 0) { + av_log(avctx, AV_LOG_ERROR, "Bitrate must be set for %s " + "RC mode.\n", rc_mode->name); + return AVERROR(EINVAL); + } + + if (rc_mode->maxrate) { + if (avctx->rc_max_rate > 0) { + if (avctx->rc_max_rate < avctx->bit_rate) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitrate settings: " + "bitrate (%"PRId64") must not be greater than " + "maxrate (%"PRId64").\n", avctx->bit_rate, + avctx->rc_max_rate); + return AVERROR(EINVAL); + } + rc_target_bitrate = avctx->bit_rate; + rc_peak_bitrate = avctx->rc_max_rate; + } else { + // We only have a target bitrate, but this mode requires + // that a maximum rate be supplied as well. Since the + // user does not want this to be a constraint, arbitrarily + // pick a maximum rate of double the target rate. + rc_target_bitrate = avctx->bit_rate; + rc_peak_bitrate = 2 * avctx->bit_rate; + } + } else { + if (avctx->rc_max_rate > avctx->bit_rate) { + av_log(avctx, AV_LOG_WARNING, "Max bitrate is ignored " + "in %s RC mode.\n", rc_mode->name); + } + rc_target_bitrate = avctx->bit_rate; + rc_peak_bitrate = 0; + } + } else { + rc_target_bitrate = 0; + rc_peak_bitrate = 0; + } + + if (rc_mode->quality) { + if (ctx->explicit_qp) { + rc_quality = ctx->explicit_qp; + } else if (avctx->global_quality > 0) { + if (avctx->flags & AV_CODEC_FLAG_QSCALE) + rc_quality = avctx->global_quality / FF_QP2LAMBDA; + else + rc_quality = avctx->global_quality; + } else { + rc_quality = ctx->codec->default_quality; + av_log(avctx, AV_LOG_WARNING, "No quality level set; " + "using default (%d).\n", rc_quality); + } + } else { + rc_quality = 0; + } + + if (rc_mode->hrd) { + if (avctx->rc_buffer_size) + hrd_buffer_size = avctx->rc_buffer_size; + else if (avctx->rc_max_rate > 0) + hrd_buffer_size = avctx->rc_max_rate; + else + hrd_buffer_size = avctx->bit_rate; + if (avctx->rc_initial_buffer_occupancy) { + if (avctx->rc_initial_buffer_occupancy > hrd_buffer_size) { + av_log(avctx, AV_LOG_ERROR, "Invalid RC buffer settings: " + "must have initial buffer size (%d) <= " + "buffer size (%"PRId64").\n", + avctx->rc_initial_buffer_occupancy, hrd_buffer_size); + return AVERROR(EINVAL); + } + hrd_initial_buffer_fullness = avctx->rc_initial_buffer_occupancy; + } else { + hrd_initial_buffer_fullness = hrd_buffer_size * 3 / 4; + } + } else { + if (avctx->rc_buffer_size || avctx->rc_initial_buffer_occupancy) { + av_log(avctx, AV_LOG_WARNING, "Buffering settings are ignored " + "in %s RC mode.\n", rc_mode->name); + } + + hrd_buffer_size = 0; + hrd_initial_buffer_fullness = 0; + } + + if (rc_target_bitrate > UINT32_MAX || + hrd_buffer_size > UINT32_MAX || + hrd_initial_buffer_fullness > UINT32_MAX) { + av_log(avctx, AV_LOG_ERROR, "RC parameters of 2^32 or " + "greater are not supported by D3D12.\n"); + return AVERROR(EINVAL); + } + + ctx->rc_quality = rc_quality; + + av_log(avctx, AV_LOG_VERBOSE, "RC mode: %s.\n", rc_mode->name); + + if (rc_mode->quality) + av_log(avctx, AV_LOG_VERBOSE, "RC quality: %d.\n", rc_quality); + + if (rc_mode->hrd) { + av_log(avctx, AV_LOG_VERBOSE, "RC buffer: %"PRId64" bits, " + "initial fullness %"PRId64" bits.\n", + hrd_buffer_size, hrd_initial_buffer_fullness); + } + + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + av_reduce(&fr_num, &fr_den, + avctx->framerate.num, avctx->framerate.den, 65535); + else + av_reduce(&fr_num, &fr_den, + avctx->time_base.den, avctx->time_base.num, 65535); + + av_log(avctx, AV_LOG_VERBOSE, "RC framerate: %d/%d (%.2f fps).\n", + fr_num, fr_den, (double)fr_num / fr_den); + + ctx->rc.Flags = D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_NONE; + ctx->rc.TargetFrameRate.Numerator = fr_num; + ctx->rc.TargetFrameRate.Denominator = fr_den; + ctx->rc.Mode = rc_mode->d3d12_mode; + + switch (rc_mode->mode) { + case RC_MODE_CQP: + // cqp ConfigParams will be updated in ctx->codec->configure. + break; + + case RC_MODE_CBR: + D3D12_VIDEO_ENCODER_RATE_CONTROL_CBR *cbr_ctl; + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_CBR); + cbr_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!cbr_ctl) + return AVERROR(ENOMEM); + + cbr_ctl->TargetBitRate = rc_target_bitrate; + cbr_ctl->VBVCapacity = hrd_buffer_size; + cbr_ctl->InitialVBVFullness = hrd_initial_buffer_fullness; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_VBV_SIZES; + + if (avctx->qmin > 0 || avctx->qmax > 0) { + cbr_ctl->MinQP = avctx->qmin; + cbr_ctl->MaxQP = avctx->qmax; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_QP_RANGE; + } + + ctx->rc.ConfigParams.pConfiguration_CBR = cbr_ctl; + break; + + case RC_MODE_VBR: + D3D12_VIDEO_ENCODER_RATE_CONTROL_VBR *vbr_ctl; + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_VBR); + vbr_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!vbr_ctl) + return AVERROR(ENOMEM); + + vbr_ctl->TargetAvgBitRate = rc_target_bitrate; + vbr_ctl->PeakBitRate = rc_peak_bitrate; + vbr_ctl->VBVCapacity = hrd_buffer_size; + vbr_ctl->InitialVBVFullness = hrd_initial_buffer_fullness; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_VBV_SIZES; + + if (avctx->qmin > 0 || avctx->qmax > 0) { + vbr_ctl->MinQP = avctx->qmin; + vbr_ctl->MaxQP = avctx->qmax; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_QP_RANGE; + } + + ctx->rc.ConfigParams.pConfiguration_VBR = vbr_ctl; + break; + + case RC_MODE_QVBR: + D3D12_VIDEO_ENCODER_RATE_CONTROL_QVBR *qvbr_ctl; + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_QVBR); + qvbr_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!qvbr_ctl) + return AVERROR(ENOMEM); + + qvbr_ctl->TargetAvgBitRate = rc_target_bitrate; + qvbr_ctl->PeakBitRate = rc_peak_bitrate; + qvbr_ctl->ConstantQualityTarget = rc_quality; + + if (avctx->qmin > 0 || avctx->qmax > 0) { + qvbr_ctl->MinQP = avctx->qmin; + qvbr_ctl->MaxQP = avctx->qmax; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_QP_RANGE; + } + + ctx->rc.ConfigParams.pConfiguration_QVBR = qvbr_ctl; + break; + + default: + break; + } + return 0; +} + +static int d3d12va_encode_init_gop_structure(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + uint32_t ref_l0, ref_l1; + int err; + HRESULT hr; + D3D12_FEATURE_DATA_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT support; + union { + D3D12_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT_H264 h264; + D3D12_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT_HEVC hevc; + } codec_support; + + support.NodeIndex = 0; + support.Codec = ctx->codec->d3d12_codec; + support.Profile = ctx->profile->d3d12_profile; + + switch (ctx->codec->d3d12_codec) { + case D3D12_VIDEO_ENCODER_CODEC_H264: + support.PictureSupport.DataSize = sizeof(codec_support.h264); + support.PictureSupport.pH264Support = &codec_support.h264; + break; + + case D3D12_VIDEO_ENCODER_CODEC_HEVC: + support.PictureSupport.DataSize = sizeof(codec_support.hevc); + support.PictureSupport.pHEVCSupport = &codec_support.hevc; + break; + + default: + av_assert0(0); + } + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT, + &support, sizeof(support)); + if (FAILED(hr)) + return AVERROR(EINVAL); + + if (support.IsSupported) { + switch (ctx->codec->d3d12_codec) { + case D3D12_VIDEO_ENCODER_CODEC_H264: + ref_l0 = FFMIN(support.PictureSupport.pH264Support->MaxL0ReferencesForP, + support.PictureSupport.pH264Support->MaxL1ReferencesForB); + ref_l1 = support.PictureSupport.pH264Support->MaxL1ReferencesForB; + break; + + case D3D12_VIDEO_ENCODER_CODEC_HEVC: + ref_l0 = FFMIN(support.PictureSupport.pHEVCSupport->MaxL0ReferencesForP, + support.PictureSupport.pHEVCSupport->MaxL1ReferencesForB); + ref_l1 = support.PictureSupport.pHEVCSupport->MaxL1ReferencesForB; + break; + + default: + av_assert0(0); + } + } else { + ref_l0 = ref_l1 = 0; + } + + if (ref_l0 > 0 && ref_l1 > 0 && ctx->bi_not_empty) { + base_ctx->p_to_gpb = 1; + av_log(avctx, AV_LOG_VERBOSE, "Driver does not support P-frames, " + "replacing them with B-frames.\n"); + } + + err = ff_hw_base_init_gop_structure(avctx, ref_l0, ref_l1, ctx->codec->flags, 0); + if (err < 0) + return err; + + return 0; +} + +static int d3d12va_create_encoder(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + AVD3D12VAFramesContext *frames_hwctx = base_ctx->input_frames->hwctx; + HRESULT hr; + + D3D12_VIDEO_ENCODER_DESC desc = { + .NodeMask = 0, + .Flags = D3D12_VIDEO_ENCODER_FLAG_NONE, + .EncodeCodec = ctx->codec->d3d12_codec, + .EncodeProfile = ctx->profile->d3d12_profile, + .InputFormat = frames_hwctx->format, + .CodecConfiguration = ctx->codec_conf, + .MaxMotionEstimationPrecision = D3D12_VIDEO_ENCODER_MOTION_ESTIMATION_PRECISION_MODE_MAXIMUM, + }; + + hr = ID3D12VideoDevice3_CreateVideoEncoder(ctx->video_device3, &desc, &IID_ID3D12VideoEncoder, + (void **)&ctx->encoder); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create encoder.\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int d3d12va_create_encoder_heap(AVCodecContext* avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + HRESULT hr; + + D3D12_VIDEO_ENCODER_HEAP_DESC desc = { + .NodeMask = 0, + .Flags = D3D12_VIDEO_ENCODER_FLAG_NONE, + .EncodeCodec = ctx->codec->d3d12_codec, + .EncodeProfile = ctx->profile->d3d12_profile, + .EncodeLevel = ctx->level, + .ResolutionsListCount = 1, + .pResolutionList = &ctx->resolution, + }; + + hr = ID3D12VideoDevice3_CreateVideoEncoderHeap(ctx->video_device3, &desc, + &IID_ID3D12VideoEncoderHeap, (void **)&ctx->encoder_heap); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create encoder heap.\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static void d3d12va_encode_free_buffer(void *opaque, uint8_t *data) +{ + ID3D12Resource *pResource; + + pResource = (ID3D12Resource *)data; + D3D12_OBJECT_RELEASE(pResource); +} + +static AVBufferRef *d3d12va_encode_alloc_output_buffer(void *opaque, size_t size) +{ + AVCodecContext *avctx = opaque; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + ID3D12Resource *pResource = NULL; + HRESULT hr; + AVBufferRef *ref; + D3D12_HEAP_PROPERTIES heap_props; + D3D12_HEAP_TYPE heap_type = D3D12_HEAP_TYPE_READBACK; + + D3D12_RESOURCE_DESC desc = { + .Dimension = D3D12_RESOURCE_DIMENSION_BUFFER, + .Alignment = 0, + .Width = FFALIGN(3 * base_ctx->surface_width * base_ctx->surface_height + (1 << 16), + D3D12_TEXTURE_DATA_PLACEMENT_ALIGNMENT), + .Height = 1, + .DepthOrArraySize = 1, + .MipLevels = 1, + .Format = DXGI_FORMAT_UNKNOWN, + .SampleDesc = { .Count = 1, .Quality = 0 }, + .Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR, + .Flags = D3D12_RESOURCE_FLAG_NONE, + }; + + ctx->hwctx->device->lpVtbl->GetCustomHeapProperties(ctx->hwctx->device, &heap_props, 0, heap_type); + + hr = ID3D12Device_CreateCommittedResource(ctx->hwctx->device, &heap_props, D3D12_HEAP_FLAG_NONE, + &desc, D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, + (void **)&pResource); + + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create d3d12 buffer.\n"); + return NULL; + } + + ref = av_buffer_create((uint8_t *)(uintptr_t)pResource, + sizeof(pResource), + &d3d12va_encode_free_buffer, + avctx, AV_BUFFER_FLAG_READONLY); + if (!ref) { + D3D12_OBJECT_RELEASE(pResource); + return NULL; + } + + return ref; +} + +static int d3d12va_encode_prepare_output_buffers(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + AVD3D12VAFramesContext *frames_ctx = base_ctx->input_frames->hwctx; + HRESULT hr; + + ctx->req.NodeIndex = 0; + ctx->req.Codec = ctx->codec->d3d12_codec; + ctx->req.Profile = ctx->profile->d3d12_profile; + ctx->req.InputFormat = frames_ctx->format; + ctx->req.PictureTargetResolution = ctx->resolution; + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, + D3D12_FEATURE_VIDEO_ENCODER_RESOURCE_REQUIREMENTS, + &ctx->req, sizeof(ctx->req)); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to check encoder resource requirements support.\n"); + return AVERROR(EINVAL); + } + + if (!ctx->req.IsSupported) { + av_log(avctx, AV_LOG_ERROR, "Encoder resource requirements unsupported.\n"); + return AVERROR(EINVAL); + } + + ctx->output_buffer_pool = av_buffer_pool_init2(sizeof(ID3D12Resource *), avctx, + &d3d12va_encode_alloc_output_buffer, NULL); + if (!ctx->output_buffer_pool) + return AVERROR(ENOMEM); + + return 0; +} + +static int d3d12va_encode_create_command_objects(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + ID3D12CommandAllocator *command_allocator = NULL; + int err; + HRESULT hr; + + D3D12_COMMAND_QUEUE_DESC queue_desc = { + .Type = D3D12_COMMAND_LIST_TYPE_VIDEO_ENCODE, + .Priority = 0, + .Flags = D3D12_COMMAND_QUEUE_FLAG_NONE, + .NodeMask = 0, + }; + + ctx->allocator_queue = av_fifo_alloc2(D3D12VA_VIDEO_ENC_ASYNC_DEPTH, + sizeof(CommandAllocator), AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->allocator_queue) + return AVERROR(ENOMEM); + + hr = ID3D12Device_CreateFence(ctx->hwctx->device, 0, D3D12_FENCE_FLAG_NONE, + &IID_ID3D12Fence, (void **)&ctx->sync_ctx.fence); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create fence(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + ctx->sync_ctx.event = CreateEvent(NULL, FALSE, FALSE, NULL); + if (!ctx->sync_ctx.event) + goto fail; + + err = d3d12va_get_valid_command_allocator(avctx, &command_allocator); + if (err < 0) + goto fail; + + hr = ID3D12Device_CreateCommandQueue(ctx->hwctx->device, &queue_desc, + &IID_ID3D12CommandQueue, (void **)&ctx->command_queue); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create command queue(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12Device_CreateCommandList(ctx->hwctx->device, 0, queue_desc.Type, + command_allocator, NULL, &IID_ID3D12CommandList, + (void **)&ctx->command_list); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create command list(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12VideoEncodeCommandList2_Close(ctx->command_list); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to close the command list(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + ID3D12CommandQueue_ExecuteCommandLists(ctx->command_queue, 1, (ID3D12CommandList **)&ctx->command_list); + + err = d3d12va_sync_with_gpu(avctx); + if (err < 0) + goto fail; + + err = d3d12va_discard_command_allocator(avctx, command_allocator, ctx->sync_ctx.fence_value); + if (err < 0) + goto fail; + + return 0; + +fail: + D3D12_OBJECT_RELEASE(command_allocator); + return err; +} + +static int d3d12va_encode_create_recon_frames(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + AVD3D12VAFramesContext *hwctx; + enum AVPixelFormat recon_format; + int err; + + err = ff_hw_base_get_recon_format(avctx, NULL, &recon_format); + if (err < 0) + return err; + + base_ctx->recon_frames_ref = av_hwframe_ctx_alloc(base_ctx->device_ref); + if (!base_ctx->recon_frames_ref) + return AVERROR(ENOMEM); + + base_ctx->recon_frames = (AVHWFramesContext *)base_ctx->recon_frames_ref->data; + hwctx = (AVD3D12VAFramesContext *)base_ctx->recon_frames->hwctx; + + base_ctx->recon_frames->format = AV_PIX_FMT_D3D12; + base_ctx->recon_frames->sw_format = recon_format; + base_ctx->recon_frames->width = base_ctx->surface_width; + base_ctx->recon_frames->height = base_ctx->surface_height; + + hwctx->flags = D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY | + D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE; + + err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialise reconstructed " + "frame context: %d.\n", err); + return err; + } + + return 0; +} + +static const HWEncodePictureOperation d3d12va_type = { + .alloc = &d3d12va_encode_alloc, + + .issue = &d3d12va_encode_issue, + + .output = &d3d12va_encode_output, + + .free = &d3d12va_encode_free, +}; + +int ff_d3d12va_encode_init(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12_FEATURE_DATA_VIDEO_FEATURE_AREA_SUPPORT support = { 0 }; + int err; + HRESULT hr; + + err = ff_hw_base_encode_init(avctx); + if (err < 0) + goto fail; + + base_ctx->op = &d3d12va_type; + + ctx->hwctx = base_ctx->device->hwctx; + + ctx->resolution.Width = base_ctx->input_frames->width; + ctx->resolution.Height = base_ctx->input_frames->height; + + hr = ID3D12Device_QueryInterface(ctx->hwctx->device, &IID_ID3D12Device3, (void **)&ctx->device3); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "ID3D12Device3 interface is not supported.\n"); + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12Device3_QueryInterface(ctx->device3, &IID_ID3D12VideoDevice3, (void **)&ctx->video_device3); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "ID3D12VideoDevice3 interface is not supported.\n"); + err = AVERROR_UNKNOWN; + goto fail; + } + + if (FAILED(ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_FEATURE_AREA_SUPPORT, + &support, sizeof(support))) && !support.VideoEncodeSupport) { + av_log(avctx, AV_LOG_ERROR, "D3D12 video device has no video encoder support.\n"); + err = AVERROR(EINVAL); + goto fail; + } + + err = d3d12va_encode_set_profile(avctx); + if (err < 0) + goto fail; + + err = d3d12va_encode_init_rate_control(avctx); + if (err < 0) + goto fail; + + if (ctx->codec->get_encoder_caps) { + err = ctx->codec->get_encoder_caps(avctx); + if (err < 0) + goto fail; + } + + err = d3d12va_encode_init_gop_structure(avctx); + if (err < 0) + goto fail; + + if (!(ctx->codec->flags & FLAG_SLICE_CONTROL) && avctx->slices > 0) { + av_log(avctx, AV_LOG_WARNING, "Multiple slices were requested " + "but this codec does not support controlling slices.\n"); + } + + err = d3d12va_encode_create_command_objects(avctx); + if (err < 0) + goto fail; + + err = d3d12va_encode_create_recon_frames(avctx); + if (err < 0) + goto fail; + + err = d3d12va_encode_prepare_output_buffers(avctx); + if (err < 0) + goto fail; + + if (ctx->codec->configure) { + err = ctx->codec->configure(avctx); + if (err < 0) + goto fail; + } + + if (ctx->codec->init_sequence_params) { + err = ctx->codec->init_sequence_params(avctx); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Codec sequence initialisation " + "failed: %d.\n", err); + goto fail; + } + } + + if (ctx->codec->set_level) { + err = ctx->codec->set_level(avctx); + if (err < 0) + goto fail; + } + + base_ctx->output_delay = base_ctx->b_per_p; + base_ctx->decode_delay = base_ctx->max_b_depth; + + err = d3d12va_create_encoder(avctx); + if (err < 0) + goto fail; + + err = d3d12va_create_encoder_heap(avctx); + if (err < 0) + goto fail; + + base_ctx->async_encode = 1; + base_ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, + sizeof(D3D12VAEncodePicture *), 0); + if (!base_ctx->encode_fifo) + return AVERROR(ENOMEM); + + return 0; + +fail: + return err; +} + +int ff_d3d12va_encode_close(AVCodecContext *avctx) +{ + int num_allocator = 0; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *next; + CommandAllocator allocator; + + if (!base_ctx->frame) + return 0; + + for (pic = base_ctx->pic_start; pic; pic = next) { + next = pic->next; + d3d12va_encode_free(avctx, pic); + } + + d3d12va_encode_free_rc_params(avctx); + + av_buffer_pool_uninit(&ctx->output_buffer_pool); + + D3D12_OBJECT_RELEASE(ctx->command_list); + D3D12_OBJECT_RELEASE(ctx->command_queue); + + if (ctx->allocator_queue) { + while (av_fifo_read(ctx->allocator_queue, &allocator, 1) >= 0) { + num_allocator++; + D3D12_OBJECT_RELEASE(allocator.command_allocator); + } + + av_log(avctx, AV_LOG_VERBOSE, "Total number of command allocators reused: %d\n", num_allocator); + } + + av_fifo_freep2(&ctx->allocator_queue); + + D3D12_OBJECT_RELEASE(ctx->sync_ctx.fence); + if (ctx->sync_ctx.event) + CloseHandle(ctx->sync_ctx.event); + + D3D12_OBJECT_RELEASE(ctx->encoder_heap); + D3D12_OBJECT_RELEASE(ctx->encoder); + D3D12_OBJECT_RELEASE(ctx->video_device3); + D3D12_OBJECT_RELEASE(ctx->device3); + + ff_hw_base_encode_close(avctx); + + return 0; +} diff --git a/libavcodec/d3d12va_encode.h b/libavcodec/d3d12va_encode.h new file mode 100644 index 0000000000..510c36a5eb --- /dev/null +++ b/libavcodec/d3d12va_encode.h @@ -0,0 +1,334 @@ +/* + * Direct3D 12 HW acceleration video encoder + * + * Copyright (c) 2024 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_D3D12VA_ENCODE_H +#define AVCODEC_D3D12VA_ENCODE_H + +#include "libavutil/fifo.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_d3d12va_internal.h" +#include "libavutil/hwcontext_d3d12va.h" +#include "avcodec.h" +#include "internal.h" +#include "hwconfig.h" +#include "hw_base_encode.h" + +struct D3D12VAEncodeType; + +extern const AVCodecHWConfigInternal *const ff_d3d12va_encode_hw_configs[]; + +#define MAX_PARAM_BUFFER_SIZE 4096 +#define D3D12VA_VIDEO_ENC_ASYNC_DEPTH 8 + +typedef struct D3D12VAEncodePicture { + HWBaseEncodePicture base; + + int header_size; + + AVD3D12VAFrame *input_surface; + AVD3D12VAFrame *recon_surface; + + AVBufferRef *output_buffer_ref; + ID3D12Resource *output_buffer; + + ID3D12Resource *encoded_metadata; + ID3D12Resource *resolved_metadata; + + D3D12_VIDEO_ENCODER_PICTURE_CONTROL_CODEC_DATA pic_ctl; + + int fence_value; +} D3D12VAEncodePicture; + +typedef struct D3D12VAEncodeProfile { + /** + * lavc profile value (AV_PROFILE_*). + */ + int av_profile; + + /** + * Supported bit depth. + */ + int depth; + + /** + * Number of components. + */ + int nb_components; + + /** + * Chroma subsampling in width dimension. + */ + int log2_chroma_w; + + /** + * Chroma subsampling in height dimension. + */ + int log2_chroma_h; + + /** + * D3D12 profile value. + */ + D3D12_VIDEO_ENCODER_PROFILE_DESC d3d12_profile; +} D3D12VAEncodeProfile; + +enum { + RC_MODE_AUTO, + RC_MODE_CQP, + RC_MODE_CBR, + RC_MODE_VBR, + RC_MODE_QVBR, + RC_MODE_MAX = RC_MODE_QVBR, +}; + + +typedef struct D3D12VAEncodeRCMode { + /** + * Mode from above enum (RC_MODE_*). + */ + int mode; + + /** + * Name. + * + */ + const char *name; + + /** + * Uses bitrate parameters. + * + */ + int bitrate; + + /** + * Supports maxrate distinct from bitrate. + * + */ + int maxrate; + + /** + * Uses quality value. + * + */ + int quality; + + /** + * Supports HRD/VBV parameters. + * + */ + int hrd; + + /** + * D3D12 mode value. + */ + D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE d3d12_mode; +} D3D12VAEncodeRCMode; + +typedef struct D3D12VAEncodeContext { + HWBaseEncodeContext base; + + /** + * Codec-specific hooks. + */ + const struct D3D12VAEncodeType *codec; + + /** + * Explicitly set RC mode (otherwise attempt to pick from + * available modes). + */ + int explicit_rc_mode; + + /** + * Explicitly-set QP, for use with the "qp" options. + * (Forces CQP mode when set, overriding everything else.) + */ + int explicit_qp; + + /** + * RC quality level - meaning depends on codec and RC mode. + * In CQP mode this sets the fixed quantiser value. + */ + int rc_quality; + + /** + * Chosen encoding profile details. + */ + const D3D12VAEncodeProfile *profile; + + AVD3D12VADeviceContext *hwctx; + + /** + * ID3D12Device3 interface. + */ + ID3D12Device3 *device3; + + /** + * ID3D12VideoDevice3 interface. + */ + ID3D12VideoDevice3 *video_device3; + + /** + * Pool of (reusable) bitstream output buffers. + */ + AVBufferPool *output_buffer_pool; + + /** + * D3D12 video encoder. + */ + AVBufferRef *encoder_ref; + + ID3D12VideoEncoder *encoder; + + /** + * D3D12 video encoder heap. + */ + ID3D12VideoEncoderHeap *encoder_heap; + + /** + * A cached queue for reusing the D3D12 command allocators. + * + * @see https://learn.microsoft.com/en-us/windows/win32/direct3d12/recording-command-lists-and-bundles#id3d12commandallocator + */ + AVFifo *allocator_queue; + + /** + * D3D12 command queue. + */ + ID3D12CommandQueue *command_queue; + + /** + * D3D12 video encode command list. + */ + ID3D12VideoEncodeCommandList2 *command_list; + + /** + * The sync context used to sync command queue. + */ + AVD3D12VASyncContext sync_ctx; + + /** + * The bi_not_empty feature. + */ + int bi_not_empty; + + /** + * D3D12_FEATURE structures. + */ + D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS req; + + D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOLUTION_SUPPORT_LIMITS res_limits; + + /** + * D3D12_VIDEO_ENCODER structures. + */ + D3D12_VIDEO_ENCODER_PICTURE_RESOLUTION_DESC resolution; + + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION codec_conf; + + D3D12_VIDEO_ENCODER_RATE_CONTROL rc; + + D3D12_VIDEO_ENCODER_SEQUENCE_GOP_STRUCTURE gop; + + D3D12_VIDEO_ENCODER_LEVEL_SETTING level; +} D3D12VAEncodeContext; + +typedef struct D3D12VAEncodeType { + /** + * List of supported profiles. + */ + const D3D12VAEncodeProfile *profiles; + + /** + * D3D12 codec name. + */ + D3D12_VIDEO_ENCODER_CODEC d3d12_codec; + + /** + * Codec feature flags. + */ + int flags; + + /** + * Default quality for this codec - used as quantiser or RC quality + * factor depending on RC mode. + */ + int default_quality; + + /** + * Query codec configuration and determine encode parameters like + * block sizes for surface alignment and slices. If not set, assume + * that all blocks are 16x16 and that surfaces should be aligned to match + * this. + */ + int (*get_encoder_caps)(AVCodecContext *avctx); + + /** + * Perform any extra codec-specific configuration. + */ + int (*configure)(AVCodecContext *avctx); + + /** + * Set codec-specific level setting. + */ + int (*set_level)(AVCodecContext *avctx); + + /** + * The size of any private data structure associated with each + * picture (can be zero if not required). + */ + size_t picture_priv_data_size; + + /** + * Fill the corresponding parameters. + */ + int (*init_sequence_params)(AVCodecContext *avctx); + + int (*init_picture_params)(AVCodecContext *avctx, + D3D12VAEncodePicture *pic); + + void (*free_picture_params)(D3D12VAEncodePicture *pic); + + /** + * Write the packed header data to the provided buffer. + */ + int (*write_sequence_header)(AVCodecContext *avctx, + char *data, size_t *data_len); +} D3D12VAEncodeType; + +int ff_d3d12va_encode_init(AVCodecContext *avctx); +int ff_d3d12va_encode_close(AVCodecContext *avctx); + +#define D3D12VA_ENCODE_RC_MODE(name, desc) \ + { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \ + 0, 0, FLAGS, .unit = "rc_mode" } +#define D3D12VA_ENCODE_RC_OPTIONS \ + { "rc_mode",\ + "Set rate control mode", \ + OFFSET(common.explicit_rc_mode), AV_OPT_TYPE_INT, \ + { .i64 = RC_MODE_AUTO }, RC_MODE_AUTO, RC_MODE_MAX, FLAGS, .unit = "rc_mode" }, \ + { "auto", "Choose mode automatically based on other parameters", \ + 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_AUTO }, 0, 0, FLAGS, .unit = "rc_mode" }, \ + D3D12VA_ENCODE_RC_MODE(CQP, "Constant-quality"), \ + D3D12VA_ENCODE_RC_MODE(CBR, "Constant-bitrate"), \ + D3D12VA_ENCODE_RC_MODE(VBR, "Variable-bitrate"), \ + D3D12VA_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate") + +#endif /* AVCODEC_D3D12VA_ENCODE_H */ diff --git a/libavcodec/d3d12va_encode_hevc.c b/libavcodec/d3d12va_encode_hevc.c new file mode 100644 index 0000000000..7f7d98db17 --- /dev/null +++ b/libavcodec/d3d12va_encode_hevc.c @@ -0,0 +1,1007 @@ +/* + * Direct3D 12 HW acceleration video encoder + * + * Copyright (c) 2024 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ +#include "libavutil/opt.h" +#include "libavutil/common.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" + +#include "avcodec.h" +#include "cbs.h" +#include "cbs_h265.h" +#include "h2645data.h" +#include "h265_profile_level.h" +#include "codec_internal.h" +#include "d3d12va_encode.h" + +typedef struct D3D12VAEncodeHEVCPicture { + int pic_order_cnt; + int64_t last_idr_frame; +} D3D12VAEncodeHEVCPicture; + +typedef struct D3D12VAEncodeHEVCContext { + D3D12VAEncodeContext common; + + // User options. + int qp; + int profile; + int tier; + int level; + + // Writer structures. + H265RawVPS raw_vps; + H265RawSPS raw_sps; + H265RawPPS raw_pps; + + CodedBitstreamContext *cbc; + CodedBitstreamFragment current_access_unit; +} D3D12VAEncodeHEVCContext; + +typedef struct D3D12VAEncodeHEVCLevel { + int level; + D3D12_VIDEO_ENCODER_LEVELS_HEVC d3d12_level; +} D3D12VAEncodeHEVCLevel; + +static const D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC hevc_config_support_sets[] = +{ + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 3, + 3, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 0, + 0, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 2, + 2, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 2, + 2, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 4, + 4, + }, +}; + +static const D3D12VAEncodeHEVCLevel hevc_levels[] = { + { 30, D3D12_VIDEO_ENCODER_LEVELS_HEVC_1 }, + { 60, D3D12_VIDEO_ENCODER_LEVELS_HEVC_2 }, + { 63, D3D12_VIDEO_ENCODER_LEVELS_HEVC_21 }, + { 90, D3D12_VIDEO_ENCODER_LEVELS_HEVC_3 }, + { 93, D3D12_VIDEO_ENCODER_LEVELS_HEVC_31 }, + { 120, D3D12_VIDEO_ENCODER_LEVELS_HEVC_4 }, + { 123, D3D12_VIDEO_ENCODER_LEVELS_HEVC_41 }, + { 150, D3D12_VIDEO_ENCODER_LEVELS_HEVC_5 }, + { 153, D3D12_VIDEO_ENCODER_LEVELS_HEVC_51 }, + { 156, D3D12_VIDEO_ENCODER_LEVELS_HEVC_52 }, + { 180, D3D12_VIDEO_ENCODER_LEVELS_HEVC_6 }, + { 183, D3D12_VIDEO_ENCODER_LEVELS_HEVC_61 }, + { 186, D3D12_VIDEO_ENCODER_LEVELS_HEVC_62 }, +}; + +static const D3D12_VIDEO_ENCODER_PROFILE_HEVC profile_main = D3D12_VIDEO_ENCODER_PROFILE_HEVC_MAIN; +static const D3D12_VIDEO_ENCODER_PROFILE_HEVC profile_main10 = D3D12_VIDEO_ENCODER_PROFILE_HEVC_MAIN10; + +#define D3D_PROFILE_DESC(name) \ + { sizeof(D3D12_VIDEO_ENCODER_PROFILE_HEVC), { .pHEVCProfile = (D3D12_VIDEO_ENCODER_PROFILE_HEVC *)&profile_ ## name } } +static const D3D12VAEncodeProfile d3d12va_encode_hevc_profiles[] = { + { AV_PROFILE_HEVC_MAIN, 8, 3, 1, 1, D3D_PROFILE_DESC(main) }, + { AV_PROFILE_HEVC_MAIN_10, 10, 3, 1, 1, D3D_PROFILE_DESC(main10) }, + { AV_PROFILE_UNKNOWN }, +}; + +static uint8_t d3d12va_encode_hevc_map_cusize(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE cusize) +{ + switch (cusize) { + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8: return 8; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_16x16: return 16; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32: return 32; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64: return 64; + default: av_assert0(0); + } + return 0; +} + +static uint8_t d3d12va_encode_hevc_map_tusize(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE tusize) +{ + switch (tusize) { + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4: return 4; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_8x8: return 8; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_16x16: return 16; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32: return 32; + default: av_assert0(0); + } + return 0; +} + +static int d3d12va_encode_hevc_write_access_unit(AVCodecContext *avctx, + char *data, size_t *data_len, + CodedBitstreamFragment *au) +{ + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + int err; + + err = ff_cbs_write_fragment_data(priv->cbc, au); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write packed header.\n"); + return err; + } + + if (*data_len < 8 * au->data_size - au->data_bit_padding) { + av_log(avctx, AV_LOG_ERROR, "Access unit too large: " + "%zu < %zu.\n", *data_len, + 8 * au->data_size - au->data_bit_padding); + return AVERROR(ENOSPC); + } + + memcpy(data, au->data, au->data_size); + *data_len = 8 * au->data_size - au->data_bit_padding; + + return 0; +} + +static int d3d12va_encode_hevc_add_nal(AVCodecContext *avctx, + CodedBitstreamFragment *au, + void *nal_unit) +{ + H265RawNALUnitHeader *header = nal_unit; + int err; + + err = ff_cbs_insert_unit_content(au, -1, + header->nal_unit_type, nal_unit, NULL); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to add NAL unit: " + "type = %d.\n", header->nal_unit_type); + return err; + } + + return 0; +} + +static int d3d12va_encode_hevc_write_sequence_header(AVCodecContext *avctx, + char *data, size_t *data_len) +{ + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + CodedBitstreamFragment *au = &priv->current_access_unit; + int err; + + err = d3d12va_encode_hevc_add_nal(avctx, au, &priv->raw_vps); + if (err < 0) + goto fail; + + err = d3d12va_encode_hevc_add_nal(avctx, au, &priv->raw_sps); + if (err < 0) + goto fail; + + err = d3d12va_encode_hevc_add_nal(avctx, au, &priv->raw_pps); + if (err < 0) + goto fail; + + err = d3d12va_encode_hevc_write_access_unit(avctx, data, data_len, au); +fail: + ff_cbs_fragment_reset(au); + return err; + +} + +static int d3d12va_encode_hevc_init_sequence_params(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + AVD3D12VAFramesContext *hwctx = base_ctx->input_frames->hwctx; + H265RawVPS *vps = &priv->raw_vps; + H265RawSPS *sps = &priv->raw_sps; + H265RawPPS *pps = &priv->raw_pps; + H265RawProfileTierLevel *ptl = &vps->profile_tier_level; + H265RawVUI *vui = &sps->vui; + D3D12_VIDEO_ENCODER_PROFILE_HEVC profile = D3D12_VIDEO_ENCODER_PROFILE_HEVC_MAIN; + D3D12_VIDEO_ENCODER_LEVEL_TIER_CONSTRAINTS_HEVC level = { 0 }; + const AVPixFmtDescriptor *desc; + uint8_t min_cu_size, max_cu_size, min_tu_size, max_tu_size; + int chroma_format, bit_depth; + HRESULT hr; + int i; + + D3D12_FEATURE_DATA_VIDEO_ENCODER_SUPPORT support = { + .NodeIndex = 0, + .Codec = D3D12_VIDEO_ENCODER_CODEC_HEVC, + .InputFormat = hwctx->format, + .RateControl = ctx->rc, + .IntraRefresh = D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE, + .SubregionFrameEncoding = D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME, + .ResolutionsListCount = 1, + .pResolutionList = &ctx->resolution, + .CodecGopSequence = ctx->gop, + .MaxReferenceFramesInDPB = MAX_DPB_SIZE - 1, + .CodecConfiguration = ctx->codec_conf, + .SuggestedProfile.DataSize = sizeof(D3D12_VIDEO_ENCODER_PROFILE_HEVC), + .SuggestedProfile.pHEVCProfile = &profile, + .SuggestedLevel.DataSize = sizeof(D3D12_VIDEO_ENCODER_LEVEL_TIER_CONSTRAINTS_HEVC), + .SuggestedLevel.pHEVCLevelSetting = &level, + .pResolutionDependentSupport = &ctx->res_limits, + }; + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_ENCODER_SUPPORT, + &support, sizeof(support)); + + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to check encoder support(%lx).\n", (long)hr); + return AVERROR(EINVAL); + } + + if (!(support.SupportFlags & D3D12_VIDEO_ENCODER_SUPPORT_FLAG_GENERAL_SUPPORT_OK)) { + av_log(avctx, AV_LOG_ERROR, "Driver does not support some request features. %#x\n", + support.ValidationFlags); + return AVERROR(EINVAL); + } + + if (support.SupportFlags & D3D12_VIDEO_ENCODER_SUPPORT_FLAG_RECONSTRUCTED_FRAMES_REQUIRE_TEXTURE_ARRAYS) { + av_log(avctx, AV_LOG_ERROR, "D3D12 video encode on this device requires texture array support, " + "but it's not implemented.\n"); + return AVERROR_PATCHWELCOME; + } + + memset(vps, 0, sizeof(*vps)); + memset(sps, 0, sizeof(*sps)); + memset(pps, 0, sizeof(*pps)); + + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); + av_assert0(desc); + if (desc->nb_components == 1) { + chroma_format = 0; + } else { + if (desc->log2_chroma_w == 1 && desc->log2_chroma_h == 1) { + chroma_format = 1; + } else if (desc->log2_chroma_w == 1 && desc->log2_chroma_h == 0) { + chroma_format = 2; + } else if (desc->log2_chroma_w == 0 && desc->log2_chroma_h == 0) { + chroma_format = 3; + } else { + av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " + "%s is not supported.\n", desc->name); + return AVERROR(EINVAL); + } + } + bit_depth = desc->comp[0].depth; + + min_cu_size = d3d12va_encode_hevc_map_cusize(ctx->codec_conf.pHEVCConfig->MinLumaCodingUnitSize); + max_cu_size = d3d12va_encode_hevc_map_cusize(ctx->codec_conf.pHEVCConfig->MaxLumaCodingUnitSize); + min_tu_size = d3d12va_encode_hevc_map_tusize(ctx->codec_conf.pHEVCConfig->MinLumaTransformUnitSize); + max_tu_size = d3d12va_encode_hevc_map_tusize(ctx->codec_conf.pHEVCConfig->MaxLumaTransformUnitSize); + + // VPS + + vps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_VPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + vps->vps_video_parameter_set_id = 0; + + vps->vps_base_layer_internal_flag = 1; + vps->vps_base_layer_available_flag = 1; + vps->vps_max_layers_minus1 = 0; + vps->vps_max_sub_layers_minus1 = 0; + vps->vps_temporal_id_nesting_flag = 1; + + ptl->general_profile_space = 0; + ptl->general_profile_idc = avctx->profile; + ptl->general_tier_flag = priv->tier; + + ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1; + + ptl->general_progressive_source_flag = 1; + ptl->general_interlaced_source_flag = 0; + ptl->general_non_packed_constraint_flag = 1; + ptl->general_frame_only_constraint_flag = 1; + + ptl->general_max_14bit_constraint_flag = bit_depth <= 14; + ptl->general_max_12bit_constraint_flag = bit_depth <= 12; + ptl->general_max_10bit_constraint_flag = bit_depth <= 10; + ptl->general_max_8bit_constraint_flag = bit_depth == 8; + + ptl->general_max_422chroma_constraint_flag = chroma_format <= 2; + ptl->general_max_420chroma_constraint_flag = chroma_format <= 1; + ptl->general_max_monochrome_constraint_flag = chroma_format == 0; + + ptl->general_intra_constraint_flag = base_ctx->gop_size == 1; + ptl->general_one_picture_only_constraint_flag = 0; + + ptl->general_lower_bit_rate_constraint_flag = 1; + + if (avctx->level != FF_LEVEL_UNKNOWN) { + ptl->general_level_idc = avctx->level; + } else { + const H265LevelDescriptor *level; + + level = ff_h265_guess_level(ptl, avctx->bit_rate, + base_ctx->surface_width, base_ctx->surface_height, + 1, 1, 1, (base_ctx->b_per_p > 0) + 1); + if (level) { + av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); + ptl->general_level_idc = level->level_idc; + } else { + av_log(avctx, AV_LOG_VERBOSE, "Stream will not conform to " + "any normal level; using level 8.5.\n"); + ptl->general_level_idc = 255; + // The tier flag must be set in level 8.5. + ptl->general_tier_flag = 1; + } + avctx->level = ptl->general_level_idc; + } + + vps->vps_sub_layer_ordering_info_present_flag = 0; + vps->vps_max_dec_pic_buffering_minus1[0] = base_ctx->max_b_depth + 1; + vps->vps_max_num_reorder_pics[0] = base_ctx->max_b_depth; + vps->vps_max_latency_increase_plus1[0] = 0; + + vps->vps_max_layer_id = 0; + vps->vps_num_layer_sets_minus1 = 0; + vps->layer_id_included_flag[0][0] = 1; + + vps->vps_timing_info_present_flag = 1; + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { + vps->vps_num_units_in_tick = avctx->framerate.den; + vps->vps_time_scale = avctx->framerate.num; + vps->vps_poc_proportional_to_timing_flag = 1; + vps->vps_num_ticks_poc_diff_one_minus1 = 0; + } else { + vps->vps_num_units_in_tick = avctx->time_base.num; + vps->vps_time_scale = avctx->time_base.den; + vps->vps_poc_proportional_to_timing_flag = 0; + } + vps->vps_num_hrd_parameters = 0; + + // SPS + + sps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_SPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + sps->sps_video_parameter_set_id = vps->vps_video_parameter_set_id; + + sps->sps_max_sub_layers_minus1 = vps->vps_max_sub_layers_minus1; + sps->sps_temporal_id_nesting_flag = vps->vps_temporal_id_nesting_flag; + + sps->profile_tier_level = vps->profile_tier_level; + + sps->sps_seq_parameter_set_id = 0; + + sps->chroma_format_idc = chroma_format; + sps->separate_colour_plane_flag = 0; + + av_assert0(ctx->res_limits.SubregionBlockPixelsSize % min_cu_size == 0); + + sps->pic_width_in_luma_samples = FFALIGN(base_ctx->surface_width, + ctx->res_limits.SubregionBlockPixelsSize); + sps->pic_height_in_luma_samples = FFALIGN(base_ctx->surface_height, + ctx->res_limits.SubregionBlockPixelsSize); + + if (avctx->width != sps->pic_width_in_luma_samples || + avctx->height != sps->pic_height_in_luma_samples) { + sps->conformance_window_flag = 1; + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = + (sps->pic_width_in_luma_samples - avctx->width) >> desc->log2_chroma_w; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = + (sps->pic_height_in_luma_samples - avctx->height) >> desc->log2_chroma_h; + } else { + sps->conformance_window_flag = 0; + } + + sps->bit_depth_luma_minus8 = bit_depth - 8; + sps->bit_depth_chroma_minus8 = bit_depth - 8; + + sps->log2_max_pic_order_cnt_lsb_minus4 = ctx->gop.pHEVCGroupOfPictures->log2_max_pic_order_cnt_lsb_minus4; + + sps->sps_sub_layer_ordering_info_present_flag = + vps->vps_sub_layer_ordering_info_present_flag; + for (i = 0; i <= sps->sps_max_sub_layers_minus1; i++) { + sps->sps_max_dec_pic_buffering_minus1[i] = + vps->vps_max_dec_pic_buffering_minus1[i]; + sps->sps_max_num_reorder_pics[i] = + vps->vps_max_num_reorder_pics[i]; + sps->sps_max_latency_increase_plus1[i] = + vps->vps_max_latency_increase_plus1[i]; + } + + sps->log2_min_luma_coding_block_size_minus3 = (uint8_t)(av_log2(min_cu_size) - 3); + sps->log2_diff_max_min_luma_coding_block_size = (uint8_t)(av_log2(max_cu_size) - av_log2(min_cu_size)); + sps->log2_min_luma_transform_block_size_minus2 = (uint8_t)(av_log2(min_tu_size) - 2); + sps->log2_diff_max_min_luma_transform_block_size = (uint8_t)(av_log2(max_tu_size) - av_log2(min_tu_size)); + + sps->max_transform_hierarchy_depth_inter = ctx->codec_conf.pHEVCConfig->max_transform_hierarchy_depth_inter; + sps->max_transform_hierarchy_depth_intra = ctx->codec_conf.pHEVCConfig->max_transform_hierarchy_depth_intra; + + sps->amp_enabled_flag = !!(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_USE_ASYMETRIC_MOTION_PARTITION); + sps->sample_adaptive_offset_enabled_flag = !!(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_SAO_FILTER); + sps->sps_temporal_mvp_enabled_flag = 0; + sps->pcm_enabled_flag = 0; + + sps->vui_parameters_present_flag = 1; + + if (avctx->sample_aspect_ratio.num != 0 && + avctx->sample_aspect_ratio.den != 0) { + int num, den, i; + av_reduce(&num, &den, avctx->sample_aspect_ratio.num, + avctx->sample_aspect_ratio.den, 65535); + for (i = 0; i < FF_ARRAY_ELEMS(ff_h2645_pixel_aspect); i++) { + if (num == ff_h2645_pixel_aspect[i].num && + den == ff_h2645_pixel_aspect[i].den) { + vui->aspect_ratio_idc = i; + break; + } + } + if (i >= FF_ARRAY_ELEMS(ff_h2645_pixel_aspect)) { + vui->aspect_ratio_idc = 255; + vui->sar_width = num; + vui->sar_height = den; + } + vui->aspect_ratio_info_present_flag = 1; + } + + // Unspecified video format, from table E-2. + vui->video_format = 5; + vui->video_full_range_flag = + avctx->color_range == AVCOL_RANGE_JPEG; + vui->colour_primaries = avctx->color_primaries; + vui->transfer_characteristics = avctx->color_trc; + vui->matrix_coefficients = avctx->colorspace; + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED || + avctx->color_trc != AVCOL_TRC_UNSPECIFIED || + avctx->colorspace != AVCOL_SPC_UNSPECIFIED) + vui->colour_description_present_flag = 1; + if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED || + vui->colour_description_present_flag) + vui->video_signal_type_present_flag = 1; + + if (avctx->chroma_sample_location != AVCHROMA_LOC_UNSPECIFIED) { + vui->chroma_loc_info_present_flag = 1; + vui->chroma_sample_loc_type_top_field = + vui->chroma_sample_loc_type_bottom_field = + avctx->chroma_sample_location - 1; + } + + vui->vui_timing_info_present_flag = 1; + vui->vui_num_units_in_tick = vps->vps_num_units_in_tick; + vui->vui_time_scale = vps->vps_time_scale; + vui->vui_poc_proportional_to_timing_flag = vps->vps_poc_proportional_to_timing_flag; + vui->vui_num_ticks_poc_diff_one_minus1 = vps->vps_num_ticks_poc_diff_one_minus1; + vui->vui_hrd_parameters_present_flag = 0; + + vui->bitstream_restriction_flag = 1; + vui->motion_vectors_over_pic_boundaries_flag = 1; + vui->restricted_ref_pic_lists_flag = 1; + vui->max_bytes_per_pic_denom = 0; + vui->max_bits_per_min_cu_denom = 0; + vui->log2_max_mv_length_horizontal = 15; + vui->log2_max_mv_length_vertical = 15; + + // PPS + + pps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_PPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + pps->pps_pic_parameter_set_id = 0; + pps->pps_seq_parameter_set_id = sps->sps_seq_parameter_set_id; + + pps->cabac_init_present_flag = 1; + + pps->num_ref_idx_l0_default_active_minus1 = 0; + pps->num_ref_idx_l1_default_active_minus1 = 0; + + pps->init_qp_minus26 = 0; + + pps->transform_skip_enabled_flag = !!(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_TRANSFORM_SKIPPING); + + // cu_qp_delta always required to be 1 in https://github.com/microsoft/DirectX-Specs/blob/master/d3d/D3D12VideoEncoding.md + pps->cu_qp_delta_enabled_flag = 1; + + pps->diff_cu_qp_delta_depth = 0; + + pps->pps_slice_chroma_qp_offsets_present_flag = 1; + + pps->tiles_enabled_flag = 0; // no tiling in D3D12 + + pps->pps_loop_filter_across_slices_enabled_flag = !(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_DISABLE_LOOP_FILTER_ACROSS_SLICES); + pps->deblocking_filter_control_present_flag = 1; + + return 0; +} + +static int d3d12va_encode_hevc_get_encoder_caps(AVCodecContext *avctx) +{ + int i; + HRESULT hr; + uint8_t min_cu_size, max_cu_size; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC *config; + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC hevc_caps; + + D3D12_FEATURE_DATA_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT codec_caps = { + .NodeIndex = 0, + .Codec = D3D12_VIDEO_ENCODER_CODEC_HEVC, + .Profile = ctx->profile->d3d12_profile, + .CodecSupportLimits.DataSize = sizeof(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC), + }; + + for (i = 0; i < FF_ARRAY_ELEMS(hevc_config_support_sets); i++) { + hevc_caps = hevc_config_support_sets[i]; + codec_caps.CodecSupportLimits.pHEVCSupport = &hevc_caps; + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT, + &codec_caps, sizeof(codec_caps)); + if (SUCCEEDED(hr) && codec_caps.IsSupported) + break; + } + + if (i == FF_ARRAY_ELEMS(hevc_config_support_sets)) { + av_log(avctx, AV_LOG_ERROR, "Unsupported codec configuration\n"); + return AVERROR(EINVAL); + } + + ctx->codec_conf.DataSize = sizeof(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC); + ctx->codec_conf.pHEVCConfig = av_mallocz(ctx->codec_conf.DataSize); + if (!ctx->codec_conf.pHEVCConfig) + return AVERROR(ENOMEM); + + config = ctx->codec_conf.pHEVCConfig; + + config->ConfigurationFlags = D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_NONE; + config->MinLumaCodingUnitSize = hevc_caps.MinLumaCodingUnitSize; + config->MaxLumaCodingUnitSize = hevc_caps.MaxLumaCodingUnitSize; + config->MinLumaTransformUnitSize = hevc_caps.MinLumaTransformUnitSize; + config->MaxLumaTransformUnitSize = hevc_caps.MaxLumaTransformUnitSize; + config->max_transform_hierarchy_depth_inter = hevc_caps.max_transform_hierarchy_depth_inter; + config->max_transform_hierarchy_depth_intra = hevc_caps.max_transform_hierarchy_depth_intra; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_ASYMETRIC_MOTION_PARTITION_SUPPORT || + hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_ASYMETRIC_MOTION_PARTITION_REQUIRED) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_USE_ASYMETRIC_MOTION_PARTITION; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_SAO_FILTER_SUPPORT) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_SAO_FILTER; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_DISABLING_LOOP_FILTER_ACROSS_SLICES_SUPPORT) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_DISABLE_LOOP_FILTER_ACROSS_SLICES; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_TRANSFORM_SKIP_SUPPORT) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_TRANSFORM_SKIPPING; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_P_FRAMES_IMPLEMENTED_AS_LOW_DELAY_B_FRAMES) + ctx->bi_not_empty = 1; + + // block sizes + min_cu_size = d3d12va_encode_hevc_map_cusize(hevc_caps.MinLumaCodingUnitSize); + max_cu_size = d3d12va_encode_hevc_map_cusize(hevc_caps.MaxLumaCodingUnitSize); + + av_log(avctx, AV_LOG_VERBOSE, "Using CTU size %dx%d, " + "min CB size %dx%d.\n", max_cu_size, max_cu_size, + min_cu_size, min_cu_size); + + base_ctx->surface_width = FFALIGN(avctx->width, min_cu_size); + base_ctx->surface_height = FFALIGN(avctx->height, min_cu_size); + + return 0; +} + +static int d3d12va_encode_hevc_configure(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + int fixed_qp_idr, fixed_qp_p, fixed_qp_b; + int err; + + err = ff_cbs_init(&priv->cbc, AV_CODEC_ID_HEVC, avctx); + if (err < 0) + return err; + + // Rate control + if (ctx->rc.Mode == D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CQP) { + D3D12_VIDEO_ENCODER_RATE_CONTROL_CQP *cqp_ctl; + fixed_qp_p = av_clip(ctx->rc_quality, 1, 51); + if (avctx->i_quant_factor > 0.0) + fixed_qp_idr = av_clip((avctx->i_quant_factor * fixed_qp_p + + avctx->i_quant_offset) + 0.5, 1, 51); + else + fixed_qp_idr = fixed_qp_p; + if (avctx->b_quant_factor > 0.0) + fixed_qp_b = av_clip((avctx->b_quant_factor * fixed_qp_p + + avctx->b_quant_offset) + 0.5, 1, 51); + else + fixed_qp_b = fixed_qp_p; + + av_log(avctx, AV_LOG_DEBUG, "Using fixed QP = " + "%d / %d / %d for IDR- / P- / B-frames.\n", + fixed_qp_idr, fixed_qp_p, fixed_qp_b); + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_CQP); + cqp_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!cqp_ctl) + return AVERROR(ENOMEM); + + cqp_ctl->ConstantQP_FullIntracodedFrame = fixed_qp_idr; + cqp_ctl->ConstantQP_InterPredictedFrame_PrevRefOnly = fixed_qp_p; + cqp_ctl->ConstantQP_InterPredictedFrame_BiDirectionalRef = fixed_qp_b; + + ctx->rc.ConfigParams.pConfiguration_CQP = cqp_ctl; + } + + // GOP + ctx->gop.DataSize = sizeof(D3D12_VIDEO_ENCODER_SEQUENCE_GOP_STRUCTURE_HEVC); + ctx->gop.pHEVCGroupOfPictures = av_mallocz(ctx->gop.DataSize); + if (!ctx->gop.pHEVCGroupOfPictures) + return AVERROR(ENOMEM); + + ctx->gop.pHEVCGroupOfPictures->GOPLength = base_ctx->gop_size; + ctx->gop.pHEVCGroupOfPictures->PPicturePeriod = base_ctx->b_per_p + 1; + // Power of 2 + if (base_ctx->gop_size & base_ctx->gop_size - 1 == 0) + ctx->gop.pHEVCGroupOfPictures->log2_max_pic_order_cnt_lsb_minus4 = + FFMAX(av_log2(base_ctx->gop_size) - 4, 0); + else + ctx->gop.pHEVCGroupOfPictures->log2_max_pic_order_cnt_lsb_minus4 = + FFMAX(av_log2(base_ctx->gop_size) - 3, 0); + + return 0; +} + +static int d3d12va_encode_hevc_set_level(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + int i; + + ctx->level.DataSize = sizeof(D3D12_VIDEO_ENCODER_LEVEL_TIER_CONSTRAINTS_HEVC); + ctx->level.pHEVCLevelSetting = av_mallocz(ctx->level.DataSize); + if (!ctx->level.pHEVCLevelSetting) + return AVERROR(ENOMEM); + + for (i = 0; i < FF_ARRAY_ELEMS(hevc_levels); i++) { + if (avctx->level == hevc_levels[i].level) { + ctx->level.pHEVCLevelSetting->Level = hevc_levels[i].d3d12_level; + break; + } + } + + if (i == FF_ARRAY_ELEMS(hevc_levels)) { + av_log(avctx, AV_LOG_ERROR, "Invalid level %d.\n", avctx->level); + return AVERROR(EINVAL); + } + + ctx->level.pHEVCLevelSetting->Tier = priv->raw_vps.profile_tier_level.general_tier_flag == 0 ? + D3D12_VIDEO_ENCODER_TIER_HEVC_MAIN : + D3D12_VIDEO_ENCODER_TIER_HEVC_HIGH; + + return 0; +} + +static void d3d12va_encode_hevc_free_picture_params(D3D12VAEncodePicture *pic) +{ + if (!pic->pic_ctl.pHEVCPicData) + return; + + av_freep(&pic->pic_ctl.pHEVCPicData->pList0ReferenceFrames); + av_freep(&pic->pic_ctl.pHEVCPicData->pList1ReferenceFrames); + av_freep(&pic->pic_ctl.pHEVCPicData->pReferenceFramesReconPictureDescriptors); + av_freep(&pic->pic_ctl.pHEVCPicData); +} + +static int d3d12va_encode_hevc_init_picture_params(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + HWBaseEncodePicture *base_pic = &pic->base; + D3D12VAEncodeHEVCPicture *hpic = base_pic->priv_data; + HWBaseEncodePicture *prev = base_pic->prev; + D3D12VAEncodeHEVCPicture *hprev = prev ? prev->priv_data : NULL; + D3D12_VIDEO_ENCODER_REFERENCE_PICTURE_DESCRIPTOR_HEVC *pd = NULL; + UINT *ref_list0 = NULL, *ref_list1 = NULL; + int i, idx = 0; + + pic->pic_ctl.DataSize = sizeof(D3D12_VIDEO_ENCODER_PICTURE_CONTROL_CODEC_DATA_HEVC); + pic->pic_ctl.pHEVCPicData = av_mallocz(pic->pic_ctl.DataSize); + if (!pic->pic_ctl.pHEVCPicData) + return AVERROR(ENOMEM); + + if (base_pic->type == PICTURE_TYPE_IDR) { + av_assert0(base_pic->display_order == base_pic->encode_order); + hpic->last_idr_frame = base_pic->display_order; + } else { + av_assert0(prev); + hpic->last_idr_frame = hprev->last_idr_frame; + } + hpic->pic_order_cnt = base_pic->display_order - hpic->last_idr_frame; + + switch(base_pic->type) { + case PICTURE_TYPE_IDR: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_IDR_FRAME; + break; + case PICTURE_TYPE_I: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_I_FRAME; + break; + case PICTURE_TYPE_P: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_P_FRAME; + break; + case PICTURE_TYPE_B: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_B_FRAME; + break; + default: + av_assert0(0 && "invalid picture type"); + } + + pic->pic_ctl.pHEVCPicData->slice_pic_parameter_set_id = 0; + pic->pic_ctl.pHEVCPicData->PictureOrderCountNumber = hpic->pic_order_cnt; + + if (base_pic->type == PICTURE_TYPE_P || base_pic->type == PICTURE_TYPE_B) { + pd = av_calloc(MAX_PICTURE_REFERENCES, sizeof(*pd)); + if (!pd) + return AVERROR(ENOMEM); + + ref_list0 = av_calloc(MAX_PICTURE_REFERENCES, sizeof(*ref_list0)); + if (!ref_list0) + return AVERROR(ENOMEM); + + pic->pic_ctl.pHEVCPicData->List0ReferenceFramesCount = base_pic->nb_refs[0]; + for (i = 0; i < base_pic->nb_refs[0]; i++) { + HWBaseEncodePicture *ref = base_pic->refs[0][i]; + D3D12VAEncodeHEVCPicture *href; + + av_assert0(ref && ref->encode_order < base_pic->encode_order); + href = ref->priv_data; + + ref_list0[i] = idx; + pd[idx].ReconstructedPictureResourceIndex = idx; + pd[idx].IsRefUsedByCurrentPic = TRUE; + pd[idx].PictureOrderCountNumber = href->pic_order_cnt; + idx++; + } + } + + if (base_pic->type == PICTURE_TYPE_B) { + ref_list1 = av_calloc(MAX_PICTURE_REFERENCES, sizeof(*ref_list1)); + if (!ref_list1) + return AVERROR(ENOMEM); + + pic->pic_ctl.pHEVCPicData->List1ReferenceFramesCount = base_pic->nb_refs[1]; + for (i = 0; i < base_pic->nb_refs[1]; i++) { + HWBaseEncodePicture *ref = base_pic->refs[1][i]; + D3D12VAEncodeHEVCPicture *href; + + av_assert0(ref && ref->encode_order < base_pic->encode_order); + href = ref->priv_data; + + ref_list1[i] = idx; + pd[idx].ReconstructedPictureResourceIndex = idx; + pd[idx].IsRefUsedByCurrentPic = TRUE; + pd[idx].PictureOrderCountNumber = href->pic_order_cnt; + idx++; + } + } + + pic->pic_ctl.pHEVCPicData->pList0ReferenceFrames = ref_list0; + pic->pic_ctl.pHEVCPicData->pList1ReferenceFrames = ref_list1; + pic->pic_ctl.pHEVCPicData->ReferenceFramesReconPictureDescriptorsCount = idx; + pic->pic_ctl.pHEVCPicData->pReferenceFramesReconPictureDescriptors = pd; + + return 0; +} + +static const D3D12VAEncodeType d3d12va_encode_type_hevc = { + .profiles = d3d12va_encode_hevc_profiles, + + .d3d12_codec = D3D12_VIDEO_ENCODER_CODEC_HEVC, + + .flags = FLAG_B_PICTURES | + FLAG_B_PICTURE_REFERENCES | + FLAG_NON_IDR_KEY_PICTURES, + + .default_quality = 25, + + .get_encoder_caps = &d3d12va_encode_hevc_get_encoder_caps, + + .configure = &d3d12va_encode_hevc_configure, + + .set_level = &d3d12va_encode_hevc_set_level, + + .picture_priv_data_size = sizeof(D3D12VAEncodeHEVCPicture), + + .init_sequence_params = &d3d12va_encode_hevc_init_sequence_params, + + .init_picture_params = &d3d12va_encode_hevc_init_picture_params, + + .free_picture_params = &d3d12va_encode_hevc_free_picture_params, + + .write_sequence_header = &d3d12va_encode_hevc_write_sequence_header, +}; + +static int d3d12va_encode_hevc_init(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + + ctx->codec = &d3d12va_encode_type_hevc; + + if (avctx->profile == AV_PROFILE_UNKNOWN) + avctx->profile = priv->profile; + if (avctx->level == FF_LEVEL_UNKNOWN) + avctx->level = priv->level; + + if (avctx->level != FF_LEVEL_UNKNOWN && avctx->level & ~0xff) { + av_log(avctx, AV_LOG_ERROR, "Invalid level %d: must fit " + "in 8-bit unsigned integer.\n", avctx->level); + return AVERROR(EINVAL); + } + + if (priv->qp > 0) + ctx->explicit_qp = priv->qp; + + return ff_d3d12va_encode_init(avctx); +} + +static int d3d12va_encode_hevc_close(AVCodecContext *avctx) +{ + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + + ff_cbs_fragment_free(&priv->current_access_unit); + ff_cbs_close(&priv->cbc); + + av_freep(&priv->common.codec_conf.pHEVCConfig); + av_freep(&priv->common.gop.pHEVCGroupOfPictures); + av_freep(&priv->common.level.pHEVCLevelSetting); + + return ff_d3d12va_encode_close(avctx); +} + +#define OFFSET(x) offsetof(D3D12VAEncodeHEVCContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) +static const AVOption d3d12va_encode_hevc_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, + D3D12VA_ENCODE_RC_OPTIONS, + + { "qp", "Constant QP (for P-frames; scaled by qfactor/qoffset for I/B)", + OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 52, FLAGS }, + + { "profile", "Set profile (general_profile_idc)", + OFFSET(profile), AV_OPT_TYPE_INT, + { .i64 = AV_PROFILE_UNKNOWN }, AV_PROFILE_UNKNOWN, 0xff, FLAGS, "profile" }, + +#define PROFILE(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, "profile" + { PROFILE("main", AV_PROFILE_HEVC_MAIN) }, + { PROFILE("main10", AV_PROFILE_HEVC_MAIN_10) }, +#undef PROFILE + + { "tier", "Set tier (general_tier_flag)", + OFFSET(tier), AV_OPT_TYPE_INT, + { .i64 = 0 }, 0, 1, FLAGS, "tier" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = 0 }, 0, 0, FLAGS, "tier" }, + { "high", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = 1 }, 0, 0, FLAGS, "tier" }, + + { "level", "Set level (general_level_idc)", + OFFSET(level), AV_OPT_TYPE_INT, + { .i64 = FF_LEVEL_UNKNOWN }, FF_LEVEL_UNKNOWN, 0xff, FLAGS, "level" }, + +#define LEVEL(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, "level" + { LEVEL("1", 30) }, + { LEVEL("2", 60) }, + { LEVEL("2.1", 63) }, + { LEVEL("3", 90) }, + { LEVEL("3.1", 93) }, + { LEVEL("4", 120) }, + { LEVEL("4.1", 123) }, + { LEVEL("5", 150) }, + { LEVEL("5.1", 153) }, + { LEVEL("5.2", 156) }, + { LEVEL("6", 180) }, + { LEVEL("6.1", 183) }, + { LEVEL("6.2", 186) }, +#undef LEVEL + + { NULL }, +}; + +static const FFCodecDefault d3d12va_encode_hevc_defaults[] = { + { "b", "0" }, + { "bf", "2" }, + { "g", "120" }, + { "i_qfactor", "1" }, + { "i_qoffset", "0" }, + { "b_qfactor", "1" }, + { "b_qoffset", "0" }, + { "qmin", "-1" }, + { "qmax", "-1" }, + { NULL }, +}; + +static const AVClass d3d12va_encode_hevc_class = { + .class_name = "hevc_d3d12va", + .item_name = av_default_item_name, + .option = d3d12va_encode_hevc_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_hevc_d3d12va_encoder = { + .p.name = "hevc_d3d12va", + CODEC_LONG_NAME("D3D12VA hevc encoder"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_HEVC, + .priv_data_size = sizeof(D3D12VAEncodeHEVCContext), + .init = &d3d12va_encode_hevc_init, + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), + .close = &d3d12va_encode_hevc_close, + .p.priv_class = &d3d12va_encode_hevc_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | + AV_CODEC_CAP_DR1 | AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | + FF_CODEC_CAP_INIT_CLEANUP, + .defaults = d3d12va_encode_hevc_defaults, + .p.pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_D3D12, + AV_PIX_FMT_NONE, + }, + .hw_configs = ff_d3d12va_encode_hw_configs, + .p.wrapper_name = "d3d12va", +}; From patchwork Mon May 20 14:52:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49063 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:3a48:b0:1af:fc2d:ff5a with SMTP id zu8csp4515143pzb; Mon, 20 May 2024 07:56:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXhqfF8Te+nflSvAaUz3jZAgrdHokZJre01pf7F6eLKmgWTnUUUDGDCojJ58vOz6oWPhqE7M4KFrd9yVug1PHTxhijSulAC9sY8iA== X-Google-Smtp-Source: AGHT+IFjzMmUqbTbbEsWMJWaoTSMNh5KlFjy2uM62zgm4cjG6B1V5InwDJJPO4ZSpa5dBSlJ93BK X-Received: by 2002:a05:6512:3b91:b0:51a:cdc4:dd48 with SMTP id 2adb3069b0e04-524079a81d3mr2582102e87.3.1716216972514; Mon, 20 May 2024 07:56:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716216972; cv=none; d=google.com; s=arc-20160816; b=kks+JbFZADlDxmA1LxQ6MwVLfHIynbm3GEbgqc6W5S9odELsCNb5M0oX6UBVHcDzgV phetUkL29g6V5ABotaM/kQMQPBn1k0Bnx67ndTx2m3tGPPpwN5WmJ6IvWBzba5Xtqm3/ hAqu/EQqL5pEdH3IHHKT4JxmRv1LJfZaBzXYm+x4xnYS1uB7Et8n4OQ02pUBTEZJ/MY+ YhT81H7zvUWZvcrKJcsLYdsFEDvBPfDAsrtprCNyYCA3jiVkBnB3RNg16FOfsxqoRNma JiRuLVRHY920il0ezIreOHN8YsWQQstIJsje0Jq6MLcnu9QWM4AGbeZW2s8jbHDmwtoT FBoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=FrVsZRUnAwN8hfhJZmf90AL8Ip8ggVO4cPSwyIv78hc=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=Y2wsm00SI0BN3YH754KFtOMUDG/bbJkih6NeQ5CK+R6ayNgrU1kcrCQ6dXF/b38mHi VhXF3RfSIratsWc4W5dU6SZ55zdo73mV4mBGyxOXpbMTDowalNDs9a/8BjryduZXXgE4 Hu1E3iiwk+Qrh+CN+dry+z955hP8j3/STmlKtPUiuUL8HsZ4fXONCubf1IS3j+KQ59wk kRPW1EJZAzrII9jhRktik13x71tSMDNNIqz7XKs13nfRdvehH+2UKtVVizwoDf/8e/cD xgtWLoqwYD2yBE8uOpAJEMW/Qg0y86TrpPkgIQV7ZLto+UQu8uz+4YPH3GJjo3ufYRFo RgOg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=N5TejpMB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-521f38d43bdsi8498973e87.202.2024.05.20.07.56.12; Mon, 20 May 2024 07:56:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=N5TejpMB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 84E1668D1CA; Mon, 20 May 2024 17:54:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A839468D274 for ; Mon, 20 May 2024 17:54:14 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716216855; x=1747752855; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=An+9AH09UChkDb+LkOssEWdiuCAe+bYjk5wXEhVCmNg=; b=N5TejpMBR8d7TvlLxFXbc4cYEIGx8pRm+CQtDOrAu1UmzfXmAEXdjWsF yvww5GAKqks0b8KTktss3qHrAfph2FBF20y8Sn4Y+w3cWdO5aLR5666fs yppN05Pm+cJhX4EvFKV6H4um0VEz1IjWag2QiPd5ZzdNk7hAN2Zfks2B0 9X/wNqVxHFNiP8SGGEeufcZ+1zKsonSKWDhqSOyQz6962oc8pdALp+EDD Njg+snCecIQITop2NC71r/B9p/v4IENCoi0muK1dA1KpcdeXd2YuA04Eh 3gq1dl6JMysJFZN19fnR3bjJqj5I7Imt+zUVb4PXmoa70+KLcxGHZK24k w==; X-CSE-ConnectionGUID: T11HJH/oQ022NwE09aIlFg== X-CSE-MsgGUID: umzWwcsQRkqCfdSyiwGLkw== X-IronPort-AV: E=McAfee;i="6600,9927,11078"; a="12197259" X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="12197259" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 May 2024 07:54:08 -0700 X-CSE-ConnectionGUID: mSj8yzeOTIyaGU/5j75FyQ== X-CSE-MsgGUID: lDrwXBrVQlaowgmXYQ03aA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,175,1712646000"; d="scan'208";a="55787933" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by fmviesa002.fm.intel.com with ESMTP; 20 May 2024 07:54:07 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 May 2024 22:52:22 +0800 Message-ID: <20240520145222.291-13-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240520145222.291-1-tong1.wu@intel.com> References: <20240520145222.291-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v9 13/13] Changelog: add D3D12VA HEVC encoder changelog X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3uVdjf2jfvfa From: Tong Wu Signed-off-by: Tong Wu --- Changelog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Changelog b/Changelog index dd25715d6b..c3f802617f 100644 --- a/Changelog +++ b/Changelog @@ -10,7 +10,7 @@ version : - vf_scale supports secondary ref input and framesync options - vf_scale2ref deprecated - qsv_params option added for QSV encoders - +- D3D12VA HEVC encoder version 7.0: - DXV DXT1 encoder