From patchwork Wed May 22 01:26:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49116 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp81764pzb; Tue, 21 May 2024 18:28:13 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUkEo7F8hmhtUnGPrRjiY1JRrzNLbjEFcZkFHpHh07SlMH9ptiJiNU8jnzgvIr6wLPKG6lsrjtLiH3SbLUjizfvY0x3AHSfFf1cuQ== X-Google-Smtp-Source: AGHT+IF2AhfcuVGMk0BZVSPPC0ylElD9AY85DUnmTbqKMWywk0ZbdURBJwib674CyPFiKvAmxQO3 X-Received: by 2002:a17:906:d9c7:b0:a5a:580f:8e74 with SMTP id a640c23a62f3a-a6228178b07mr24484866b.5.1716341293435; Tue, 21 May 2024 18:28:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341293; cv=none; d=google.com; s=arc-20160816; b=RckVTGbaQ6hYMqmtEjEjlTw4CqV33tf57npToN4GxJ0X7S7DU9SCx7SOLQLM+qV6D5 HGeiq0dP4noiWrfIBjF63/bhniTkTOWU9KwzqAU7BDhUZsobdggtQufGN2vs6dYF3bZ+ JkTskJ0HcsDyzo33jVBgEeeK8kQ8nOcbW15OGQwc2e3i6zMo3yViPQfmDm6cqEU/zoth szG8RYGqDSmJNQDtymCWY7xdWz5Z79ssDG16y4JFq6MVqWqB/a6Yj4WtWP1t5D9tw2Sj 4npAajtLWk9kTDghrhwmrgIqfUPGDQQYwDLzvvtDzSpfh8P+GE81kv8/wplEAu5tjpGm TjbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=wsJ3TJuX1NfALfWGavJ3p9g3UvnVOe/26HSaQtWVUjs=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=er22nxvHWLL9HxHMiABmAGpuXKUoZ6gYbjNbT2yHXwRgi/z+EpK5DqhCBGQzOecTBi UpTXSfStw5N9Q23WTlyxNEVhnuUNF5mTqrXlw9l5AEECNEFkmYrb+WuANlQTSg75gl6n ev/zuTLScnev/cX/ZqaCgcTZmmfvI8retgeJl5OTMIIuh9k/BtQjVHVolcZ7qbt017WN gKMyfDd4bWrgue1fbAx0ifTfajYntCsz+p/P6dwmfM+nOKwG9COxb03brt0V6o0GoYs+ JlURKvfQ29cYCLsl+69b8vV2LjLqFI42vlJ5PZ7OHKY2CgsOHHCWZaOYkKOs5SzNa2WC 3ZHA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=W58OHG6A; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17b2187esi1568850966b.475.2024.05.21.18.28.12; Tue, 21 May 2024 18:28:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=W58OHG6A; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E25AF68A7ED; Wed, 22 May 2024 04:28:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9229668A7ED for ; Wed, 22 May 2024 04:28:02 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341288; x=1747877288; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=x7XrFxP5LibwB3DN39MXwR2Z3n8fd79D2curPjW9Dv4=; b=W58OHG6A+F+52tGsGXDZS+lUyZAmH2ThUzeZgoOmgHJdy140pvVT4TJN NPWw041MARa7x+jzP5H+SXAWZDAAfzqGhJaLgmQiQqe2sCsUEnxxUAITh SjDGnYAFXSmAY3pxbpyF/BoLIKaSlND7emDTl+WQ5JqNSlvZmIXNF1VXx zyzcaaMOZpbtoa6UUH4fkqWt+AteQtxTiEPAka4v76jm+IFThSkWGzbCb KyLD/0Ia4YoZsLW7o8wJ+s/F5BaRVcFQ3Zl4ItTLzhsZbUXYVYHCvY4Hi YSZkgx13vDY4dVAEFl7x/16RzTcP0kdee66IUgBTnPCgmjJnJo1wkGKkO w==; X-CSE-ConnectionGUID: 2/7z7C7/Rmy22gU/LqWsWA== X-CSE-MsgGUID: P5oi9/dQS5Kv672B1zBbMQ== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797184" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797184" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:01 -0700 X-CSE-ConnectionGUID: PJiNXxZtRy+FATZpkGXYgg== X-CSE-MsgGUID: KsyO8j1DSKmG3LO9YSGXAA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537417" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:27:59 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:14 +0800 Message-ID: <20240522012627.1386-1-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 01/13] avcodec/vaapi_encode: introduce a base layer for vaapi encode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OsV83dF7loJ7 From: Tong Wu Since VAAPI and future D3D12VA implementation may share some common parameters, a base layer encode context is introduced as vaapi context's base. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.h | 56 +++++++++++++++++++++++++++++++++++++ libavcodec/vaapi_encode.h | 39 +++++--------------------- 2 files changed, 63 insertions(+), 32 deletions(-) create mode 100644 libavcodec/hw_base_encode.h diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h new file mode 100644 index 0000000000..1996179456 --- /dev/null +++ b/libavcodec/hw_base_encode.h @@ -0,0 +1,56 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_HW_BASE_ENCODE_H +#define AVCODEC_HW_BASE_ENCODE_H + +#define MAX_DPB_SIZE 16 +#define MAX_PICTURE_REFERENCES 2 +#define MAX_REORDER_DELAY 16 +#define MAX_ASYNC_DEPTH 64 +#define MAX_REFERENCE_LIST_NUM 2 + +enum { + PICTURE_TYPE_IDR = 0, + PICTURE_TYPE_I = 1, + PICTURE_TYPE_P = 2, + PICTURE_TYPE_B = 3, +}; + +enum { + // Codec supports controlling the subdivision of pictures into slices. + FLAG_SLICE_CONTROL = 1 << 0, + // Codec only supports constant quality (no rate control). + FLAG_CONSTANT_QUALITY_ONLY = 1 << 1, + // Codec is intra-only. + FLAG_INTRA_ONLY = 1 << 2, + // Codec supports B-pictures. + FLAG_B_PICTURES = 1 << 3, + // Codec supports referencing B-pictures. + FLAG_B_PICTURE_REFERENCES = 1 << 4, + // Codec supports non-IDR key pictures (that is, key pictures do + // not necessarily empty the DPB). + FLAG_NON_IDR_KEY_PICTURES = 1 << 5, +}; + +typedef struct HWBaseEncodeContext { + const AVClass *class; +} HWBaseEncodeContext; + +#endif /* AVCODEC_HW_BASE_ENCODE_H */ + diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 0eed9691ca..f5c9be8973 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -33,34 +33,27 @@ #include "avcodec.h" #include "hwconfig.h" +#include "hw_base_encode.h" struct VAAPIEncodeType; struct VAAPIEncodePicture; +// Codec output packet without timestamp delay, which means the +// output packet has same PTS and DTS. +#define FLAG_TIMESTAMP_NO_DELAY 1 << 6 + enum { MAX_CONFIG_ATTRIBUTES = 4, MAX_GLOBAL_PARAMS = 4, - MAX_DPB_SIZE = 16, - MAX_PICTURE_REFERENCES = 2, - MAX_REORDER_DELAY = 16, MAX_PARAM_BUFFER_SIZE = 1024, // A.4.1: table A.6 allows at most 22 tile rows for any level. MAX_TILE_ROWS = 22, // A.4.1: table A.6 allows at most 20 tile columns for any level. MAX_TILE_COLS = 20, - MAX_ASYNC_DEPTH = 64, - MAX_REFERENCE_LIST_NUM = 2, }; extern const AVCodecHWConfigInternal *const ff_vaapi_encode_hw_configs[]; -enum { - PICTURE_TYPE_IDR = 0, - PICTURE_TYPE_I = 1, - PICTURE_TYPE_P = 2, - PICTURE_TYPE_B = 3, -}; - typedef struct VAAPIEncodeSlice { int index; int row_start; @@ -193,7 +186,8 @@ typedef struct VAAPIEncodeRCMode { } VAAPIEncodeRCMode; typedef struct VAAPIEncodeContext { - const AVClass *class; + // Base context. + HWBaseEncodeContext base; // Codec-specific hooks. const struct VAAPIEncodeType *codec; @@ -397,25 +391,6 @@ typedef struct VAAPIEncodeContext { AVPacket *tail_pkt; } VAAPIEncodeContext; -enum { - // Codec supports controlling the subdivision of pictures into slices. - FLAG_SLICE_CONTROL = 1 << 0, - // Codec only supports constant quality (no rate control). - FLAG_CONSTANT_QUALITY_ONLY = 1 << 1, - // Codec is intra-only. - FLAG_INTRA_ONLY = 1 << 2, - // Codec supports B-pictures. - FLAG_B_PICTURES = 1 << 3, - // Codec supports referencing B-pictures. - FLAG_B_PICTURE_REFERENCES = 1 << 4, - // Codec supports non-IDR key pictures (that is, key pictures do - // not necessarily empty the DPB). - FLAG_NON_IDR_KEY_PICTURES = 1 << 5, - // Codec output packet without timestamp delay, which means the - // output packet has same PTS and DTS. - FLAG_TIMESTAMP_NO_DELAY = 1 << 6, -}; - typedef struct VAAPIEncodeType { // List of supported profiles and corresponding VAAPI profiles. // (Must end with AV_PROFILE_UNKNOWN.) From patchwork Wed May 22 01:26:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49117 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp81817pzb; Tue, 21 May 2024 18:28:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU395LY3kGWlgsFJrQuusB3UaEUT/mNNd4O/1mljDkKOnLGk/03xz2xFJzN7Cm13iKPslTNrAPpiRSnglsx7o7Mz/jnoWf1b9sxjw== X-Google-Smtp-Source: AGHT+IGoOPdwcH37D8RPVBnVQUL61JMMQn1AfXJxhTdJdXo9HIFDd0XwVAjOgRV2LyaD8XoLwlF4 X-Received: by 2002:a50:ccd9:0:b0:574:ecf3:f7d1 with SMTP id 4fb4d7f45d1cf-57830ce38e3mr615713a12.0.1716341303595; Tue, 21 May 2024 18:28:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341303; cv=none; d=google.com; s=arc-20160816; b=DtV5kYWgOfdOc1gmezY0O221MwK9aS2lgKswJIe8MyA/8CWoPqYeRDMB9tfV9viO55 EP6VNtWrTJIJ94bOxBvWtHJ6+dSMC4k8texbRTIVQ+FB7tcVK0N7K41IVcIs1c2kie3Y rl5qHt7Baqcdln/ukcPAGRsJSqFtiJufUDdKYdiQWfmoQAV5yII3C+maTqkPGbBJ5lZp IPu5sOcXd1dnVKeb9Joz4hO0VI6lXMoB1rOBB3zuLEdSZp83CRsl/wz3DAeg6vV0uRI8 s2uss+pzT/Dkj/mVK5ARN5ZznfZMyZtIBVUT4SEu2z0+tEmrvvzHtQlHdFJE6vWVZPRH 4nkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=HKgbMHJaVakAuq/2qpGaujnrPFpo0XBj6fSfCo3c7mM=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=EnMWY8OE19QHOa80eIh3RVHZOC4BaZeHOJg+G4oQPTby72AsUprcgr3CoYayBdsn51 7Eu2CKlgDVKHmxiNp/C75WB9Oy/iuCAshTHshlZQwSMPN0VvLd/oM95NxfjM2UGwWMoS GhwUtbTkJG3nScDN9Z0JN1IfGEivNR6VhCJIVWrinJsxS0/Ksj2VZnsyIfbcyOv3Khvl /M1+zpYOszVHD3iGkriAM2DulUs4EuutVoqfahrWinSH9JneEtsh+lPGJ4HRLE53SIX/ yxVuxTcvgDkUz7/ecgF5z6Exn69wCsbAgM+4D6FDJjVbvSW8jGFpPt9E6Z7gKrvsrnvU LUNQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ajqHDrIv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5782a4227f7si401067a12.673.2024.05.21.18.28.22; Tue, 21 May 2024 18:28:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=ajqHDrIv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1D84968D3E5; Wed, 22 May 2024 04:28:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9AB9B68D3C9 for ; Wed, 22 May 2024 04:28:04 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341290; x=1747877290; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L+pX56Ae6U7EXYvxYNkP42r/F/Bqry4GF5aUtsvc24w=; b=ajqHDrIvf38zmvYQpIT3ipUf3rIYVPkIRCoxgkYfLy8xmMWIir15ur6v ImZe8umS3RWzRNp9kEOn0lA6rPBwYSXp3qhfT67Ugq15X7mnGQQe0cvXY Htl00uHCaPhM+g6dKGJ7W08rMvel6e195FX0m0dyubN6lT9NPKTQBOzOQ 8zCwQUlVGHRImvVWNIKuDOtGd5w21jnsFEyDrS+W6dfMDZRB8rX7hAfz3 lhZcpCNUGWHRpM73BinweBvJh7OO3qiBJp2G7nyxS5UYYcWAn/BN/0+Fn kDdr4EOwCalkIQRqQ+53RNcfLTLa7xICNXuWyEvR4ElycyZP56YQPwSlO g==; X-CSE-ConnectionGUID: zyW9/JVbTxiGZETZ2uTYnA== X-CSE-MsgGUID: rEktRgBVQ8aJLdJQUAIxJg== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797191" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797191" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:02 -0700 X-CSE-ConnectionGUID: FACjUc+9Qg6nSqkCCM6xBw== X-CSE-MsgGUID: IuMBbWlkTdOkq/HYXR3QgQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537427" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:01 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:15 +0800 Message-ID: <20240522012627.1386-2-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 02/13] avcodec/vaapi_encode: add async_depth to common options X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +idxJ4s8HWlE From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.h | 10 +++++++++- libavcodec/vaapi_encode.c | 13 ++++++++----- libavcodec/vaapi_encode.h | 7 ------- libavcodec/vaapi_encode_av1.c | 1 + libavcodec/vaapi_encode_h264.c | 1 + libavcodec/vaapi_encode_h265.c | 1 + libavcodec/vaapi_encode_mjpeg.c | 1 + libavcodec/vaapi_encode_mpeg2.c | 1 + libavcodec/vaapi_encode_vp8.c | 1 + libavcodec/vaapi_encode_vp9.c | 1 + 10 files changed, 24 insertions(+), 13 deletions(-) diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 1996179456..5272f2836d 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -50,7 +50,15 @@ enum { typedef struct HWBaseEncodeContext { const AVClass *class; + + // Max number of frame buffered in encoder. + int async_depth; } HWBaseEncodeContext; -#endif /* AVCODEC_HW_BASE_ENCODE_H */ +#define HW_BASE_ENCODE_COMMON_OPTIONS \ + { "async_depth", "Maximum processing parallelism. " \ + "Increase this to improve single channel performance.", \ + OFFSET(common.base.async_depth), AV_OPT_TYPE_INT, \ + { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS } +#endif /* AVCODEC_HW_BASE_ENCODE_H */ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f54b2579ec..9373512417 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -669,7 +669,8 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, VAAPIEncodePicture *pic, AVPacket *pkt) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; if (pic->type == PICTURE_TYPE_IDR) pkt->flags |= AV_PKT_FLAG_KEY; @@ -699,7 +700,7 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; } else { pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % - (3 * ctx->output_delay + ctx->async_depth)]; + (3 * ctx->output_delay + base_ctx->async_depth)]; } return 0; @@ -1320,6 +1321,7 @@ static int vaapi_encode_check_frame(AVCodecContext *avctx, static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodePicture *pic; int err; @@ -1365,7 +1367,7 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) ctx->dts_pts_diff = pic->pts - ctx->first_pts; if (ctx->output_delay > 0) ctx->ts_ring[ctx->input_order % - (3 * ctx->output_delay + ctx->async_depth)] = pic->pts; + (3 * ctx->output_delay + base_ctx->async_depth)] = pic->pts; pic->display_order = ctx->input_order; ++ctx->input_order; @@ -2773,7 +2775,8 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; AVVAAPIFramesContext *recon_hwctx = NULL; VAStatus vas; int err; @@ -2966,7 +2969,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) vas = vaSyncBuffer(ctx->hwctx->display, VA_INVALID_ID, 0); if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) { ctx->has_sync_buffer_func = 1; - ctx->encode_fifo = av_fifo_alloc2(ctx->async_depth, + ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, sizeof(VAAPIEncodePicture *), 0); if (!ctx->encode_fifo) diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index f5c9be8973..02410c72ec 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -374,8 +374,6 @@ typedef struct VAAPIEncodeContext { int has_sync_buffer_func; // Store buffered pic AVFifo *encode_fifo; - // Max number of frame buffered in encoder. - int async_depth; /** Head data for current output pkt, used only for AV1. */ //void *header_data; @@ -491,11 +489,6 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "Maximum B-frame reference depth", \ OFFSET(common.desired_b_depth), AV_OPT_TYPE_INT, \ { .i64 = 1 }, 1, INT_MAX, FLAGS }, \ - { "async_depth", "Maximum processing parallelism. " \ - "Increase this to improve single channel performance. This option " \ - "doesn't work if driver doesn't implement vaSyncBuffer function.", \ - OFFSET(common.async_depth), AV_OPT_TYPE_INT, \ - { .i64 = 2 }, 1, MAX_ASYNC_DEPTH, FLAGS }, \ { "max_frame_size", \ "Maximum frame size (in bytes)",\ OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index b868f5b66a..a5a475dca6 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -965,6 +965,7 @@ static av_cold int vaapi_encode_av1_close(AVCodecContext *avctx) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_av1_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, { "profile", "Set profile (seq_profile)", diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index d656b1020f..ebb1760cd3 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -1276,6 +1276,7 @@ static av_cold int vaapi_encode_h264_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeH264Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_h264_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 2f59161346..77bd5e31af 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -1394,6 +1394,7 @@ static av_cold int vaapi_encode_h265_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeH265Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_h265_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index c17747e3a9..fb5c0d34c6 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -540,6 +540,7 @@ static av_cold int vaapi_encode_mjpeg_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeMJPEGContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_mjpeg_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, { "jfif", "Include JFIF header", diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index c9b16fbcfc..d0980c52b0 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -639,6 +639,7 @@ static av_cold int vaapi_encode_mpeg2_close(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeMPEG2Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_mpeg2_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index 8a557b967e..4e284f86e2 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -216,6 +216,7 @@ static av_cold int vaapi_encode_vp8_init(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeVP8Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_vp8_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index c2a8dec71b..88f951652c 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -273,6 +273,7 @@ static av_cold int vaapi_encode_vp9_init(AVCodecContext *avctx) #define OFFSET(x) offsetof(VAAPIEncodeVP9Context, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) static const AVOption vaapi_encode_vp9_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_COMMON_OPTIONS, VAAPI_ENCODE_RC_OPTIONS, From patchwork Wed May 22 01:26:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49118 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp81856pzb; Tue, 21 May 2024 18:28:33 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVwp/KP+Ou9iVKJqA9Aq5Qzr1FqxTSF45GSz9edXR4qheS1eZdL1rYwea0LjWuKHKlrtjNfg/b+6fMiF7F7zbnW81Z7EavAjfFofQ== X-Google-Smtp-Source: AGHT+IHjZ0cSaHznRZBQpRDy8jTbs07p6vJNUQZZhfpjuYXTA10dbRutBUe2Gm+sT8AakhS4iSED X-Received: by 2002:a50:d581:0:b0:572:5f4b:bf2e with SMTP id 4fb4d7f45d1cf-57832a3fa43mr410550a12.23.1716341313409; Tue, 21 May 2024 18:28:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341313; cv=none; d=google.com; s=arc-20160816; b=DRqOAI2J+ixXL0GFwcR+6MEhtXUNDDZY87UiTjuj3whffQk+2IMInjgozlM43zowGB e3wHfLWXYv7AKYDlNpBjt0rVbRxnA//mZIAzihOTkNWi82ccI6bcIj71CQOSShp/btx8 yBmaRVXxTNFFSROHiF9l+mc0aYclhAWgzpD/sgVX8XC7ysonmKKGO9gJA874wY6WeTHq twMqVjFd0d9QnmS7G7CfrX+MC6nWV+VnCoHu/aLKnbhICpshv7nijKFH8wyLBhMl0+Gz yzMfeNaEG+QmYR0e3g1QsGXjFi92cfcJ/i2wuz1mNWs1uTve6ZJafBLArVu40ciMzAk+ rbKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=q1kdfZ2WVoEFjfviXQUkQo/qA20jDHG+TCQ8dQEuWeI=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=KcpKrQ15rIxbbz8wg9NIQDK+eaweb6KksBGd4KZkVLsmx5mP0ChUr01eOX3U47wMzg gH1LU9f4X97SIAuZ949zU/1JWnPzZeLtK5SlfULLIcM4zfFTgxcOUedu4KPgfnHWGGCn Oz9VcVZY+6uQFZOz099ain1MKd2ny3sEd4zQbVPFHLLuWcL0kIqfw0ORXtau9IECnbEr mnjNHEH1IQeT8XKNgfTXWmcnQZiFDhnvlPxYvroSmL7o6SyxB4KCin0CZWVExLqhaEz3 cg3xE5sKROSOkq2qxm4kHewG7YQg82v4bJLuZUhAgm2BNh7NFAqRRJ1iMc2IUaoDRmPN 3aow==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=EWBYIOhH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5733c37850bsi14111991a12.629.2024.05.21.18.28.32; Tue, 21 May 2024 18:28:33 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=EWBYIOhH; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7B6FA68D3FA; Wed, 22 May 2024 04:28:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B212868D3C6 for ; Wed, 22 May 2024 04:28:05 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341291; x=1747877291; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KAG/yivTgGEHchufiYiAVJxZSI/iQ/DiNn32BP+e50Y=; b=EWBYIOhHqEjMREWuHf1D2eQUm/QM8mF4+mPzagmfYS5tbF+38k1mj1jR xiLds+SN7In4zvYDJFheHYiwugtPbMkOlvy3LTtMh+Vq8Mw9pCqv9WVFv LHN+hjs8xis1jCIUEU4hhnyNk5o+Rew6nKFfTPROFR1+TQ9M5BQTexKK5 SQn3CMX9m3VqXPamH5pZLCyY1t2++7e2W+IL6rSpXXrbYJZNtKBQC0vEr xw9q6uksdHCG81/QXMQg7lPb6gLfwQLrL+31/5f86pbGr5LhV++ZFFWu+ C35mpcSMHysvcsTfMMNHA0DMR/VIqsIpNjUuNR8IaKV2JkG7fHqXaCO/v w==; X-CSE-ConnectionGUID: zYrjvbR6Su+WqJaTcD71OA== X-CSE-MsgGUID: xDN7EHFTTKyfMsT9HXUXQQ== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797198" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797198" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:03 -0700 X-CSE-ConnectionGUID: 86di0l1tQqawx0GZeFoZwg== X-CSE-MsgGUID: KCRaRggjTQ2T9J/vgs4tkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537440" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:02 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:16 +0800 Message-ID: <20240522012627.1386-3-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 03/13] avcodec/vaapi_encode: add picture type name to base X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vdc0jC3axZMR From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.h | 5 +++++ libavcodec/vaapi_encode.c | 4 +--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 5272f2836d..a578db8c06 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -25,6 +25,11 @@ #define MAX_ASYNC_DEPTH 64 #define MAX_REFERENCE_LIST_NUM 2 +static inline const char *ff_hw_base_encode_get_pictype_name(const int type) { + const char * const picture_type_name[] = { "IDR", "I", "P", "B" }; + return picture_type_name[type]; +} + enum { PICTURE_TYPE_IDR = 0, PICTURE_TYPE_I = 1, diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 9373512417..2d22e4bd85 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -38,8 +38,6 @@ const AVCodecHWConfigInternal *const ff_vaapi_encode_hw_configs[] = { NULL, }; -static const char * const picture_type_name[] = { "IDR", "I", "P", "B" }; - static int vaapi_encode_make_packed_header(AVCodecContext *avctx, VAAPIEncodePicture *pic, int type, char *data, size_t bit_len) @@ -277,7 +275,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " "as type %s.\n", pic->display_order, pic->encode_order, - picture_type_name[pic->type]); + ff_hw_base_encode_get_pictype_name(pic->type)); if (pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0) { av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); } else { From patchwork Wed May 22 01:26:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49119 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp81886pzb; Tue, 21 May 2024 18:28:41 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWpf1S8NUq1gO28CWC2/v7yUAuwRSMIE4NXfC0Q/mnk7Q/KU3XaHxnNr2z3FHP+WGLIuMNdkuTmKZt/lFZwwkitbayPDBx0zy6PFQ== X-Google-Smtp-Source: AGHT+IHWD8/Jy8A6a7tZcSEmX6dSMwyS38eBBRz5pYAFLuQG2ZHu9ETJRAAk8E8GPxRqBFSRb+Tc X-Received: by 2002:a05:6402:2710:b0:572:cfa4:57ea with SMTP id 4fb4d7f45d1cf-57832c87c28mr203759a12.4.1716341321603; Tue, 21 May 2024 18:28:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341321; cv=none; d=google.com; s=arc-20160816; b=YdcfOrrmK1xtHnJRZd3uVNQo69Xn+DOokLntA9s5RxrfoXOFpqpml+TCHakKaUby/9 ySps2Tqjjgu9U7kP2qmpSo/qgMXkSy6islNLsKjV3Bkr7LdUQ45dWpex2jzjjb0ysnnV pdP/BELiGUN2OqSJhXxxAiCkAUyYdNYAlhB1IWptj4tx0Co9W0DO8lVuAiZZZFe4dlif 1V998uIRd0BZlbb0+CnIt/EfgsOme22vpUrF+qi6mjbqGRhV2FTFDX2hDMObbCNCv6B4 U6iu+hOQY+o0SzNEBy7KkVqMO8K1iw5bfgxYwP08DobWCQvx52fow12o6iqTtgutRto7 HTTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=DruZqgCM0Ll/K9SNdY0r0CexfVTk23BkLKZDcD+w+o4=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=Qcb3IEHYneAcg0SYcmousjOo9gXwYMssw3PflnAdLx7rDVQeSIwF9dfUOUQyqRBsm5 4xhnhm++20Zk8PXlbTgCRATWYoGjzSLPYw/MqiSMS9YTGFAElii3ZpgisHCtvW63sh75 HhU05lD/BqKGTV5/rPY1iCbu9YNLKESoACQt5hJi9iSF6ccBZgYFFWFKSC5JYK9QnDxf RNNa20ZT3DMKkCqM9qdyldqHd6iITZhafrafLGPaiQC/iO9QQ13lGhOsID+/bGciC4nB VY2FMoHUP0xfcVglF96Zjg0SLXyucJ3G0xz1ZfAgqkVlMqWiygxujzwL1jhZrwqUTA6U BQ7g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=b2BsZk5E; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-574ece5ca7esi8363573a12.498.2024.05.21.18.28.41; Tue, 21 May 2024 18:28:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=b2BsZk5E; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 118BF68D41D; Wed, 22 May 2024 04:28:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B20E168D3D0 for ; Wed, 22 May 2024 04:28:08 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341294; x=1747877294; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k+Cd3SbMsZI2AOBFYmvXlX99kdjYu6SRZFiVO28Uq+c=; b=b2BsZk5E+aovP9V6vq+DWMog2xUR1ll1b5crm8kpmzPt4yWfBtE+71+0 hMULF39FwhpSUC7FwKjBIliXB2KpjlbZWsQpbmet0XAmGWRl7Y0XoAZ9k zTPYJA86y0X05YtyGxZRiJ43gjCgy9FaA+a9k6reGQdAJH0BCtHRmGnmr 59XGZHm4Obz7E7KVXKLcTLEzXS3f6PP7NgXTgI+6VU58MiMr9LlAPf7rJ QtlimqkI0Oql8H6+JDneGoHtLbSd5gTfagoATcdJf4LK35n40XnfKsd00 vIGrj1D87sMdmbam8F8kZozx4nsF+3rHvv/+8nTtaaRri3Db2na28H1E6 g==; X-CSE-ConnectionGUID: cIOgi5i3QG2x6PEitcJ9mw== X-CSE-MsgGUID: k9i1fmfjQo+Aqy+1w7DPvg== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797200" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797200" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:04 -0700 X-CSE-ConnectionGUID: 59kBYwqzQUW6uGVe8ierdw== X-CSE-MsgGUID: z4X7yHehS76v7fl96eo2qw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537447" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:03 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:17 +0800 Message-ID: <20240522012627.1386-4-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 04/13] avcodec/vaapi_encode: move pic->input_surface initialization to encode_alloc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: AL1ug7O4Dnjq From: Tong Wu When allocating the VAAPIEncodePicture, pic->input_surface can be initialized right in the place. This movement simplifies the send_frame logic and is the preparation for moving vaapi_encode_send_frame to the base layer. Signed-off-by: Tong Wu --- libavcodec/vaapi_encode.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 2d22e4bd85..227cccae64 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -878,7 +878,8 @@ static int vaapi_encode_discard(AVCodecContext *avctx, return 0; } -static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx) +static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, + const AVFrame *frame) { VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodePicture *pic; @@ -895,7 +896,7 @@ static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx) } } - pic->input_surface = VA_INVALID_ID; + pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3]; pic->recon_surface = VA_INVALID_ID; pic->output_buffer = VA_INVALID_ID; @@ -1332,7 +1333,7 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) if (err < 0) return err; - pic = vaapi_encode_alloc(avctx); + pic = vaapi_encode_alloc(avctx, frame); if (!pic) return AVERROR(ENOMEM); @@ -1345,7 +1346,6 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I) pic->force_idr = 1; - pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3]; pic->pts = frame->pts; pic->duration = frame->duration; From patchwork Wed May 22 01:26:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49124 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82034pzb; Tue, 21 May 2024 18:29:11 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVcyoVV70Z2zJm87IJ0Zub7lkBXInvjdmPkikxZeA+B6htLq/rtK8H4cRzyvcyCH1WtTGr97PgwVRCKHVdExsl+Fcx7FmmAWVeOrQ== X-Google-Smtp-Source: AGHT+IHFfVmVWWD12ZIXKG6szyEfFzjkwWg/cJUxBdD05i0otJushH7U/H4uUs+pJv/8Qvvvni4t X-Received: by 2002:ac2:5ecc:0:b0:51f:488d:274b with SMTP id 2adb3069b0e04-526bebb3991mr242871e87.9.1716341350781; Tue, 21 May 2024 18:29:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341350; cv=none; d=google.com; s=arc-20160816; b=epw2Ca5RtSBx3RqaZL6nVt+QTxEMr6otId4atx8MMw6udtmXx+Plsr4vUE+c+2g7/W x75VV13Q2AsB/Cn4OzlfhSloS+8Jo1/oT+9DWAIfW9lt9Ja8clKF1k2EO1onw4dOiekp TCX5jJEBNRZWyQOws3dU9Oienlmj/85x5b5k66ef78/1u2kpgIt/VkT8mG2iukiw+h1y 6Ae6ITOQnBF/Nr5nXe09DeP/i1Ve6K+RmWvUkeCZmkuwjMUmiBtehIxqSVKTY+fq4X2w 9ClWI5T9BdjhGOzEbVRHs13nUX6eTYoZQKe8X6tPdNgncgGHx2xLbxpA2ovr0GFCxZIn yWxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=CfVRKteViZZY2At9Yw1ZhjxmQ6NuXTS6IdNSAuUSibc=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=JMQncHah8s+tUttB2GDIhpsOt8fJYxPIQIqQoT1FB6jD1GfBjApG9KijQi11RPF68n Ct8S34U3kqRLafB7qopxcuqb3pt4phIDWuT5YtTo1Oh0lp6XSPMhHMRbytLf+xxa/+96 d/uYrJh3hPH8V5S/URqkTYHU/XRT8OKfvpCyT7wdT4zcdseujUlVzm/8NpgWO9fB9dEu HXWB6MTylB3ngUvpcZLjVTgtoNOFXllI+b2wNkuRMbpPLrPFJCLkD7RmGU1TF0/cWCdi YMpNFAy53eZK+/44xYMkUfxTq1woeBwnbey6Z4yhjw39SV78f+30Vf4snBq+Iad2ZywQ q8Bg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=en5Qdgjp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-521f35b1186si9032562e87.112.2024.05.21.18.29.10; Tue, 21 May 2024 18:29:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=en5Qdgjp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 42A8368D477; Wed, 22 May 2024 04:28:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 85D2168D288 for ; Wed, 22 May 2024 04:28:10 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341296; x=1747877296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZDiQ9vIROS3oVAcDgdhanC41aRj2Ab8ZKq7BlYsercA=; b=en5QdgjpF7zNc1wJ0lEK2Jm9QYE8lEfqCXyJaoD2sLTnGQW9q3f6UFCZ l4b7ptmmJ7PdAwWyWdcb3FtNeefcUXY9jL4aCKyi+bUpheMrtfEVApI0y +miS/YSgpj/WWm/Tl+dqU2McjM9uLyJsZ8QmHubs6XnuUciDwUE05MTyU Ryl5El9Uc9XFwMGauEve3jhLK3kVF4Be5iruG/UunPamGecHlx6tIv3G1 N2M32TJEv80HOI6RnEaCExxWbScjWx4m+hLjElD8PilFuDhef3syWbPl/ 4HgsxVVDXhATIFYvayvcXGtsHPycAv7D7E4HyfrKR8H97RsaJXzxhPD0r g==; X-CSE-ConnectionGUID: T66eCOoaTuCUEBojWHoBXw== X-CSE-MsgGUID: WCiMUpqpR0KbUhwT/PArDw== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797201" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797201" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:05 -0700 X-CSE-ConnectionGUID: E8RUW9VSR4K80Wy7hLB96A== X-CSE-MsgGUID: mnesI5TuTf2ij9kTj9ck8A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537460" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:04 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:18 +0800 Message-ID: <20240522012627.1386-5-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 05/13] avcodec/vaapi_encode: move the dpb logic from VAAPI to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: X3LcLYIJrOJ6 From: Tong Wu Move receive_packet function to base. This requires adding *alloc, *issue, *output, *free as hardware callbacks. HWBaseEncodePicture is introduced as the base layer structure. The related parameters in VAAPIEncodeContext are also extracted to HWBaseEncodeContext. Then DPB management logic can be fully extracted to base layer as-is. Signed-off-by: Tong Wu --- libavcodec/Makefile | 2 +- libavcodec/hw_base_encode.c | 597 ++++++++++++++++++++++++ libavcodec/hw_base_encode.h | 124 +++++ libavcodec/vaapi_encode.c | 793 +++++--------------------------- libavcodec/vaapi_encode.h | 102 +--- libavcodec/vaapi_encode_av1.c | 35 +- libavcodec/vaapi_encode_h264.c | 84 ++-- libavcodec/vaapi_encode_h265.c | 53 ++- libavcodec/vaapi_encode_mjpeg.c | 13 +- libavcodec/vaapi_encode_mpeg2.c | 33 +- libavcodec/vaapi_encode_vp8.c | 18 +- libavcodec/vaapi_encode_vp9.c | 24 +- 12 files changed, 988 insertions(+), 890 deletions(-) create mode 100644 libavcodec/hw_base_encode.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 2443d2c6fd..998f6b7e12 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -165,7 +165,7 @@ OBJS-$(CONFIG_STARTCODE) += startcode.o OBJS-$(CONFIG_TEXTUREDSP) += texturedsp.o OBJS-$(CONFIG_TEXTUREDSPENC) += texturedspenc.o OBJS-$(CONFIG_TPELDSP) += tpeldsp.o -OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o +OBJS-$(CONFIG_VAAPI_ENCODE) += vaapi_encode.o hw_base_encode.o OBJS-$(CONFIG_AV1_AMF_ENCODER) += amfenc_av1.o OBJS-$(CONFIG_VC1DSP) += vc1dsp.o OBJS-$(CONFIG_VIDEODSP) += videodsp.o diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c new file mode 100644 index 0000000000..ec7178d2dc --- /dev/null +++ b/libavcodec/hw_base_encode.c @@ -0,0 +1,597 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/internal.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" + +#include "encode.h" +#include "avcodec.h" +#include "hw_base_encode.h" + +static void hw_base_encode_add_ref(AVCodecContext *avctx, + HWBaseEncodePicture *pic, + HWBaseEncodePicture *target, + int is_ref, int in_dpb, int prev) +{ + int refs = 0; + + if (is_ref) { + av_assert0(pic != target); + av_assert0(pic->nb_refs[0] < MAX_PICTURE_REFERENCES && + pic->nb_refs[1] < MAX_PICTURE_REFERENCES); + if (target->display_order < pic->display_order) + pic->refs[0][pic->nb_refs[0]++] = target; + else + pic->refs[1][pic->nb_refs[1]++] = target; + ++refs; + } + + if (in_dpb) { + av_assert0(pic->nb_dpb_pics < MAX_DPB_SIZE); + pic->dpb[pic->nb_dpb_pics++] = target; + ++refs; + } + + if (prev) { + av_assert0(!pic->prev); + pic->prev = target; + ++refs; + } + + target->ref_count[0] += refs; + target->ref_count[1] += refs; +} + +static void hw_base_encode_remove_refs(AVCodecContext *avctx, + HWBaseEncodePicture *pic, + int level) +{ + int i; + + if (pic->ref_removed[level]) + return; + + for (i = 0; i < pic->nb_refs[0]; i++) { + av_assert0(pic->refs[0][i]); + --pic->refs[0][i]->ref_count[level]; + av_assert0(pic->refs[0][i]->ref_count[level] >= 0); + } + + for (i = 0; i < pic->nb_refs[1]; i++) { + av_assert0(pic->refs[1][i]); + --pic->refs[1][i]->ref_count[level]; + av_assert0(pic->refs[1][i]->ref_count[level] >= 0); + } + + for (i = 0; i < pic->nb_dpb_pics; i++) { + av_assert0(pic->dpb[i]); + --pic->dpb[i]->ref_count[level]; + av_assert0(pic->dpb[i]->ref_count[level] >= 0); + } + + av_assert0(pic->prev || pic->type == PICTURE_TYPE_IDR); + if (pic->prev) { + --pic->prev->ref_count[level]; + av_assert0(pic->prev->ref_count[level] >= 0); + } + + pic->ref_removed[level] = 1; +} + +static void hw_base_encode_set_b_pictures(AVCodecContext *avctx, + HWBaseEncodePicture *start, + HWBaseEncodePicture *end, + HWBaseEncodePicture *prev, + int current_depth, + HWBaseEncodePicture **last) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *next, *ref; + int i, len; + + av_assert0(start && end && start != end && start->next != end); + + // If we are at the maximum depth then encode all pictures as + // non-referenced B-pictures. Also do this if there is exactly one + // picture left, since there will be nothing to reference it. + if (current_depth == ctx->max_b_depth || start->next->next == end) { + for (pic = start->next; pic; pic = pic->next) { + if (pic == end) + break; + pic->type = PICTURE_TYPE_B; + pic->b_depth = current_depth; + + hw_base_encode_add_ref(avctx, pic, start, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, end, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, prev, 0, 0, 1); + + for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) + hw_base_encode_add_ref(avctx, pic, ref, 0, 1, 0); + } + *last = prev; + + } else { + // Split the current list at the midpoint with a referenced + // B-picture, then descend into each side separately. + len = 0; + for (pic = start->next; pic != end; pic = pic->next) + ++len; + for (pic = start->next, i = 1; 2 * i < len; pic = pic->next, i++); + + pic->type = PICTURE_TYPE_B; + pic->b_depth = current_depth; + + pic->is_reference = 1; + + hw_base_encode_add_ref(avctx, pic, pic, 0, 1, 0); + hw_base_encode_add_ref(avctx, pic, start, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, end, 1, 1, 0); + hw_base_encode_add_ref(avctx, pic, prev, 0, 0, 1); + + for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) + hw_base_encode_add_ref(avctx, pic, ref, 0, 1, 0); + + if (i > 1) + hw_base_encode_set_b_pictures(avctx, start, pic, pic, + current_depth + 1, &next); + else + next = pic; + + hw_base_encode_set_b_pictures(avctx, pic, end, next, + current_depth + 1, last); + } +} + +static void hw_base_encode_add_next_prev(AVCodecContext *avctx, + HWBaseEncodePicture *pic) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + int i; + + if (!pic) + return; + + if (pic->type == PICTURE_TYPE_IDR) { + for (i = 0; i < ctx->nb_next_prev; i++) { + --ctx->next_prev[i]->ref_count[0]; + ctx->next_prev[i] = NULL; + } + ctx->next_prev[0] = pic; + ++pic->ref_count[0]; + ctx->nb_next_prev = 1; + + return; + } + + if (ctx->nb_next_prev < MAX_PICTURE_REFERENCES) { + ctx->next_prev[ctx->nb_next_prev++] = pic; + ++pic->ref_count[0]; + } else { + --ctx->next_prev[0]->ref_count[0]; + for (i = 0; i < MAX_PICTURE_REFERENCES - 1; i++) + ctx->next_prev[i] = ctx->next_prev[i + 1]; + ctx->next_prev[i] = pic; + ++pic->ref_count[0]; + } +} + +static int hw_base_encode_pick_next(AVCodecContext *avctx, + HWBaseEncodePicture **pic_out) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic = NULL, *prev = NULL, *next, *start; + int i, b_counter, closed_gop_end; + + // If there are any B-frames already queued, the next one to encode + // is the earliest not-yet-issued frame for which all references are + // available. + for (pic = ctx->pic_start; pic; pic = pic->next) { + if (pic->encode_issued) + continue; + if (pic->type != PICTURE_TYPE_B) + continue; + for (i = 0; i < pic->nb_refs[0]; i++) { + if (!pic->refs[0][i]->encode_issued) + break; + } + if (i != pic->nb_refs[0]) + continue; + + for (i = 0; i < pic->nb_refs[1]; i++) { + if (!pic->refs[1][i]->encode_issued) + break; + } + if (i == pic->nb_refs[1]) + break; + } + + if (pic) { + av_log(avctx, AV_LOG_DEBUG, "Pick B-picture at depth %d to " + "encode next.\n", pic->b_depth); + *pic_out = pic; + return 0; + } + + // Find the B-per-Pth available picture to become the next picture + // on the top layer. + start = NULL; + b_counter = 0; + closed_gop_end = ctx->closed_gop || + ctx->idr_counter == ctx->gop_per_idr; + for (pic = ctx->pic_start; pic; pic = next) { + next = pic->next; + if (pic->encode_issued) { + start = pic; + continue; + } + // If the next available picture is force-IDR, encode it to start + // a new GOP immediately. + if (pic->force_idr) + break; + if (b_counter == ctx->b_per_p) + break; + // If this picture ends a closed GOP or starts a new GOP then it + // needs to be in the top layer. + if (ctx->gop_counter + b_counter + closed_gop_end >= ctx->gop_size) + break; + // If the picture after this one is force-IDR, we need to encode + // this one in the top layer. + if (next && next->force_idr) + break; + ++b_counter; + } + + // At the end of the stream the last picture must be in the top layer. + if (!pic && ctx->end_of_stream) { + --b_counter; + pic = ctx->pic_end; + if (pic->encode_complete) + return AVERROR_EOF; + else if (pic->encode_issued) + return AVERROR(EAGAIN); + } + + if (!pic) { + av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " + "need more input for reference pictures.\n"); + return AVERROR(EAGAIN); + } + if (ctx->input_order <= ctx->decode_delay && !ctx->end_of_stream) { + av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " + "need more input for timestamps.\n"); + return AVERROR(EAGAIN); + } + + if (pic->force_idr) { + av_log(avctx, AV_LOG_DEBUG, "Pick forced IDR-picture to " + "encode next.\n"); + pic->type = PICTURE_TYPE_IDR; + ctx->idr_counter = 1; + ctx->gop_counter = 1; + + } else if (ctx->gop_counter + b_counter >= ctx->gop_size) { + if (ctx->idr_counter == ctx->gop_per_idr) { + av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP IDR-picture to " + "encode next.\n"); + pic->type = PICTURE_TYPE_IDR; + ctx->idr_counter = 1; + } else { + av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP I-picture to " + "encode next.\n"); + pic->type = PICTURE_TYPE_I; + ++ctx->idr_counter; + } + ctx->gop_counter = 1; + + } else { + if (ctx->gop_counter + b_counter + closed_gop_end == ctx->gop_size) { + av_log(avctx, AV_LOG_DEBUG, "Pick group-end P-picture to " + "encode next.\n"); + } else { + av_log(avctx, AV_LOG_DEBUG, "Pick normal P-picture to " + "encode next.\n"); + } + pic->type = PICTURE_TYPE_P; + av_assert0(start); + ctx->gop_counter += 1 + b_counter; + } + pic->is_reference = 1; + *pic_out = pic; + + hw_base_encode_add_ref(avctx, pic, pic, 0, 1, 0); + if (pic->type != PICTURE_TYPE_IDR) { + // TODO: apply both previous and forward multi reference for all vaapi encoders. + // And L0/L1 reference frame number can be set dynamically through query + // VAConfigAttribEncMaxRefFrames attribute. + if (avctx->codec_id == AV_CODEC_ID_AV1) { + for (i = 0; i < ctx->nb_next_prev; i++) + hw_base_encode_add_ref(avctx, pic, ctx->next_prev[i], + pic->type == PICTURE_TYPE_P, + b_counter > 0, 0); + } else + hw_base_encode_add_ref(avctx, pic, start, + pic->type == PICTURE_TYPE_P, + b_counter > 0, 0); + + hw_base_encode_add_ref(avctx, pic, ctx->next_prev[ctx->nb_next_prev - 1], 0, 0, 1); + } + + if (b_counter > 0) { + hw_base_encode_set_b_pictures(avctx, start, pic, pic, 1, + &prev); + } else { + prev = pic; + } + hw_base_encode_add_next_prev(avctx, prev); + + return 0; +} + +static int hw_base_encode_clear_old(AVCodecContext *avctx) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *prev, *next; + + av_assert0(ctx->pic_start); + + // Remove direct references once each picture is complete. + for (pic = ctx->pic_start; pic; pic = pic->next) { + if (pic->encode_complete && pic->next) + hw_base_encode_remove_refs(avctx, pic, 0); + } + + // Remove indirect references once a picture has no direct references. + for (pic = ctx->pic_start; pic; pic = pic->next) { + if (pic->encode_complete && pic->ref_count[0] == 0) + hw_base_encode_remove_refs(avctx, pic, 1); + } + + // Clear out all complete pictures with no remaining references. + prev = NULL; + for (pic = ctx->pic_start; pic; pic = next) { + next = pic->next; + if (pic->encode_complete && pic->ref_count[1] == 0) { + av_assert0(pic->ref_removed[0] && pic->ref_removed[1]); + if (prev) + prev->next = next; + else + ctx->pic_start = next; + ctx->op->free(avctx, pic); + } else { + prev = pic; + } + } + + return 0; +} + +static int hw_base_encode_check_frame(AVCodecContext *avctx, + const AVFrame *frame) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + if ((frame->crop_top || frame->crop_bottom || + frame->crop_left || frame->crop_right) && !ctx->crop_warned) { + av_log(avctx, AV_LOG_WARNING, "Cropping information on input " + "frames ignored due to lack of API support.\n"); + ctx->crop_warned = 1; + } + + if (!ctx->roi_allowed) { + AVFrameSideData *sd = + av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); + + if (sd && !ctx->roi_warned) { + av_log(avctx, AV_LOG_WARNING, "ROI side data on input " + "frames ignored due to lack of driver support.\n"); + ctx->roi_warned = 1; + } + } + + return 0; +} + +static int hw_base_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic; + int err; + + if (frame) { + av_log(avctx, AV_LOG_DEBUG, "Input frame: %ux%u (%"PRId64").\n", + frame->width, frame->height, frame->pts); + + err = hw_base_encode_check_frame(avctx, frame); + if (err < 0) + return err; + + pic = ctx->op->alloc(avctx, frame); + if (!pic) + return AVERROR(ENOMEM); + + pic->input_image = av_frame_alloc(); + if (!pic->input_image) { + err = AVERROR(ENOMEM); + goto fail; + } + + pic->recon_image = av_frame_alloc(); + if (!pic->recon_image) { + err = AVERROR(ENOMEM); + goto fail; + } + + if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I) + pic->force_idr = 1; + + pic->pts = frame->pts; + pic->duration = frame->duration; + + if (avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { + err = av_buffer_replace(&pic->opaque_ref, frame->opaque_ref); + if (err < 0) + goto fail; + + pic->opaque = frame->opaque; + } + + av_frame_move_ref(pic->input_image, frame); + + if (ctx->input_order == 0) + ctx->first_pts = pic->pts; + if (ctx->input_order == ctx->decode_delay) + ctx->dts_pts_diff = pic->pts - ctx->first_pts; + if (ctx->output_delay > 0) + ctx->ts_ring[ctx->input_order % + (3 * ctx->output_delay + ctx->async_depth)] = pic->pts; + + pic->display_order = ctx->input_order; + ++ctx->input_order; + + if (ctx->pic_start) { + ctx->pic_end->next = pic; + ctx->pic_end = pic; + } else { + ctx->pic_start = pic; + ctx->pic_end = pic; + } + + } else { + ctx->end_of_stream = 1; + + // Fix timestamps if we hit end-of-stream before the initial decode + // delay has elapsed. + if (ctx->input_order < ctx->decode_delay) + ctx->dts_pts_diff = ctx->pic_end->pts - ctx->first_pts; + } + + return 0; + +fail: + ctx->op->free(avctx, pic); + return err; +} + +int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic = NULL; + AVFrame *frame = ctx->frame; + int err; + + av_assert0(ctx->op && ctx->op->alloc && ctx->op->issue && + ctx->op->output && ctx->op->free); + +start: + /** if no B frame before repeat P frame, sent repeat P frame out. */ + if (ctx->tail_pkt->size) { + for (HWBaseEncodePicture *tmp = ctx->pic_start; tmp; tmp = tmp->next) { + if (tmp->type == PICTURE_TYPE_B && tmp->pts < ctx->tail_pkt->pts) + break; + else if (!tmp->next) { + av_packet_move_ref(pkt, ctx->tail_pkt); + goto end; + } + } + } + + err = ff_encode_get_frame(avctx, frame); + if (err < 0 && err != AVERROR_EOF) + return err; + + if (err == AVERROR_EOF) + frame = NULL; + + err = hw_base_encode_send_frame(avctx, frame); + if (err < 0) + return err; + + if (!ctx->pic_start) { + if (ctx->end_of_stream) + return AVERROR_EOF; + else + return AVERROR(EAGAIN); + } + + if (ctx->async_encode) { + if (av_fifo_can_write(ctx->encode_fifo)) { + err = hw_base_encode_pick_next(avctx, &pic); + if (!err) { + av_assert0(pic); + pic->encode_order = ctx->encode_order + + av_fifo_can_read(ctx->encode_fifo); + err = ctx->op->issue(avctx, pic); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); + return err; + } + pic->encode_issued = 1; + av_fifo_write(ctx->encode_fifo, &pic, 1); + } + } + + if (!av_fifo_can_read(ctx->encode_fifo)) + return err; + + // More frames can be buffered + if (av_fifo_can_write(ctx->encode_fifo) && !ctx->end_of_stream) + return AVERROR(EAGAIN); + + av_fifo_read(ctx->encode_fifo, &pic, 1); + ctx->encode_order = pic->encode_order + 1; + } else { + err = hw_base_encode_pick_next(avctx, &pic); + if (err < 0) + return err; + av_assert0(pic); + + pic->encode_order = ctx->encode_order++; + + err = ctx->op->issue(avctx, pic); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); + return err; + } + + pic->encode_issued = 1; + } + + err = ctx->op->output(avctx, pic, pkt); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Output failed: %d.\n", err); + return err; + } + + ctx->output_order = pic->encode_order; + hw_base_encode_clear_old(avctx); + + /** loop to get an available pkt in encoder flushing. */ + if (ctx->end_of_stream && !pkt->size) + goto start; + +end: + if (pkt->size) + av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " + "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); + + return 0; +} diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index a578db8c06..2667ae61cd 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -19,6 +19,8 @@ #ifndef AVCODEC_HW_BASE_ENCODE_H #define AVCODEC_HW_BASE_ENCODE_H +#include "libavutil/fifo.h" + #define MAX_DPB_SIZE 16 #define MAX_PICTURE_REFERENCES 2 #define MAX_REORDER_DELAY 16 @@ -53,13 +55,135 @@ enum { FLAG_NON_IDR_KEY_PICTURES = 1 << 5, }; +typedef struct HWBaseEncodePicture { + struct HWBaseEncodePicture *next; + + int64_t display_order; + int64_t encode_order; + int64_t pts; + int64_t duration; + int force_idr; + + void *opaque; + AVBufferRef *opaque_ref; + + int type; + int b_depth; + int encode_issued; + int encode_complete; + + AVFrame *input_image; + AVFrame *recon_image; + + void *priv_data; + + // Whether this picture is a reference picture. + int is_reference; + + // The contents of the DPB after this picture has been decoded. + // This will contain the picture itself if it is a reference picture, + // but not if it isn't. + int nb_dpb_pics; + struct HWBaseEncodePicture *dpb[MAX_DPB_SIZE]; + // The reference pictures used in decoding this picture. If they are + // used by later pictures they will also appear in the DPB. ref[0][] for + // previous reference frames. ref[1][] for future reference frames. + int nb_refs[MAX_REFERENCE_LIST_NUM]; + struct HWBaseEncodePicture *refs[MAX_REFERENCE_LIST_NUM][MAX_PICTURE_REFERENCES]; + // The previous reference picture in encode order. Must be in at least + // one of the reference list and DPB list. + struct HWBaseEncodePicture *prev; + // Reference count for other pictures referring to this one through + // the above pointers, directly from incomplete pictures and indirectly + // through completed pictures. + int ref_count[2]; + int ref_removed[2]; +} HWBaseEncodePicture; + +typedef struct HWEncodePictureOperation { + // Alloc memory for the picture structure and initialize the API-specific internals + // based of the given frame. + HWBaseEncodePicture * (*alloc)(AVCodecContext *avctx, const AVFrame *frame); + // Issue the picture structure, which will send the frame surface to HW Encode API. + int (*issue)(AVCodecContext *avctx, const HWBaseEncodePicture *base_pic); + // Get the output AVPacket. + int (*output)(AVCodecContext *avctx, const HWBaseEncodePicture *base_pic, AVPacket *pkt); + // Free the picture structure. + int (*free)(AVCodecContext *avctx, HWBaseEncodePicture *base_pic); +} HWEncodePictureOperation; + typedef struct HWBaseEncodeContext { const AVClass *class; + // Hardware-specific hooks. + const struct HWEncodePictureOperation *op; + + // Current encoding window, in display (input) order. + HWBaseEncodePicture *pic_start, *pic_end; + // The next picture to use as the previous reference picture in + // encoding order. Order from small to large in encoding order. + HWBaseEncodePicture *next_prev[MAX_PICTURE_REFERENCES]; + int nb_next_prev; + + // Next input order index (display order). + int64_t input_order; + // Number of frames that output is behind input. + int64_t output_delay; + // Next encode order index. + int64_t encode_order; + // Number of frames decode output will need to be delayed. + int64_t decode_delay; + // Next output order index (in encode order). + int64_t output_order; + + // Timestamp handling. + int64_t first_pts; + int64_t dts_pts_diff; + int64_t ts_ring[MAX_REORDER_DELAY * 3 + + MAX_ASYNC_DEPTH]; + + // Frame type decision. + int gop_size; + int closed_gop; + int gop_per_idr; + int p_per_i; + int max_b_depth; + int b_per_p; + int force_idr; + int idr_counter; + int gop_counter; + int end_of_stream; + int p_to_gpb; + + // Whether the driver supports ROI at all. + int roi_allowed; + + // The encoder does not support cropping information, so warn about + // it the first time we encounter any nonzero crop fields. + int crop_warned; + // If the driver does not support ROI then warn the first time we + // encounter a frame with ROI side data. + int roi_warned; + + // The frame to be filled with data. + AVFrame *frame; + + // Whether the HW supports sync buffer function. + // If supported, encode_fifo/async_depth will be used together. + // Used for output buffer synchronization. + int async_encode; + + // Store buffered pic. + AVFifo *encode_fifo; // Max number of frame buffered in encoder. int async_depth; + + /** Tail data of a pic, now only used for av1 repeat frame header. */ + AVPacket *tail_pkt; } HWBaseEncodeContext; +int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); + #define HW_BASE_ENCODE_COMMON_OPTIONS \ { "async_depth", "Maximum processing parallelism. " \ "Increase this to improve single channel performance.", \ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 227cccae64..c4bb93c520 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -138,22 +138,26 @@ static int vaapi_encode_make_misc_param_buffer(AVCodecContext *avctx, static int vaapi_encode_wait(AVCodecContext *avctx, VAAPIEncodePicture *pic) { +#if VA_CHECK_VERSION(1, 9, 0) + HWBaseEncodeContext *base_ctx = avctx->priv_data; +#endif VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *base_pic = &pic->base; VAStatus vas; - av_assert0(pic->encode_issued); + av_assert0(base_pic->encode_issued); - if (pic->encode_complete) { + if (base_pic->encode_complete) { // Already waited for this picture. return 0; } av_log(avctx, AV_LOG_DEBUG, "Sync to pic %"PRId64"/%"PRId64" " - "(input surface %#x).\n", pic->display_order, - pic->encode_order, pic->input_surface); + "(input surface %#x).\n", base_pic->display_order, + base_pic->encode_order, pic->input_surface); #if VA_CHECK_VERSION(1, 9, 0) - if (ctx->has_sync_buffer_func) { + if (base_ctx->async_encode) { vas = vaSyncBuffer(ctx->hwctx->display, pic->output_buffer, VA_TIMEOUT_INFINITE); @@ -174,9 +178,9 @@ static int vaapi_encode_wait(AVCodecContext *avctx, } // Input is definitely finished with now. - av_frame_free(&pic->input_image); + av_frame_free(&base_pic->input_image); - pic->encode_complete = 1; + base_pic->encode_complete = 1; return 0; } @@ -263,9 +267,11 @@ static int vaapi_encode_make_tile_slice(AVCodecContext *avctx, } static int vaapi_encode_issue(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + const HWBaseEncodePicture *base_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodePicture *pic = (VAAPIEncodePicture*)base_pic; VAAPIEncodeSlice *slice; VAStatus vas; int err, i; @@ -274,52 +280,46 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_unused AVFrameSideData *sd; av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " - "as type %s.\n", pic->display_order, pic->encode_order, - ff_hw_base_encode_get_pictype_name(pic->type)); - if (pic->nb_refs[0] == 0 && pic->nb_refs[1] == 0) { + "as type %s.\n", base_pic->display_order, base_pic->encode_order, + ff_hw_base_encode_get_pictype_name(base_pic->type)); + if (base_pic->nb_refs[0] == 0 && base_pic->nb_refs[1] == 0) { av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); } else { av_log(avctx, AV_LOG_DEBUG, "L0 refers to"); - for (i = 0; i < pic->nb_refs[0]; i++) { + for (i = 0; i < base_pic->nb_refs[0]; i++) { av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, - pic->refs[0][i]->display_order, pic->refs[0][i]->encode_order); + base_pic->refs[0][i]->display_order, base_pic->refs[0][i]->encode_order); } av_log(avctx, AV_LOG_DEBUG, ".\n"); - if (pic->nb_refs[1]) { + if (base_pic->nb_refs[1]) { av_log(avctx, AV_LOG_DEBUG, "L1 refers to"); - for (i = 0; i < pic->nb_refs[1]; i++) { + for (i = 0; i < base_pic->nb_refs[1]; i++) { av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, - pic->refs[1][i]->display_order, pic->refs[1][i]->encode_order); + base_pic->refs[1][i]->display_order, base_pic->refs[1][i]->encode_order); } av_log(avctx, AV_LOG_DEBUG, ".\n"); } } - av_assert0(!pic->encode_issued); - for (i = 0; i < pic->nb_refs[0]; i++) { - av_assert0(pic->refs[0][i]); - av_assert0(pic->refs[0][i]->encode_issued); + av_assert0(!base_pic->encode_issued); + for (i = 0; i < base_pic->nb_refs[0]; i++) { + av_assert0(base_pic->refs[0][i]); + av_assert0(base_pic->refs[0][i]->encode_issued); } - for (i = 0; i < pic->nb_refs[1]; i++) { - av_assert0(pic->refs[1][i]); - av_assert0(pic->refs[1][i]->encode_issued); + for (i = 0; i < base_pic->nb_refs[1]; i++) { + av_assert0(base_pic->refs[1][i]); + av_assert0(base_pic->refs[1][i]->encode_issued); } av_log(avctx, AV_LOG_DEBUG, "Input surface is %#x.\n", pic->input_surface); - pic->recon_image = av_frame_alloc(); - if (!pic->recon_image) { - err = AVERROR(ENOMEM); - goto fail; - } - - err = av_hwframe_get_buffer(ctx->recon_frames_ref, pic->recon_image, 0); + err = av_hwframe_get_buffer(ctx->recon_frames_ref, base_pic->recon_image, 0); if (err < 0) { err = AVERROR(ENOMEM); goto fail; } - pic->recon_surface = (VASurfaceID)(uintptr_t)pic->recon_image->data[3]; + pic->recon_surface = (VASurfaceID)(uintptr_t)base_pic->recon_image->data[3]; av_log(avctx, AV_LOG_DEBUG, "Recon surface is %#x.\n", pic->recon_surface); pic->output_buffer_ref = ff_refstruct_pool_get(ctx->output_buffer_pool); @@ -343,7 +343,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, pic->nb_param_buffers = 0; - if (pic->type == PICTURE_TYPE_IDR && ctx->codec->init_sequence_params) { + if (base_pic->type == PICTURE_TYPE_IDR && ctx->codec->init_sequence_params) { err = vaapi_encode_make_param_buffer(avctx, pic, VAEncSequenceParameterBufferType, ctx->codec_sequence_params, @@ -352,7 +352,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, goto fail; } - if (pic->type == PICTURE_TYPE_IDR) { + if (base_pic->type == PICTURE_TYPE_IDR) { for (i = 0; i < ctx->nb_global_params; i++) { err = vaapi_encode_make_misc_param_buffer(avctx, pic, ctx->global_params_type[i], @@ -389,7 +389,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } #endif - if (pic->type == PICTURE_TYPE_IDR) { + if (base_pic->type == PICTURE_TYPE_IDR) { if (ctx->va_packed_headers & VA_ENC_PACKED_HEADER_SEQUENCE && ctx->codec->write_sequence_header) { bit_len = 8 * sizeof(data); @@ -529,9 +529,9 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } #if VA_CHECK_VERSION(1, 0, 0) - sd = av_frame_get_side_data(pic->input_image, + sd = av_frame_get_side_data(base_pic->input_image, AV_FRAME_DATA_REGIONS_OF_INTEREST); - if (sd && ctx->roi_allowed) { + if (sd && base_ctx->roi_allowed) { const AVRegionOfInterest *roi; uint32_t roi_size; VAEncMiscParameterBufferROI param_roi; @@ -542,11 +542,11 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_assert0(roi_size && sd->size % roi_size == 0); nb_roi = sd->size / roi_size; if (nb_roi > ctx->roi_max_regions) { - if (!ctx->roi_warned) { + if (!base_ctx->roi_warned) { av_log(avctx, AV_LOG_WARNING, "More ROIs set than " "supported by driver (%d > %d).\n", nb_roi, ctx->roi_max_regions); - ctx->roi_warned = 1; + base_ctx->roi_warned = 1; } nb_roi = ctx->roi_max_regions; } @@ -639,8 +639,6 @@ static int vaapi_encode_issue(AVCodecContext *avctx, } } - pic->encode_issued = 1; - return 0; fail_with_picture: @@ -657,14 +655,13 @@ fail_at_end: av_freep(&pic->param_buffers); av_freep(&pic->slices); av_freep(&pic->roi); - av_frame_free(&pic->recon_image); ff_refstruct_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; return err; } static int vaapi_encode_set_output_property(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + HWBaseEncodePicture *pic, AVPacket *pkt) { HWBaseEncodeContext *base_ctx = avctx->priv_data; @@ -689,16 +686,16 @@ static int vaapi_encode_set_output_property(AVCodecContext *avctx, return 0; } - if (ctx->output_delay == 0) { + if (base_ctx->output_delay == 0) { pkt->dts = pkt->pts; - } else if (pic->encode_order < ctx->decode_delay) { - if (ctx->ts_ring[pic->encode_order] < INT64_MIN + ctx->dts_pts_diff) + } else if (pic->encode_order < base_ctx->decode_delay) { + if (base_ctx->ts_ring[pic->encode_order] < INT64_MIN + base_ctx->dts_pts_diff) pkt->dts = INT64_MIN; else - pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; + pkt->dts = base_ctx->ts_ring[pic->encode_order] - base_ctx->dts_pts_diff; } else { - pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % - (3 * ctx->output_delay + base_ctx->async_depth)]; + pkt->dts = base_ctx->ts_ring[(pic->encode_order - base_ctx->decode_delay) % + (3 * base_ctx->output_delay + base_ctx->async_depth)]; } return 0; @@ -817,9 +814,11 @@ end: } static int vaapi_encode_output(AVCodecContext *avctx, - VAAPIEncodePicture *pic, AVPacket *pkt) + const HWBaseEncodePicture *base_pic, AVPacket *pkt) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodePicture *pic = (VAAPIEncodePicture*)base_pic; AVPacket *pkt_ptr = pkt; int err; @@ -832,17 +831,17 @@ static int vaapi_encode_output(AVCodecContext *avctx, ctx->coded_buffer_ref = ff_refstruct_ref(pic->output_buffer_ref); if (pic->tail_size) { - if (ctx->tail_pkt->size) { + if (base_ctx->tail_pkt->size) { err = AVERROR_BUG; goto end; } - err = ff_get_encode_buffer(avctx, ctx->tail_pkt, pic->tail_size, 0); + err = ff_get_encode_buffer(avctx, base_ctx->tail_pkt, pic->tail_size, 0); if (err < 0) goto end; - memcpy(ctx->tail_pkt->data, pic->tail_data, pic->tail_size); - pkt_ptr = ctx->tail_pkt; + memcpy(base_ctx->tail_pkt->data, pic->tail_data, pic->tail_size); + pkt_ptr = base_ctx->tail_pkt; } } else { err = vaapi_encode_get_coded_data(avctx, pic, pkt); @@ -851,9 +850,9 @@ static int vaapi_encode_output(AVCodecContext *avctx, } av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", - pic->display_order, pic->encode_order); + base_pic->display_order, base_pic->encode_order); - vaapi_encode_set_output_property(avctx, pic, pkt_ptr); + vaapi_encode_set_output_property(avctx, (HWBaseEncodePicture*)pic, pkt_ptr); end: ff_refstruct_unref(&pic->output_buffer_ref); @@ -864,12 +863,14 @@ end: static int vaapi_encode_discard(AVCodecContext *avctx, VAAPIEncodePicture *pic) { + HWBaseEncodePicture *base_pic = &pic->base; + vaapi_encode_wait(avctx, pic); if (pic->output_buffer_ref) { av_log(avctx, AV_LOG_DEBUG, "Discard output for pic " "%"PRId64"/%"PRId64".\n", - pic->display_order, pic->encode_order); + base_pic->display_order, base_pic->encode_order); ff_refstruct_unref(&pic->output_buffer_ref); pic->output_buffer = VA_INVALID_ID; @@ -878,8 +879,8 @@ static int vaapi_encode_discard(AVCodecContext *avctx, return 0; } -static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, - const AVFrame *frame) +static HWBaseEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, + const AVFrame *frame) { VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodePicture *pic; @@ -889,8 +890,8 @@ static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, return NULL; if (ctx->codec->picture_priv_data_size > 0) { - pic->priv_data = av_mallocz(ctx->codec->picture_priv_data_size); - if (!pic->priv_data) { + pic->base.priv_data = av_mallocz(ctx->codec->picture_priv_data_size); + if (!pic->base.priv_data) { av_freep(&pic); return NULL; } @@ -900,15 +901,16 @@ static VAAPIEncodePicture *vaapi_encode_alloc(AVCodecContext *avctx, pic->recon_surface = VA_INVALID_ID; pic->output_buffer = VA_INVALID_ID; - return pic; + return &pic->base; } static int vaapi_encode_free(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + HWBaseEncodePicture *base_pic) { + VAAPIEncodePicture *pic = (VAAPIEncodePicture*)base_pic; int i; - if (pic->encode_issued) + if (base_pic->encode_issued) vaapi_encode_discard(avctx, pic); if (pic->slices) { @@ -916,17 +918,17 @@ static int vaapi_encode_free(AVCodecContext *avctx, av_freep(&pic->slices[i].codec_slice_params); } - av_frame_free(&pic->input_image); - av_frame_free(&pic->recon_image); + av_frame_free(&base_pic->input_image); + av_frame_free(&base_pic->recon_image); - av_buffer_unref(&pic->opaque_ref); + av_buffer_unref(&base_pic->opaque_ref); av_freep(&pic->param_buffers); av_freep(&pic->slices); // Output buffer should already be destroyed. av_assert0(pic->output_buffer == VA_INVALID_ID); - av_freep(&pic->priv_data); + av_freep(&base_pic->priv_data); av_freep(&pic->codec_picture_params); av_freep(&pic->roi); @@ -935,564 +937,6 @@ static int vaapi_encode_free(AVCodecContext *avctx, return 0; } -static void vaapi_encode_add_ref(AVCodecContext *avctx, - VAAPIEncodePicture *pic, - VAAPIEncodePicture *target, - int is_ref, int in_dpb, int prev) -{ - int refs = 0; - - if (is_ref) { - av_assert0(pic != target); - av_assert0(pic->nb_refs[0] < MAX_PICTURE_REFERENCES && - pic->nb_refs[1] < MAX_PICTURE_REFERENCES); - if (target->display_order < pic->display_order) - pic->refs[0][pic->nb_refs[0]++] = target; - else - pic->refs[1][pic->nb_refs[1]++] = target; - ++refs; - } - - if (in_dpb) { - av_assert0(pic->nb_dpb_pics < MAX_DPB_SIZE); - pic->dpb[pic->nb_dpb_pics++] = target; - ++refs; - } - - if (prev) { - av_assert0(!pic->prev); - pic->prev = target; - ++refs; - } - - target->ref_count[0] += refs; - target->ref_count[1] += refs; -} - -static void vaapi_encode_remove_refs(AVCodecContext *avctx, - VAAPIEncodePicture *pic, - int level) -{ - int i; - - if (pic->ref_removed[level]) - return; - - for (i = 0; i < pic->nb_refs[0]; i++) { - av_assert0(pic->refs[0][i]); - --pic->refs[0][i]->ref_count[level]; - av_assert0(pic->refs[0][i]->ref_count[level] >= 0); - } - - for (i = 0; i < pic->nb_refs[1]; i++) { - av_assert0(pic->refs[1][i]); - --pic->refs[1][i]->ref_count[level]; - av_assert0(pic->refs[1][i]->ref_count[level] >= 0); - } - - for (i = 0; i < pic->nb_dpb_pics; i++) { - av_assert0(pic->dpb[i]); - --pic->dpb[i]->ref_count[level]; - av_assert0(pic->dpb[i]->ref_count[level] >= 0); - } - - av_assert0(pic->prev || pic->type == PICTURE_TYPE_IDR); - if (pic->prev) { - --pic->prev->ref_count[level]; - av_assert0(pic->prev->ref_count[level] >= 0); - } - - pic->ref_removed[level] = 1; -} - -static void vaapi_encode_set_b_pictures(AVCodecContext *avctx, - VAAPIEncodePicture *start, - VAAPIEncodePicture *end, - VAAPIEncodePicture *prev, - int current_depth, - VAAPIEncodePicture **last) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic, *next, *ref; - int i, len; - - av_assert0(start && end && start != end && start->next != end); - - // If we are at the maximum depth then encode all pictures as - // non-referenced B-pictures. Also do this if there is exactly one - // picture left, since there will be nothing to reference it. - if (current_depth == ctx->max_b_depth || start->next->next == end) { - for (pic = start->next; pic; pic = pic->next) { - if (pic == end) - break; - pic->type = PICTURE_TYPE_B; - pic->b_depth = current_depth; - - vaapi_encode_add_ref(avctx, pic, start, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); - - for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) - vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); - } - *last = prev; - - } else { - // Split the current list at the midpoint with a referenced - // B-picture, then descend into each side separately. - len = 0; - for (pic = start->next; pic != end; pic = pic->next) - ++len; - for (pic = start->next, i = 1; 2 * i < len; pic = pic->next, i++); - - pic->type = PICTURE_TYPE_B; - pic->b_depth = current_depth; - - pic->is_reference = 1; - - vaapi_encode_add_ref(avctx, pic, pic, 0, 1, 0); - vaapi_encode_add_ref(avctx, pic, start, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, end, 1, 1, 0); - vaapi_encode_add_ref(avctx, pic, prev, 0, 0, 1); - - for (ref = end->refs[1][0]; ref; ref = ref->refs[1][0]) - vaapi_encode_add_ref(avctx, pic, ref, 0, 1, 0); - - if (i > 1) - vaapi_encode_set_b_pictures(avctx, start, pic, pic, - current_depth + 1, &next); - else - next = pic; - - vaapi_encode_set_b_pictures(avctx, pic, end, next, - current_depth + 1, last); - } -} - -static void vaapi_encode_add_next_prev(AVCodecContext *avctx, - VAAPIEncodePicture *pic) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - int i; - - if (!pic) - return; - - if (pic->type == PICTURE_TYPE_IDR) { - for (i = 0; i < ctx->nb_next_prev; i++) { - --ctx->next_prev[i]->ref_count[0]; - ctx->next_prev[i] = NULL; - } - ctx->next_prev[0] = pic; - ++pic->ref_count[0]; - ctx->nb_next_prev = 1; - - return; - } - - if (ctx->nb_next_prev < MAX_PICTURE_REFERENCES) { - ctx->next_prev[ctx->nb_next_prev++] = pic; - ++pic->ref_count[0]; - } else { - --ctx->next_prev[0]->ref_count[0]; - for (i = 0; i < MAX_PICTURE_REFERENCES - 1; i++) - ctx->next_prev[i] = ctx->next_prev[i + 1]; - ctx->next_prev[i] = pic; - ++pic->ref_count[0]; - } -} - -static int vaapi_encode_pick_next(AVCodecContext *avctx, - VAAPIEncodePicture **pic_out) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic = NULL, *prev = NULL, *next, *start; - int i, b_counter, closed_gop_end; - - // If there are any B-frames already queued, the next one to encode - // is the earliest not-yet-issued frame for which all references are - // available. - for (pic = ctx->pic_start; pic; pic = pic->next) { - if (pic->encode_issued) - continue; - if (pic->type != PICTURE_TYPE_B) - continue; - for (i = 0; i < pic->nb_refs[0]; i++) { - if (!pic->refs[0][i]->encode_issued) - break; - } - if (i != pic->nb_refs[0]) - continue; - - for (i = 0; i < pic->nb_refs[1]; i++) { - if (!pic->refs[1][i]->encode_issued) - break; - } - if (i == pic->nb_refs[1]) - break; - } - - if (pic) { - av_log(avctx, AV_LOG_DEBUG, "Pick B-picture at depth %d to " - "encode next.\n", pic->b_depth); - *pic_out = pic; - return 0; - } - - // Find the B-per-Pth available picture to become the next picture - // on the top layer. - start = NULL; - b_counter = 0; - closed_gop_end = ctx->closed_gop || - ctx->idr_counter == ctx->gop_per_idr; - for (pic = ctx->pic_start; pic; pic = next) { - next = pic->next; - if (pic->encode_issued) { - start = pic; - continue; - } - // If the next available picture is force-IDR, encode it to start - // a new GOP immediately. - if (pic->force_idr) - break; - if (b_counter == ctx->b_per_p) - break; - // If this picture ends a closed GOP or starts a new GOP then it - // needs to be in the top layer. - if (ctx->gop_counter + b_counter + closed_gop_end >= ctx->gop_size) - break; - // If the picture after this one is force-IDR, we need to encode - // this one in the top layer. - if (next && next->force_idr) - break; - ++b_counter; - } - - // At the end of the stream the last picture must be in the top layer. - if (!pic && ctx->end_of_stream) { - --b_counter; - pic = ctx->pic_end; - if (pic->encode_complete) - return AVERROR_EOF; - else if (pic->encode_issued) - return AVERROR(EAGAIN); - } - - if (!pic) { - av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " - "need more input for reference pictures.\n"); - return AVERROR(EAGAIN); - } - if (ctx->input_order <= ctx->decode_delay && !ctx->end_of_stream) { - av_log(avctx, AV_LOG_DEBUG, "Pick nothing to encode next - " - "need more input for timestamps.\n"); - return AVERROR(EAGAIN); - } - - if (pic->force_idr) { - av_log(avctx, AV_LOG_DEBUG, "Pick forced IDR-picture to " - "encode next.\n"); - pic->type = PICTURE_TYPE_IDR; - ctx->idr_counter = 1; - ctx->gop_counter = 1; - - } else if (ctx->gop_counter + b_counter >= ctx->gop_size) { - if (ctx->idr_counter == ctx->gop_per_idr) { - av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP IDR-picture to " - "encode next.\n"); - pic->type = PICTURE_TYPE_IDR; - ctx->idr_counter = 1; - } else { - av_log(avctx, AV_LOG_DEBUG, "Pick new-GOP I-picture to " - "encode next.\n"); - pic->type = PICTURE_TYPE_I; - ++ctx->idr_counter; - } - ctx->gop_counter = 1; - - } else { - if (ctx->gop_counter + b_counter + closed_gop_end == ctx->gop_size) { - av_log(avctx, AV_LOG_DEBUG, "Pick group-end P-picture to " - "encode next.\n"); - } else { - av_log(avctx, AV_LOG_DEBUG, "Pick normal P-picture to " - "encode next.\n"); - } - pic->type = PICTURE_TYPE_P; - av_assert0(start); - ctx->gop_counter += 1 + b_counter; - } - pic->is_reference = 1; - *pic_out = pic; - - vaapi_encode_add_ref(avctx, pic, pic, 0, 1, 0); - if (pic->type != PICTURE_TYPE_IDR) { - // TODO: apply both previous and forward multi reference for all vaapi encoders. - // And L0/L1 reference frame number can be set dynamically through query - // VAConfigAttribEncMaxRefFrames attribute. - if (avctx->codec_id == AV_CODEC_ID_AV1) { - for (i = 0; i < ctx->nb_next_prev; i++) - vaapi_encode_add_ref(avctx, pic, ctx->next_prev[i], - pic->type == PICTURE_TYPE_P, - b_counter > 0, 0); - } else - vaapi_encode_add_ref(avctx, pic, start, - pic->type == PICTURE_TYPE_P, - b_counter > 0, 0); - - vaapi_encode_add_ref(avctx, pic, ctx->next_prev[ctx->nb_next_prev - 1], 0, 0, 1); - } - - if (b_counter > 0) { - vaapi_encode_set_b_pictures(avctx, start, pic, pic, 1, - &prev); - } else { - prev = pic; - } - vaapi_encode_add_next_prev(avctx, prev); - - return 0; -} - -static int vaapi_encode_clear_old(AVCodecContext *avctx) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic, *prev, *next; - - av_assert0(ctx->pic_start); - - // Remove direct references once each picture is complete. - for (pic = ctx->pic_start; pic; pic = pic->next) { - if (pic->encode_complete && pic->next) - vaapi_encode_remove_refs(avctx, pic, 0); - } - - // Remove indirect references once a picture has no direct references. - for (pic = ctx->pic_start; pic; pic = pic->next) { - if (pic->encode_complete && pic->ref_count[0] == 0) - vaapi_encode_remove_refs(avctx, pic, 1); - } - - // Clear out all complete pictures with no remaining references. - prev = NULL; - for (pic = ctx->pic_start; pic; pic = next) { - next = pic->next; - if (pic->encode_complete && pic->ref_count[1] == 0) { - av_assert0(pic->ref_removed[0] && pic->ref_removed[1]); - if (prev) - prev->next = next; - else - ctx->pic_start = next; - vaapi_encode_free(avctx, pic); - } else { - prev = pic; - } - } - - return 0; -} - -static int vaapi_encode_check_frame(AVCodecContext *avctx, - const AVFrame *frame) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - - if ((frame->crop_top || frame->crop_bottom || - frame->crop_left || frame->crop_right) && !ctx->crop_warned) { - av_log(avctx, AV_LOG_WARNING, "Cropping information on input " - "frames ignored due to lack of API support.\n"); - ctx->crop_warned = 1; - } - - if (!ctx->roi_allowed) { - AVFrameSideData *sd = - av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); - - if (sd && !ctx->roi_warned) { - av_log(avctx, AV_LOG_WARNING, "ROI side data on input " - "frames ignored due to lack of driver support.\n"); - ctx->roi_warned = 1; - } - } - - return 0; -} - -static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame) -{ - HWBaseEncodeContext *base_ctx = avctx->priv_data; - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic; - int err; - - if (frame) { - av_log(avctx, AV_LOG_DEBUG, "Input frame: %ux%u (%"PRId64").\n", - frame->width, frame->height, frame->pts); - - err = vaapi_encode_check_frame(avctx, frame); - if (err < 0) - return err; - - pic = vaapi_encode_alloc(avctx, frame); - if (!pic) - return AVERROR(ENOMEM); - - pic->input_image = av_frame_alloc(); - if (!pic->input_image) { - err = AVERROR(ENOMEM); - goto fail; - } - - if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I) - pic->force_idr = 1; - - pic->pts = frame->pts; - pic->duration = frame->duration; - - if (avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { - err = av_buffer_replace(&pic->opaque_ref, frame->opaque_ref); - if (err < 0) - goto fail; - - pic->opaque = frame->opaque; - } - - av_frame_move_ref(pic->input_image, frame); - - if (ctx->input_order == 0) - ctx->first_pts = pic->pts; - if (ctx->input_order == ctx->decode_delay) - ctx->dts_pts_diff = pic->pts - ctx->first_pts; - if (ctx->output_delay > 0) - ctx->ts_ring[ctx->input_order % - (3 * ctx->output_delay + base_ctx->async_depth)] = pic->pts; - - pic->display_order = ctx->input_order; - ++ctx->input_order; - - if (ctx->pic_start) { - ctx->pic_end->next = pic; - ctx->pic_end = pic; - } else { - ctx->pic_start = pic; - ctx->pic_end = pic; - } - - } else { - ctx->end_of_stream = 1; - - // Fix timestamps if we hit end-of-stream before the initial decode - // delay has elapsed. - if (ctx->input_order < ctx->decode_delay) - ctx->dts_pts_diff = ctx->pic_end->pts - ctx->first_pts; - } - - return 0; - -fail: - vaapi_encode_free(avctx, pic); - return err; -} - -int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) -{ - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic = NULL; - AVFrame *frame = ctx->frame; - int err; - -start: - /** if no B frame before repeat P frame, sent repeat P frame out. */ - if (ctx->tail_pkt->size) { - for (VAAPIEncodePicture *tmp = ctx->pic_start; tmp; tmp = tmp->next) { - if (tmp->type == PICTURE_TYPE_B && tmp->pts < ctx->tail_pkt->pts) - break; - else if (!tmp->next) { - av_packet_move_ref(pkt, ctx->tail_pkt); - goto end; - } - } - } - - err = ff_encode_get_frame(avctx, frame); - if (err < 0 && err != AVERROR_EOF) - return err; - - if (err == AVERROR_EOF) - frame = NULL; - - err = vaapi_encode_send_frame(avctx, frame); - if (err < 0) - return err; - - if (!ctx->pic_start) { - if (ctx->end_of_stream) - return AVERROR_EOF; - else - return AVERROR(EAGAIN); - } - - if (ctx->has_sync_buffer_func) { - if (av_fifo_can_write(ctx->encode_fifo)) { - err = vaapi_encode_pick_next(avctx, &pic); - if (!err) { - av_assert0(pic); - pic->encode_order = ctx->encode_order + - av_fifo_can_read(ctx->encode_fifo); - err = vaapi_encode_issue(avctx, pic); - if (err < 0) { - av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); - return err; - } - av_fifo_write(ctx->encode_fifo, &pic, 1); - } - } - - if (!av_fifo_can_read(ctx->encode_fifo)) - return err; - - // More frames can be buffered - if (av_fifo_can_write(ctx->encode_fifo) && !ctx->end_of_stream) - return AVERROR(EAGAIN); - - av_fifo_read(ctx->encode_fifo, &pic, 1); - ctx->encode_order = pic->encode_order + 1; - } else { - err = vaapi_encode_pick_next(avctx, &pic); - if (err < 0) - return err; - av_assert0(pic); - - pic->encode_order = ctx->encode_order++; - - err = vaapi_encode_issue(avctx, pic); - if (err < 0) { - av_log(avctx, AV_LOG_ERROR, "Encode failed: %d.\n", err); - return err; - } - } - - err = vaapi_encode_output(avctx, pic, pkt); - if (err < 0) { - av_log(avctx, AV_LOG_ERROR, "Output failed: %d.\n", err); - return err; - } - - ctx->output_order = pic->encode_order; - vaapi_encode_clear_old(avctx); - - /** loop to get an available pkt in encoder flushing. */ - if (ctx->end_of_stream && !pkt->size) - goto start; - -end: - if (pkt->size) - av_log(avctx, AV_LOG_DEBUG, "Output packet: pts %"PRId64", dts %"PRId64", " - "size %d bytes.\n", pkt->pts, pkt->dts, pkt->size); - - return 0; -} - static av_cold void vaapi_encode_add_global_param(AVCodecContext *avctx, int type, void *buffer, size_t size) { @@ -2188,7 +1632,8 @@ static av_cold int vaapi_encode_init_max_frame_size(AVCodecContext *avctx) static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; VAStatus vas; VAConfigAttrib attr = { VAConfigAttribEncMaxRefFrames }; uint32_t ref_l0, ref_l1; @@ -2211,7 +1656,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) ref_l1 = attr.value >> 16 & 0xffff; } - ctx->p_to_gpb = 0; + base_ctx->p_to_gpb = 0; prediction_pre_only = 0; #if VA_CHECK_VERSION(1, 9, 0) @@ -2247,7 +1692,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) if (attr.value & VA_PREDICTION_DIRECTION_BI_NOT_EMPTY) { if (ref_l0 > 0 && ref_l1 > 0) { - ctx->p_to_gpb = 1; + base_ctx->p_to_gpb = 1; av_log(avctx, AV_LOG_VERBOSE, "Driver does not support P-frames, " "replacing them with B-frames.\n"); } @@ -2259,7 +1704,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) if (ctx->codec->flags & FLAG_INTRA_ONLY || avctx->gop_size <= 1) { av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); - ctx->gop_size = 1; + base_ctx->gop_size = 1; } else if (ref_l0 < 1) { av_log(avctx, AV_LOG_ERROR, "Driver does not support any " "reference frames.\n"); @@ -2267,41 +1712,41 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || ref_l1 < 1 || avctx->max_b_frames < 1 || prediction_pre_only) { - if (ctx->p_to_gpb) + if (base_ctx->p_to_gpb) av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); else av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); - ctx->gop_size = avctx->gop_size; - ctx->p_per_i = INT_MAX; - ctx->b_per_p = 0; + base_ctx->gop_size = avctx->gop_size; + base_ctx->p_per_i = INT_MAX; + base_ctx->b_per_p = 0; } else { - if (ctx->p_to_gpb) + if (base_ctx->p_to_gpb) av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); else av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " "(supported references: %d / %d).\n", ref_l0, ref_l1); - ctx->gop_size = avctx->gop_size; - ctx->p_per_i = INT_MAX; - ctx->b_per_p = avctx->max_b_frames; + base_ctx->gop_size = avctx->gop_size; + base_ctx->p_per_i = INT_MAX; + base_ctx->b_per_p = avctx->max_b_frames; if (ctx->codec->flags & FLAG_B_PICTURE_REFERENCES) { - ctx->max_b_depth = FFMIN(ctx->desired_b_depth, - av_log2(ctx->b_per_p) + 1); + base_ctx->max_b_depth = FFMIN(ctx->desired_b_depth, + av_log2(base_ctx->b_per_p) + 1); } else { - ctx->max_b_depth = 1; + base_ctx->max_b_depth = 1; } } if (ctx->codec->flags & FLAG_NON_IDR_KEY_PICTURES) { - ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); - ctx->gop_per_idr = ctx->idr_interval + 1; + base_ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); + base_ctx->gop_per_idr = ctx->idr_interval + 1; } else { - ctx->closed_gop = 1; - ctx->gop_per_idr = 1; + base_ctx->closed_gop = 1; + base_ctx->gop_per_idr = 1; } return 0; @@ -2614,7 +2059,8 @@ static av_cold int vaapi_encode_init_quality(AVCodecContext *avctx) static av_cold int vaapi_encode_init_roi(AVCodecContext *avctx) { #if VA_CHECK_VERSION(1, 0, 0) - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; VAStatus vas; VAConfigAttrib attr = { VAConfigAttribEncROI }; @@ -2629,14 +2075,14 @@ static av_cold int vaapi_encode_init_roi(AVCodecContext *avctx) } if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { - ctx->roi_allowed = 0; + base_ctx->roi_allowed = 0; } else { VAConfigAttribValEncROI roi = { .value = attr.value, }; ctx->roi_max_regions = roi.bits.num_roi_regions; - ctx->roi_allowed = ctx->roi_max_regions > 0 && + base_ctx->roi_allowed = ctx->roi_max_regions > 0 && (ctx->va_rc_mode == VA_RC_CQP || roi.bits.roi_rc_qp_delta_support); } @@ -2771,6 +2217,16 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) return err; } +static const HWEncodePictureOperation vaapi_op = { + .alloc = &vaapi_encode_alloc, + + .issue = &vaapi_encode_issue, + + .output = &vaapi_encode_output, + + .free = &vaapi_encode_free, +}; + av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *base_ctx = avctx->priv_data; @@ -2782,10 +2238,12 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) ctx->va_config = VA_INVALID_ID; ctx->va_context = VA_INVALID_ID; + base_ctx->op = &vaapi_op; + /* If you add something that can fail above this av_frame_alloc(), * modify ff_vaapi_encode_close() accordingly. */ - ctx->frame = av_frame_alloc(); - if (!ctx->frame) { + base_ctx->frame = av_frame_alloc(); + if (!base_ctx->frame) { return AVERROR(ENOMEM); } @@ -2810,8 +2268,8 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) ctx->device = (AVHWDeviceContext*)ctx->device_ref->data; ctx->hwctx = ctx->device->hwctx; - ctx->tail_pkt = av_packet_alloc(); - if (!ctx->tail_pkt) { + base_ctx->tail_pkt = av_packet_alloc(); + if (!base_ctx->tail_pkt) { err = AVERROR(ENOMEM); goto fail; } @@ -2910,8 +2368,8 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } - ctx->output_delay = ctx->b_per_p; - ctx->decode_delay = ctx->max_b_depth; + base_ctx->output_delay = base_ctx->b_per_p; + base_ctx->decode_delay = base_ctx->max_b_depth; if (ctx->codec->sequence_params_size > 0) { ctx->codec_sequence_params = @@ -2966,11 +2424,11 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) // check vaSyncBuffer function vas = vaSyncBuffer(ctx->hwctx->display, VA_INVALID_ID, 0); if (vas != VA_STATUS_ERROR_UNIMPLEMENTED) { - ctx->has_sync_buffer_func = 1; - ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, - sizeof(VAAPIEncodePicture *), - 0); - if (!ctx->encode_fifo) + base_ctx->async_encode = 1; + base_ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, + sizeof(VAAPIEncodePicture*), + 0); + if (!base_ctx->encode_fifo) return AVERROR(ENOMEM); } #endif @@ -2983,15 +2441,16 @@ fail: av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodePicture *pic, *next; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *next; /* We check ctx->frame to know whether ff_vaapi_encode_init() * has been called and va_config/va_context initialized. */ - if (!ctx->frame) + if (!base_ctx->frame) return 0; - for (pic = ctx->pic_start; pic; pic = next) { + for (pic = base_ctx->pic_start; pic; pic = next) { next = pic->next; vaapi_encode_free(avctx, pic); } @@ -3008,12 +2467,12 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) ctx->va_config = VA_INVALID_ID; } - av_frame_free(&ctx->frame); - av_packet_free(&ctx->tail_pkt); + av_frame_free(&base_ctx->frame); + av_packet_free(&base_ctx->tail_pkt); av_freep(&ctx->codec_sequence_params); av_freep(&ctx->codec_picture_params); - av_fifo_freep2(&ctx->encode_fifo); + av_fifo_freep2(&base_ctx->encode_fifo); av_buffer_unref(&ctx->recon_frames_ref); av_buffer_unref(&ctx->input_frames_ref); diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 02410c72ec..13ccad8e47 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -29,7 +29,6 @@ #include "libavutil/hwcontext.h" #include "libavutil/hwcontext_vaapi.h" -#include "libavutil/fifo.h" #include "avcodec.h" #include "hwconfig.h" @@ -64,16 +63,7 @@ typedef struct VAAPIEncodeSlice { } VAAPIEncodeSlice; typedef struct VAAPIEncodePicture { - struct VAAPIEncodePicture *next; - - int64_t display_order; - int64_t encode_order; - int64_t pts; - int64_t duration; - int force_idr; - - void *opaque; - AVBufferRef *opaque_ref; + HWBaseEncodePicture base; #if VA_CHECK_VERSION(1, 0, 0) // ROI regions. @@ -82,15 +72,7 @@ typedef struct VAAPIEncodePicture { void *roi; #endif - int type; - int b_depth; - int encode_issued; - int encode_complete; - - AVFrame *input_image; VASurfaceID input_surface; - - AVFrame *recon_image; VASurfaceID recon_surface; int nb_param_buffers; @@ -100,31 +82,8 @@ typedef struct VAAPIEncodePicture { VABufferID *output_buffer_ref; VABufferID output_buffer; - void *priv_data; void *codec_picture_params; - // Whether this picture is a reference picture. - int is_reference; - - // The contents of the DPB after this picture has been decoded. - // This will contain the picture itself if it is a reference picture, - // but not if it isn't. - int nb_dpb_pics; - struct VAAPIEncodePicture *dpb[MAX_DPB_SIZE]; - // The reference pictures used in decoding this picture. If they are - // used by later pictures they will also appear in the DPB. ref[0][] for - // previous reference frames. ref[1][] for future reference frames. - int nb_refs[MAX_REFERENCE_LIST_NUM]; - struct VAAPIEncodePicture *refs[MAX_REFERENCE_LIST_NUM][MAX_PICTURE_REFERENCES]; - // The previous reference picture in encode order. Must be in at least - // one of the reference list and DPB list. - struct VAAPIEncodePicture *prev; - // Reference count for other pictures referring to this one through - // the above pointers, directly from incomplete pictures and indirectly - // through completed pictures. - int ref_count[2]; - int ref_removed[2]; - int nb_slices; VAAPIEncodeSlice *slices; @@ -298,30 +257,6 @@ typedef struct VAAPIEncodeContext { // structure (VAEncPictureParameterBuffer*). void *codec_picture_params; - // Current encoding window, in display (input) order. - VAAPIEncodePicture *pic_start, *pic_end; - // The next picture to use as the previous reference picture in - // encoding order. Order from small to large in encoding order. - VAAPIEncodePicture *next_prev[MAX_PICTURE_REFERENCES]; - int nb_next_prev; - - // Next input order index (display order). - int64_t input_order; - // Number of frames that output is behind input. - int64_t output_delay; - // Next encode order index. - int64_t encode_order; - // Number of frames decode output will need to be delayed. - int64_t decode_delay; - // Next output order index (in encode order). - int64_t output_order; - - // Timestamp handling. - int64_t first_pts; - int64_t dts_pts_diff; - int64_t ts_ring[MAX_REORDER_DELAY * 3 + - MAX_ASYNC_DEPTH]; - // Slice structure. int slice_block_rows; int slice_block_cols; @@ -340,41 +275,12 @@ typedef struct VAAPIEncodeContext { // Location of the i-th tile row boundary. int row_bd[MAX_TILE_ROWS + 1]; - // Frame type decision. - int gop_size; - int closed_gop; - int gop_per_idr; - int p_per_i; - int max_b_depth; - int b_per_p; - int force_idr; - int idr_counter; - int gop_counter; - int end_of_stream; - int p_to_gpb; - - // Whether the driver supports ROI at all. - int roi_allowed; // Maximum number of regions supported by the driver. int roi_max_regions; // Quantisation range for offset calculations. Set by codec-specific // code, as it may change based on parameters. int roi_quant_range; - // The encoder does not support cropping information, so warn about - // it the first time we encounter any nonzero crop fields. - int crop_warned; - // If the driver does not support ROI then warn the first time we - // encounter a frame with ROI side data. - int roi_warned; - - AVFrame *frame; - - // Whether the driver support vaSyncBuffer - int has_sync_buffer_func; - // Store buffered pic - AVFifo *encode_fifo; - /** Head data for current output pkt, used only for AV1. */ //void *header_data; //size_t header_data_size; @@ -384,9 +290,6 @@ typedef struct VAAPIEncodeContext { * This is a RefStruct reference. */ VABufferID *coded_buffer_ref; - - /** Tail data of a pic, now only used for av1 repeat frame header. */ - AVPacket *tail_pkt; } VAAPIEncodeContext; typedef struct VAAPIEncodeType { @@ -468,9 +371,6 @@ typedef struct VAAPIEncodeType { char *data, size_t *data_len); } VAAPIEncodeType; - -int ff_vaapi_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); - int ff_vaapi_encode_init(AVCodecContext *avctx); int ff_vaapi_encode_close(AVCodecContext *avctx); diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index a5a475dca6..f12fe54cff 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -360,6 +360,7 @@ static int vaapi_encode_av1_write_sequence_header(AVCodecContext *avctx, static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeAV1Context *priv = avctx->priv_data; AV1RawOBU *sh_obu = &priv->sh; @@ -441,8 +442,8 @@ static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) vseq->seq_level_idx = sh->seq_level_idx[0]; vseq->seq_tier = sh->seq_tier[0]; vseq->order_hint_bits_minus_1 = sh->order_hint_bits_minus_1; - vseq->intra_period = ctx->gop_size; - vseq->ip_period = ctx->b_per_p + 1; + vseq->intra_period = base_ctx->gop_size; + vseq->ip_period = base_ctx->b_per_p + 1; vseq->seq_fields.bits.enable_order_hint = sh->enable_order_hint; @@ -465,16 +466,17 @@ end: } static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeAV1Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeAV1Picture *hpic = pic->priv_data; AV1RawOBU *fh_obu = &priv->fh; AV1RawFrameHeader *fh = &fh_obu->obu.frame.header; - VAEncPictureParameterBufferAV1 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferAV1 *vpic = vaapi_pic->codec_picture_params; CodedBitstreamFragment *obu = &priv->current_obu; - VAAPIEncodePicture *ref; + HWBaseEncodePicture *ref; VAAPIEncodeAV1Picture *href; int slot, i; int ret; @@ -482,8 +484,8 @@ static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, { 1, 0, 0, 0, -1, 0, -1, -1 }; memset(fh_obu, 0, sizeof(*fh_obu)); - pic->nb_slices = priv->tile_groups; - pic->non_independent_frame = pic->encode_order < pic->display_order; + vaapi_pic->nb_slices = priv->tile_groups; + vaapi_pic->non_independent_frame = pic->encode_order < pic->display_order; fh_obu->header.obu_type = AV1_OBU_FRAME_HEADER; fh_obu->header.obu_has_size_field = 1; @@ -601,8 +603,8 @@ static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, vpic->frame_width_minus_1 = fh->frame_width_minus_1; vpic->frame_height_minus_1 = fh->frame_height_minus_1; vpic->primary_ref_frame = fh->primary_ref_frame; - vpic->reconstructed_frame = pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->reconstructed_frame = vaapi_pic->recon_surface; + vpic->coded_buf = vaapi_pic->output_buffer; vpic->tile_cols = fh->tile_cols; vpic->tile_rows = fh->tile_rows; vpic->order_hint = fh->order_hint; @@ -630,12 +632,12 @@ static int vaapi_encode_av1_init_picture_params(AVCodecContext *avctx, for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { for (int j = 0; j < pic->nb_refs[i]; j++) { - VAAPIEncodePicture *ref_pic = pic->refs[i][j]; + HWBaseEncodePicture *ref_pic = pic->refs[i][j]; slot = ((VAAPIEncodeAV1Picture*)ref_pic->priv_data)->slot; av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); - vpic->reference_frames[slot] = ref_pic->recon_surface; + vpic->reference_frames[slot] = ((VAAPIEncodePicture *)ref_pic)->recon_surface; } } @@ -752,7 +754,7 @@ static int vaapi_encode_av1_init_slice_params(AVCodecContext *avctx, } static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, char *data, size_t *data_len) { VAAPIEncodeAV1Context *priv = avctx->priv_data; @@ -760,10 +762,11 @@ static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, CodedBitstreamAV1Context *cbctx = priv->cbc->priv_data; AV1RawOBU *fh_obu = &priv->fh; AV1RawFrameHeader *rep_fh = &fh_obu->obu.frame_header; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeAV1Picture *href; int ret = 0; - pic->tail_size = 0; + vaapi_pic->tail_size = 0; /** Pack repeat frame header. */ if (pic->display_order > pic->encode_order) { memset(fh_obu, 0, sizeof(*fh_obu)); @@ -785,11 +788,11 @@ static int vaapi_encode_av1_write_picture_header(AVCodecContext *avctx, if (ret < 0) goto end; - ret = vaapi_encode_av1_write_obu(avctx, pic->tail_data, &pic->tail_size, obu); + ret = vaapi_encode_av1_write_obu(avctx, vaapi_pic->tail_data, &vaapi_pic->tail_size, obu); if (ret < 0) goto end; - pic->tail_size /= 8; + vaapi_pic->tail_size /= 8; } memcpy(data, &priv->fh_data, MAX_PARAM_BUFFER_SIZE * sizeof(char)); @@ -1038,7 +1041,7 @@ const FFCodec ff_av1_vaapi_encoder = { .p.id = AV_CODEC_ID_AV1, .priv_data_size = sizeof(VAAPIEncodeAV1Context), .init = &vaapi_encode_av1_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_av1_close, .p.priv_class = &vaapi_encode_av1_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index ebb1760cd3..9182483762 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -233,7 +233,7 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, goto fail; } if (priv->sei_needed & SEI_TIMING) { - if (pic->type == PICTURE_TYPE_IDR) { + if (pic->base.type == PICTURE_TYPE_IDR) { err = ff_cbs_sei_add_message(priv->cbc, au, 1, SEI_TYPE_BUFFERING_PERIOD, &priv->sei_buffering_period, NULL); @@ -295,6 +295,7 @@ fail: static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH264Context *priv = avctx->priv_data; H264RawSPS *sps = &priv->raw_sps; @@ -326,18 +327,18 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) sps->constraint_set1_flag = 1; if (avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) - sps->constraint_set3_flag = ctx->gop_size == 1; + sps->constraint_set3_flag = base_ctx->gop_size == 1; if (avctx->profile == AV_PROFILE_H264_MAIN || avctx->profile == AV_PROFILE_H264_HIGH || avctx->profile == AV_PROFILE_H264_HIGH_10) { sps->constraint_set4_flag = 1; - sps->constraint_set5_flag = ctx->b_per_p == 0; + sps->constraint_set5_flag = base_ctx->b_per_p == 0; } - if (ctx->gop_size == 1) + if (base_ctx->gop_size == 1) priv->dpb_frames = 0; else - priv->dpb_frames = 1 + ctx->max_b_depth; + priv->dpb_frames = 1 + base_ctx->max_b_depth; if (avctx->level != AV_LEVEL_UNKNOWN) { sps->level_idc = avctx->level; @@ -374,7 +375,7 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) sps->bit_depth_chroma_minus8 = bit_depth - 8; sps->log2_max_frame_num_minus4 = 4; - sps->pic_order_cnt_type = ctx->max_b_depth ? 0 : 2; + sps->pic_order_cnt_type = base_ctx->max_b_depth ? 0 : 2; if (sps->pic_order_cnt_type == 0) { sps->log2_max_pic_order_cnt_lsb_minus4 = 4; } @@ -501,8 +502,8 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) sps->vui.motion_vectors_over_pic_boundaries_flag = 1; sps->vui.log2_max_mv_length_horizontal = 15; sps->vui.log2_max_mv_length_vertical = 15; - sps->vui.max_num_reorder_frames = ctx->max_b_depth; - sps->vui.max_dec_frame_buffering = ctx->max_b_depth + 1; + sps->vui.max_num_reorder_frames = base_ctx->max_b_depth; + sps->vui.max_dec_frame_buffering = base_ctx->max_b_depth + 1; pps->nal_unit_header.nal_ref_idc = 3; pps->nal_unit_header.nal_unit_type = H264_NAL_PPS; @@ -535,9 +536,9 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) *vseq = (VAEncSequenceParameterBufferH264) { .seq_parameter_set_id = sps->seq_parameter_set_id, .level_idc = sps->level_idc, - .intra_period = ctx->gop_size, - .intra_idr_period = ctx->gop_size, - .ip_period = ctx->b_per_p + 1, + .intra_period = base_ctx->gop_size, + .intra_idr_period = base_ctx->gop_size, + .ip_period = base_ctx->b_per_p + 1, .bits_per_second = ctx->va_bit_rate, .max_num_ref_frames = sps->max_num_ref_frames, @@ -619,14 +620,15 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeH264Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH264Picture *hpic = pic->priv_data; - VAAPIEncodePicture *prev = pic->prev; + HWBaseEncodePicture *prev = pic->prev; VAAPIEncodeH264Picture *hprev = prev ? prev->priv_data : NULL; - VAEncPictureParameterBufferH264 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferH264 *vpic = vaapi_pic->codec_picture_params; int i, j = 0; if (pic->type == PICTURE_TYPE_IDR) { @@ -662,7 +664,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, hpic->pic_order_cnt *= 2; } - hpic->dpb_delay = pic->display_order - pic->encode_order + ctx->max_b_depth; + hpic->dpb_delay = pic->display_order - pic->encode_order + base_ctx->max_b_depth; hpic->cpb_delay = pic->encode_order - hpic->last_idr_frame; if (priv->aud) { @@ -699,7 +701,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_recovery_point = (H264RawSEIRecoveryPoint) { .recovery_frame_cnt = 0, .exact_match_flag = 1, - .broken_link_flag = ctx->b_per_p > 0, + .broken_link_flag = base_ctx->b_per_p > 0, }; priv->sei_needed |= SEI_RECOVERY_POINT; @@ -722,7 +724,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, } vpic->CurrPic = (VAPictureH264) { - .picture_id = pic->recon_surface, + .picture_id = vaapi_pic->recon_surface, .frame_idx = hpic->frame_num, .flags = 0, .TopFieldOrderCnt = hpic->pic_order_cnt, @@ -730,14 +732,14 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, }; for (int k = 0; k < MAX_REFERENCE_LIST_NUM; k++) { for (i = 0; i < pic->nb_refs[k]; i++) { - VAAPIEncodePicture *ref = pic->refs[k][i]; + HWBaseEncodePicture *ref = pic->refs[k][i]; VAAPIEncodeH264Picture *href; av_assert0(ref && ref->encode_order < pic->encode_order); href = ref->priv_data; vpic->ReferenceFrames[j++] = (VAPictureH264) { - .picture_id = ref->recon_surface, + .picture_id = ((VAAPIEncodePicture *)ref)->recon_surface, .frame_idx = href->frame_num, .flags = VA_PICTURE_H264_SHORT_TERM_REFERENCE, .TopFieldOrderCnt = href->pic_order_cnt, @@ -753,7 +755,7 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, }; } - vpic->coded_buf = pic->output_buffer; + vpic->coded_buf = vaapi_pic->output_buffer; vpic->frame_num = hpic->frame_num; @@ -764,12 +766,13 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, } static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, VAAPIEncodePicture **rpl0, VAAPIEncodePicture **rpl1, int *rpl_size) { - VAAPIEncodePicture *prev; + HWBaseEncodePicture *pic = &vaapi_pic->base; + HWBaseEncodePicture *prev; VAAPIEncodeH264Picture *hp, *hn, *hc; int i, j, n = 0; @@ -783,17 +786,17 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, if (pic->type == PICTURE_TYPE_P) { for (j = n; j > 0; j--) { - hc = rpl0[j - 1]->priv_data; + hc = rpl0[j - 1]->base.priv_data; av_assert0(hc->frame_num != hn->frame_num); if (hc->frame_num > hn->frame_num) break; rpl0[j] = rpl0[j - 1]; } - rpl0[j] = prev->dpb[i]; + rpl0[j] = (VAAPIEncodePicture *)prev->dpb[i]; } else if (pic->type == PICTURE_TYPE_B) { for (j = n; j > 0; j--) { - hc = rpl0[j - 1]->priv_data; + hc = rpl0[j - 1]->base.priv_data; av_assert0(hc->pic_order_cnt != hp->pic_order_cnt); if (hc->pic_order_cnt < hp->pic_order_cnt) { if (hn->pic_order_cnt > hp->pic_order_cnt || @@ -805,10 +808,10 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, } rpl0[j] = rpl0[j - 1]; } - rpl0[j] = prev->dpb[i]; + rpl0[j] = (VAAPIEncodePicture *)prev->dpb[i]; for (j = n; j > 0; j--) { - hc = rpl1[j - 1]->priv_data; + hc = rpl1[j - 1]->base.priv_data; av_assert0(hc->pic_order_cnt != hp->pic_order_cnt); if (hc->pic_order_cnt > hp->pic_order_cnt) { if (hn->pic_order_cnt < hp->pic_order_cnt || @@ -820,7 +823,7 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, } rpl1[j] = rpl1[j - 1]; } - rpl1[j] = prev->dpb[i]; + rpl1[j] = (VAAPIEncodePicture *)prev->dpb[i]; } ++n; @@ -840,7 +843,7 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Default RefPicList0 for fn=%d/poc=%d:", hp->frame_num, hp->pic_order_cnt); for (i = 0; i < n; i++) { - hn = rpl0[i]->priv_data; + hn = rpl0[i]->base.priv_data; av_log(avctx, AV_LOG_DEBUG, " fn=%d/poc=%d", hn->frame_num, hn->pic_order_cnt); } @@ -850,7 +853,7 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Default RefPicList1 for fn=%d/poc=%d:", hp->frame_num, hp->pic_order_cnt); for (i = 0; i < n; i++) { - hn = rpl1[i]->priv_data; + hn = rpl1[i]->base.priv_data; av_log(avctx, AV_LOG_DEBUG, " fn=%d/poc=%d", hn->frame_num, hn->pic_order_cnt); } @@ -861,16 +864,17 @@ static void vaapi_encode_h264_default_ref_pic_list(AVCodecContext *avctx, } static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, VAAPIEncodeSlice *slice) { VAAPIEncodeH264Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH264Picture *hpic = pic->priv_data; - VAAPIEncodePicture *prev = pic->prev; + HWBaseEncodePicture *prev = pic->prev; H264RawSPS *sps = &priv->raw_sps; H264RawPPS *pps = &priv->raw_pps; H264RawSliceHeader *sh = &priv->raw_slice.header; - VAEncPictureParameterBufferH264 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferH264 *vpic = vaapi_pic->codec_picture_params; VAEncSliceParameterBufferH264 *vslice = slice->codec_slice_params; int i, j; @@ -903,7 +907,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, sh->slice_qp_delta = priv->fixed_qp_idr - (pps->pic_init_qp_minus26 + 26); if (pic->is_reference && pic->type != PICTURE_TYPE_IDR) { - VAAPIEncodePicture *discard_list[MAX_DPB_SIZE]; + HWBaseEncodePicture *discard_list[MAX_DPB_SIZE]; int discard = 0, keep = 0; // Discard everything which is in the DPB of the previous frame but @@ -944,14 +948,14 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, VAAPIEncodeH264Picture *href; int n; - vaapi_encode_h264_default_ref_pic_list(avctx, pic, + vaapi_encode_h264_default_ref_pic_list(avctx, vaapi_pic, def_l0, def_l1, &n); if (pic->type == PICTURE_TYPE_P) { int need_rplm = 0; for (i = 0; i < pic->nb_refs[0]; i++) { av_assert0(pic->refs[0][i]); - if (pic->refs[0][i] != def_l0[i]) + if (pic->refs[0][i] != (HWBaseEncodePicture *)def_l0[i]) need_rplm = 1; } @@ -982,7 +986,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, av_assert0(pic->refs[0][i]); href = pic->refs[0][i]->priv_data; av_assert0(href->pic_order_cnt < hpic->pic_order_cnt); - if (pic->refs[0][i] != def_l0[n0]) + if (pic->refs[0][i] != (HWBaseEncodePicture *)def_l0[n0]) need_rplm_l0 = 1; ++n0; } @@ -991,7 +995,7 @@ static int vaapi_encode_h264_init_slice_params(AVCodecContext *avctx, av_assert0(pic->refs[1][i]); href = pic->refs[1][i]->priv_data; av_assert0(href->pic_order_cnt > hpic->pic_order_cnt); - if (pic->refs[1][i] != def_l1[n1]) + if (pic->refs[1][i] != (HWBaseEncodePicture *)def_l1[n1]) need_rplm_l1 = 1; ++n1; } @@ -1380,7 +1384,7 @@ const FFCodec ff_h264_vaapi_encoder = { .p.id = AV_CODEC_ID_H264, .priv_data_size = sizeof(VAAPIEncodeH264Context), .init = &vaapi_encode_h264_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_h264_close, .p.priv_class = &vaapi_encode_h264_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 77bd5e31af..1f9d9c56a4 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -259,6 +259,7 @@ fail: static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; H265RawVPS *vps = &priv->raw_vps; @@ -340,7 +341,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) ptl->general_max_420chroma_constraint_flag = chroma_format <= 1; ptl->general_max_monochrome_constraint_flag = chroma_format == 0; - ptl->general_intra_constraint_flag = ctx->gop_size == 1; + ptl->general_intra_constraint_flag = base_ctx->gop_size == 1; ptl->general_one_picture_only_constraint_flag = 0; ptl->general_lower_bit_rate_constraint_flag = 1; @@ -353,7 +354,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) level = ff_h265_guess_level(ptl, avctx->bit_rate, ctx->surface_width, ctx->surface_height, ctx->nb_slices, ctx->tile_rows, ctx->tile_cols, - (ctx->b_per_p > 0) + 1); + (base_ctx->b_per_p > 0) + 1); if (level) { av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); ptl->general_level_idc = level->level_idc; @@ -367,8 +368,8 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) } vps->vps_sub_layer_ordering_info_present_flag = 0; - vps->vps_max_dec_pic_buffering_minus1[0] = ctx->max_b_depth + 1; - vps->vps_max_num_reorder_pics[0] = ctx->max_b_depth; + vps->vps_max_dec_pic_buffering_minus1[0] = base_ctx->max_b_depth + 1; + vps->vps_max_num_reorder_pics[0] = base_ctx->max_b_depth; vps->vps_max_latency_increase_plus1[0] = 0; vps->vps_max_layer_id = 0; @@ -642,9 +643,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) .general_level_idc = vps->profile_tier_level.general_level_idc, .general_tier_flag = vps->profile_tier_level.general_tier_flag, - .intra_period = ctx->gop_size, - .intra_idr_period = ctx->gop_size, - .ip_period = ctx->b_per_p + 1, + .intra_period = base_ctx->gop_size, + .intra_idr_period = base_ctx->gop_size, + .ip_period = base_ctx->b_per_p + 1, .bits_per_second = ctx->va_bit_rate, .pic_width_in_luma_samples = sps->pic_width_in_luma_samples, @@ -755,14 +756,15 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; + HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH265Picture *hpic = pic->priv_data; - VAAPIEncodePicture *prev = pic->prev; + HWBaseEncodePicture *prev = pic->prev; VAAPIEncodeH265Picture *hprev = prev ? prev->priv_data : NULL; - VAEncPictureParameterBufferHEVC *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferHEVC *vpic = vaapi_pic->codec_picture_params; int i, j = 0; if (pic->type == PICTURE_TYPE_IDR) { @@ -787,13 +789,13 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, hpic->slice_type = HEVC_SLICE_P; hpic->pic_type = 1; } else { - VAAPIEncodePicture *irap_ref; + HWBaseEncodePicture *irap_ref; av_assert0(pic->refs[0][0] && pic->refs[1][0]); for (irap_ref = pic; irap_ref; irap_ref = irap_ref->refs[1][0]) { if (irap_ref->type == PICTURE_TYPE_I) break; } - if (pic->b_depth == ctx->max_b_depth) { + if (pic->b_depth == base_ctx->max_b_depth) { hpic->slice_nal_unit = irap_ref ? HEVC_NAL_RASL_N : HEVC_NAL_TRAIL_N; } else { @@ -909,21 +911,21 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, } vpic->decoded_curr_pic = (VAPictureHEVC) { - .picture_id = pic->recon_surface, + .picture_id = vaapi_pic->recon_surface, .pic_order_cnt = hpic->pic_order_cnt, .flags = 0, }; for (int k = 0; k < MAX_REFERENCE_LIST_NUM; k++) { for (i = 0; i < pic->nb_refs[k]; i++) { - VAAPIEncodePicture *ref = pic->refs[k][i]; + HWBaseEncodePicture *ref = pic->refs[k][i]; VAAPIEncodeH265Picture *href; av_assert0(ref && ref->encode_order < pic->encode_order); href = ref->priv_data; vpic->reference_frames[j++] = (VAPictureHEVC) { - .picture_id = ref->recon_surface, + .picture_id = ((VAAPIEncodePicture *)ref)->recon_surface, .pic_order_cnt = href->pic_order_cnt, .flags = (ref->display_order < pic->display_order ? VA_PICTURE_HEVC_RPS_ST_CURR_BEFORE : 0) | @@ -940,7 +942,7 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, }; } - vpic->coded_buf = pic->output_buffer; + vpic->coded_buf = vaapi_pic->output_buffer; vpic->nal_unit_type = hpic->slice_nal_unit; @@ -970,16 +972,17 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, } static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic, + VAAPIEncodePicture *vaapi_pic, VAAPIEncodeSlice *slice) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeH265Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeH265Picture *hpic = pic->priv_data; const H265RawSPS *sps = &priv->raw_sps; const H265RawPPS *pps = &priv->raw_pps; H265RawSliceHeader *sh = &priv->raw_slice.header; - VAEncPictureParameterBufferHEVC *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferHEVC *vpic = vaapi_pic->codec_picture_params; VAEncSliceParameterBufferHEVC *vslice = slice->codec_slice_params; int i; @@ -996,7 +999,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, sh->slice_type = hpic->slice_type; - if (sh->slice_type == HEVC_SLICE_P && ctx->p_to_gpb) + if (sh->slice_type == HEVC_SLICE_P && base_ctx->p_to_gpb) sh->slice_type = HEVC_SLICE_B; sh->slice_pic_order_cnt_lsb = hpic->pic_order_cnt & @@ -1140,7 +1143,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, .slice_tc_offset_div2 = sh->slice_tc_offset_div2, .slice_fields.bits = { - .last_slice_of_pic_flag = slice->index == pic->nb_slices - 1, + .last_slice_of_pic_flag = slice->index == vaapi_pic->nb_slices - 1, .dependent_slice_segment_flag = sh->dependent_slice_segment_flag, .colour_plane_id = sh->colour_plane_id, .slice_temporal_mvp_enabled_flag = @@ -1171,7 +1174,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_P || pic->type == PICTURE_TYPE_B); vslice->ref_pic_list0[0] = vpic->reference_frames[0]; - if (ctx->p_to_gpb && pic->type == PICTURE_TYPE_P) + if (base_ctx->p_to_gpb && pic->type == PICTURE_TYPE_P) // Reference for GPB B-frame, L0 == L1 vslice->ref_pic_list1[0] = vpic->reference_frames[0]; } @@ -1181,7 +1184,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, vslice->ref_pic_list1[0] = vpic->reference_frames[1]; } - if (pic->type == PICTURE_TYPE_P && ctx->p_to_gpb) { + if (pic->type == PICTURE_TYPE_P && base_ctx->p_to_gpb) { vslice->slice_type = HEVC_SLICE_B; for (i = 0; i < FF_ARRAY_ELEMS(vslice->ref_pic_list0); i++) { vslice->ref_pic_list1[i].picture_id = vslice->ref_pic_list0[i].picture_id; @@ -1494,7 +1497,7 @@ const FFCodec ff_hevc_vaapi_encoder = { .p.id = AV_CODEC_ID_HEVC, .priv_data_size = sizeof(VAAPIEncodeH265Context), .init = &vaapi_encode_h265_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_h265_close, .p.priv_class = &vaapi_encode_h265_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index fb5c0d34c6..6726006774 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -220,12 +220,13 @@ static int vaapi_encode_mjpeg_write_extra_buffer(AVCodecContext *avctx, } static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { VAAPIEncodeMJPEGContext *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; JPEGRawFrameHeader *fh = &priv->frame_header; JPEGRawScanHeader *sh = &priv->scan.header; - VAEncPictureParameterBufferJPEG *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferJPEG *vpic = vaapi_pic->codec_picture_params; const AVPixFmtDescriptor *desc; const uint8_t components_rgb[3] = { 'R', 'G', 'B' }; const uint8_t components_yuv[3] = { 1, 2, 3 }; @@ -377,8 +378,8 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, *vpic = (VAEncPictureParameterBufferJPEG) { - .reconstructed_picture = pic->recon_surface, - .coded_buf = pic->output_buffer, + .reconstructed_picture = vaapi_pic->recon_surface, + .coded_buf = vaapi_pic->output_buffer, .picture_width = fh->X, .picture_height = fh->Y, @@ -406,7 +407,7 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, vpic->quantiser_table_selector[i] = fh->Tq[i]; } - pic->nb_slices = 1; + vaapi_pic->nb_slices = 1; return 0; } @@ -572,7 +573,7 @@ const FFCodec ff_mjpeg_vaapi_encoder = { .p.id = AV_CODEC_ID_MJPEG, .priv_data_size = sizeof(VAAPIEncodeMJPEGContext), .init = &vaapi_encode_mjpeg_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_mjpeg_close, .p.priv_class = &vaapi_encode_mjpeg_class, .p.capabilities = AV_CODEC_CAP_HARDWARE | AV_CODEC_CAP_DR1 | diff --git a/libavcodec/vaapi_encode_mpeg2.c b/libavcodec/vaapi_encode_mpeg2.c index d0980c52b0..c9d3b60762 100644 --- a/libavcodec/vaapi_encode_mpeg2.c +++ b/libavcodec/vaapi_encode_mpeg2.c @@ -166,6 +166,7 @@ fail: static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAAPIEncodeMPEG2Context *priv = avctx->priv_data; MPEG2RawSequenceHeader *sh = &priv->sequence_header; @@ -281,7 +282,7 @@ static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) se->bit_rate_extension = priv->bit_rate >> 18; se->vbv_buffer_size_extension = priv->vbv_buffer_size >> 10; - se->low_delay = ctx->b_per_p == 0; + se->low_delay = base_ctx->b_per_p == 0; se->frame_rate_extension_n = ext_n; se->frame_rate_extension_d = ext_d; @@ -353,8 +354,8 @@ static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) *vseq = (VAEncSequenceParameterBufferMPEG2) { - .intra_period = ctx->gop_size, - .ip_period = ctx->b_per_p + 1, + .intra_period = base_ctx->gop_size, + .ip_period = base_ctx->b_per_p + 1, .picture_width = avctx->width, .picture_height = avctx->height, @@ -417,12 +418,13 @@ static int vaapi_encode_mpeg2_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { VAAPIEncodeMPEG2Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; MPEG2RawPictureHeader *ph = &priv->picture_header; MPEG2RawPictureCodingExtension *pce = &priv->picture_coding_extension.data.picture_coding; - VAEncPictureParameterBufferMPEG2 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferMPEG2 *vpic = vaapi_pic->codec_picture_params; if (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I) { ph->temporal_reference = 0; @@ -448,8 +450,8 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, pce->f_code[1][1] = 15; } - vpic->reconstructed_picture = pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->reconstructed_picture = vaapi_pic->recon_surface; + vpic->coded_buf = vaapi_pic->output_buffer; switch (pic->type) { case PICTURE_TYPE_IDR: @@ -458,12 +460,12 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, break; case PICTURE_TYPE_P: vpic->picture_type = VAEncPictureTypePredictive; - vpic->forward_reference_picture = pic->refs[0][0]->recon_surface; + vpic->forward_reference_picture = ((VAAPIEncodePicture *)pic->refs[0][0])->recon_surface; break; case PICTURE_TYPE_B: vpic->picture_type = VAEncPictureTypeBidirectional; - vpic->forward_reference_picture = pic->refs[0][0]->recon_surface; - vpic->backward_reference_picture = pic->refs[1][0]->recon_surface; + vpic->forward_reference_picture = ((VAAPIEncodePicture *)pic->refs[0][0])->recon_surface; + vpic->backward_reference_picture = ((VAAPIEncodePicture *)pic->refs[1][0])->recon_surface; break; default: av_assert0(0 && "invalid picture type"); @@ -479,11 +481,12 @@ static int vaapi_encode_mpeg2_init_picture_params(AVCodecContext *avctx, } static int vaapi_encode_mpeg2_init_slice_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic, - VAAPIEncodeSlice *slice) + VAAPIEncodePicture *vaapi_pic, + VAAPIEncodeSlice *slice) { - VAAPIEncodeMPEG2Context *priv = avctx->priv_data; - VAEncSliceParameterBufferMPEG2 *vslice = slice->codec_slice_params; + const HWBaseEncodePicture *pic = &vaapi_pic->base; + VAAPIEncodeMPEG2Context *priv = avctx->priv_data; + VAEncSliceParameterBufferMPEG2 *vslice = slice->codec_slice_params; int qp; vslice->macroblock_address = slice->block_start; @@ -695,7 +698,7 @@ const FFCodec ff_mpeg2_vaapi_encoder = { .p.id = AV_CODEC_ID_MPEG2VIDEO, .priv_data_size = sizeof(VAAPIEncodeMPEG2Context), .init = &vaapi_encode_mpeg2_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &vaapi_encode_mpeg2_close, .p.priv_class = &vaapi_encode_mpeg2_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_vp8.c b/libavcodec/vaapi_encode_vp8.c index 4e284f86e2..c533536ceb 100644 --- a/libavcodec/vaapi_encode_vp8.c +++ b/libavcodec/vaapi_encode_vp8.c @@ -52,6 +52,7 @@ typedef struct VAAPIEncodeVP8Context { static int vaapi_encode_vp8_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAEncSequenceParameterBufferVP8 *vseq = ctx->codec_sequence_params; @@ -66,22 +67,23 @@ static int vaapi_encode_vp8_init_sequence_params(AVCodecContext *avctx) if (!(ctx->va_rc_mode & VA_RC_CQP)) { vseq->bits_per_second = ctx->va_bit_rate; - vseq->intra_period = ctx->gop_size; + vseq->intra_period = base_ctx->gop_size; } return 0; } static int vaapi_encode_vp8_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeVP8Context *priv = avctx->priv_data; - VAEncPictureParameterBufferVP8 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferVP8 *vpic = vaapi_pic->codec_picture_params; int i; - vpic->reconstructed_frame = pic->recon_surface; + vpic->reconstructed_frame = vaapi_pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->coded_buf = vaapi_pic->output_buffer; switch (pic->type) { case PICTURE_TYPE_IDR: @@ -101,7 +103,7 @@ static int vaapi_encode_vp8_init_picture_params(AVCodecContext *avctx, vpic->ref_last_frame = vpic->ref_gf_frame = vpic->ref_arf_frame = - pic->refs[0][0]->recon_surface; + ((VAAPIEncodePicture *)pic->refs[0][0])->recon_surface; break; default: av_assert0(0 && "invalid picture type"); @@ -145,7 +147,7 @@ static int vaapi_encode_vp8_write_quant_table(AVCodecContext *avctx, memset(&quant, 0, sizeof(quant)); - if (pic->type == PICTURE_TYPE_P) + if (pic->base.type == PICTURE_TYPE_P) q = priv->q_index_p; else q = priv->q_index_i; @@ -250,7 +252,7 @@ const FFCodec ff_vp8_vaapi_encoder = { .p.id = AV_CODEC_ID_VP8, .priv_data_size = sizeof(VAAPIEncodeVP8Context), .init = &vaapi_encode_vp8_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &ff_vaapi_encode_close, .p.priv_class = &vaapi_encode_vp8_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 88f951652c..36dafea2c0 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -53,6 +53,7 @@ typedef struct VAAPIEncodeVP9Context { static int vaapi_encode_vp9_init_sequence_params(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAEncSequenceParameterBufferVP9 *vseq = ctx->codec_sequence_params; VAEncPictureParameterBufferVP9 *vpic = ctx->codec_picture_params; @@ -64,7 +65,7 @@ static int vaapi_encode_vp9_init_sequence_params(AVCodecContext *avctx) if (!(ctx->va_rc_mode & VA_RC_CQP)) { vseq->bits_per_second = ctx->va_bit_rate; - vseq->intra_period = ctx->gop_size; + vseq->intra_period = base_ctx->gop_size; } vpic->frame_width_src = avctx->width; @@ -76,17 +77,18 @@ static int vaapi_encode_vp9_init_sequence_params(AVCodecContext *avctx) } static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, - VAAPIEncodePicture *pic) + VAAPIEncodePicture *vaapi_pic) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeVP9Context *priv = avctx->priv_data; + const HWBaseEncodePicture *pic = &vaapi_pic->base; VAAPIEncodeVP9Picture *hpic = pic->priv_data; - VAEncPictureParameterBufferVP9 *vpic = pic->codec_picture_params; + VAEncPictureParameterBufferVP9 *vpic = vaapi_pic->codec_picture_params; int i; int num_tile_columns; - vpic->reconstructed_frame = pic->recon_surface; - vpic->coded_buf = pic->output_buffer; + vpic->reconstructed_frame = vaapi_pic->recon_surface; + vpic->coded_buf = vaapi_pic->output_buffer; // Maximum width of a tile in units of superblocks is MAX_TILE_WIDTH_B64(64) // So the number of tile columns is related to the width of the picture. @@ -107,7 +109,7 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, VAAPIEncodeVP9Picture *href = pic->refs[0][0]->priv_data; av_assert0(href->slot == 0 || href->slot == 1); - if (ctx->max_b_depth > 0) { + if (base_ctx->max_b_depth > 0) { hpic->slot = !href->slot; vpic->refresh_frame_flags = 1 << hpic->slot | 0xfc; } else { @@ -127,7 +129,7 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, av_assert0(href0->slot < pic->b_depth + 1 && href1->slot < pic->b_depth + 1); - if (pic->b_depth == ctx->max_b_depth) { + if (pic->b_depth == base_ctx->max_b_depth) { // Unreferenced frame. vpic->refresh_frame_flags = 0x00; hpic->slot = 8; @@ -159,11 +161,11 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, for (i = 0; i < MAX_REFERENCE_LIST_NUM; i++) { for (int j = 0; j < pic->nb_refs[i]; j++) { - VAAPIEncodePicture *ref_pic = pic->refs[i][j]; + HWBaseEncodePicture *ref_pic = pic->refs[i][j]; int slot; slot = ((VAAPIEncodeVP9Picture*)ref_pic->priv_data)->slot; av_assert0(vpic->reference_frames[slot] == VA_INVALID_SURFACE); - vpic->reference_frames[slot] = ref_pic->recon_surface; + vpic->reference_frames[slot] = ((VAAPIEncodePicture *)ref_pic)->recon_surface; } } @@ -307,7 +309,7 @@ const FFCodec ff_vp9_vaapi_encoder = { .p.id = AV_CODEC_ID_VP9, .priv_data_size = sizeof(VAAPIEncodeVP9Context), .init = &vaapi_encode_vp9_init, - FF_CODEC_RECEIVE_PACKET_CB(&ff_vaapi_encode_receive_packet), + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), .close = &ff_vaapi_encode_close, .p.priv_class = &vaapi_encode_vp9_class, .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | From patchwork Wed May 22 01:26:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49125 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82079pzb; Tue, 21 May 2024 18:29:20 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVGiLDXW6d3eSBCmukwVzilvuLFdm+DykuRNDKHDN4es5NTn/WEhz0CzJXC0Y1fmDLMBgWI5giEm+YdPJCDA4tjV6lLdbmzRPxXPQ== X-Google-Smtp-Source: AGHT+IHuaDq9AGZ9n8Cr514ExQa1lNZh0wbP4wMsFKbSkn9cEiwAzKO3MDjtdQ827bpBdR686g6n X-Received: by 2002:a50:cd19:0:b0:572:a049:c797 with SMTP id 4fb4d7f45d1cf-5783107a1bfmr468250a12.12.1716341359963; Tue, 21 May 2024 18:29:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341359; cv=none; d=google.com; s=arc-20160816; b=JLbAAxWBh50InuQPaLCLHaTx/daQarm+FqHuzwMKw/Zjj7o+yd37kPYq3GonDHZjs/ ITlwWq3wWiUjGLBAEUK5tkS/b1U3EaZHYr+m5y+AK1s3ZIcC8rHnf9YjefhG3WYPSSM9 Efc+2GAIvVDh7YPS9iAxYWcXm/0PNymYzfV6opwnhN7gjbpGQ1iEEeU85KJViblWrJvt l0ve7zMY1W9V6C6aS/cZYH4KTOYw4vMBiX37gzCj7p97ivUq8kJM4DdSFhwL2UyECtTA TVKNVWtMgWwOSUUb5q4ThRMUjj2pwjhuF9taNBk1bFGoU0x4Zt/cjEyE7pJ5W/EzpWyF x14w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=uUhs56DzUW8z9FwL2dpuphlluFhwDv1pdQLIhD4+66Q=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=GBEFiWItk39tafIDwOKirrDNTdmhpQHto5H8nZzqTZcs/bdd+dbP3fDK7474PLSQpS epQOOvNGHldSmzrMWUfwO1j94X1UqCyzu0R+YGGWeC6AiBI9X76n4Nl+4YtpRVc9d4uv rTIJ6KL6tXNr9IeY8dsEbkIf/UwjDTEYjbR5YjDt93UayIQH6qhK2By6eCfx2bdccY8X 6AhupD0w5QNrl2BTNrEMwtBMOhEN9ezUO6djfvn96rNlIHDKGPuZMjq3XJXdR/UpWFpL hyvbAqff/18ErPI//nXb2d9KxB5KO5EEO3HalKbgQLd95I6uIXkOg1yeRSM3ljIeNm9r byDA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=MkDD9g02; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5733c2d48a4si14256271a12.222.2024.05.21.18.29.19; Tue, 21 May 2024 18:29:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=MkDD9g02; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 434F568D3BC; Wed, 22 May 2024 04:28:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A00FA68D41C for ; Wed, 22 May 2024 04:28:11 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341297; x=1747877297; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TInwZ7xeuVyQT0K5b+SylT+s11hTZniuj6SiPJ2pxhc=; b=MkDD9g02mGXOQqIm6gnocrDVin7rGZPzIJ+MLKOT3crf9fWIwu+2Vcnz 5Y6vUIWvBcx/CSQbqEYazsietY2DW9NkxDG/d8WyNtBqRojc0CqZKvMiV 9eEBY2i/HeIkY6oLRoXsE0MKjEdMb4cixze6mmFtIf1/60HJd7Ts8M4ik 2NvQZO7aR/G2O9MXKMwp82V+QptgeM3siDHDADC0bPB+yOm0X88Dt7LeQ eBHxgR1A3VS3rYdfEte5YXU+6OpUfT86DDLf3d68XuklsePEcSm63Sc3S TyypBkL2J9I/1To8v8uIbKYSw+oX8Uw3zPdOA5w23JrLuXpp62qwiy8pf g==; X-CSE-ConnectionGUID: 4FIdtJHxQn+FvqzX6q86aQ== X-CSE-MsgGUID: bOYzXNotRoeFUC1qG63TzQ== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797205" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797205" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:07 -0700 X-CSE-ConnectionGUID: 4Z68QRk6SAqUjQJtUpN97A== X-CSE-MsgGUID: r8+d4hl9RPq57Zgsd5AeWw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537479" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:06 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:19 +0800 Message-ID: <20240522012627.1386-6-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 06/13] avcodec/vaapi_encode: extract the init and close function to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kFnqHScMKLEm From: Tong Wu Related parameters such as device context, frame context are also moved to base layer. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 49 ++++++++++++++++++ libavcodec/hw_base_encode.h | 17 +++++++ libavcodec/vaapi_encode.c | 90 +++++++++++---------------------- libavcodec/vaapi_encode.h | 10 ---- libavcodec/vaapi_encode_av1.c | 2 +- libavcodec/vaapi_encode_h264.c | 2 +- libavcodec/vaapi_encode_h265.c | 2 +- libavcodec/vaapi_encode_mjpeg.c | 6 ++- 8 files changed, 102 insertions(+), 76 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index ec7178d2dc..f2b6ef4a3a 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -595,3 +595,52 @@ end: return 0; } + +int ff_hw_base_encode_init(AVCodecContext *avctx) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + ctx->frame = av_frame_alloc(); + if (!ctx->frame) + return AVERROR(ENOMEM); + + if (!avctx->hw_frames_ctx) { + av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is " + "required to associate the encoding device.\n"); + return AVERROR(EINVAL); + } + + ctx->input_frames_ref = av_buffer_ref(avctx->hw_frames_ctx); + if (!ctx->input_frames_ref) + return AVERROR(ENOMEM); + + ctx->input_frames = (AVHWFramesContext *)ctx->input_frames_ref->data; + + ctx->device_ref = av_buffer_ref(ctx->input_frames->device_ref); + if (!ctx->device_ref) + return AVERROR(ENOMEM); + + ctx->device = (AVHWDeviceContext *)ctx->device_ref->data; + + ctx->tail_pkt = av_packet_alloc(); + if (!ctx->tail_pkt) + return AVERROR(ENOMEM); + + return 0; +} + +int ff_hw_base_encode_close(AVCodecContext *avctx) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + av_fifo_freep2(&ctx->encode_fifo); + + av_frame_free(&ctx->frame); + av_packet_free(&ctx->tail_pkt); + + av_buffer_unref(&ctx->device_ref); + av_buffer_unref(&ctx->input_frames_ref); + av_buffer_unref(&ctx->recon_frames_ref); + + return 0; +} diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 2667ae61cd..7e20f57b9c 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -19,6 +19,7 @@ #ifndef AVCODEC_HW_BASE_ENCODE_H #define AVCODEC_HW_BASE_ENCODE_H +#include "libavutil/hwcontext.h" #include "libavutil/fifo.h" #define MAX_DPB_SIZE 16 @@ -118,6 +119,18 @@ typedef struct HWBaseEncodeContext { // Hardware-specific hooks. const struct HWEncodePictureOperation *op; + // The hardware device context. + AVBufferRef *device_ref; + AVHWDeviceContext *device; + + // The hardware frame context containing the input frames. + AVBufferRef *input_frames_ref; + AVHWFramesContext *input_frames; + + // The hardware frame context containing the reconstructed frames. + AVBufferRef *recon_frames_ref; + AVHWFramesContext *recon_frames; + // Current encoding window, in display (input) order. HWBaseEncodePicture *pic_start, *pic_end; // The next picture to use as the previous reference picture in @@ -184,6 +197,10 @@ typedef struct HWBaseEncodeContext { int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); +int ff_hw_base_encode_init(AVCodecContext *avctx); + +int ff_hw_base_encode_close(AVCodecContext *avctx); + #define HW_BASE_ENCODE_COMMON_OPTIONS \ { "async_depth", "Maximum processing parallelism. " \ "Increase this to improve single channel performance.", \ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index c4bb93c520..c3ab0fc192 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -314,7 +314,7 @@ static int vaapi_encode_issue(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Input surface is %#x.\n", pic->input_surface); - err = av_hwframe_get_buffer(ctx->recon_frames_ref, base_pic->recon_image, 0); + err = av_hwframe_get_buffer(base_ctx->recon_frames_ref, base_pic->recon_image, 0); if (err < 0) { err = AVERROR(ENOMEM); goto fail; @@ -996,9 +996,10 @@ static const VAEntrypoint vaapi_encode_entrypoints_low_power[] = { static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAProfile *va_profiles = NULL; - VAEntrypoint *va_entrypoints = NULL; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAProfile *va_profiles = NULL; + VAEntrypoint *va_entrypoints = NULL; VAStatus vas; const VAEntrypoint *usable_entrypoints; const VAAPIEncodeProfile *profile; @@ -1021,10 +1022,10 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) usable_entrypoints = vaapi_encode_entrypoints_normal; } - desc = av_pix_fmt_desc_get(ctx->input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); if (!desc) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%d).\n", - ctx->input_frames->sw_format); + base_ctx->input_frames->sw_format); return AVERROR(EINVAL); } depth = desc->comp[0].depth; @@ -2131,20 +2132,21 @@ static int vaapi_encode_alloc_output_buffer(FFRefStructOpaque opaque, void *obj) static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; AVVAAPIHWConfig *hwconfig = NULL; AVHWFramesConstraints *constraints = NULL; enum AVPixelFormat recon_format; int err, i; - hwconfig = av_hwdevice_hwconfig_alloc(ctx->device_ref); + hwconfig = av_hwdevice_hwconfig_alloc(base_ctx->device_ref); if (!hwconfig) { err = AVERROR(ENOMEM); goto fail; } hwconfig->config_id = ctx->va_config; - constraints = av_hwdevice_get_hwframe_constraints(ctx->device_ref, + constraints = av_hwdevice_get_hwframe_constraints(base_ctx->device_ref, hwconfig); if (!constraints) { err = AVERROR(ENOMEM); @@ -2157,9 +2159,9 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) recon_format = AV_PIX_FMT_NONE; if (constraints->valid_sw_formats) { for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; i++) { - if (ctx->input_frames->sw_format == + if (base_ctx->input_frames->sw_format == constraints->valid_sw_formats[i]) { - recon_format = ctx->input_frames->sw_format; + recon_format = base_ctx->input_frames->sw_format; break; } } @@ -2170,7 +2172,7 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) } } else { // No idea what to use; copy input format. - recon_format = ctx->input_frames->sw_format; + recon_format = base_ctx->input_frames->sw_format; } av_log(avctx, AV_LOG_DEBUG, "Using %s as format of " "reconstructed frames.\n", av_get_pix_fmt_name(recon_format)); @@ -2191,19 +2193,19 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) av_freep(&hwconfig); av_hwframe_constraints_free(&constraints); - ctx->recon_frames_ref = av_hwframe_ctx_alloc(ctx->device_ref); - if (!ctx->recon_frames_ref) { + base_ctx->recon_frames_ref = av_hwframe_ctx_alloc(base_ctx->device_ref); + if (!base_ctx->recon_frames_ref) { err = AVERROR(ENOMEM); goto fail; } - ctx->recon_frames = (AVHWFramesContext*)ctx->recon_frames_ref->data; + base_ctx->recon_frames = (AVHWFramesContext*)base_ctx->recon_frames_ref->data; - ctx->recon_frames->format = AV_PIX_FMT_VAAPI; - ctx->recon_frames->sw_format = recon_format; - ctx->recon_frames->width = ctx->surface_width; - ctx->recon_frames->height = ctx->surface_height; + base_ctx->recon_frames->format = AV_PIX_FMT_VAAPI; + base_ctx->recon_frames->sw_format = recon_format; + base_ctx->recon_frames->width = ctx->surface_width; + base_ctx->recon_frames->height = ctx->surface_height; - err = av_hwframe_ctx_init(ctx->recon_frames_ref); + err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); if (err < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to initialise reconstructed " "frame context: %d.\n", err); @@ -2235,44 +2237,16 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) VAStatus vas; int err; + err = ff_hw_base_encode_init(avctx); + if (err < 0) + goto fail; + ctx->va_config = VA_INVALID_ID; ctx->va_context = VA_INVALID_ID; base_ctx->op = &vaapi_op; - /* If you add something that can fail above this av_frame_alloc(), - * modify ff_vaapi_encode_close() accordingly. */ - base_ctx->frame = av_frame_alloc(); - if (!base_ctx->frame) { - return AVERROR(ENOMEM); - } - - if (!avctx->hw_frames_ctx) { - av_log(avctx, AV_LOG_ERROR, "A hardware frames reference is " - "required to associate the encoding device.\n"); - return AVERROR(EINVAL); - } - - ctx->input_frames_ref = av_buffer_ref(avctx->hw_frames_ctx); - if (!ctx->input_frames_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - ctx->input_frames = (AVHWFramesContext*)ctx->input_frames_ref->data; - - ctx->device_ref = av_buffer_ref(ctx->input_frames->device_ref); - if (!ctx->device_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - ctx->device = (AVHWDeviceContext*)ctx->device_ref->data; - ctx->hwctx = ctx->device->hwctx; - - base_ctx->tail_pkt = av_packet_alloc(); - if (!base_ctx->tail_pkt) { - err = AVERROR(ENOMEM); - goto fail; - } + ctx->hwctx = base_ctx->device->hwctx; err = vaapi_encode_profile_entrypoint(avctx); if (err < 0) @@ -2339,7 +2313,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) if (err < 0) goto fail; - recon_hwctx = ctx->recon_frames->hwctx; + recon_hwctx = base_ctx->recon_frames->hwctx; vas = vaCreateContext(ctx->hwctx->display, ctx->va_config, ctx->surface_width, ctx->surface_height, VA_PROGRESSIVE, @@ -2467,16 +2441,10 @@ av_cold int ff_vaapi_encode_close(AVCodecContext *avctx) ctx->va_config = VA_INVALID_ID; } - av_frame_free(&base_ctx->frame); - av_packet_free(&base_ctx->tail_pkt); - av_freep(&ctx->codec_sequence_params); av_freep(&ctx->codec_picture_params); - av_fifo_freep2(&base_ctx->encode_fifo); - av_buffer_unref(&ctx->recon_frames_ref); - av_buffer_unref(&ctx->input_frames_ref); - av_buffer_unref(&ctx->device_ref); + ff_hw_base_encode_close(avctx); return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 13ccad8e47..c25eab1cb7 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -219,18 +219,8 @@ typedef struct VAAPIEncodeContext { VAConfigID va_config; VAContextID va_context; - AVBufferRef *device_ref; - AVHWDeviceContext *device; AVVAAPIDeviceContext *hwctx; - // The hardware frame context containing the input frames. - AVBufferRef *input_frames_ref; - AVHWFramesContext *input_frames; - - // The hardware frame context containing the reconstructed frames. - AVBufferRef *recon_frames_ref; - AVHWFramesContext *recon_frames; - // Pool of (reusable) bitstream output buffers. struct FFRefStructPool *output_buffer_pool; diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index f12fe54cff..a15b74f409 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -373,7 +373,7 @@ static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) memset(sh_obu, 0, sizeof(*sh_obu)); sh_obu->header.obu_type = AV1_OBU_SEQUENCE_HEADER; - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); sh->seq_profile = avctx->profile; diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index 9182483762..d41b9649e8 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -308,7 +308,7 @@ static int vaapi_encode_h264_init_sequence_params(AVCodecContext *avctx) memset(sps, 0, sizeof(*sps)); memset(pps, 0, sizeof(*pps)); - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); if (desc->nb_components == 1 || desc->log2_chroma_w != 1 || desc->log2_chroma_h != 1) { av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1f9d9c56a4..ec48773d95 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -278,7 +278,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) memset(pps, 0, sizeof(*pps)); - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); if (desc->nb_components == 1) { chroma_format = 0; diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index 6726006774..24acc88a21 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -222,6 +222,7 @@ static int vaapi_encode_mjpeg_write_extra_buffer(AVCodecContext *avctx, static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, VAAPIEncodePicture *vaapi_pic) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeMJPEGContext *priv = avctx->priv_data; const HWBaseEncodePicture *pic = &vaapi_pic->base; JPEGRawFrameHeader *fh = &priv->frame_header; @@ -235,7 +236,7 @@ static int vaapi_encode_mjpeg_init_picture_params(AVCodecContext *avctx, av_assert0(pic->type == PICTURE_TYPE_IDR); - desc = av_pix_fmt_desc_get(priv->common.input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); if (desc->flags & AV_PIX_FMT_FLAG_RGB) components = components_rgb; @@ -437,10 +438,11 @@ static int vaapi_encode_mjpeg_init_slice_params(AVCodecContext *avctx, static av_cold int vaapi_encode_mjpeg_get_encoder_caps(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; const AVPixFmtDescriptor *desc; - desc = av_pix_fmt_desc_get(ctx->input_frames->sw_format); + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); From patchwork Wed May 22 01:26:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49122 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp81937pzb; Tue, 21 May 2024 18:28:51 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWRAcMa2S+/RSL4E8XIU40Qb4W/cNLy3pjKTTR6KGbhhYtZOSaEC7dZASbELWnlgrBeY0mdkE+C3+JhBsJczNExt79fj3Gx3AcaNg== X-Google-Smtp-Source: AGHT+IGxbhGkWscurrDOPiiUj9pEHTqz1zYPFkCqpTEJK+n/0C+hAUkHiIliz7mKnA45fHhXuamH X-Received: by 2002:ac2:47e2:0:b0:51c:348:3ba9 with SMTP id 2adb3069b0e04-526bee935b3mr249469e87.22.1716341331173; Tue, 21 May 2024 18:28:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341331; cv=none; d=google.com; s=arc-20160816; b=LJU4YiV7WVmF5krdthLiQmJDE2Xq0wWb4jfQhA/PNLDULJPivGdk4VnCIgxtcdf5Am od4Sz9EchgQEcSEk6Pjrg4I4fb9VHdrV5712EkeetsaLRiaaRdAjFBm3azQkjpF89sy9 z//6UPhI+mVfaP6u48ibxXBMYd1yEwf5f5FU15rAQVtkmBu320fbcH5wVQqlkcDJWtlm iL8zqc1U04FIuI+4PW7WTQsOh4mQzTSgditX17sOtDJg9rtqsyAWfv++TrrhKxZrvwyL 3A3GIZUgL2N0uxslg31Gw5UGQ+fswRgLiuHvFBt3dRk8jNfrmFP3x4U0W+sMs7pqQFJ3 AhLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=SLiWgWqqTs0LVUIa/xKlR1BLjfUmEq8sOhkqZaBjFfA=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=mMTKyqe68Fx0d3WH8PzBGYDfgn23nyotmQa2q+0DlLT8q4Q3Ik3hz8aAmhhildQvCF 7COXe6spPKhOIaibaCljhHWC/+2g5NTDHpBoLqetg/jwFencyzfoUstN4hr7lCrxxCLH xy377pO+R3UoF1AUSpFUIdVhVtW/35KJ+LAprP6mw4lb3zqvChX02sVWYN6dR6y1uCMp tV9hVaig9Jevyut3dOiEvmlAAmN9jipBYU2ML4Bfb89sBQllQ3ghpVj+vUmbHm0A7z5K aiRTOCN4gCvEWpJCcqjS2pW229UfNMqcsIC9jk3tlTqQPVB6VOoBJ1gOLQQ70bKpgUIU NwUw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=jXx37lMd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-57828739510si570075a12.92.2024.05.21.18.28.50; Tue, 21 May 2024 18:28:51 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=jXx37lMd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4BCB568D422; Wed, 22 May 2024 04:28:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C464168D288 for ; Wed, 22 May 2024 04:28:14 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341295; x=1747877295; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2HxOvL7oTDqLNpaxVXEDZz1WorudUaVdNSw2U2Jbk8E=; b=jXx37lMdXnHMb80wa91o0DNZ6cVAoI1tLw5F2ILQFf2AgiN8WtsJBuZl utjKSR9wX6rb6TPATMRYDEHi3mzHagemIa7P00YwrbT7QEy8+lbJToZzb omr/wtfDh0ryBbPr2r8lsnubzMy0DVcl/dlvwtaLOoX89PYXzZ3r8rRYf Sn1ck3ps3OLtbm/5Gf85CYH6RfIT0vQI+LCtjhXrOYRKyfRlp7yIVS7HF Gnk5vQxRhLrqA7Nxxl/LcAsRTucUp8MmeAH81htU31q+r6PT8q2FdQ0/p cyR/YWzahwANDeGpAy3Aoc58oWA3bHXZXHQ7BGpcxxzarjfzSivXhvxdA w==; X-CSE-ConnectionGUID: sD99bAM1QGOkyaVN6oLiVQ== X-CSE-MsgGUID: drLcP0bRRsWeMWkb8TOQUg== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797207" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797207" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:08 -0700 X-CSE-ConnectionGUID: Pypbqu0YRv+hAS/I4xHW5Q== X-CSE-MsgGUID: vCwG6esbQL+WkfhfWLydEw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537488" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:07 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:20 +0800 Message-ID: <20240522012627.1386-7-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 07/13] avcodec/vaapi_encode: extract gop configuration and two options to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zIexxqtC1EBb From: Tong Wu idr_interval and desired_b_depth are moved to HW_BASE_ENCODE_COMMON_OPTIONS. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 54 +++++++++++++++++++++++++++++++++++++ libavcodec/hw_base_encode.h | 19 +++++++++++++ libavcodec/vaapi_encode.c | 52 +++-------------------------------- libavcodec/vaapi_encode.h | 16 ----------- 4 files changed, 77 insertions(+), 64 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index f2b6ef4a3a..dfe20c4e67 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -596,6 +596,60 @@ end: return 0; } +int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, + int flags, int prediction_pre_only) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + if (flags & FLAG_INTRA_ONLY || avctx->gop_size <= 1) { + av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); + ctx->gop_size = 1; + } else if (ref_l0 < 1) { + av_log(avctx, AV_LOG_ERROR, "Driver does not support any " + "reference frames.\n"); + return AVERROR(EINVAL); + } else if (!(flags & FLAG_B_PICTURES) || ref_l1 < 1 || + avctx->max_b_frames < 1 || prediction_pre_only) { + if (ctx->p_to_gpb) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " + "(supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); + ctx->gop_size = avctx->gop_size; + ctx->p_per_i = INT_MAX; + ctx->b_per_p = 0; + } else { + if (ctx->p_to_gpb) + av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " + "(supported references: %d / %d).\n", + ref_l0, ref_l1); + else + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " + "(supported references: %d / %d).\n", ref_l0, ref_l1); + ctx->gop_size = avctx->gop_size; + ctx->p_per_i = INT_MAX; + ctx->b_per_p = avctx->max_b_frames; + if (flags & FLAG_B_PICTURE_REFERENCES) { + ctx->max_b_depth = FFMIN(ctx->desired_b_depth, + av_log2(ctx->b_per_p) + 1); + } else { + ctx->max_b_depth = 1; + } + } + + if (flags & FLAG_NON_IDR_KEY_PICTURES) { + ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); + ctx->gop_per_idr = ctx->idr_interval + 1; + } else { + ctx->closed_gop = 1; + ctx->gop_per_idr = 1; + } + + return 0; +} + int ff_hw_base_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 7e20f57b9c..07936bf9bb 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -119,6 +119,14 @@ typedef struct HWBaseEncodeContext { // Hardware-specific hooks. const struct HWEncodePictureOperation *op; + // Global options. + + // Number of I frames between IDR frames. + int idr_interval; + + // Desired B frame reference depth. + int desired_b_depth; + // The hardware device context. AVBufferRef *device_ref; AVHWDeviceContext *device; @@ -197,11 +205,22 @@ typedef struct HWBaseEncodeContext { int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); +int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, + int flags, int prediction_pre_only); + int ff_hw_base_encode_init(AVCodecContext *avctx); int ff_hw_base_encode_close(AVCodecContext *avctx); #define HW_BASE_ENCODE_COMMON_OPTIONS \ + { "idr_interval", \ + "Distance (in I-frames) between key frames", \ + OFFSET(common.base.idr_interval), AV_OPT_TYPE_INT, \ + { .i64 = 0 }, 0, INT_MAX, FLAGS }, \ + { "b_depth", \ + "Maximum B-frame reference depth", \ + OFFSET(common.base.desired_b_depth), AV_OPT_TYPE_INT, \ + { .i64 = 1 }, 1, INT_MAX, FLAGS }, \ { "async_depth", "Maximum processing parallelism. " \ "Increase this to improve single channel performance.", \ OFFSET(common.base.async_depth), AV_OPT_TYPE_INT, \ diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index c3ab0fc192..4db64b686b 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1638,7 +1638,7 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) VAStatus vas; VAConfigAttrib attr = { VAConfigAttribEncMaxRefFrames }; uint32_t ref_l0, ref_l1; - int prediction_pre_only; + int prediction_pre_only, err; vas = vaGetConfigAttributes(ctx->hwctx->display, ctx->va_profile, @@ -1702,53 +1702,9 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) } #endif - if (ctx->codec->flags & FLAG_INTRA_ONLY || - avctx->gop_size <= 1) { - av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); - base_ctx->gop_size = 1; - } else if (ref_l0 < 1) { - av_log(avctx, AV_LOG_ERROR, "Driver does not support any " - "reference frames.\n"); - return AVERROR(EINVAL); - } else if (!(ctx->codec->flags & FLAG_B_PICTURES) || - ref_l1 < 1 || avctx->max_b_frames < 1 || - prediction_pre_only) { - if (base_ctx->p_to_gpb) - av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " - "(supported references: %d / %d).\n", - ref_l0, ref_l1); - else - av_log(avctx, AV_LOG_VERBOSE, "Using intra and P-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); - base_ctx->gop_size = avctx->gop_size; - base_ctx->p_per_i = INT_MAX; - base_ctx->b_per_p = 0; - } else { - if (base_ctx->p_to_gpb) - av_log(avctx, AV_LOG_VERBOSE, "Using intra and B-frames " - "(supported references: %d / %d).\n", - ref_l0, ref_l1); - else - av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " - "(supported references: %d / %d).\n", ref_l0, ref_l1); - base_ctx->gop_size = avctx->gop_size; - base_ctx->p_per_i = INT_MAX; - base_ctx->b_per_p = avctx->max_b_frames; - if (ctx->codec->flags & FLAG_B_PICTURE_REFERENCES) { - base_ctx->max_b_depth = FFMIN(ctx->desired_b_depth, - av_log2(base_ctx->b_per_p) + 1); - } else { - base_ctx->max_b_depth = 1; - } - } - - if (ctx->codec->flags & FLAG_NON_IDR_KEY_PICTURES) { - base_ctx->closed_gop = !!(avctx->flags & AV_CODEC_FLAG_CLOSED_GOP); - base_ctx->gop_per_idr = ctx->idr_interval + 1; - } else { - base_ctx->closed_gop = 1; - base_ctx->gop_per_idr = 1; - } + err = ff_hw_base_init_gop_structure(avctx, ref_l0, ref_l1, ctx->codec->flags, prediction_pre_only); + if (err < 0) + return err; return 0; } diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index c25eab1cb7..76fb645d71 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -151,17 +151,9 @@ typedef struct VAAPIEncodeContext { // Codec-specific hooks. const struct VAAPIEncodeType *codec; - // Global options. - // Use low power encoding mode. int low_power; - // Number of I frames between IDR frames. - int idr_interval; - - // Desired B frame reference depth. - int desired_b_depth; - // Max Frame Size int max_frame_size; @@ -371,14 +363,6 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); "may not support all encoding features)", \ OFFSET(common.low_power), AV_OPT_TYPE_BOOL, \ { .i64 = 0 }, 0, 1, FLAGS }, \ - { "idr_interval", \ - "Distance (in I-frames) between IDR frames", \ - OFFSET(common.idr_interval), AV_OPT_TYPE_INT, \ - { .i64 = 0 }, 0, INT_MAX, FLAGS }, \ - { "b_depth", \ - "Maximum B-frame reference depth", \ - OFFSET(common.desired_b_depth), AV_OPT_TYPE_INT, \ - { .i64 = 1 }, 1, INT_MAX, FLAGS }, \ { "max_frame_size", \ "Maximum frame size (in bytes)",\ OFFSET(common.max_frame_size), AV_OPT_TYPE_INT, \ From patchwork Wed May 22 01:26:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49123 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp81983pzb; Tue, 21 May 2024 18:28:59 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUYbkGDWIORhuMXOjdMAoq/Uo7+h870/kv821SIh4iWlbTX4ykwEHSII2o28l0xq4YyePAwOQp9FiZbzR8FdmaVAMsG9OXiUBNqHA== X-Google-Smtp-Source: AGHT+IGw2O+XALQ8Gi077sENrUKzLk0XFwclfUlcsFn4LgYrEEfG+SD61xOKDQ52TZUKuEEfsbUU X-Received: by 2002:a19:4301:0:b0:523:54a2:3836 with SMTP id 2adb3069b0e04-526bf738461mr224651e87.33.1716341339356; Tue, 21 May 2024 18:28:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341339; cv=none; d=google.com; s=arc-20160816; b=uoBAB0RlQFlmXq9U51KJKxKt1uDafFsfyMoc5gCulYeftcF3mTS+hKq7KExjkSs9Fp ot+Qq2+R6/Zmje5OKmfFxPhw2Q2q/th4dH55/YtR2OLloeIOrhth8DyMMBjvcRL/XfyW mYfdHSUg1HjbJ09uFUY9wtoG2qmqq5vzI5RoEw/bdpfi+vNsJsaQe7LzWUpNWAHSXcgj VuXGtbQotbj58ti8AMrJdQYAhA4ijaiimKRB/EqqFEQ9vwmYiU8SybGP3rb4xAtdoQyQ 9mtWHg2GwVjBt0V/CW8ziyEkhyvlKTt34dd3IZOeHRW7UBZ0LQ7XG8J3zPece9UgE/gm UUSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=1QmX/9xo4/hc8pm9VjpzI+I+H3vm2NTPpDWCyb9mwzA=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=UrrcN3Neb+CCtJ9BTJl1KrhRs4E9S8ZH8CQy0OxinyjjaPyQpLlm4SKBLXnDjj/nBt y/uD9EE5AB7tTprNAm5y80LYIEFZo2H4F+dIxvjDoxcdhOX4gva5K/Hwnz9wwT1uMtZD 1+LNFrfrs4+uhxZBFI6bjL+9JZg1JIHkRl3jDQKAF7vGeLGy7AAZl1RLpvryTgq7rErk pkcq3ctMtOxzuvZJmrwdmIRLCfAWH/ClTQ55njfWOaI1hBGFDYhbVZvm5wah+CrtVLD8 u3cM//czBPs5JXj/W7hLdX6acCOld0dpInBQtHMQT3eYBUGVZHjd13/PGEDpYNGuAC05 +tWQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=nX60xzfa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a2e6518d3si1364133666b.986.2024.05.21.18.28.58; Tue, 21 May 2024 18:28:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=nX60xzfa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2735268D446; Wed, 22 May 2024 04:28:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DAB1E68D41C for ; Wed, 22 May 2024 04:28:15 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341296; x=1747877296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ILKhRkw+7e2FkdCBOJoN+eocfS2wScMEDAqCDKDZeAc=; b=nX60xzfaCO7eEr6uZVSW1/zet/vzNIbkcumf/5s7Bgz19g011Q/2SNAM 08HmfVk9Xwyejh54qmlJjgSVbJnylajDnt3l117E6H/8RpJEka2Aio1wE vGNF0pRk7ObPb3rd/jPLvNkkT/idXcvHXbjr8guCGJgFBeMgdjParwC6i ieGBaFcR4yr30d6UF6/q+n8o/Xfn8+HxafGbtS2i8NlYyZcSBHGBWROPF 81vjdKWrZhZ+DdPmIvLG7tUBqz+OPj5vYfpqI96XKVMeukUY2gy36vN47 DLeUQdaC7vMwVlInNFatpuFmH2SsXCdRnRPYOjtfSnOk953f3ZWfaQm+3 w==; X-CSE-ConnectionGUID: MdHA9kLtTemiTkELxdUbLw== X-CSE-MsgGUID: gUVu3ad2Su6ZlajNDSc10g== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797211" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797211" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:09 -0700 X-CSE-ConnectionGUID: QjdT/VF9RrmOHzey3tthXQ== X-CSE-MsgGUID: AeXHEV6MR+a/4W63X4lRQQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537492" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:08 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:21 +0800 Message-ID: <20240522012627.1386-8-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 08/13] avcodec/vaapi_encode: extract set_output_property to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: g9LXei/0/VtS From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 40 +++++++++++++++++++++++++++++++++ libavcodec/hw_base_encode.h | 3 +++ libavcodec/vaapi_encode.c | 44 ++----------------------------------- 3 files changed, 45 insertions(+), 42 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index dfe20c4e67..df820c2f83 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -491,6 +491,46 @@ fail: return err; } +int ff_hw_base_encode_set_output_property(AVCodecContext *avctx, + HWBaseEncodePicture *pic, + AVPacket *pkt, int flag_no_delay) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + + if (pic->type == PICTURE_TYPE_IDR) + pkt->flags |= AV_PKT_FLAG_KEY; + + pkt->pts = pic->pts; + pkt->duration = pic->duration; + + // for no-delay encoders this is handled in generic codec + if (avctx->codec->capabilities & AV_CODEC_CAP_DELAY && + avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { + pkt->opaque = pic->opaque; + pkt->opaque_ref = pic->opaque_ref; + pic->opaque_ref = NULL; + } + + if (flag_no_delay) { + pkt->dts = pkt->pts; + return 0; + } + + if (ctx->output_delay == 0) { + pkt->dts = pkt->pts; + } else if (pic->encode_order < ctx->decode_delay) { + if (ctx->ts_ring[pic->encode_order] < INT64_MIN + ctx->dts_pts_diff) + pkt->dts = INT64_MIN; + else + pkt->dts = ctx->ts_ring[pic->encode_order] - ctx->dts_pts_diff; + } else { + pkt->dts = ctx->ts_ring[(pic->encode_order - ctx->decode_delay) % + (3 * ctx->output_delay + ctx->async_depth)]; + } + + return 0; +} + int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 07936bf9bb..d363819329 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -203,6 +203,9 @@ typedef struct HWBaseEncodeContext { AVPacket *tail_pkt; } HWBaseEncodeContext; +int ff_hw_base_encode_set_output_property(AVCodecContext *avctx, HWBaseEncodePicture *pic, + AVPacket *pkt, int flag_no_delay); + int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 4db64b686b..6ab1b633ed 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -660,47 +660,6 @@ fail_at_end: return err; } -static int vaapi_encode_set_output_property(AVCodecContext *avctx, - HWBaseEncodePicture *pic, - AVPacket *pkt) -{ - HWBaseEncodeContext *base_ctx = avctx->priv_data; - VAAPIEncodeContext *ctx = avctx->priv_data; - - if (pic->type == PICTURE_TYPE_IDR) - pkt->flags |= AV_PKT_FLAG_KEY; - - pkt->pts = pic->pts; - pkt->duration = pic->duration; - - // for no-delay encoders this is handled in generic codec - if (avctx->codec->capabilities & AV_CODEC_CAP_DELAY && - avctx->flags & AV_CODEC_FLAG_COPY_OPAQUE) { - pkt->opaque = pic->opaque; - pkt->opaque_ref = pic->opaque_ref; - pic->opaque_ref = NULL; - } - - if (ctx->codec->flags & FLAG_TIMESTAMP_NO_DELAY) { - pkt->dts = pkt->pts; - return 0; - } - - if (base_ctx->output_delay == 0) { - pkt->dts = pkt->pts; - } else if (pic->encode_order < base_ctx->decode_delay) { - if (base_ctx->ts_ring[pic->encode_order] < INT64_MIN + base_ctx->dts_pts_diff) - pkt->dts = INT64_MIN; - else - pkt->dts = base_ctx->ts_ring[pic->encode_order] - base_ctx->dts_pts_diff; - } else { - pkt->dts = base_ctx->ts_ring[(pic->encode_order - base_ctx->decode_delay) % - (3 * base_ctx->output_delay + base_ctx->async_depth)]; - } - - return 0; -} - static int vaapi_encode_get_coded_buffer_size(AVCodecContext *avctx, VABufferID buf_id) { VAAPIEncodeContext *ctx = avctx->priv_data; @@ -852,7 +811,8 @@ static int vaapi_encode_output(AVCodecContext *avctx, av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", base_pic->display_order, base_pic->encode_order); - vaapi_encode_set_output_property(avctx, (HWBaseEncodePicture*)pic, pkt_ptr); + ff_hw_base_encode_set_output_property(avctx, (HWBaseEncodePicture*)base_pic, pkt_ptr, + ctx->codec->flags & FLAG_TIMESTAMP_NO_DELAY); end: ff_refstruct_unref(&pic->output_buffer_ref); From patchwork Wed May 22 01:26:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49126 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82145pzb; Tue, 21 May 2024 18:29:29 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXFSZ1RXBph1lqddp2SoHWC4n4NfNMi8AcuvyKMWs77lU6MMRw/RNG+WAJhk9u8Zc32mmXBbqAb5DJBQbtrL/4hwGfFfZA54WiJjQ== X-Google-Smtp-Source: AGHT+IGST+HQhqsjdXKWbBTLh2LPrgP3Wfex17mhpDO5utEgizGiwA7zh3V/YFG/onSgkmboGlXZ X-Received: by 2002:a17:907:7844:b0:a5a:88ac:fbbe with SMTP id a640c23a62f3a-a62280a2be2mr27620766b.46.1716341369456; Tue, 21 May 2024 18:29:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341369; cv=none; d=google.com; s=arc-20160816; b=nrKScsys8M0cVQBM0l772hswo3xrJQqJkrlyw9Og0bYZihetWlsQNoHyeJpVF5bNzj yFelBUMJUcRwPPZJIc1/mrVWCkVnLTVQ2k4CnLoZnckO1Ib4ukvSXbM3s04rdwuatTlS eGke4RaeDdLnlhoXP0QBTeBZYA+sI2ZMP0c3V6NycRdgODNUCu2+S4no+2Ki938NjcBZ k663vdXjFjdCATCI/L1DVUuF6lJHOu+qP1/RdY4VYf9WaaLizb7L7h+U4QMqYCq+9uxT B2ADV95jaNsKvyvfbdZuP3i2wxOP4ZRRmU7s8Q0QkWqjGhl8ca8MM+AxO/AjUMi7gDjK r/gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=MvrM3lM7/mlEwlvPMcGxITsWo5q2/lZ1TEP4NTSdJyI=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=CQ4GD+gh5Mx+sb2Z7RKNmEvVAe9pwEGBU36R8V+70BIPnhncbOquASgsPdYfIt32CS dtKYW15IMznK6j4IBF1y9wlB8rKxxUMZGFtry/4TIV+YCFGUcQs/Lu7KxmsJ+gD3zagD YPb72wMQWloksr2o8OvmkVMBn1EIJy+elKiaIG6l6V+qUwsWBz1HlDhJsCb2RrKOnYHw wXsTKodzMk0YJ0jQqf8mFczRic5gJFxM83kuHc1AbWs7jwV8itp7SESzefYsLLlsllmF jG4l1LBf0pWJO5FSMKwXgwj/Watd4GwzG9kRrHswg526usTO0zn1tmUJfV/8fwhUJHe/ nwqw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=hwnG6tZI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17945ffesi1387881666b.11.2024.05.21.18.29.28; Tue, 21 May 2024 18:29:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=hwnG6tZI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2CD9E68D428; Wed, 22 May 2024 04:28:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EBE8968D288 for ; Wed, 22 May 2024 04:28:16 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341297; x=1747877297; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o6JQCmdbK+Pobn9rQYnsXpsrrgpYHkAr8T3wtER+ckw=; b=hwnG6tZIacP7SVx2gnWp1F4m7UQ0eMB1sKiyR+B3RzZWJTUzwtOXstLh IUGe120cWYQz3fBVmJe7h4md5G82/Ei/20Y0JEdMdafhv1vBuAokTYbZl b/eEpa84eXE831rBXkHWLdnYLc2s1P4gnWirfygF+yosSSxyjn8CwDAtu wAUtbB2yOxyKOyWVGbC68wrmvcpSqnLuvevuepBLV3CrnUX9q4yBh65gq +/lbpoXhcojzhmmXkQK4KEkKqloYmpm3PKUIwUoN3cdaYuQLkAJoDn98/ lYw7PNK2CZLDkdb6gfSZvcq3UPm68r6XrqEu7lHuOOiFbBR5beVmbISBj g==; X-CSE-ConnectionGUID: gdyKwrPuQf2/HMnoyHJGzw== X-CSE-MsgGUID: 2M2X7THRQHaxG+aZEpzIEA== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797213" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797213" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:10 -0700 X-CSE-ConnectionGUID: WqHet3dzS6mn8mtmJVhSxQ== X-CSE-MsgGUID: bIaDHcThS9KWKVHOXr97dA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537499" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:09 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:22 +0800 Message-ID: <20240522012627.1386-9-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 09/13] avcodec/vaapi_encode: extract a get_recon_format function to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4R72J8L/hNBD From: Tong Wu Surface size and block size parameters are also moved to base layer. Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 58 +++++++++++++++++++++++ libavcodec/hw_base_encode.h | 12 +++++ libavcodec/vaapi_encode.c | 81 ++++++++------------------------- libavcodec/vaapi_encode.h | 10 ---- libavcodec/vaapi_encode_av1.c | 10 ++-- libavcodec/vaapi_encode_h264.c | 11 +++-- libavcodec/vaapi_encode_h265.c | 25 +++++----- libavcodec/vaapi_encode_mjpeg.c | 5 +- libavcodec/vaapi_encode_vp9.c | 6 +-- 9 files changed, 118 insertions(+), 100 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index df820c2f83..42c40cb48f 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -690,6 +690,64 @@ int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32 return 0; } +int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void *hwconfig, enum AVPixelFormat *fmt) +{ + HWBaseEncodeContext *ctx = avctx->priv_data; + AVHWFramesConstraints *constraints = NULL; + enum AVPixelFormat recon_format; + int err, i; + + constraints = av_hwdevice_get_hwframe_constraints(ctx->device_ref, + hwconfig); + if (!constraints) { + err = AVERROR(ENOMEM); + goto fail; + } + + // Probably we can use the input surface format as the surface format + // of the reconstructed frames. If not, we just pick the first (only?) + // format in the valid list and hope that it all works. + recon_format = AV_PIX_FMT_NONE; + if (constraints->valid_sw_formats) { + for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; i++) { + if (ctx->input_frames->sw_format == + constraints->valid_sw_formats[i]) { + recon_format = ctx->input_frames->sw_format; + break; + } + } + if (recon_format == AV_PIX_FMT_NONE) { + // No match. Just use the first in the supported list and + // hope for the best. + recon_format = constraints->valid_sw_formats[0]; + } + } else { + // No idea what to use; copy input format. + recon_format = ctx->input_frames->sw_format; + } + av_log(avctx, AV_LOG_DEBUG, "Using %s as format of " + "reconstructed frames.\n", av_get_pix_fmt_name(recon_format)); + + if (ctx->surface_width < constraints->min_width || + ctx->surface_height < constraints->min_height || + ctx->surface_width > constraints->max_width || + ctx->surface_height > constraints->max_height) { + av_log(avctx, AV_LOG_ERROR, "Hardware does not support encoding at " + "size %dx%d (constraints: width %d-%d height %d-%d).\n", + ctx->surface_width, ctx->surface_height, + constraints->min_width, constraints->max_width, + constraints->min_height, constraints->max_height); + err = AVERROR(EINVAL); + goto fail; + } + + *fmt = recon_format; + err = 0; +fail: + av_hwframe_constraints_free(&constraints); + return err; +} + int ff_hw_base_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index d363819329..7d40da039c 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -127,6 +127,16 @@ typedef struct HWBaseEncodeContext { // Desired B frame reference depth. int desired_b_depth; + // The required size of surfaces. This is probably the input + // size (AVCodecContext.width|height) aligned up to whatever + // block size is required by the codec. + int surface_width; + int surface_height; + + // The block size for slice calculations. + int slice_block_width; + int slice_block_height; + // The hardware device context. AVBufferRef *device_ref; AVHWDeviceContext *device; @@ -211,6 +221,8 @@ int ff_hw_base_encode_receive_packet(AVCodecContext *avctx, AVPacket *pkt); int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32_t ref_l1, int flags, int prediction_pre_only); +int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void *hwconfig, enum AVPixelFormat *fmt); + int ff_hw_base_encode_init(AVCodecContext *avctx); int ff_hw_base_encode_close(AVCodecContext *avctx); diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 6ab1b633ed..98b8c82da3 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1777,6 +1777,7 @@ static av_cold int vaapi_encode_init_tile_slice_structure(AVCodecContext *avctx, static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) { + HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; VAConfigAttrib attr[3] = { { VAConfigAttribEncMaxSlices }, { VAConfigAttribEncSliceStructure }, @@ -1796,12 +1797,12 @@ static av_cold int vaapi_encode_init_slice_structure(AVCodecContext *avctx) return 0; } - av_assert0(ctx->slice_block_height > 0 && ctx->slice_block_width > 0); + av_assert0(base_ctx->slice_block_height > 0 && base_ctx->slice_block_width > 0); - ctx->slice_block_rows = (avctx->height + ctx->slice_block_height - 1) / - ctx->slice_block_height; - ctx->slice_block_cols = (avctx->width + ctx->slice_block_width - 1) / - ctx->slice_block_width; + ctx->slice_block_rows = (avctx->height + base_ctx->slice_block_height - 1) / + base_ctx->slice_block_height; + ctx->slice_block_cols = (avctx->width + base_ctx->slice_block_width - 1) / + base_ctx->slice_block_width; if (avctx->slices <= 1 && !ctx->tile_rows && !ctx->tile_cols) { ctx->nb_slices = 1; @@ -2023,7 +2024,8 @@ static void vaapi_encode_free_output_buffer(FFRefStructOpaque opaque, static int vaapi_encode_alloc_output_buffer(FFRefStructOpaque opaque, void *obj) { AVCodecContext *avctx = opaque.nc; - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; VABufferID *buffer_id = obj; VAStatus vas; @@ -2033,7 +2035,7 @@ static int vaapi_encode_alloc_output_buffer(FFRefStructOpaque opaque, void *obj) // bound on that. vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, VAEncCodedBufferType, - 3 * ctx->surface_width * ctx->surface_height + + 3 * base_ctx->surface_width * base_ctx->surface_height + (1 << 16), 1, 0, buffer_id); if (vas != VA_STATUS_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Failed to create bitstream " @@ -2051,9 +2053,8 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) HWBaseEncodeContext *base_ctx = avctx->priv_data; VAAPIEncodeContext *ctx = avctx->priv_data; AVVAAPIHWConfig *hwconfig = NULL; - AVHWFramesConstraints *constraints = NULL; enum AVPixelFormat recon_format; - int err, i; + int err; hwconfig = av_hwdevice_hwconfig_alloc(base_ctx->device_ref); if (!hwconfig) { @@ -2062,52 +2063,9 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) } hwconfig->config_id = ctx->va_config; - constraints = av_hwdevice_get_hwframe_constraints(base_ctx->device_ref, - hwconfig); - if (!constraints) { - err = AVERROR(ENOMEM); - goto fail; - } - - // Probably we can use the input surface format as the surface format - // of the reconstructed frames. If not, we just pick the first (only?) - // format in the valid list and hope that it all works. - recon_format = AV_PIX_FMT_NONE; - if (constraints->valid_sw_formats) { - for (i = 0; constraints->valid_sw_formats[i] != AV_PIX_FMT_NONE; i++) { - if (base_ctx->input_frames->sw_format == - constraints->valid_sw_formats[i]) { - recon_format = base_ctx->input_frames->sw_format; - break; - } - } - if (recon_format == AV_PIX_FMT_NONE) { - // No match. Just use the first in the supported list and - // hope for the best. - recon_format = constraints->valid_sw_formats[0]; - } - } else { - // No idea what to use; copy input format. - recon_format = base_ctx->input_frames->sw_format; - } - av_log(avctx, AV_LOG_DEBUG, "Using %s as format of " - "reconstructed frames.\n", av_get_pix_fmt_name(recon_format)); - - if (ctx->surface_width < constraints->min_width || - ctx->surface_height < constraints->min_height || - ctx->surface_width > constraints->max_width || - ctx->surface_height > constraints->max_height) { - av_log(avctx, AV_LOG_ERROR, "Hardware does not support encoding at " - "size %dx%d (constraints: width %d-%d height %d-%d).\n", - ctx->surface_width, ctx->surface_height, - constraints->min_width, constraints->max_width, - constraints->min_height, constraints->max_height); - err = AVERROR(EINVAL); + err = ff_hw_base_get_recon_format(avctx, (const void*)hwconfig, &recon_format); + if (err < 0) goto fail; - } - - av_freep(&hwconfig); - av_hwframe_constraints_free(&constraints); base_ctx->recon_frames_ref = av_hwframe_ctx_alloc(base_ctx->device_ref); if (!base_ctx->recon_frames_ref) { @@ -2118,8 +2076,8 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) base_ctx->recon_frames->format = AV_PIX_FMT_VAAPI; base_ctx->recon_frames->sw_format = recon_format; - base_ctx->recon_frames->width = ctx->surface_width; - base_ctx->recon_frames->height = ctx->surface_height; + base_ctx->recon_frames->width = base_ctx->surface_width; + base_ctx->recon_frames->height = base_ctx->surface_height; err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); if (err < 0) { @@ -2131,7 +2089,6 @@ static av_cold int vaapi_encode_create_recon_frames(AVCodecContext *avctx) err = 0; fail: av_freep(&hwconfig); - av_hwframe_constraints_free(&constraints); return err; } @@ -2174,11 +2131,11 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) goto fail; } else { // Assume 16x16 blocks. - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); + base_ctx->surface_width = FFALIGN(avctx->width, 16); + base_ctx->surface_height = FFALIGN(avctx->height, 16); if (ctx->codec->flags & FLAG_SLICE_CONTROL) { - ctx->slice_block_width = 16; - ctx->slice_block_height = 16; + base_ctx->slice_block_width = 16; + base_ctx->slice_block_height = 16; } } @@ -2231,7 +2188,7 @@ av_cold int ff_vaapi_encode_init(AVCodecContext *avctx) recon_hwctx = base_ctx->recon_frames->hwctx; vas = vaCreateContext(ctx->hwctx->display, ctx->va_config, - ctx->surface_width, ctx->surface_height, + base_ctx->surface_width, base_ctx->surface_height, VA_PROGRESSIVE, recon_hwctx->surface_ids, recon_hwctx->nb_surfaces, diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index 76fb645d71..8c7568e9ae 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -171,16 +171,6 @@ typedef struct VAAPIEncodeContext { // Desired packed headers. unsigned int desired_packed_headers; - // The required size of surfaces. This is probably the input - // size (AVCodecContext.width|height) aligned up to whatever - // block size is required by the codec. - int surface_width; - int surface_height; - - // The block size for slice calculations. - int slice_block_width; - int slice_block_height; - // Everything above this point must be set before calling // ff_vaapi_encode_init(). diff --git a/libavcodec/vaapi_encode_av1.c b/libavcodec/vaapi_encode_av1.c index a15b74f409..a767d22d37 100644 --- a/libavcodec/vaapi_encode_av1.c +++ b/libavcodec/vaapi_encode_av1.c @@ -112,12 +112,12 @@ static void vaapi_encode_av1_trace_write_log(void *ctx, static av_cold int vaapi_encode_av1_get_encoder_caps(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodeAV1Context *priv = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeAV1Context *priv = avctx->priv_data; // Surfaces must be aligned to superblock boundaries. - ctx->surface_width = FFALIGN(avctx->width, priv->use_128x128_superblock ? 128 : 64); - ctx->surface_height = FFALIGN(avctx->height, priv->use_128x128_superblock ? 128 : 64); + base_ctx->surface_width = FFALIGN(avctx->width, priv->use_128x128_superblock ? 128 : 64); + base_ctx->surface_height = FFALIGN(avctx->height, priv->use_128x128_superblock ? 128 : 64); return 0; } @@ -425,7 +425,7 @@ static int vaapi_encode_av1_init_sequence_params(AVCodecContext *avctx) framerate = 0; level = ff_av1_guess_level(avctx->bit_rate, priv->tier, - ctx->surface_width, ctx->surface_height, + base_ctx->surface_width, base_ctx->surface_height, priv->tile_rows * priv->tile_cols, priv->tile_cols, framerate); if (level) { diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index d41b9649e8..f69bbeebc1 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -1205,8 +1205,9 @@ static const VAAPIEncodeType vaapi_encode_type_h264 = { static av_cold int vaapi_encode_h264_init(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodeH264Context *priv = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeContext *ctx = avctx->priv_data; + VAAPIEncodeH264Context *priv = avctx->priv_data; ctx->codec = &vaapi_encode_type_h264; @@ -1254,10 +1255,10 @@ static av_cold int vaapi_encode_h264_init(AVCodecContext *avctx) VA_ENC_PACKED_HEADER_SLICE | // Slice headers. VA_ENC_PACKED_HEADER_MISC; // SEI. - ctx->surface_width = FFALIGN(avctx->width, 16); - ctx->surface_height = FFALIGN(avctx->height, 16); + base_ctx->surface_width = FFALIGN(avctx->width, 16); + base_ctx->surface_height = FFALIGN(avctx->height, 16); - ctx->slice_block_height = ctx->slice_block_width = 16; + base_ctx->slice_block_height = base_ctx->slice_block_width = 16; if (priv->qp > 0) ctx->explicit_qp = priv->qp; diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index ec48773d95..ff153f778a 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -352,7 +352,7 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) const H265LevelDescriptor *level; level = ff_h265_guess_level(ptl, avctx->bit_rate, - ctx->surface_width, ctx->surface_height, + base_ctx->surface_width, base_ctx->surface_height, ctx->nb_slices, ctx->tile_rows, ctx->tile_cols, (base_ctx->b_per_p > 0) + 1); if (level) { @@ -410,18 +410,18 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) sps->chroma_format_idc = chroma_format; sps->separate_colour_plane_flag = 0; - sps->pic_width_in_luma_samples = ctx->surface_width; - sps->pic_height_in_luma_samples = ctx->surface_height; + sps->pic_width_in_luma_samples = base_ctx->surface_width; + sps->pic_height_in_luma_samples = base_ctx->surface_height; - if (avctx->width != ctx->surface_width || - avctx->height != ctx->surface_height) { + if (avctx->width != base_ctx->surface_width || + avctx->height != base_ctx->surface_height) { sps->conformance_window_flag = 1; sps->conf_win_left_offset = 0; sps->conf_win_right_offset = - (ctx->surface_width - avctx->width) >> desc->log2_chroma_w; + (base_ctx->surface_width - avctx->width) >> desc->log2_chroma_w; sps->conf_win_top_offset = 0; sps->conf_win_bottom_offset = - (ctx->surface_height - avctx->height) >> desc->log2_chroma_h; + (base_ctx->surface_height - avctx->height) >> desc->log2_chroma_h; } else { sps->conformance_window_flag = 0; } @@ -1197,11 +1197,12 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; - VAAPIEncodeH265Context *priv = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + VAAPIEncodeH265Context *priv = avctx->priv_data; #if VA_CHECK_VERSION(1, 13, 0) { + VAAPIEncodeContext *ctx = avctx->priv_data; VAConfigAttribValEncHEVCBlockSizes block_size; VAConfigAttrib attr; VAStatus vas; @@ -1249,10 +1250,10 @@ static av_cold int vaapi_encode_h265_get_encoder_caps(AVCodecContext *avctx) "min CB size %dx%d.\n", priv->ctu_size, priv->ctu_size, priv->min_cb_size, priv->min_cb_size); - ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); - ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); + base_ctx->surface_width = FFALIGN(avctx->width, priv->min_cb_size); + base_ctx->surface_height = FFALIGN(avctx->height, priv->min_cb_size); - ctx->slice_block_width = ctx->slice_block_height = priv->ctu_size; + base_ctx->slice_block_width = base_ctx->slice_block_height = priv->ctu_size; return 0; } diff --git a/libavcodec/vaapi_encode_mjpeg.c b/libavcodec/vaapi_encode_mjpeg.c index 24acc88a21..6ed58d7118 100644 --- a/libavcodec/vaapi_encode_mjpeg.c +++ b/libavcodec/vaapi_encode_mjpeg.c @@ -439,14 +439,13 @@ static int vaapi_encode_mjpeg_init_slice_params(AVCodecContext *avctx, static av_cold int vaapi_encode_mjpeg_get_encoder_caps(AVCodecContext *avctx) { HWBaseEncodeContext *base_ctx = avctx->priv_data; - VAAPIEncodeContext *ctx = avctx->priv_data; const AVPixFmtDescriptor *desc; desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); av_assert0(desc); - ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); - ctx->surface_height = FFALIGN(avctx->height, 8 << desc->log2_chroma_h); + base_ctx->surface_width = FFALIGN(avctx->width, 8 << desc->log2_chroma_w); + base_ctx->surface_height = FFALIGN(avctx->height, 8 << desc->log2_chroma_h); return 0; } diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 36dafea2c0..5e1bf3a7c4 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -190,11 +190,11 @@ static int vaapi_encode_vp9_init_picture_params(AVCodecContext *avctx, static av_cold int vaapi_encode_vp9_get_encoder_caps(AVCodecContext *avctx) { - VAAPIEncodeContext *ctx = avctx->priv_data; + HWBaseEncodeContext *base_ctx = avctx->priv_data; // Surfaces must be aligned to 64x64 superblock boundaries. - ctx->surface_width = FFALIGN(avctx->width, 64); - ctx->surface_height = FFALIGN(avctx->height, 64); + base_ctx->surface_width = FFALIGN(avctx->width, 64); + base_ctx->surface_height = FFALIGN(avctx->height, 64); return 0; } From patchwork Wed May 22 01:26:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49121 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82336pzb; Tue, 21 May 2024 18:30:05 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVkPmhH8OmztUva04QZ/f2uEEO7WmNeSe9lbdRwCwGRX1FirRSNeXUzsGi3Hr1P5rLQB2K7jHWGAdbDirDIzcS2S7AMnAYcxavo4g== X-Google-Smtp-Source: AGHT+IFrj0WBmOpFlEfjNbm8uScLKxXuosk4OHEuoJ3E0gybb6T6o/9tyk+Gb2z9wzDbgj7VKUXP X-Received: by 2002:aa7:c659:0:b0:578:3510:993a with SMTP id 4fb4d7f45d1cf-57835109c57mr37806a12.5.1716341405330; Tue, 21 May 2024 18:30:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341405; cv=none; d=google.com; s=arc-20160816; b=0QY7aCFrer+HnD6d76tTHJMo6tsGpqLpU8DtPWng+B6CQ0czJunCD/6ePIk7mi9mTB Fr3IBHKmVqOT3gJoQ+zy+NpBar7Y6IyG1iURAyp1ZxvAA9DTk/YZ0Nwq+pgg7MWpSUXI aMCnUAVdKqkXnyxWBtMm7QySliC0cSpBGfS7N9/I0wTfnxa1VB4b3Aqzcb5CkCSnz9qK D8NVamnThzmeuW1S8lGff0fruLqSvXb3dAqpGahBYBTiZjhUPEY+uVZeMUDmqJc4zyoP 9iLZaYq8r5HiGk03J1D4QJZd4PsfQCVyVoB/gJ3mfUs+Ozvmmr8XZNXkwPkaMi2RDs8U wa8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=wfyUQ2gNra6aUKJRctOZaKmOEBfF+ZhD29YktJwxr7Q=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=uSDqziY7PLd7j08GMkkahNESJmm36s28aGkC9hbLNmIB0WXUGheqmCKzlRQ4n1ufUI yrxEdDKleAfRSWutdulj4EIn8A2P+NlvtaKBNjUKxvPvfGCgZbUeiADXa3kZOs99lwNh 1rVBIuSYNgPRC+YzEV01t/Ksy1rsMpwm1ykAJ3bzyfn8yR0F6QkM+7GrXfAR1Vj8xEXc kqlxIRWWoO4JVtf3/sVmKFe7ZBrM7C9cghSbtUQAdnWT/zH+McuMSUApcL6HU4PSVUcX mPEaMrd3jfanzYUmA/ogStrJDl/aNk5HTQUgBniyWL/1CDCe8jScXTaYjlHRjFn3vkXW tQHg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Cnb1lY5Q; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5733c37825asi15171985a12.645.2024.05.21.18.29.38; Tue, 21 May 2024 18:30:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=Cnb1lY5Q; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 11A0F68D48A; Wed, 22 May 2024 04:28:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5259168D422 for ; Wed, 22 May 2024 04:28:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341297; x=1747877297; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cQWovVr5I/lD/RbrUYvuZefltPgzt8wh96+kMJzho40=; b=Cnb1lY5QecyzVXwg85sRGlgSpFIZYd1w9O+XxouC2xAaRXH4slLwTHAV WXvzE3Fv8FrjI2vo9h/Tyc4Vq9x7rYGBv5RTAKj9boDzdANevHaj7O9Wr q9MWAoIb/WtlF0tB1Wd2tyhZOD3RL6G2+RPtPcc/OnMZwDUHOPTvvpCVo gd0EWRYxe3cfzg5043d2moasQzPhXPKQUnzcDrhAgaliKdO8ReOARYR0B jWCWuPscwS+Jm7u2oyB71nK79W35BnqpjgyLGCKQ3OkSqRwNUO5hPW0ct TXNb8m2dt3Uj2TfTN5t5120AqUca/3on/TgmHx7J4jnO+0paVhHXw5HjZ Q==; X-CSE-ConnectionGUID: QBTrM/yISSu8lzcupEHrHw== X-CSE-MsgGUID: wA8GgWRQTH+EYeC3EllNfg== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797214" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797214" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:11 -0700 X-CSE-ConnectionGUID: /xhSmMGIROG1TEMaDRVz1w== X-CSE-MsgGUID: 97V9lnmJTdi/PMOsTuEjNA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537504" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:10 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:23 +0800 Message-ID: <20240522012627.1386-10-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 10/13] avcodec/vaapi_encode: extract a free funtion to base layer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: H08m98AqHi+b From: Tong Wu Signed-off-by: Tong Wu --- libavcodec/hw_base_encode.c | 11 +++++++++++ libavcodec/hw_base_encode.h | 2 ++ libavcodec/vaapi_encode.c | 6 +----- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/libavcodec/hw_base_encode.c b/libavcodec/hw_base_encode.c index 42c40cb48f..f743d119cd 100644 --- a/libavcodec/hw_base_encode.c +++ b/libavcodec/hw_base_encode.c @@ -748,6 +748,17 @@ fail: return err; } +int ff_hw_base_encode_free(AVCodecContext *avctx, HWBaseEncodePicture *pic) +{ + av_frame_free(&pic->input_image); + av_frame_free(&pic->recon_image); + + av_buffer_unref(&pic->opaque_ref); + av_freep(&pic->priv_data); + + return 0; +} + int ff_hw_base_encode_init(AVCodecContext *avctx) { HWBaseEncodeContext *ctx = avctx->priv_data; diff --git a/libavcodec/hw_base_encode.h b/libavcodec/hw_base_encode.h index 7d40da039c..76a39e2d97 100644 --- a/libavcodec/hw_base_encode.h +++ b/libavcodec/hw_base_encode.h @@ -223,6 +223,8 @@ int ff_hw_base_init_gop_structure(AVCodecContext *avctx, uint32_t ref_l0, uint32 int ff_hw_base_get_recon_format(AVCodecContext *avctx, const void *hwconfig, enum AVPixelFormat *fmt); +int ff_hw_base_encode_free(AVCodecContext *avctx, HWBaseEncodePicture *pic); + int ff_hw_base_encode_init(AVCodecContext *avctx); int ff_hw_base_encode_close(AVCodecContext *avctx); diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index 98b8c82da3..e89d6e01af 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -878,17 +878,13 @@ static int vaapi_encode_free(AVCodecContext *avctx, av_freep(&pic->slices[i].codec_slice_params); } - av_frame_free(&base_pic->input_image); - av_frame_free(&base_pic->recon_image); - - av_buffer_unref(&base_pic->opaque_ref); + ff_hw_base_encode_free(avctx, base_pic); av_freep(&pic->param_buffers); av_freep(&pic->slices); // Output buffer should already be destroyed. av_assert0(pic->output_buffer == VA_INVALID_ID); - av_freep(&base_pic->priv_data); av_freep(&pic->codec_picture_params); av_freep(&pic->roi); From patchwork Wed May 22 01:26:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49120 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82243pzb; Tue, 21 May 2024 18:29:49 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXFUYCdYzg2niGZYLYaSoEUjcOy3BU17NtccscNiS1uhweywIJq67J4y/mPfo0BUhvI2t4OTKpKUm4jWbBkKPGx3FPShzviaucEqA== X-Google-Smtp-Source: AGHT+IE4ugZUBAUepbJ9dtiyWPvholvwxPAGMcwByTVxOAU8DeYv2oxYpXb1plXK7HcPY5oVy4xO X-Received: by 2002:a19:5e58:0:b0:51b:6ed6:af3f with SMTP id 2adb3069b0e04-526bd693330mr177665e87.16.1716341388837; Tue, 21 May 2024 18:29:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341388; cv=none; d=google.com; s=arc-20160816; b=QJLBoDmuD7lF+Ipiwii7bMZQsJMVPujTsOOBT/kki0g7AEOhO2EFHAsRvsYumqQV62 J7Yic6NDENgVMFGK+D13djzKfBBi4SkGmd/qe/0KMLWArnKrK95gvUIG8aiSpFMFPoFs 9++zLEfQx+sWI9uKRtpSsiZnky7DEuIgwnPquBMqAZBZrj2fxQs7GYPDxKtTVyLllOLI 3YWovE5m+zpvC0pLg/rXebB18cIhBDXPZ9xcbtJiWz1SOK9m+vots4499Skfq2fjaFcB 9B4YYLzwX51sOl8QxYjo0LyOYk7+/SBZvApSsU/cImbuWJuNdbn4Lu0K3odAKuzN6sl4 pvIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=yypiYOnv+IJ+I7evKTUKZNuo5QAR/BCv0zTqjeywCz4=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=g9CyxePtc5/2L3GfQ/AVV+w2AO0GM3iq6PUDzb+gKbSNkyrATCAENc7VcUKmec62I5 Q2HtbOJXYEVf7x5AN1RhEWPSlDhcTbt2DCg5XSKaRSV7mJ8lk1TVrkuCO2cD9BxqMVSh 5pAUdVDb/NoMzzCXWR13qArpe28WyjSuz4H24cZGMXFLL4/FtQ63t2IHeMC8PSF41Kgj Z2ILpev/wSeCQhvCPKvnuwRD9uy2IK/C3HuuiPrW7vQ7N2RkmHfz0KMl9ofY1iM1TjcF 9yfy6RDLJkWYzlRCtgKZ/wLL8nkp82qeFpwuUl5fbKq8XH9io3v8MprisuFJ4luNBDyN rVVA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=DUnrfVys; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17b21d4bsi1463355966b.301.2024.05.21.18.29.48; Tue, 21 May 2024 18:29:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=DUnrfVys; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E3B7B68D495; Wed, 22 May 2024 04:28:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BBAB968D288 for ; Wed, 22 May 2024 04:28:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341298; x=1747877298; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=w4JvU19VzxO1x2UpzFOcslapest2i4We4Imaj7OWmFk=; b=DUnrfVysImuTd8aW1IIJgMYejbfTp7rFApAEA98ffj1YpTMv/1KD8TwL gO+6CQ/rhG6ayy1P59Jb7jQxnYkikWD3ULkWHMkiMkJaB5strhdmzJVGY fuFIuWrrrCtKf/XFRLC0MEixwTH30YJJ1V/SMflzWJB/IyoQF84KkcBux QX2jD9YW5ghckWuQL7YzOzHy76ScuWliQEGwvuHKTPuPqq1/2iYCbs1B9 bhBzmi6+G1QbKZ7BGlU/FILrMm6BI++MCU94QMW/nSzzufbHzUuSZJbYl r9JVvskhVyFhcP6ImZE82E8/yIvCSBdzwAVhqPblIhCrn0Jhm/Aq2YJdr w==; X-CSE-ConnectionGUID: FcuDmOe5T/CyzXSl/EBE1A== X-CSE-MsgGUID: tAZRFI7RR2it9cb8juw7Yw== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797216" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797216" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:12 -0700 X-CSE-ConnectionGUID: sGHl8YzlSqGvhtyk5jmp5w== X-CSE-MsgGUID: khsNtzU/QsagGKxUTos5oQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537507" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:11 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:24 +0800 Message-ID: <20240522012627.1386-11-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 11/13] avutil/hwcontext_d3d12va: add Flags for resource creation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kIJ1iztuaJ9p From: Tong Wu Flags field is added to support diffferent resource creation. Signed-off-by: Tong Wu --- doc/APIchanges | 3 +++ libavutil/hwcontext_d3d12va.c | 2 +- libavutil/hwcontext_d3d12va.h | 8 ++++++++ libavutil/version.h | 2 +- 4 files changed, 13 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 269fd36559..808ba02f2d 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-01-xx - xxxxxxxxxx - lavu 59.20.100 - hwcontext_d3d12va.h + Add AVD3D12VAFramesContext.flags + 2024-05-xx - xxxxxxxxxx - lavu 59.19.100 - hwcontext_qsv.h Add AVQSVFramesContext.info diff --git a/libavutil/hwcontext_d3d12va.c b/libavutil/hwcontext_d3d12va.c index cfc016315d..6507cf69c1 100644 --- a/libavutil/hwcontext_d3d12va.c +++ b/libavutil/hwcontext_d3d12va.c @@ -247,7 +247,7 @@ static AVBufferRef *d3d12va_pool_alloc(void *opaque, size_t size) .Format = hwctx->format, .SampleDesc = {.Count = 1, .Quality = 0 }, .Layout = D3D12_TEXTURE_LAYOUT_UNKNOWN, - .Flags = D3D12_RESOURCE_FLAG_NONE, + .Flags = hwctx->flags, }; frame = av_mallocz(sizeof(AVD3D12VAFrame)); diff --git a/libavutil/hwcontext_d3d12va.h b/libavutil/hwcontext_d3d12va.h index ff06e6f2ef..212a6a6146 100644 --- a/libavutil/hwcontext_d3d12va.h +++ b/libavutil/hwcontext_d3d12va.h @@ -129,6 +129,14 @@ typedef struct AVD3D12VAFramesContext { * If unset, will be automatically set. */ DXGI_FORMAT format; + + /** + * Options for working with resources. + * If unset, this will be D3D12_RESOURCE_FLAG_NONE. + * + * @see https://learn.microsoft.com/en-us/windows/win32/api/d3d12/ne-d3d12-d3d12_resource_flags + */ + D3D12_RESOURCE_FLAGS flags; } AVD3D12VAFramesContext; #endif /* AVUTIL_HWCONTEXT_D3D12VA_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 3221c4c592..9c7146c228 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 19 +#define LIBAVUTIL_VERSION_MINOR 20 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Wed May 22 01:26:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49127 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82353pzb; Tue, 21 May 2024 18:30:08 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXMuZG5IXy7WfyYqSWc5KWU9dnfvh077OSW/VvRGAGdVFTWdZPPzZd4SzSKljuzuGvs6y9P1JGp05f5Fg4MVD/vR5pbbf/Kfi/0LQ== X-Google-Smtp-Source: AGHT+IHmp+uCz08h+gZsd9Vy5SdzmeY4ElZ8Lfi6XA8zo4E1VvyYnHFWZkVvK0lCzLf8vJIen2RP X-Received: by 2002:a17:906:fc16:b0:a5c:deb3:19ee with SMTP id a640c23a62f3a-a6228179553mr22975766b.6.1716341407849; Tue, 21 May 2024 18:30:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341407; cv=none; d=google.com; s=arc-20160816; b=e7wGoroEnGqLghz0IxFWc0VlSYRJpipyrOxOJ9UHSUXoYDhYTWb5gkyJWdibjmsqWK vX5nUj+7buxIK24u1wUapYwxINmAcgkfhiSLe065eMjABjTQOEQj7eUQS1yiPgQrc5Xm 6lG3rfOtTE82tJbLkeNWWHEucB6B7MSbG1MumFMBufE9vKcOVCpkDh9r6xO374TVdF58 8BhGj3m0YkhQQBlphNuhtY08+hCaGwtNxVgQrxM0I/iluOqIQyAgzjOhugW8vfZpBtCY Jt26QQeDBjP0GW65gbikX45Lo9VR4tTXgTE0wjK0T/YhhTU2ZgwE0/XDIl31SpF/wKV5 AnMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=8rCW+mWUToVM/ZVam6C/TnBWqRalwgiyBK1hJHZUe00=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=uGKXpddq5GtYRQHoZlJ4E3h74BASTQ8GO3RFyaY6NL0tY2TwF67gZix6k6uQF5QfRW +zsJQPxGukm8E9FGAvZ8iLb2Ey3dKAtP0h48oqMAUxJBnmZCfdDyvQKJQWYyRK1+noYn Fa6TpHpAW/Shv4ojIJ+Tg21N8r5WyhXuK/qIWA4U0Y8HHNLhGsuqIoI/wI6vhTAAloHq up+62zfjbzn3FM/ZCvFS//VbjqRIuakx0m0KodkzKNM0lRFeEpMtt8lzmeTSBk9KCAl+ v2rqVcyLbKCwCIciBFQVi4N4anYAodiCUacxdJ30gH5G2mGJ7zlhhzzdGmSNCKmSbDW2 yVLA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=luJSGpPb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a17b21af3si1399795466b.303.2024.05.21.18.29.59; Tue, 21 May 2024 18:30:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=luJSGpPb; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F0B2968D288; Wed, 22 May 2024 04:28:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D9E9668D42A for ; Wed, 22 May 2024 04:28:17 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341298; x=1747877298; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9hJwgHP4vPl9C5DBCzmOUtLDISkDKKFLyu89+pdbtNg=; b=luJSGpPbcSv7/6ke5Wh4G0wBgy0zjO0bax3sCP6CzxaSkErRaSvNo0wI cx5NyRAiCYdyew/HVcI8gKBAMVw28h0iRuRKajxvx5ygbWv0LxZ8jlWJZ jIjjQR30jxRh8dR928ff9JcilwD1wT/7kX6c5B55Awm6OeMSvyruJwDx7 JxCblDUaKRRRaoGcU2+d/MHIl4MUos65sRIvdpMNcza3bNJPQN/4ojrsd 8Fv23mTUQK+WtWFZikNvF9QCo5Z0iPonMeg/W0fZ7O3KE0Zbeym5sWnXl mPStEfZ/YRZt2QGQs7mqnms1/flxJxxsUXu1kgykGZyrFNq7XBXjlc5LR w==; X-CSE-ConnectionGUID: //toG+C8Tt2LK7Ce+KJVyA== X-CSE-MsgGUID: K1AIIOs0QOyNj386SX0lrQ== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797218" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797218" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:13 -0700 X-CSE-ConnectionGUID: TwCRaU9EQmWXTLJ9eYpfxw== X-CSE-MsgGUID: EWCAk3AER06SMnCMPUL41w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537512" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:12 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:25 +0800 Message-ID: <20240522012627.1386-12-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 12/13] avcodec: add D3D12VA hardware HEVC encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PSHezqBjza/0 From: Tong Wu This implementation is based on D3D12 Video Encoding Spec: https://microsoft.github.io/DirectX-Specs/d3d/D3D12VideoEncoding.html Sample command line for transcoding: ffmpeg.exe -hwaccel d3d12va -hwaccel_output_format d3d12 -i input.mp4 -c:v hevc_d3d12va output.mp4 Signed-off-by: Tong Wu --- configure | 6 + libavcodec/Makefile | 5 +- libavcodec/allcodecs.c | 1 + libavcodec/d3d12va_encode.c | 1558 ++++++++++++++++++++++++++++++ libavcodec/d3d12va_encode.h | 334 +++++++ libavcodec/d3d12va_encode_hevc.c | 1007 +++++++++++++++++++ 6 files changed, 2910 insertions(+), 1 deletion(-) create mode 100644 libavcodec/d3d12va_encode.c create mode 100644 libavcodec/d3d12va_encode.h create mode 100644 libavcodec/d3d12va_encode_hevc.c diff --git a/configure b/configure index b16722d83d..127d68e60c 100755 --- a/configure +++ b/configure @@ -2551,6 +2551,7 @@ CONFIG_EXTRA=" cbs_mpeg2 cbs_vp8 cbs_vp9 + d3d12va_encode deflate_wrapper dirac_parse dnn @@ -3287,6 +3288,7 @@ wmv3_vaapi_hwaccel_select="vc1_vaapi_hwaccel" wmv3_vdpau_hwaccel_select="vc1_vdpau_hwaccel" # hardware-accelerated codecs +d3d12va_encode_deps="d3d12va ID3D12VideoEncoder d3d12_encoder_feature" mediafoundation_deps="mftransform_h MFCreateAlignedMemoryBuffer" omx_deps="libdl pthreads" omx_rpi_select="omx" @@ -3354,6 +3356,7 @@ h264_v4l2m2m_encoder_deps="v4l2_m2m h264_v4l2_m2m" hevc_amf_encoder_deps="amf" hevc_cuvid_decoder_deps="cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" +hevc_d3d12va_encoder_select="cbs_h265 d3d12va_encode" hevc_mediacodec_decoder_deps="mediacodec" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" hevc_mediacodec_encoder_deps="mediacodec" @@ -6725,6 +6728,9 @@ check_type "windows.h d3d11.h" "ID3D11VideoDecoder" check_type "windows.h d3d11.h" "ID3D11VideoContext" check_type "windows.h d3d12.h" "ID3D12Device" check_type "windows.h d3d12video.h" "ID3D12VideoDecoder" +check_type "windows.h d3d12video.h" "ID3D12VideoEncoder" +test_code cc "windows.h d3d12video.h" "D3D12_FEATURE_VIDEO feature = D3D12_FEATURE_VIDEO_ENCODER_CODEC" && \ +test_code cc "windows.h d3d12video.h" "D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS req" && enable d3d12_encoder_feature check_type "windows.h" "DPI_AWARENESS_CONTEXT" -D_WIN32_WINNT=0x0A00 check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -D_WIN32_WINNT=0x0602 check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 998f6b7e12..6c4500ce6d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -86,6 +86,7 @@ OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP8) += cbs_vp8.o vp8data.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o +OBJS-$(CONFIG_D3D12VA_ENCODE) += d3d12va_encode.o hw_base_encode.o OBJS-$(CONFIG_DEFLATE_WRAPPER) += zlib_wrapper.o OBJS-$(CONFIG_DOVI_RPUDEC) += dovi_rpu.o dovi_rpudec.o OBJS-$(CONFIG_DOVI_RPUENC) += dovi_rpu.o dovi_rpuenc.o @@ -436,6 +437,8 @@ OBJS-$(CONFIG_HEVC_DECODER) += hevcdec.o hevc_mvs.o \ h274.o aom_film_grain.o OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o +OBJS-$(CONFIG_HEVC_D3D12VA_ENCODER) += d3d12va_encode_hevc.o h265_profile_level.o \ + h2645data.o OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_HEVC_MF_ENCODER) += mfenc.o mf_utils.o @@ -1265,7 +1268,7 @@ SKIPHEADERS += %_tablegen.h \ SKIPHEADERS-$(CONFIG_AMF) += amfenc.h SKIPHEADERS-$(CONFIG_D3D11VA) += d3d11va.h dxva2_internal.h -SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h +SKIPHEADERS-$(CONFIG_D3D12VA) += d3d12va_decode.h d3d12va_encode.h SKIPHEADERS-$(CONFIG_DXVA2) += dxva2.h dxva2_internal.h SKIPHEADERS-$(CONFIG_JNI) += ffjni.h SKIPHEADERS-$(CONFIG_LCMS2) += fflcms2.h diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index b102a8069e..463ffbbd08 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -848,6 +848,7 @@ extern const FFCodec ff_h264_vaapi_encoder; extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; extern const FFCodec ff_hevc_cuvid_decoder; +extern const FFCodec ff_hevc_d3d12va_encoder; extern const FFCodec ff_hevc_mediacodec_decoder; extern const FFCodec ff_hevc_mediacodec_encoder; extern const FFCodec ff_hevc_mf_encoder; diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c new file mode 100644 index 0000000000..a20e6cc961 --- /dev/null +++ b/libavcodec/d3d12va_encode.c @@ -0,0 +1,1558 @@ +/* + * Direct3D 12 HW acceleration video encoder + * + * Copyright (c) 2024 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/internal.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" +#include "libavutil/hwcontext_d3d12va.h" + +#include "avcodec.h" +#include "d3d12va_encode.h" +#include "encode.h" + +const AVCodecHWConfigInternal *const ff_d3d12va_encode_hw_configs[] = { + HW_CONFIG_ENCODER_FRAMES(D3D12, D3D12VA), + NULL, +}; + +static int d3d12va_fence_completion(AVD3D12VASyncContext *psync_ctx) +{ + uint64_t completion = ID3D12Fence_GetCompletedValue(psync_ctx->fence); + if (completion < psync_ctx->fence_value) { + if (FAILED(ID3D12Fence_SetEventOnCompletion(psync_ctx->fence, psync_ctx->fence_value, psync_ctx->event))) + return AVERROR(EINVAL); + + WaitForSingleObjectEx(psync_ctx->event, INFINITE, FALSE); + } + + return 0; +} + +static int d3d12va_sync_with_gpu(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + + DX_CHECK(ID3D12CommandQueue_Signal(ctx->command_queue, ctx->sync_ctx.fence, ++ctx->sync_ctx.fence_value)); + return d3d12va_fence_completion(&ctx->sync_ctx); + +fail: + return AVERROR(EINVAL); +} + +typedef struct CommandAllocator { + ID3D12CommandAllocator *command_allocator; + uint64_t fence_value; +} CommandAllocator; + +static int d3d12va_get_valid_command_allocator(AVCodecContext *avctx, ID3D12CommandAllocator **ppAllocator) +{ + HRESULT hr; + D3D12VAEncodeContext *ctx = avctx->priv_data; + CommandAllocator allocator; + + if (av_fifo_peek(ctx->allocator_queue, &allocator, 1, 0) >= 0) { + uint64_t completion = ID3D12Fence_GetCompletedValue(ctx->sync_ctx.fence); + if (completion >= allocator.fence_value) { + *ppAllocator = allocator.command_allocator; + av_fifo_read(ctx->allocator_queue, &allocator, 1); + return 0; + } + } + + hr = ID3D12Device_CreateCommandAllocator(ctx->hwctx->device, D3D12_COMMAND_LIST_TYPE_VIDEO_ENCODE, + &IID_ID3D12CommandAllocator, (void **)ppAllocator); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create a new command allocator!\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int d3d12va_discard_command_allocator(AVCodecContext *avctx, ID3D12CommandAllocator *pAllocator, uint64_t fence_value) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + + CommandAllocator allocator = { + .command_allocator = pAllocator, + .fence_value = fence_value, + }; + + av_fifo_write(ctx->allocator_queue, &allocator, 1); + + return 0; +} + +static int d3d12va_encode_wait(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *base_pic = &pic->base; + uint64_t completion; + + av_assert0(base_pic->encode_issued); + + if (base_pic->encode_complete) { + // Already waited for this picture. + return 0; + } + + completion = ID3D12Fence_GetCompletedValue(ctx->sync_ctx.fence); + if (completion < pic->fence_value) { + if (FAILED(ID3D12Fence_SetEventOnCompletion(ctx->sync_ctx.fence, pic->fence_value, + ctx->sync_ctx.event))) + return AVERROR(EINVAL); + + WaitForSingleObjectEx(ctx->sync_ctx.event, INFINITE, FALSE); + } + + av_log(avctx, AV_LOG_DEBUG, "Sync to pic %"PRId64"/%"PRId64" " + "(input surface %p).\n", base_pic->display_order, + base_pic->encode_order, pic->input_surface->texture); + + av_frame_free(&base_pic->input_image); + + base_pic->encode_complete = 1; + return 0; +} + +static int d3d12va_encode_create_metadata_buffers(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + int width = sizeof(D3D12_VIDEO_ENCODER_OUTPUT_METADATA) + sizeof(D3D12_VIDEO_ENCODER_FRAME_SUBREGION_METADATA); + D3D12_HEAP_PROPERTIES encoded_meta_props = { .Type = D3D12_HEAP_TYPE_DEFAULT }, resolved_meta_props; + D3D12_HEAP_TYPE resolved_heap_type = D3D12_HEAP_TYPE_READBACK; + HRESULT hr; + + D3D12_RESOURCE_DESC meta_desc = { + .Dimension = D3D12_RESOURCE_DIMENSION_BUFFER, + .Alignment = 0, + .Width = ctx->req.MaxEncoderOutputMetadataBufferSize, + .Height = 1, + .DepthOrArraySize = 1, + .MipLevels = 1, + .Format = DXGI_FORMAT_UNKNOWN, + .SampleDesc = { .Count = 1, .Quality = 0 }, + .Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR, + .Flags = D3D12_RESOURCE_FLAG_NONE, + }; + + hr = ID3D12Device_CreateCommittedResource(ctx->hwctx->device, &encoded_meta_props, D3D12_HEAP_FLAG_NONE, + &meta_desc, D3D12_RESOURCE_STATE_COMMON, NULL, + &IID_ID3D12Resource, (void **)&pic->encoded_metadata); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create metadata buffer.\n"); + return AVERROR_UNKNOWN; + } + + ctx->hwctx->device->lpVtbl->GetCustomHeapProperties(ctx->hwctx->device, &resolved_meta_props, 0, resolved_heap_type); + + meta_desc.Width = width; + + hr = ID3D12Device_CreateCommittedResource(ctx->hwctx->device, &resolved_meta_props, D3D12_HEAP_FLAG_NONE, + &meta_desc, D3D12_RESOURCE_STATE_COMMON, NULL, + &IID_ID3D12Resource, (void **)&pic->resolved_metadata); + + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create output metadata buffer.\n"); + return AVERROR_UNKNOWN; + } + + return 0; +} + +static int d3d12va_encode_issue(AVCodecContext *avctx, + const HWBaseEncodePicture *base_pic) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + AVD3D12VAFramesContext *frames_hwctx = base_ctx->input_frames->hwctx; + D3D12VAEncodePicture *pic = (D3D12VAEncodePicture *)base_pic; + int err, i, j; + HRESULT hr; + char data[MAX_PARAM_BUFFER_SIZE]; + void *ptr; + size_t bit_len; + ID3D12CommandAllocator *command_allocator = NULL; + ID3D12VideoEncodeCommandList2 *cmd_list = ctx->command_list; + D3D12_RESOURCE_BARRIER barriers[32] = { 0 }; + D3D12_VIDEO_ENCODE_REFERENCE_FRAMES d3d12_refs = { 0 }; + + D3D12_VIDEO_ENCODER_ENCODEFRAME_INPUT_ARGUMENTS input_args = { + .SequenceControlDesc = { + .Flags = D3D12_VIDEO_ENCODER_SEQUENCE_CONTROL_FLAG_NONE, + .IntraRefreshConfig = { 0 }, + .RateControl = ctx->rc, + .PictureTargetResolution = ctx->resolution, + .SelectedLayoutMode = D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME, + .FrameSubregionsLayoutData = { 0 }, + .CodecGopSequence = ctx->gop, + }, + .pInputFrame = pic->input_surface->texture, + .InputFrameSubresource = 0, + }; + + D3D12_VIDEO_ENCODER_ENCODEFRAME_OUTPUT_ARGUMENTS output_args = { 0 }; + + D3D12_VIDEO_ENCODER_RESOLVE_METADATA_INPUT_ARGUMENTS input_metadata = { + .EncoderCodec = ctx->codec->d3d12_codec, + .EncoderProfile = ctx->profile->d3d12_profile, + .EncoderInputFormat = frames_hwctx->format, + .EncodedPictureEffectiveResolution = ctx->resolution, + }; + + D3D12_VIDEO_ENCODER_RESOLVE_METADATA_OUTPUT_ARGUMENTS output_metadata = { 0 }; + + memset(data, 0, sizeof(data)); + + av_log(avctx, AV_LOG_DEBUG, "Issuing encode for pic %"PRId64"/%"PRId64" " + "as type %s.\n", base_pic->display_order, base_pic->encode_order, + ff_hw_base_encode_get_pictype_name(base_pic->type)); + if (base_pic->nb_refs[0] == 0 && base_pic->nb_refs[1] == 0) { + av_log(avctx, AV_LOG_DEBUG, "No reference pictures.\n"); + } else { + av_log(avctx, AV_LOG_DEBUG, "L0 refers to"); + for (i = 0; i < base_pic->nb_refs[0]; i++) { + av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, + base_pic->refs[0][i]->display_order, base_pic->refs[0][i]->encode_order); + } + av_log(avctx, AV_LOG_DEBUG, ".\n"); + + if (base_pic->nb_refs[1]) { + av_log(avctx, AV_LOG_DEBUG, "L1 refers to"); + for (i = 0; i < base_pic->nb_refs[1]; i++) { + av_log(avctx, AV_LOG_DEBUG, " %"PRId64"/%"PRId64, + base_pic->refs[1][i]->display_order, base_pic->refs[1][i]->encode_order); + } + av_log(avctx, AV_LOG_DEBUG, ".\n"); + } + } + + av_assert0(!base_pic->encode_issued); + for (i = 0; i < base_pic->nb_refs[0]; i++) { + av_assert0(base_pic->refs[0][i]); + av_assert0(base_pic->refs[0][i]->encode_issued); + } + for (i = 0; i < base_pic->nb_refs[1]; i++) { + av_assert0(base_pic->refs[1][i]); + av_assert0(base_pic->refs[1][i]->encode_issued); + } + + av_log(avctx, AV_LOG_DEBUG, "Input surface is %p.\n", pic->input_surface->texture); + + err = av_hwframe_get_buffer(base_ctx->recon_frames_ref, base_pic->recon_image, 0); + if (err < 0) { + err = AVERROR(ENOMEM); + goto fail; + } + + pic->recon_surface = (AVD3D12VAFrame *)base_pic->recon_image->data[0]; + av_log(avctx, AV_LOG_DEBUG, "Recon surface is %p.\n", + pic->recon_surface->texture); + + pic->output_buffer_ref = av_buffer_pool_get(ctx->output_buffer_pool); + if (!pic->output_buffer_ref) { + err = AVERROR(ENOMEM); + goto fail; + } + pic->output_buffer = (ID3D12Resource *)pic->output_buffer_ref->data; + av_log(avctx, AV_LOG_DEBUG, "Output buffer is %p.\n", + pic->output_buffer); + + err = d3d12va_encode_create_metadata_buffers(avctx, pic); + if (err < 0) + goto fail; + + if (ctx->codec->init_picture_params) { + err = ctx->codec->init_picture_params(avctx, pic); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialise picture " + "parameters: %d.\n", err); + goto fail; + } + } + + if (base_pic->type == PICTURE_TYPE_IDR) { + if (ctx->codec->write_sequence_header) { + bit_len = 8 * sizeof(data); + err = ctx->codec->write_sequence_header(avctx, data, &bit_len); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write per-sequence " + "header: %d.\n", err); + goto fail; + } + } + + pic->header_size = (int)bit_len / 8; + pic->header_size = pic->header_size % ctx->req.CompressedBitstreamBufferAccessAlignment ? + FFALIGN(pic->header_size, ctx->req.CompressedBitstreamBufferAccessAlignment) : + pic->header_size; + + hr = ID3D12Resource_Map(pic->output_buffer, 0, NULL, (void **)&ptr); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + memcpy(ptr, data, pic->header_size); + ID3D12Resource_Unmap(pic->output_buffer, 0, NULL); + } + + d3d12_refs.NumTexture2Ds = base_pic->nb_refs[0] + base_pic->nb_refs[1]; + if (d3d12_refs.NumTexture2Ds) { + d3d12_refs.ppTexture2Ds = av_calloc(d3d12_refs.NumTexture2Ds, + sizeof(*d3d12_refs.ppTexture2Ds)); + if (!d3d12_refs.ppTexture2Ds) { + err = AVERROR(ENOMEM); + goto fail; + } + + i = 0; + for (j = 0; j < base_pic->nb_refs[0]; j++) + d3d12_refs.ppTexture2Ds[i++] = ((D3D12VAEncodePicture *)base_pic->refs[0][j])->recon_surface->texture; + for (j = 0; j < base_pic->nb_refs[1]; j++) + d3d12_refs.ppTexture2Ds[i++] = ((D3D12VAEncodePicture *)base_pic->refs[1][j])->recon_surface->texture; + } + + input_args.PictureControlDesc.IntraRefreshFrameIndex = 0; + if (base_pic->is_reference) + input_args.PictureControlDesc.Flags |= D3D12_VIDEO_ENCODER_PICTURE_CONTROL_FLAG_USED_AS_REFERENCE_PICTURE; + + input_args.PictureControlDesc.PictureControlCodecData = pic->pic_ctl; + input_args.PictureControlDesc.ReferenceFrames = d3d12_refs; + input_args.CurrentFrameBitstreamMetadataSize = pic->header_size; + + output_args.Bitstream.pBuffer = pic->output_buffer; + output_args.Bitstream.FrameStartOffset = pic->header_size; + output_args.ReconstructedPicture.pReconstructedPicture = pic->recon_surface->texture; + output_args.ReconstructedPicture.ReconstructedPictureSubresource = 0; + output_args.EncoderOutputMetadata.pBuffer = pic->encoded_metadata; + output_args.EncoderOutputMetadata.Offset = 0; + + input_metadata.HWLayoutMetadata.pBuffer = pic->encoded_metadata; + input_metadata.HWLayoutMetadata.Offset = 0; + + output_metadata.ResolvedLayoutMetadata.pBuffer = pic->resolved_metadata; + output_metadata.ResolvedLayoutMetadata.Offset = 0; + + err = d3d12va_get_valid_command_allocator(avctx, &command_allocator); + if (err < 0) + goto fail; + + hr = ID3D12CommandAllocator_Reset(command_allocator); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12VideoEncodeCommandList2_Reset(cmd_list, command_allocator); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + +#define TRANSITION_BARRIER(res, before, after) \ + (D3D12_RESOURCE_BARRIER) { \ + .Type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION, \ + .Flags = D3D12_RESOURCE_BARRIER_FLAG_NONE, \ + .Transition = { \ + .pResource = res, \ + .Subresource = D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES, \ + .StateBefore = before, \ + .StateAfter = after, \ + }, \ + } + + barriers[0] = TRANSITION_BARRIER(pic->input_surface->texture, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + barriers[1] = TRANSITION_BARRIER(pic->output_buffer, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + barriers[2] = TRANSITION_BARRIER(pic->recon_surface->texture, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + barriers[4] = TRANSITION_BARRIER(pic->resolved_metadata, + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 5, barriers); + + if (d3d12_refs.NumTexture2Ds) { + D3D12_RESOURCE_BARRIER refs_barriers[3]; + + for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) + refs_barriers[i] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], + D3D12_RESOURCE_STATE_COMMON, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, d3d12_refs.NumTexture2Ds, + refs_barriers); + } + + ID3D12VideoEncodeCommandList2_EncodeFrame(cmd_list, ctx->encoder, ctx->encoder_heap, + &input_args, &output_args); + + barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 1, &barriers[3]); + + ID3D12VideoEncodeCommandList2_ResolveEncoderOutputMetadata(cmd_list, &input_metadata, &output_metadata); + + if (d3d12_refs.NumTexture2Ds) { + D3D12_RESOURCE_BARRIER refs_barriers[3]; + + for (i = 0; i < d3d12_refs.NumTexture2Ds; i++) + refs_barriers[i] = TRANSITION_BARRIER(d3d12_refs.ppTexture2Ds[i], + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, + D3D12_RESOURCE_STATE_COMMON); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, d3d12_refs.NumTexture2Ds, + refs_barriers); + } + + barriers[0] = TRANSITION_BARRIER(pic->input_surface->texture, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, + D3D12_RESOURCE_STATE_COMMON); + barriers[1] = TRANSITION_BARRIER(pic->output_buffer, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_COMMON); + barriers[2] = TRANSITION_BARRIER(pic->recon_surface->texture, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_COMMON); + barriers[3] = TRANSITION_BARRIER(pic->encoded_metadata, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_READ, + D3D12_RESOURCE_STATE_COMMON); + barriers[4] = TRANSITION_BARRIER(pic->resolved_metadata, + D3D12_RESOURCE_STATE_VIDEO_ENCODE_WRITE, + D3D12_RESOURCE_STATE_COMMON); + + ID3D12VideoEncodeCommandList2_ResourceBarrier(cmd_list, 5, barriers); + + hr = ID3D12VideoEncodeCommandList2_Close(cmd_list); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12CommandQueue_Wait(ctx->command_queue, pic->input_surface->sync_ctx.fence, + pic->input_surface->sync_ctx.fence_value); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + ID3D12CommandQueue_ExecuteCommandLists(ctx->command_queue, 1, (ID3D12CommandList **)&ctx->command_list); + + hr = ID3D12CommandQueue_Signal(ctx->command_queue, pic->input_surface->sync_ctx.fence, + ++pic->input_surface->sync_ctx.fence_value); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12CommandQueue_Signal(ctx->command_queue, ctx->sync_ctx.fence, ++ctx->sync_ctx.fence_value); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto fail; + } + + err = d3d12va_discard_command_allocator(avctx, command_allocator, ctx->sync_ctx.fence_value); + if (err < 0) + goto fail; + + pic->fence_value = ctx->sync_ctx.fence_value; + + if (d3d12_refs.ppTexture2Ds) + av_freep(&d3d12_refs.ppTexture2Ds); + + return 0; + +fail: + if (command_allocator) + d3d12va_discard_command_allocator(avctx, command_allocator, ctx->sync_ctx.fence_value); + + if (d3d12_refs.ppTexture2Ds) + av_freep(&d3d12_refs.ppTexture2Ds); + + if (ctx->codec->free_picture_params) + ctx->codec->free_picture_params(pic); + + av_buffer_unref(&pic->output_buffer_ref); + pic->output_buffer = NULL; + D3D12_OBJECT_RELEASE(pic->encoded_metadata); + D3D12_OBJECT_RELEASE(pic->resolved_metadata); + return err; +} + +static int d3d12va_encode_discard(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + HWBaseEncodePicture *base_pic = &pic->base; + d3d12va_encode_wait(avctx, pic); + + if (pic->output_buffer_ref) { + av_log(avctx, AV_LOG_DEBUG, "Discard output for pic " + "%"PRId64"/%"PRId64".\n", + base_pic->display_order, base_pic->encode_order); + + av_buffer_unref(&pic->output_buffer_ref); + pic->output_buffer = NULL; + } + + D3D12_OBJECT_RELEASE(pic->encoded_metadata); + D3D12_OBJECT_RELEASE(pic->resolved_metadata); + + return 0; +} + +static int d3d12va_encode_free_rc_params(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + + switch (ctx->rc.Mode) + { + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CQP: + av_freep(&ctx->rc.ConfigParams.pConfiguration_CQP); + break; + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CBR: + av_freep(&ctx->rc.ConfigParams.pConfiguration_CBR); + break; + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_VBR: + av_freep(&ctx->rc.ConfigParams.pConfiguration_VBR); + break; + case D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_QVBR: + av_freep(&ctx->rc.ConfigParams.pConfiguration_QVBR); + break; + default: + break; + } + + return 0; +} + +static HWBaseEncodePicture *d3d12va_encode_alloc(AVCodecContext *avctx, + const AVFrame *frame) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodePicture *pic; + + pic = av_mallocz(sizeof(*pic)); + if (!pic) + return NULL; + + if (ctx->codec->picture_priv_data_size > 0) { + pic->base.priv_data = av_mallocz(ctx->codec->picture_priv_data_size); + if (!pic->base.priv_data) { + av_freep(&pic); + return NULL; + } + } + + pic->input_surface = (AVD3D12VAFrame *)frame->data[0]; + + return &pic->base; +} + +static int d3d12va_encode_free(AVCodecContext *avctx, + HWBaseEncodePicture *base_pic) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodePicture *pic = (D3D12VAEncodePicture *)base_pic; + + if (base_pic->encode_issued) + d3d12va_encode_discard(avctx, pic); + + if (ctx->codec->free_picture_params) + ctx->codec->free_picture_params(pic); + + ff_hw_base_encode_free(avctx, base_pic); + + av_free(pic); + + return 0; +} + +static int d3d12va_encode_get_buffer_size(AVCodecContext *avctx, + D3D12VAEncodePicture *pic, size_t *size) +{ + D3D12_VIDEO_ENCODER_OUTPUT_METADATA *meta = NULL; + uint8_t *data; + HRESULT hr; + int err; + + hr = ID3D12Resource_Map(pic->resolved_metadata, 0, NULL, (void **)&data); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + return err; + } + + meta = (D3D12_VIDEO_ENCODER_OUTPUT_METADATA *)data; + + if (meta->EncodeErrorFlags != D3D12_VIDEO_ENCODER_ENCODE_ERROR_FLAG_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Encode failed %"PRIu64"\n", meta->EncodeErrorFlags); + err = AVERROR(EINVAL); + return err; + } + + if (meta->EncodedBitstreamWrittenBytesCount == 0) { + av_log(avctx, AV_LOG_ERROR, "No bytes were written to encoded bitstream\n"); + err = AVERROR(EINVAL); + return err; + } + + *size = meta->EncodedBitstreamWrittenBytesCount; + + ID3D12Resource_Unmap(pic->resolved_metadata, 0, NULL); + + return 0; +} + +static int d3d12va_encode_get_coded_data(AVCodecContext *avctx, + D3D12VAEncodePicture *pic, AVPacket *pkt) +{ + int err; + uint8_t *ptr, *mapped_data; + size_t total_size = 0; + HRESULT hr; + + err = d3d12va_encode_get_buffer_size(avctx, pic, &total_size); + if (err < 0) + goto end; + + total_size += pic->header_size; + av_log(avctx, AV_LOG_DEBUG, "Output buffer size %"PRId64"\n", total_size); + + hr = ID3D12Resource_Map(pic->output_buffer, 0, NULL, (void **)&mapped_data); + if (FAILED(hr)) { + err = AVERROR_UNKNOWN; + goto end; + } + + err = ff_get_encode_buffer(avctx, pkt, total_size, 0); + if (err < 0) + goto end; + ptr = pkt->data; + + memcpy(ptr, mapped_data, total_size); + + ID3D12Resource_Unmap(pic->output_buffer, 0, NULL); + +end: + av_buffer_unref(&pic->output_buffer_ref); + pic->output_buffer = NULL; + return err; +} + +static int d3d12va_encode_output(AVCodecContext *avctx, + const HWBaseEncodePicture *base_pic, AVPacket *pkt) +{ + D3D12VAEncodePicture *pic = (D3D12VAEncodePicture *)base_pic; + AVPacket *pkt_ptr = pkt; + int err; + + err = d3d12va_encode_wait(avctx, pic); + if (err < 0) + return err; + + err = d3d12va_encode_get_coded_data(avctx, pic, pkt); + if (err < 0) + return err; + + av_log(avctx, AV_LOG_DEBUG, "Output read for pic %"PRId64"/%"PRId64".\n", + base_pic->display_order, base_pic->encode_order); + + ff_hw_base_encode_set_output_property(avctx, (HWBaseEncodePicture *)base_pic, pkt_ptr, 0); + + return 0; +} + +static int d3d12va_encode_set_profile(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + const D3D12VAEncodeProfile *profile; + const AVPixFmtDescriptor *desc; + int i, depth; + + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); + if (!desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%d).\n", + base_ctx->input_frames->sw_format); + return AVERROR(EINVAL); + } + + depth = desc->comp[0].depth; + for (i = 1; i < desc->nb_components; i++) { + if (desc->comp[i].depth != depth) { + av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", + desc->name); + return AVERROR(EINVAL); + } + } + av_log(avctx, AV_LOG_VERBOSE, "Input surface format is %s.\n", + desc->name); + + av_assert0(ctx->codec->profiles); + for (i = 0; (ctx->codec->profiles[i].av_profile != + AV_PROFILE_UNKNOWN); i++) { + profile = &ctx->codec->profiles[i]; + if (depth != profile->depth || + desc->nb_components != profile->nb_components) + continue; + if (desc->nb_components > 1 && + (desc->log2_chroma_w != profile->log2_chroma_w || + desc->log2_chroma_h != profile->log2_chroma_h)) + continue; + if (avctx->profile != profile->av_profile && + avctx->profile != AV_PROFILE_UNKNOWN) + continue; + + ctx->profile = profile; + break; + } + if (!ctx->profile) { + av_log(avctx, AV_LOG_ERROR, "No usable encoding profile found.\n"); + return AVERROR(ENOSYS); + } + + avctx->profile = profile->av_profile; + return 0; +} + +static const D3D12VAEncodeRCMode d3d12va_encode_rc_modes[] = { + // Bitrate Quality + // | Maxrate | HRD/VBV + { 0 }, // | | | | + { RC_MODE_CQP, "CQP", 0, 0, 1, 0, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CQP }, + { RC_MODE_CBR, "CBR", 1, 0, 0, 1, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CBR }, + { RC_MODE_VBR, "VBR", 1, 1, 0, 1, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_VBR }, + { RC_MODE_QVBR, "QVBR", 1, 1, 1, 1, D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_QVBR }, +}; + +static int check_rate_control_support(AVCodecContext *avctx, const D3D12VAEncodeRCMode *rc_mode) +{ + HRESULT hr; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12_FEATURE_DATA_VIDEO_ENCODER_RATE_CONTROL_MODE d3d12_rc_mode = { + .Codec = ctx->codec->d3d12_codec, + }; + + if (!rc_mode->d3d12_mode) + return 0; + + d3d12_rc_mode.IsSupported = 0; + d3d12_rc_mode.RateControlMode = rc_mode->d3d12_mode; + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, + D3D12_FEATURE_VIDEO_ENCODER_RATE_CONTROL_MODE, + &d3d12_rc_mode, sizeof(d3d12_rc_mode)); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to check rate control support.\n"); + return 0; + } + + return d3d12_rc_mode.IsSupported; +} + +static int d3d12va_encode_init_rate_control(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + int64_t rc_target_bitrate; + int64_t rc_peak_bitrate; + int rc_quality; + int64_t hrd_buffer_size; + int64_t hrd_initial_buffer_fullness; + int fr_num, fr_den; + const D3D12VAEncodeRCMode *rc_mode; + + // Rate control mode selection: + // * If the user has set a mode explicitly with the rc_mode option, + // use it and fail if it is not available. + // * If an explicit QP option has been set, use CQP. + // * If the codec is CQ-only, use CQP. + // * If the QSCALE avcodec option is set, use CQP. + // * If bitrate and quality are both set, try QVBR. + // * If quality is set, try CQP. + // * If bitrate and maxrate are set and have the same value, try CBR. + // * If a bitrate is set, try VBR, then CBR. + // * If no bitrate is set, try CQP. + +#define TRY_RC_MODE(mode, fail) do { \ + rc_mode = &d3d12va_encode_rc_modes[mode]; \ + if (!(rc_mode->d3d12_mode && check_rate_control_support(avctx, rc_mode))) { \ + if (fail) { \ + av_log(avctx, AV_LOG_ERROR, "Driver does not support %s " \ + "RC mode.\n", rc_mode->name); \ + return AVERROR(EINVAL); \ + } \ + av_log(avctx, AV_LOG_DEBUG, "Driver does not support %s " \ + "RC mode.\n", rc_mode->name); \ + rc_mode = NULL; \ + } else { \ + goto rc_mode_found; \ + } \ + } while (0) + + if (ctx->explicit_rc_mode) + TRY_RC_MODE(ctx->explicit_rc_mode, 1); + + if (ctx->explicit_qp) + TRY_RC_MODE(RC_MODE_CQP, 1); + + if (ctx->codec->flags & FLAG_CONSTANT_QUALITY_ONLY) + TRY_RC_MODE(RC_MODE_CQP, 1); + + if (avctx->flags & AV_CODEC_FLAG_QSCALE) + TRY_RC_MODE(RC_MODE_CQP, 1); + + if (avctx->bit_rate > 0 && avctx->global_quality > 0) + TRY_RC_MODE(RC_MODE_QVBR, 0); + + if (avctx->global_quality > 0) { + TRY_RC_MODE(RC_MODE_CQP, 0); + } + + if (avctx->bit_rate > 0 && avctx->rc_max_rate == avctx->bit_rate) + TRY_RC_MODE(RC_MODE_CBR, 0); + + if (avctx->bit_rate > 0) { + TRY_RC_MODE(RC_MODE_VBR, 0); + TRY_RC_MODE(RC_MODE_CBR, 0); + } else { + TRY_RC_MODE(RC_MODE_CQP, 0); + } + + av_log(avctx, AV_LOG_ERROR, "Driver does not support any " + "RC mode compatible with selected options.\n"); + return AVERROR(EINVAL); + +rc_mode_found: + if (rc_mode->bitrate) { + if (avctx->bit_rate <= 0) { + av_log(avctx, AV_LOG_ERROR, "Bitrate must be set for %s " + "RC mode.\n", rc_mode->name); + return AVERROR(EINVAL); + } + + if (rc_mode->maxrate) { + if (avctx->rc_max_rate > 0) { + if (avctx->rc_max_rate < avctx->bit_rate) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitrate settings: " + "bitrate (%"PRId64") must not be greater than " + "maxrate (%"PRId64").\n", avctx->bit_rate, + avctx->rc_max_rate); + return AVERROR(EINVAL); + } + rc_target_bitrate = avctx->bit_rate; + rc_peak_bitrate = avctx->rc_max_rate; + } else { + // We only have a target bitrate, but this mode requires + // that a maximum rate be supplied as well. Since the + // user does not want this to be a constraint, arbitrarily + // pick a maximum rate of double the target rate. + rc_target_bitrate = avctx->bit_rate; + rc_peak_bitrate = 2 * avctx->bit_rate; + } + } else { + if (avctx->rc_max_rate > avctx->bit_rate) { + av_log(avctx, AV_LOG_WARNING, "Max bitrate is ignored " + "in %s RC mode.\n", rc_mode->name); + } + rc_target_bitrate = avctx->bit_rate; + rc_peak_bitrate = 0; + } + } else { + rc_target_bitrate = 0; + rc_peak_bitrate = 0; + } + + if (rc_mode->quality) { + if (ctx->explicit_qp) { + rc_quality = ctx->explicit_qp; + } else if (avctx->global_quality > 0) { + if (avctx->flags & AV_CODEC_FLAG_QSCALE) + rc_quality = avctx->global_quality / FF_QP2LAMBDA; + else + rc_quality = avctx->global_quality; + } else { + rc_quality = ctx->codec->default_quality; + av_log(avctx, AV_LOG_WARNING, "No quality level set; " + "using default (%d).\n", rc_quality); + } + } else { + rc_quality = 0; + } + + if (rc_mode->hrd) { + if (avctx->rc_buffer_size) + hrd_buffer_size = avctx->rc_buffer_size; + else if (avctx->rc_max_rate > 0) + hrd_buffer_size = avctx->rc_max_rate; + else + hrd_buffer_size = avctx->bit_rate; + if (avctx->rc_initial_buffer_occupancy) { + if (avctx->rc_initial_buffer_occupancy > hrd_buffer_size) { + av_log(avctx, AV_LOG_ERROR, "Invalid RC buffer settings: " + "must have initial buffer size (%d) <= " + "buffer size (%"PRId64").\n", + avctx->rc_initial_buffer_occupancy, hrd_buffer_size); + return AVERROR(EINVAL); + } + hrd_initial_buffer_fullness = avctx->rc_initial_buffer_occupancy; + } else { + hrd_initial_buffer_fullness = hrd_buffer_size * 3 / 4; + } + } else { + if (avctx->rc_buffer_size || avctx->rc_initial_buffer_occupancy) { + av_log(avctx, AV_LOG_WARNING, "Buffering settings are ignored " + "in %s RC mode.\n", rc_mode->name); + } + + hrd_buffer_size = 0; + hrd_initial_buffer_fullness = 0; + } + + if (rc_target_bitrate > UINT32_MAX || + hrd_buffer_size > UINT32_MAX || + hrd_initial_buffer_fullness > UINT32_MAX) { + av_log(avctx, AV_LOG_ERROR, "RC parameters of 2^32 or " + "greater are not supported by D3D12.\n"); + return AVERROR(EINVAL); + } + + ctx->rc_quality = rc_quality; + + av_log(avctx, AV_LOG_VERBOSE, "RC mode: %s.\n", rc_mode->name); + + if (rc_mode->quality) + av_log(avctx, AV_LOG_VERBOSE, "RC quality: %d.\n", rc_quality); + + if (rc_mode->hrd) { + av_log(avctx, AV_LOG_VERBOSE, "RC buffer: %"PRId64" bits, " + "initial fullness %"PRId64" bits.\n", + hrd_buffer_size, hrd_initial_buffer_fullness); + } + + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) + av_reduce(&fr_num, &fr_den, + avctx->framerate.num, avctx->framerate.den, 65535); + else + av_reduce(&fr_num, &fr_den, + avctx->time_base.den, avctx->time_base.num, 65535); + + av_log(avctx, AV_LOG_VERBOSE, "RC framerate: %d/%d (%.2f fps).\n", + fr_num, fr_den, (double)fr_num / fr_den); + + ctx->rc.Flags = D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_NONE; + ctx->rc.TargetFrameRate.Numerator = fr_num; + ctx->rc.TargetFrameRate.Denominator = fr_den; + ctx->rc.Mode = rc_mode->d3d12_mode; + + switch (rc_mode->mode) { + case RC_MODE_CQP: + // cqp ConfigParams will be updated in ctx->codec->configure. + break; + + case RC_MODE_CBR: + D3D12_VIDEO_ENCODER_RATE_CONTROL_CBR *cbr_ctl; + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_CBR); + cbr_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!cbr_ctl) + return AVERROR(ENOMEM); + + cbr_ctl->TargetBitRate = rc_target_bitrate; + cbr_ctl->VBVCapacity = hrd_buffer_size; + cbr_ctl->InitialVBVFullness = hrd_initial_buffer_fullness; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_VBV_SIZES; + + if (avctx->qmin > 0 || avctx->qmax > 0) { + cbr_ctl->MinQP = avctx->qmin; + cbr_ctl->MaxQP = avctx->qmax; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_QP_RANGE; + } + + ctx->rc.ConfigParams.pConfiguration_CBR = cbr_ctl; + break; + + case RC_MODE_VBR: + D3D12_VIDEO_ENCODER_RATE_CONTROL_VBR *vbr_ctl; + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_VBR); + vbr_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!vbr_ctl) + return AVERROR(ENOMEM); + + vbr_ctl->TargetAvgBitRate = rc_target_bitrate; + vbr_ctl->PeakBitRate = rc_peak_bitrate; + vbr_ctl->VBVCapacity = hrd_buffer_size; + vbr_ctl->InitialVBVFullness = hrd_initial_buffer_fullness; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_VBV_SIZES; + + if (avctx->qmin > 0 || avctx->qmax > 0) { + vbr_ctl->MinQP = avctx->qmin; + vbr_ctl->MaxQP = avctx->qmax; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_QP_RANGE; + } + + ctx->rc.ConfigParams.pConfiguration_VBR = vbr_ctl; + break; + + case RC_MODE_QVBR: + D3D12_VIDEO_ENCODER_RATE_CONTROL_QVBR *qvbr_ctl; + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_QVBR); + qvbr_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!qvbr_ctl) + return AVERROR(ENOMEM); + + qvbr_ctl->TargetAvgBitRate = rc_target_bitrate; + qvbr_ctl->PeakBitRate = rc_peak_bitrate; + qvbr_ctl->ConstantQualityTarget = rc_quality; + + if (avctx->qmin > 0 || avctx->qmax > 0) { + qvbr_ctl->MinQP = avctx->qmin; + qvbr_ctl->MaxQP = avctx->qmax; + ctx->rc.Flags |= D3D12_VIDEO_ENCODER_RATE_CONTROL_FLAG_ENABLE_QP_RANGE; + } + + ctx->rc.ConfigParams.pConfiguration_QVBR = qvbr_ctl; + break; + + default: + break; + } + return 0; +} + +static int d3d12va_encode_init_gop_structure(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + uint32_t ref_l0, ref_l1; + int err; + HRESULT hr; + D3D12_FEATURE_DATA_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT support; + union { + D3D12_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT_H264 h264; + D3D12_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT_HEVC hevc; + } codec_support; + + support.NodeIndex = 0; + support.Codec = ctx->codec->d3d12_codec; + support.Profile = ctx->profile->d3d12_profile; + + switch (ctx->codec->d3d12_codec) { + case D3D12_VIDEO_ENCODER_CODEC_H264: + support.PictureSupport.DataSize = sizeof(codec_support.h264); + support.PictureSupport.pH264Support = &codec_support.h264; + break; + + case D3D12_VIDEO_ENCODER_CODEC_HEVC: + support.PictureSupport.DataSize = sizeof(codec_support.hevc); + support.PictureSupport.pHEVCSupport = &codec_support.hevc; + break; + + default: + av_assert0(0); + } + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_ENCODER_CODEC_PICTURE_CONTROL_SUPPORT, + &support, sizeof(support)); + if (FAILED(hr)) + return AVERROR(EINVAL); + + if (support.IsSupported) { + switch (ctx->codec->d3d12_codec) { + case D3D12_VIDEO_ENCODER_CODEC_H264: + ref_l0 = FFMIN(support.PictureSupport.pH264Support->MaxL0ReferencesForP, + support.PictureSupport.pH264Support->MaxL1ReferencesForB); + ref_l1 = support.PictureSupport.pH264Support->MaxL1ReferencesForB; + break; + + case D3D12_VIDEO_ENCODER_CODEC_HEVC: + ref_l0 = FFMIN(support.PictureSupport.pHEVCSupport->MaxL0ReferencesForP, + support.PictureSupport.pHEVCSupport->MaxL1ReferencesForB); + ref_l1 = support.PictureSupport.pHEVCSupport->MaxL1ReferencesForB; + break; + + default: + av_assert0(0); + } + } else { + ref_l0 = ref_l1 = 0; + } + + if (ref_l0 > 0 && ref_l1 > 0 && ctx->bi_not_empty) { + base_ctx->p_to_gpb = 1; + av_log(avctx, AV_LOG_VERBOSE, "Driver does not support P-frames, " + "replacing them with B-frames.\n"); + } + + err = ff_hw_base_init_gop_structure(avctx, ref_l0, ref_l1, ctx->codec->flags, 0); + if (err < 0) + return err; + + return 0; +} + +static int d3d12va_create_encoder(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + AVD3D12VAFramesContext *frames_hwctx = base_ctx->input_frames->hwctx; + HRESULT hr; + + D3D12_VIDEO_ENCODER_DESC desc = { + .NodeMask = 0, + .Flags = D3D12_VIDEO_ENCODER_FLAG_NONE, + .EncodeCodec = ctx->codec->d3d12_codec, + .EncodeProfile = ctx->profile->d3d12_profile, + .InputFormat = frames_hwctx->format, + .CodecConfiguration = ctx->codec_conf, + .MaxMotionEstimationPrecision = D3D12_VIDEO_ENCODER_MOTION_ESTIMATION_PRECISION_MODE_MAXIMUM, + }; + + hr = ID3D12VideoDevice3_CreateVideoEncoder(ctx->video_device3, &desc, &IID_ID3D12VideoEncoder, + (void **)&ctx->encoder); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create encoder.\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static int d3d12va_create_encoder_heap(AVCodecContext* avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + HRESULT hr; + + D3D12_VIDEO_ENCODER_HEAP_DESC desc = { + .NodeMask = 0, + .Flags = D3D12_VIDEO_ENCODER_FLAG_NONE, + .EncodeCodec = ctx->codec->d3d12_codec, + .EncodeProfile = ctx->profile->d3d12_profile, + .EncodeLevel = ctx->level, + .ResolutionsListCount = 1, + .pResolutionList = &ctx->resolution, + }; + + hr = ID3D12VideoDevice3_CreateVideoEncoderHeap(ctx->video_device3, &desc, + &IID_ID3D12VideoEncoderHeap, (void **)&ctx->encoder_heap); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create encoder heap.\n"); + return AVERROR(EINVAL); + } + + return 0; +} + +static void d3d12va_encode_free_buffer(void *opaque, uint8_t *data) +{ + ID3D12Resource *pResource; + + pResource = (ID3D12Resource *)data; + D3D12_OBJECT_RELEASE(pResource); +} + +static AVBufferRef *d3d12va_encode_alloc_output_buffer(void *opaque, size_t size) +{ + AVCodecContext *avctx = opaque; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + ID3D12Resource *pResource = NULL; + HRESULT hr; + AVBufferRef *ref; + D3D12_HEAP_PROPERTIES heap_props; + D3D12_HEAP_TYPE heap_type = D3D12_HEAP_TYPE_READBACK; + + D3D12_RESOURCE_DESC desc = { + .Dimension = D3D12_RESOURCE_DIMENSION_BUFFER, + .Alignment = 0, + .Width = FFALIGN(3 * base_ctx->surface_width * base_ctx->surface_height + (1 << 16), + D3D12_TEXTURE_DATA_PLACEMENT_ALIGNMENT), + .Height = 1, + .DepthOrArraySize = 1, + .MipLevels = 1, + .Format = DXGI_FORMAT_UNKNOWN, + .SampleDesc = { .Count = 1, .Quality = 0 }, + .Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR, + .Flags = D3D12_RESOURCE_FLAG_NONE, + }; + + ctx->hwctx->device->lpVtbl->GetCustomHeapProperties(ctx->hwctx->device, &heap_props, 0, heap_type); + + hr = ID3D12Device_CreateCommittedResource(ctx->hwctx->device, &heap_props, D3D12_HEAP_FLAG_NONE, + &desc, D3D12_RESOURCE_STATE_COMMON, NULL, &IID_ID3D12Resource, + (void **)&pResource); + + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create d3d12 buffer.\n"); + return NULL; + } + + ref = av_buffer_create((uint8_t *)(uintptr_t)pResource, + sizeof(pResource), + &d3d12va_encode_free_buffer, + avctx, AV_BUFFER_FLAG_READONLY); + if (!ref) { + D3D12_OBJECT_RELEASE(pResource); + return NULL; + } + + return ref; +} + +static int d3d12va_encode_prepare_output_buffers(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + AVD3D12VAFramesContext *frames_ctx = base_ctx->input_frames->hwctx; + HRESULT hr; + + ctx->req.NodeIndex = 0; + ctx->req.Codec = ctx->codec->d3d12_codec; + ctx->req.Profile = ctx->profile->d3d12_profile; + ctx->req.InputFormat = frames_ctx->format; + ctx->req.PictureTargetResolution = ctx->resolution; + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, + D3D12_FEATURE_VIDEO_ENCODER_RESOURCE_REQUIREMENTS, + &ctx->req, sizeof(ctx->req)); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to check encoder resource requirements support.\n"); + return AVERROR(EINVAL); + } + + if (!ctx->req.IsSupported) { + av_log(avctx, AV_LOG_ERROR, "Encoder resource requirements unsupported.\n"); + return AVERROR(EINVAL); + } + + ctx->output_buffer_pool = av_buffer_pool_init2(sizeof(ID3D12Resource *), avctx, + &d3d12va_encode_alloc_output_buffer, NULL); + if (!ctx->output_buffer_pool) + return AVERROR(ENOMEM); + + return 0; +} + +static int d3d12va_encode_create_command_objects(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + ID3D12CommandAllocator *command_allocator = NULL; + int err; + HRESULT hr; + + D3D12_COMMAND_QUEUE_DESC queue_desc = { + .Type = D3D12_COMMAND_LIST_TYPE_VIDEO_ENCODE, + .Priority = 0, + .Flags = D3D12_COMMAND_QUEUE_FLAG_NONE, + .NodeMask = 0, + }; + + ctx->allocator_queue = av_fifo_alloc2(D3D12VA_VIDEO_ENC_ASYNC_DEPTH, + sizeof(CommandAllocator), AV_FIFO_FLAG_AUTO_GROW); + if (!ctx->allocator_queue) + return AVERROR(ENOMEM); + + hr = ID3D12Device_CreateFence(ctx->hwctx->device, 0, D3D12_FENCE_FLAG_NONE, + &IID_ID3D12Fence, (void **)&ctx->sync_ctx.fence); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create fence(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + ctx->sync_ctx.event = CreateEvent(NULL, FALSE, FALSE, NULL); + if (!ctx->sync_ctx.event) + goto fail; + + err = d3d12va_get_valid_command_allocator(avctx, &command_allocator); + if (err < 0) + goto fail; + + hr = ID3D12Device_CreateCommandQueue(ctx->hwctx->device, &queue_desc, + &IID_ID3D12CommandQueue, (void **)&ctx->command_queue); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create command queue(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12Device_CreateCommandList(ctx->hwctx->device, 0, queue_desc.Type, + command_allocator, NULL, &IID_ID3D12CommandList, + (void **)&ctx->command_list); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to create command list(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12VideoEncodeCommandList2_Close(ctx->command_list); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to close the command list(%lx)\n", (long)hr); + err = AVERROR_UNKNOWN; + goto fail; + } + + ID3D12CommandQueue_ExecuteCommandLists(ctx->command_queue, 1, (ID3D12CommandList **)&ctx->command_list); + + err = d3d12va_sync_with_gpu(avctx); + if (err < 0) + goto fail; + + err = d3d12va_discard_command_allocator(avctx, command_allocator, ctx->sync_ctx.fence_value); + if (err < 0) + goto fail; + + return 0; + +fail: + D3D12_OBJECT_RELEASE(command_allocator); + return err; +} + +static int d3d12va_encode_create_recon_frames(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + AVD3D12VAFramesContext *hwctx; + enum AVPixelFormat recon_format; + int err; + + err = ff_hw_base_get_recon_format(avctx, NULL, &recon_format); + if (err < 0) + return err; + + base_ctx->recon_frames_ref = av_hwframe_ctx_alloc(base_ctx->device_ref); + if (!base_ctx->recon_frames_ref) + return AVERROR(ENOMEM); + + base_ctx->recon_frames = (AVHWFramesContext *)base_ctx->recon_frames_ref->data; + hwctx = (AVD3D12VAFramesContext *)base_ctx->recon_frames->hwctx; + + base_ctx->recon_frames->format = AV_PIX_FMT_D3D12; + base_ctx->recon_frames->sw_format = recon_format; + base_ctx->recon_frames->width = base_ctx->surface_width; + base_ctx->recon_frames->height = base_ctx->surface_height; + + hwctx->flags = D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY | + D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE; + + err = av_hwframe_ctx_init(base_ctx->recon_frames_ref); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialise reconstructed " + "frame context: %d.\n", err); + return err; + } + + return 0; +} + +static const HWEncodePictureOperation d3d12va_type = { + .alloc = &d3d12va_encode_alloc, + + .issue = &d3d12va_encode_issue, + + .output = &d3d12va_encode_output, + + .free = &d3d12va_encode_free, +}; + +int ff_d3d12va_encode_init(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12_FEATURE_DATA_VIDEO_FEATURE_AREA_SUPPORT support = { 0 }; + int err; + HRESULT hr; + + err = ff_hw_base_encode_init(avctx); + if (err < 0) + goto fail; + + base_ctx->op = &d3d12va_type; + + ctx->hwctx = base_ctx->device->hwctx; + + ctx->resolution.Width = base_ctx->input_frames->width; + ctx->resolution.Height = base_ctx->input_frames->height; + + hr = ID3D12Device_QueryInterface(ctx->hwctx->device, &IID_ID3D12Device3, (void **)&ctx->device3); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "ID3D12Device3 interface is not supported.\n"); + err = AVERROR_UNKNOWN; + goto fail; + } + + hr = ID3D12Device3_QueryInterface(ctx->device3, &IID_ID3D12VideoDevice3, (void **)&ctx->video_device3); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "ID3D12VideoDevice3 interface is not supported.\n"); + err = AVERROR_UNKNOWN; + goto fail; + } + + if (FAILED(ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_FEATURE_AREA_SUPPORT, + &support, sizeof(support))) && !support.VideoEncodeSupport) { + av_log(avctx, AV_LOG_ERROR, "D3D12 video device has no video encoder support.\n"); + err = AVERROR(EINVAL); + goto fail; + } + + err = d3d12va_encode_set_profile(avctx); + if (err < 0) + goto fail; + + err = d3d12va_encode_init_rate_control(avctx); + if (err < 0) + goto fail; + + if (ctx->codec->get_encoder_caps) { + err = ctx->codec->get_encoder_caps(avctx); + if (err < 0) + goto fail; + } + + err = d3d12va_encode_init_gop_structure(avctx); + if (err < 0) + goto fail; + + if (!(ctx->codec->flags & FLAG_SLICE_CONTROL) && avctx->slices > 0) { + av_log(avctx, AV_LOG_WARNING, "Multiple slices were requested " + "but this codec does not support controlling slices.\n"); + } + + err = d3d12va_encode_create_command_objects(avctx); + if (err < 0) + goto fail; + + err = d3d12va_encode_create_recon_frames(avctx); + if (err < 0) + goto fail; + + err = d3d12va_encode_prepare_output_buffers(avctx); + if (err < 0) + goto fail; + + if (ctx->codec->configure) { + err = ctx->codec->configure(avctx); + if (err < 0) + goto fail; + } + + if (ctx->codec->init_sequence_params) { + err = ctx->codec->init_sequence_params(avctx); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Codec sequence initialisation " + "failed: %d.\n", err); + goto fail; + } + } + + if (ctx->codec->set_level) { + err = ctx->codec->set_level(avctx); + if (err < 0) + goto fail; + } + + base_ctx->output_delay = base_ctx->b_per_p; + base_ctx->decode_delay = base_ctx->max_b_depth; + + err = d3d12va_create_encoder(avctx); + if (err < 0) + goto fail; + + err = d3d12va_create_encoder_heap(avctx); + if (err < 0) + goto fail; + + base_ctx->async_encode = 1; + base_ctx->encode_fifo = av_fifo_alloc2(base_ctx->async_depth, + sizeof(D3D12VAEncodePicture *), 0); + if (!base_ctx->encode_fifo) + return AVERROR(ENOMEM); + + return 0; + +fail: + return err; +} + +int ff_d3d12va_encode_close(AVCodecContext *avctx) +{ + int num_allocator = 0; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + HWBaseEncodePicture *pic, *next; + CommandAllocator allocator; + + if (!base_ctx->frame) + return 0; + + for (pic = base_ctx->pic_start; pic; pic = next) { + next = pic->next; + d3d12va_encode_free(avctx, pic); + } + + d3d12va_encode_free_rc_params(avctx); + + av_buffer_pool_uninit(&ctx->output_buffer_pool); + + D3D12_OBJECT_RELEASE(ctx->command_list); + D3D12_OBJECT_RELEASE(ctx->command_queue); + + if (ctx->allocator_queue) { + while (av_fifo_read(ctx->allocator_queue, &allocator, 1) >= 0) { + num_allocator++; + D3D12_OBJECT_RELEASE(allocator.command_allocator); + } + + av_log(avctx, AV_LOG_VERBOSE, "Total number of command allocators reused: %d\n", num_allocator); + } + + av_fifo_freep2(&ctx->allocator_queue); + + D3D12_OBJECT_RELEASE(ctx->sync_ctx.fence); + if (ctx->sync_ctx.event) + CloseHandle(ctx->sync_ctx.event); + + D3D12_OBJECT_RELEASE(ctx->encoder_heap); + D3D12_OBJECT_RELEASE(ctx->encoder); + D3D12_OBJECT_RELEASE(ctx->video_device3); + D3D12_OBJECT_RELEASE(ctx->device3); + + ff_hw_base_encode_close(avctx); + + return 0; +} diff --git a/libavcodec/d3d12va_encode.h b/libavcodec/d3d12va_encode.h new file mode 100644 index 0000000000..510c36a5eb --- /dev/null +++ b/libavcodec/d3d12va_encode.h @@ -0,0 +1,334 @@ +/* + * Direct3D 12 HW acceleration video encoder + * + * Copyright (c) 2024 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_D3D12VA_ENCODE_H +#define AVCODEC_D3D12VA_ENCODE_H + +#include "libavutil/fifo.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_d3d12va_internal.h" +#include "libavutil/hwcontext_d3d12va.h" +#include "avcodec.h" +#include "internal.h" +#include "hwconfig.h" +#include "hw_base_encode.h" + +struct D3D12VAEncodeType; + +extern const AVCodecHWConfigInternal *const ff_d3d12va_encode_hw_configs[]; + +#define MAX_PARAM_BUFFER_SIZE 4096 +#define D3D12VA_VIDEO_ENC_ASYNC_DEPTH 8 + +typedef struct D3D12VAEncodePicture { + HWBaseEncodePicture base; + + int header_size; + + AVD3D12VAFrame *input_surface; + AVD3D12VAFrame *recon_surface; + + AVBufferRef *output_buffer_ref; + ID3D12Resource *output_buffer; + + ID3D12Resource *encoded_metadata; + ID3D12Resource *resolved_metadata; + + D3D12_VIDEO_ENCODER_PICTURE_CONTROL_CODEC_DATA pic_ctl; + + int fence_value; +} D3D12VAEncodePicture; + +typedef struct D3D12VAEncodeProfile { + /** + * lavc profile value (AV_PROFILE_*). + */ + int av_profile; + + /** + * Supported bit depth. + */ + int depth; + + /** + * Number of components. + */ + int nb_components; + + /** + * Chroma subsampling in width dimension. + */ + int log2_chroma_w; + + /** + * Chroma subsampling in height dimension. + */ + int log2_chroma_h; + + /** + * D3D12 profile value. + */ + D3D12_VIDEO_ENCODER_PROFILE_DESC d3d12_profile; +} D3D12VAEncodeProfile; + +enum { + RC_MODE_AUTO, + RC_MODE_CQP, + RC_MODE_CBR, + RC_MODE_VBR, + RC_MODE_QVBR, + RC_MODE_MAX = RC_MODE_QVBR, +}; + + +typedef struct D3D12VAEncodeRCMode { + /** + * Mode from above enum (RC_MODE_*). + */ + int mode; + + /** + * Name. + * + */ + const char *name; + + /** + * Uses bitrate parameters. + * + */ + int bitrate; + + /** + * Supports maxrate distinct from bitrate. + * + */ + int maxrate; + + /** + * Uses quality value. + * + */ + int quality; + + /** + * Supports HRD/VBV parameters. + * + */ + int hrd; + + /** + * D3D12 mode value. + */ + D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE d3d12_mode; +} D3D12VAEncodeRCMode; + +typedef struct D3D12VAEncodeContext { + HWBaseEncodeContext base; + + /** + * Codec-specific hooks. + */ + const struct D3D12VAEncodeType *codec; + + /** + * Explicitly set RC mode (otherwise attempt to pick from + * available modes). + */ + int explicit_rc_mode; + + /** + * Explicitly-set QP, for use with the "qp" options. + * (Forces CQP mode when set, overriding everything else.) + */ + int explicit_qp; + + /** + * RC quality level - meaning depends on codec and RC mode. + * In CQP mode this sets the fixed quantiser value. + */ + int rc_quality; + + /** + * Chosen encoding profile details. + */ + const D3D12VAEncodeProfile *profile; + + AVD3D12VADeviceContext *hwctx; + + /** + * ID3D12Device3 interface. + */ + ID3D12Device3 *device3; + + /** + * ID3D12VideoDevice3 interface. + */ + ID3D12VideoDevice3 *video_device3; + + /** + * Pool of (reusable) bitstream output buffers. + */ + AVBufferPool *output_buffer_pool; + + /** + * D3D12 video encoder. + */ + AVBufferRef *encoder_ref; + + ID3D12VideoEncoder *encoder; + + /** + * D3D12 video encoder heap. + */ + ID3D12VideoEncoderHeap *encoder_heap; + + /** + * A cached queue for reusing the D3D12 command allocators. + * + * @see https://learn.microsoft.com/en-us/windows/win32/direct3d12/recording-command-lists-and-bundles#id3d12commandallocator + */ + AVFifo *allocator_queue; + + /** + * D3D12 command queue. + */ + ID3D12CommandQueue *command_queue; + + /** + * D3D12 video encode command list. + */ + ID3D12VideoEncodeCommandList2 *command_list; + + /** + * The sync context used to sync command queue. + */ + AVD3D12VASyncContext sync_ctx; + + /** + * The bi_not_empty feature. + */ + int bi_not_empty; + + /** + * D3D12_FEATURE structures. + */ + D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOURCE_REQUIREMENTS req; + + D3D12_FEATURE_DATA_VIDEO_ENCODER_RESOLUTION_SUPPORT_LIMITS res_limits; + + /** + * D3D12_VIDEO_ENCODER structures. + */ + D3D12_VIDEO_ENCODER_PICTURE_RESOLUTION_DESC resolution; + + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION codec_conf; + + D3D12_VIDEO_ENCODER_RATE_CONTROL rc; + + D3D12_VIDEO_ENCODER_SEQUENCE_GOP_STRUCTURE gop; + + D3D12_VIDEO_ENCODER_LEVEL_SETTING level; +} D3D12VAEncodeContext; + +typedef struct D3D12VAEncodeType { + /** + * List of supported profiles. + */ + const D3D12VAEncodeProfile *profiles; + + /** + * D3D12 codec name. + */ + D3D12_VIDEO_ENCODER_CODEC d3d12_codec; + + /** + * Codec feature flags. + */ + int flags; + + /** + * Default quality for this codec - used as quantiser or RC quality + * factor depending on RC mode. + */ + int default_quality; + + /** + * Query codec configuration and determine encode parameters like + * block sizes for surface alignment and slices. If not set, assume + * that all blocks are 16x16 and that surfaces should be aligned to match + * this. + */ + int (*get_encoder_caps)(AVCodecContext *avctx); + + /** + * Perform any extra codec-specific configuration. + */ + int (*configure)(AVCodecContext *avctx); + + /** + * Set codec-specific level setting. + */ + int (*set_level)(AVCodecContext *avctx); + + /** + * The size of any private data structure associated with each + * picture (can be zero if not required). + */ + size_t picture_priv_data_size; + + /** + * Fill the corresponding parameters. + */ + int (*init_sequence_params)(AVCodecContext *avctx); + + int (*init_picture_params)(AVCodecContext *avctx, + D3D12VAEncodePicture *pic); + + void (*free_picture_params)(D3D12VAEncodePicture *pic); + + /** + * Write the packed header data to the provided buffer. + */ + int (*write_sequence_header)(AVCodecContext *avctx, + char *data, size_t *data_len); +} D3D12VAEncodeType; + +int ff_d3d12va_encode_init(AVCodecContext *avctx); +int ff_d3d12va_encode_close(AVCodecContext *avctx); + +#define D3D12VA_ENCODE_RC_MODE(name, desc) \ + { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_ ## name }, \ + 0, 0, FLAGS, .unit = "rc_mode" } +#define D3D12VA_ENCODE_RC_OPTIONS \ + { "rc_mode",\ + "Set rate control mode", \ + OFFSET(common.explicit_rc_mode), AV_OPT_TYPE_INT, \ + { .i64 = RC_MODE_AUTO }, RC_MODE_AUTO, RC_MODE_MAX, FLAGS, .unit = "rc_mode" }, \ + { "auto", "Choose mode automatically based on other parameters", \ + 0, AV_OPT_TYPE_CONST, { .i64 = RC_MODE_AUTO }, 0, 0, FLAGS, .unit = "rc_mode" }, \ + D3D12VA_ENCODE_RC_MODE(CQP, "Constant-quality"), \ + D3D12VA_ENCODE_RC_MODE(CBR, "Constant-bitrate"), \ + D3D12VA_ENCODE_RC_MODE(VBR, "Variable-bitrate"), \ + D3D12VA_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate") + +#endif /* AVCODEC_D3D12VA_ENCODE_H */ diff --git a/libavcodec/d3d12va_encode_hevc.c b/libavcodec/d3d12va_encode_hevc.c new file mode 100644 index 0000000000..7f7d98db17 --- /dev/null +++ b/libavcodec/d3d12va_encode_hevc.c @@ -0,0 +1,1007 @@ +/* + * Direct3D 12 HW acceleration video encoder + * + * Copyright (c) 2024 Intel Corporation + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ +#include "libavutil/opt.h" +#include "libavutil/common.h" +#include "libavutil/mem.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext_d3d12va_internal.h" + +#include "avcodec.h" +#include "cbs.h" +#include "cbs_h265.h" +#include "h2645data.h" +#include "h265_profile_level.h" +#include "codec_internal.h" +#include "d3d12va_encode.h" + +typedef struct D3D12VAEncodeHEVCPicture { + int pic_order_cnt; + int64_t last_idr_frame; +} D3D12VAEncodeHEVCPicture; + +typedef struct D3D12VAEncodeHEVCContext { + D3D12VAEncodeContext common; + + // User options. + int qp; + int profile; + int tier; + int level; + + // Writer structures. + H265RawVPS raw_vps; + H265RawSPS raw_sps; + H265RawPPS raw_pps; + + CodedBitstreamContext *cbc; + CodedBitstreamFragment current_access_unit; +} D3D12VAEncodeHEVCContext; + +typedef struct D3D12VAEncodeHEVCLevel { + int level; + D3D12_VIDEO_ENCODER_LEVELS_HEVC d3d12_level; +} D3D12VAEncodeHEVCLevel; + +static const D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC hevc_config_support_sets[] = +{ + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 3, + 3, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 0, + 0, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 2, + 2, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 2, + 2, + }, + { + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_NONE, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4, + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32, + 4, + 4, + }, +}; + +static const D3D12VAEncodeHEVCLevel hevc_levels[] = { + { 30, D3D12_VIDEO_ENCODER_LEVELS_HEVC_1 }, + { 60, D3D12_VIDEO_ENCODER_LEVELS_HEVC_2 }, + { 63, D3D12_VIDEO_ENCODER_LEVELS_HEVC_21 }, + { 90, D3D12_VIDEO_ENCODER_LEVELS_HEVC_3 }, + { 93, D3D12_VIDEO_ENCODER_LEVELS_HEVC_31 }, + { 120, D3D12_VIDEO_ENCODER_LEVELS_HEVC_4 }, + { 123, D3D12_VIDEO_ENCODER_LEVELS_HEVC_41 }, + { 150, D3D12_VIDEO_ENCODER_LEVELS_HEVC_5 }, + { 153, D3D12_VIDEO_ENCODER_LEVELS_HEVC_51 }, + { 156, D3D12_VIDEO_ENCODER_LEVELS_HEVC_52 }, + { 180, D3D12_VIDEO_ENCODER_LEVELS_HEVC_6 }, + { 183, D3D12_VIDEO_ENCODER_LEVELS_HEVC_61 }, + { 186, D3D12_VIDEO_ENCODER_LEVELS_HEVC_62 }, +}; + +static const D3D12_VIDEO_ENCODER_PROFILE_HEVC profile_main = D3D12_VIDEO_ENCODER_PROFILE_HEVC_MAIN; +static const D3D12_VIDEO_ENCODER_PROFILE_HEVC profile_main10 = D3D12_VIDEO_ENCODER_PROFILE_HEVC_MAIN10; + +#define D3D_PROFILE_DESC(name) \ + { sizeof(D3D12_VIDEO_ENCODER_PROFILE_HEVC), { .pHEVCProfile = (D3D12_VIDEO_ENCODER_PROFILE_HEVC *)&profile_ ## name } } +static const D3D12VAEncodeProfile d3d12va_encode_hevc_profiles[] = { + { AV_PROFILE_HEVC_MAIN, 8, 3, 1, 1, D3D_PROFILE_DESC(main) }, + { AV_PROFILE_HEVC_MAIN_10, 10, 3, 1, 1, D3D_PROFILE_DESC(main10) }, + { AV_PROFILE_UNKNOWN }, +}; + +static uint8_t d3d12va_encode_hevc_map_cusize(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE cusize) +{ + switch (cusize) { + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_8x8: return 8; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_16x16: return 16; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_32x32: return 32; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_CUSIZE_64x64: return 64; + default: av_assert0(0); + } + return 0; +} + +static uint8_t d3d12va_encode_hevc_map_tusize(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE tusize) +{ + switch (tusize) { + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_4x4: return 4; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_8x8: return 8; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_16x16: return 16; + case D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_TUSIZE_32x32: return 32; + default: av_assert0(0); + } + return 0; +} + +static int d3d12va_encode_hevc_write_access_unit(AVCodecContext *avctx, + char *data, size_t *data_len, + CodedBitstreamFragment *au) +{ + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + int err; + + err = ff_cbs_write_fragment_data(priv->cbc, au); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to write packed header.\n"); + return err; + } + + if (*data_len < 8 * au->data_size - au->data_bit_padding) { + av_log(avctx, AV_LOG_ERROR, "Access unit too large: " + "%zu < %zu.\n", *data_len, + 8 * au->data_size - au->data_bit_padding); + return AVERROR(ENOSPC); + } + + memcpy(data, au->data, au->data_size); + *data_len = 8 * au->data_size - au->data_bit_padding; + + return 0; +} + +static int d3d12va_encode_hevc_add_nal(AVCodecContext *avctx, + CodedBitstreamFragment *au, + void *nal_unit) +{ + H265RawNALUnitHeader *header = nal_unit; + int err; + + err = ff_cbs_insert_unit_content(au, -1, + header->nal_unit_type, nal_unit, NULL); + if (err < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to add NAL unit: " + "type = %d.\n", header->nal_unit_type); + return err; + } + + return 0; +} + +static int d3d12va_encode_hevc_write_sequence_header(AVCodecContext *avctx, + char *data, size_t *data_len) +{ + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + CodedBitstreamFragment *au = &priv->current_access_unit; + int err; + + err = d3d12va_encode_hevc_add_nal(avctx, au, &priv->raw_vps); + if (err < 0) + goto fail; + + err = d3d12va_encode_hevc_add_nal(avctx, au, &priv->raw_sps); + if (err < 0) + goto fail; + + err = d3d12va_encode_hevc_add_nal(avctx, au, &priv->raw_pps); + if (err < 0) + goto fail; + + err = d3d12va_encode_hevc_write_access_unit(avctx, data, data_len, au); +fail: + ff_cbs_fragment_reset(au); + return err; + +} + +static int d3d12va_encode_hevc_init_sequence_params(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + AVD3D12VAFramesContext *hwctx = base_ctx->input_frames->hwctx; + H265RawVPS *vps = &priv->raw_vps; + H265RawSPS *sps = &priv->raw_sps; + H265RawPPS *pps = &priv->raw_pps; + H265RawProfileTierLevel *ptl = &vps->profile_tier_level; + H265RawVUI *vui = &sps->vui; + D3D12_VIDEO_ENCODER_PROFILE_HEVC profile = D3D12_VIDEO_ENCODER_PROFILE_HEVC_MAIN; + D3D12_VIDEO_ENCODER_LEVEL_TIER_CONSTRAINTS_HEVC level = { 0 }; + const AVPixFmtDescriptor *desc; + uint8_t min_cu_size, max_cu_size, min_tu_size, max_tu_size; + int chroma_format, bit_depth; + HRESULT hr; + int i; + + D3D12_FEATURE_DATA_VIDEO_ENCODER_SUPPORT support = { + .NodeIndex = 0, + .Codec = D3D12_VIDEO_ENCODER_CODEC_HEVC, + .InputFormat = hwctx->format, + .RateControl = ctx->rc, + .IntraRefresh = D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE, + .SubregionFrameEncoding = D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME, + .ResolutionsListCount = 1, + .pResolutionList = &ctx->resolution, + .CodecGopSequence = ctx->gop, + .MaxReferenceFramesInDPB = MAX_DPB_SIZE - 1, + .CodecConfiguration = ctx->codec_conf, + .SuggestedProfile.DataSize = sizeof(D3D12_VIDEO_ENCODER_PROFILE_HEVC), + .SuggestedProfile.pHEVCProfile = &profile, + .SuggestedLevel.DataSize = sizeof(D3D12_VIDEO_ENCODER_LEVEL_TIER_CONSTRAINTS_HEVC), + .SuggestedLevel.pHEVCLevelSetting = &level, + .pResolutionDependentSupport = &ctx->res_limits, + }; + + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_ENCODER_SUPPORT, + &support, sizeof(support)); + + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to check encoder support(%lx).\n", (long)hr); + return AVERROR(EINVAL); + } + + if (!(support.SupportFlags & D3D12_VIDEO_ENCODER_SUPPORT_FLAG_GENERAL_SUPPORT_OK)) { + av_log(avctx, AV_LOG_ERROR, "Driver does not support some request features. %#x\n", + support.ValidationFlags); + return AVERROR(EINVAL); + } + + if (support.SupportFlags & D3D12_VIDEO_ENCODER_SUPPORT_FLAG_RECONSTRUCTED_FRAMES_REQUIRE_TEXTURE_ARRAYS) { + av_log(avctx, AV_LOG_ERROR, "D3D12 video encode on this device requires texture array support, " + "but it's not implemented.\n"); + return AVERROR_PATCHWELCOME; + } + + memset(vps, 0, sizeof(*vps)); + memset(sps, 0, sizeof(*sps)); + memset(pps, 0, sizeof(*pps)); + + desc = av_pix_fmt_desc_get(base_ctx->input_frames->sw_format); + av_assert0(desc); + if (desc->nb_components == 1) { + chroma_format = 0; + } else { + if (desc->log2_chroma_w == 1 && desc->log2_chroma_h == 1) { + chroma_format = 1; + } else if (desc->log2_chroma_w == 1 && desc->log2_chroma_h == 0) { + chroma_format = 2; + } else if (desc->log2_chroma_w == 0 && desc->log2_chroma_h == 0) { + chroma_format = 3; + } else { + av_log(avctx, AV_LOG_ERROR, "Chroma format of input pixel format " + "%s is not supported.\n", desc->name); + return AVERROR(EINVAL); + } + } + bit_depth = desc->comp[0].depth; + + min_cu_size = d3d12va_encode_hevc_map_cusize(ctx->codec_conf.pHEVCConfig->MinLumaCodingUnitSize); + max_cu_size = d3d12va_encode_hevc_map_cusize(ctx->codec_conf.pHEVCConfig->MaxLumaCodingUnitSize); + min_tu_size = d3d12va_encode_hevc_map_tusize(ctx->codec_conf.pHEVCConfig->MinLumaTransformUnitSize); + max_tu_size = d3d12va_encode_hevc_map_tusize(ctx->codec_conf.pHEVCConfig->MaxLumaTransformUnitSize); + + // VPS + + vps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_VPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + vps->vps_video_parameter_set_id = 0; + + vps->vps_base_layer_internal_flag = 1; + vps->vps_base_layer_available_flag = 1; + vps->vps_max_layers_minus1 = 0; + vps->vps_max_sub_layers_minus1 = 0; + vps->vps_temporal_id_nesting_flag = 1; + + ptl->general_profile_space = 0; + ptl->general_profile_idc = avctx->profile; + ptl->general_tier_flag = priv->tier; + + ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1; + + ptl->general_progressive_source_flag = 1; + ptl->general_interlaced_source_flag = 0; + ptl->general_non_packed_constraint_flag = 1; + ptl->general_frame_only_constraint_flag = 1; + + ptl->general_max_14bit_constraint_flag = bit_depth <= 14; + ptl->general_max_12bit_constraint_flag = bit_depth <= 12; + ptl->general_max_10bit_constraint_flag = bit_depth <= 10; + ptl->general_max_8bit_constraint_flag = bit_depth == 8; + + ptl->general_max_422chroma_constraint_flag = chroma_format <= 2; + ptl->general_max_420chroma_constraint_flag = chroma_format <= 1; + ptl->general_max_monochrome_constraint_flag = chroma_format == 0; + + ptl->general_intra_constraint_flag = base_ctx->gop_size == 1; + ptl->general_one_picture_only_constraint_flag = 0; + + ptl->general_lower_bit_rate_constraint_flag = 1; + + if (avctx->level != FF_LEVEL_UNKNOWN) { + ptl->general_level_idc = avctx->level; + } else { + const H265LevelDescriptor *level; + + level = ff_h265_guess_level(ptl, avctx->bit_rate, + base_ctx->surface_width, base_ctx->surface_height, + 1, 1, 1, (base_ctx->b_per_p > 0) + 1); + if (level) { + av_log(avctx, AV_LOG_VERBOSE, "Using level %s.\n", level->name); + ptl->general_level_idc = level->level_idc; + } else { + av_log(avctx, AV_LOG_VERBOSE, "Stream will not conform to " + "any normal level; using level 8.5.\n"); + ptl->general_level_idc = 255; + // The tier flag must be set in level 8.5. + ptl->general_tier_flag = 1; + } + avctx->level = ptl->general_level_idc; + } + + vps->vps_sub_layer_ordering_info_present_flag = 0; + vps->vps_max_dec_pic_buffering_minus1[0] = base_ctx->max_b_depth + 1; + vps->vps_max_num_reorder_pics[0] = base_ctx->max_b_depth; + vps->vps_max_latency_increase_plus1[0] = 0; + + vps->vps_max_layer_id = 0; + vps->vps_num_layer_sets_minus1 = 0; + vps->layer_id_included_flag[0][0] = 1; + + vps->vps_timing_info_present_flag = 1; + if (avctx->framerate.num > 0 && avctx->framerate.den > 0) { + vps->vps_num_units_in_tick = avctx->framerate.den; + vps->vps_time_scale = avctx->framerate.num; + vps->vps_poc_proportional_to_timing_flag = 1; + vps->vps_num_ticks_poc_diff_one_minus1 = 0; + } else { + vps->vps_num_units_in_tick = avctx->time_base.num; + vps->vps_time_scale = avctx->time_base.den; + vps->vps_poc_proportional_to_timing_flag = 0; + } + vps->vps_num_hrd_parameters = 0; + + // SPS + + sps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_SPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + sps->sps_video_parameter_set_id = vps->vps_video_parameter_set_id; + + sps->sps_max_sub_layers_minus1 = vps->vps_max_sub_layers_minus1; + sps->sps_temporal_id_nesting_flag = vps->vps_temporal_id_nesting_flag; + + sps->profile_tier_level = vps->profile_tier_level; + + sps->sps_seq_parameter_set_id = 0; + + sps->chroma_format_idc = chroma_format; + sps->separate_colour_plane_flag = 0; + + av_assert0(ctx->res_limits.SubregionBlockPixelsSize % min_cu_size == 0); + + sps->pic_width_in_luma_samples = FFALIGN(base_ctx->surface_width, + ctx->res_limits.SubregionBlockPixelsSize); + sps->pic_height_in_luma_samples = FFALIGN(base_ctx->surface_height, + ctx->res_limits.SubregionBlockPixelsSize); + + if (avctx->width != sps->pic_width_in_luma_samples || + avctx->height != sps->pic_height_in_luma_samples) { + sps->conformance_window_flag = 1; + sps->conf_win_left_offset = 0; + sps->conf_win_right_offset = + (sps->pic_width_in_luma_samples - avctx->width) >> desc->log2_chroma_w; + sps->conf_win_top_offset = 0; + sps->conf_win_bottom_offset = + (sps->pic_height_in_luma_samples - avctx->height) >> desc->log2_chroma_h; + } else { + sps->conformance_window_flag = 0; + } + + sps->bit_depth_luma_minus8 = bit_depth - 8; + sps->bit_depth_chroma_minus8 = bit_depth - 8; + + sps->log2_max_pic_order_cnt_lsb_minus4 = ctx->gop.pHEVCGroupOfPictures->log2_max_pic_order_cnt_lsb_minus4; + + sps->sps_sub_layer_ordering_info_present_flag = + vps->vps_sub_layer_ordering_info_present_flag; + for (i = 0; i <= sps->sps_max_sub_layers_minus1; i++) { + sps->sps_max_dec_pic_buffering_minus1[i] = + vps->vps_max_dec_pic_buffering_minus1[i]; + sps->sps_max_num_reorder_pics[i] = + vps->vps_max_num_reorder_pics[i]; + sps->sps_max_latency_increase_plus1[i] = + vps->vps_max_latency_increase_plus1[i]; + } + + sps->log2_min_luma_coding_block_size_minus3 = (uint8_t)(av_log2(min_cu_size) - 3); + sps->log2_diff_max_min_luma_coding_block_size = (uint8_t)(av_log2(max_cu_size) - av_log2(min_cu_size)); + sps->log2_min_luma_transform_block_size_minus2 = (uint8_t)(av_log2(min_tu_size) - 2); + sps->log2_diff_max_min_luma_transform_block_size = (uint8_t)(av_log2(max_tu_size) - av_log2(min_tu_size)); + + sps->max_transform_hierarchy_depth_inter = ctx->codec_conf.pHEVCConfig->max_transform_hierarchy_depth_inter; + sps->max_transform_hierarchy_depth_intra = ctx->codec_conf.pHEVCConfig->max_transform_hierarchy_depth_intra; + + sps->amp_enabled_flag = !!(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_USE_ASYMETRIC_MOTION_PARTITION); + sps->sample_adaptive_offset_enabled_flag = !!(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_SAO_FILTER); + sps->sps_temporal_mvp_enabled_flag = 0; + sps->pcm_enabled_flag = 0; + + sps->vui_parameters_present_flag = 1; + + if (avctx->sample_aspect_ratio.num != 0 && + avctx->sample_aspect_ratio.den != 0) { + int num, den, i; + av_reduce(&num, &den, avctx->sample_aspect_ratio.num, + avctx->sample_aspect_ratio.den, 65535); + for (i = 0; i < FF_ARRAY_ELEMS(ff_h2645_pixel_aspect); i++) { + if (num == ff_h2645_pixel_aspect[i].num && + den == ff_h2645_pixel_aspect[i].den) { + vui->aspect_ratio_idc = i; + break; + } + } + if (i >= FF_ARRAY_ELEMS(ff_h2645_pixel_aspect)) { + vui->aspect_ratio_idc = 255; + vui->sar_width = num; + vui->sar_height = den; + } + vui->aspect_ratio_info_present_flag = 1; + } + + // Unspecified video format, from table E-2. + vui->video_format = 5; + vui->video_full_range_flag = + avctx->color_range == AVCOL_RANGE_JPEG; + vui->colour_primaries = avctx->color_primaries; + vui->transfer_characteristics = avctx->color_trc; + vui->matrix_coefficients = avctx->colorspace; + if (avctx->color_primaries != AVCOL_PRI_UNSPECIFIED || + avctx->color_trc != AVCOL_TRC_UNSPECIFIED || + avctx->colorspace != AVCOL_SPC_UNSPECIFIED) + vui->colour_description_present_flag = 1; + if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED || + vui->colour_description_present_flag) + vui->video_signal_type_present_flag = 1; + + if (avctx->chroma_sample_location != AVCHROMA_LOC_UNSPECIFIED) { + vui->chroma_loc_info_present_flag = 1; + vui->chroma_sample_loc_type_top_field = + vui->chroma_sample_loc_type_bottom_field = + avctx->chroma_sample_location - 1; + } + + vui->vui_timing_info_present_flag = 1; + vui->vui_num_units_in_tick = vps->vps_num_units_in_tick; + vui->vui_time_scale = vps->vps_time_scale; + vui->vui_poc_proportional_to_timing_flag = vps->vps_poc_proportional_to_timing_flag; + vui->vui_num_ticks_poc_diff_one_minus1 = vps->vps_num_ticks_poc_diff_one_minus1; + vui->vui_hrd_parameters_present_flag = 0; + + vui->bitstream_restriction_flag = 1; + vui->motion_vectors_over_pic_boundaries_flag = 1; + vui->restricted_ref_pic_lists_flag = 1; + vui->max_bytes_per_pic_denom = 0; + vui->max_bits_per_min_cu_denom = 0; + vui->log2_max_mv_length_horizontal = 15; + vui->log2_max_mv_length_vertical = 15; + + // PPS + + pps->nal_unit_header = (H265RawNALUnitHeader) { + .nal_unit_type = HEVC_NAL_PPS, + .nuh_layer_id = 0, + .nuh_temporal_id_plus1 = 1, + }; + + pps->pps_pic_parameter_set_id = 0; + pps->pps_seq_parameter_set_id = sps->sps_seq_parameter_set_id; + + pps->cabac_init_present_flag = 1; + + pps->num_ref_idx_l0_default_active_minus1 = 0; + pps->num_ref_idx_l1_default_active_minus1 = 0; + + pps->init_qp_minus26 = 0; + + pps->transform_skip_enabled_flag = !!(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_TRANSFORM_SKIPPING); + + // cu_qp_delta always required to be 1 in https://github.com/microsoft/DirectX-Specs/blob/master/d3d/D3D12VideoEncoding.md + pps->cu_qp_delta_enabled_flag = 1; + + pps->diff_cu_qp_delta_depth = 0; + + pps->pps_slice_chroma_qp_offsets_present_flag = 1; + + pps->tiles_enabled_flag = 0; // no tiling in D3D12 + + pps->pps_loop_filter_across_slices_enabled_flag = !(ctx->codec_conf.pHEVCConfig->ConfigurationFlags & + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_DISABLE_LOOP_FILTER_ACROSS_SLICES); + pps->deblocking_filter_control_present_flag = 1; + + return 0; +} + +static int d3d12va_encode_hevc_get_encoder_caps(AVCodecContext *avctx) +{ + int i; + HRESULT hr; + uint8_t min_cu_size, max_cu_size; + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC *config; + D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC hevc_caps; + + D3D12_FEATURE_DATA_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT codec_caps = { + .NodeIndex = 0, + .Codec = D3D12_VIDEO_ENCODER_CODEC_HEVC, + .Profile = ctx->profile->d3d12_profile, + .CodecSupportLimits.DataSize = sizeof(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC), + }; + + for (i = 0; i < FF_ARRAY_ELEMS(hevc_config_support_sets); i++) { + hevc_caps = hevc_config_support_sets[i]; + codec_caps.CodecSupportLimits.pHEVCSupport = &hevc_caps; + hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3, D3D12_FEATURE_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT, + &codec_caps, sizeof(codec_caps)); + if (SUCCEEDED(hr) && codec_caps.IsSupported) + break; + } + + if (i == FF_ARRAY_ELEMS(hevc_config_support_sets)) { + av_log(avctx, AV_LOG_ERROR, "Unsupported codec configuration\n"); + return AVERROR(EINVAL); + } + + ctx->codec_conf.DataSize = sizeof(D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC); + ctx->codec_conf.pHEVCConfig = av_mallocz(ctx->codec_conf.DataSize); + if (!ctx->codec_conf.pHEVCConfig) + return AVERROR(ENOMEM); + + config = ctx->codec_conf.pHEVCConfig; + + config->ConfigurationFlags = D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_NONE; + config->MinLumaCodingUnitSize = hevc_caps.MinLumaCodingUnitSize; + config->MaxLumaCodingUnitSize = hevc_caps.MaxLumaCodingUnitSize; + config->MinLumaTransformUnitSize = hevc_caps.MinLumaTransformUnitSize; + config->MaxLumaTransformUnitSize = hevc_caps.MaxLumaTransformUnitSize; + config->max_transform_hierarchy_depth_inter = hevc_caps.max_transform_hierarchy_depth_inter; + config->max_transform_hierarchy_depth_intra = hevc_caps.max_transform_hierarchy_depth_intra; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_ASYMETRIC_MOTION_PARTITION_SUPPORT || + hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_ASYMETRIC_MOTION_PARTITION_REQUIRED) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_USE_ASYMETRIC_MOTION_PARTITION; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_SAO_FILTER_SUPPORT) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_SAO_FILTER; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_DISABLING_LOOP_FILTER_ACROSS_SLICES_SUPPORT) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_DISABLE_LOOP_FILTER_ACROSS_SLICES; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_TRANSFORM_SKIP_SUPPORT) + config->ConfigurationFlags |= D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_HEVC_FLAG_ENABLE_TRANSFORM_SKIPPING; + + if (hevc_caps.SupportFlags & D3D12_VIDEO_ENCODER_CODEC_CONFIGURATION_SUPPORT_HEVC_FLAG_P_FRAMES_IMPLEMENTED_AS_LOW_DELAY_B_FRAMES) + ctx->bi_not_empty = 1; + + // block sizes + min_cu_size = d3d12va_encode_hevc_map_cusize(hevc_caps.MinLumaCodingUnitSize); + max_cu_size = d3d12va_encode_hevc_map_cusize(hevc_caps.MaxLumaCodingUnitSize); + + av_log(avctx, AV_LOG_VERBOSE, "Using CTU size %dx%d, " + "min CB size %dx%d.\n", max_cu_size, max_cu_size, + min_cu_size, min_cu_size); + + base_ctx->surface_width = FFALIGN(avctx->width, min_cu_size); + base_ctx->surface_height = FFALIGN(avctx->height, min_cu_size); + + return 0; +} + +static int d3d12va_encode_hevc_configure(AVCodecContext *avctx) +{ + HWBaseEncodeContext *base_ctx = avctx->priv_data; + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + int fixed_qp_idr, fixed_qp_p, fixed_qp_b; + int err; + + err = ff_cbs_init(&priv->cbc, AV_CODEC_ID_HEVC, avctx); + if (err < 0) + return err; + + // Rate control + if (ctx->rc.Mode == D3D12_VIDEO_ENCODER_RATE_CONTROL_MODE_CQP) { + D3D12_VIDEO_ENCODER_RATE_CONTROL_CQP *cqp_ctl; + fixed_qp_p = av_clip(ctx->rc_quality, 1, 51); + if (avctx->i_quant_factor > 0.0) + fixed_qp_idr = av_clip((avctx->i_quant_factor * fixed_qp_p + + avctx->i_quant_offset) + 0.5, 1, 51); + else + fixed_qp_idr = fixed_qp_p; + if (avctx->b_quant_factor > 0.0) + fixed_qp_b = av_clip((avctx->b_quant_factor * fixed_qp_p + + avctx->b_quant_offset) + 0.5, 1, 51); + else + fixed_qp_b = fixed_qp_p; + + av_log(avctx, AV_LOG_DEBUG, "Using fixed QP = " + "%d / %d / %d for IDR- / P- / B-frames.\n", + fixed_qp_idr, fixed_qp_p, fixed_qp_b); + + ctx->rc.ConfigParams.DataSize = sizeof(D3D12_VIDEO_ENCODER_RATE_CONTROL_CQP); + cqp_ctl = av_mallocz(ctx->rc.ConfigParams.DataSize); + if (!cqp_ctl) + return AVERROR(ENOMEM); + + cqp_ctl->ConstantQP_FullIntracodedFrame = fixed_qp_idr; + cqp_ctl->ConstantQP_InterPredictedFrame_PrevRefOnly = fixed_qp_p; + cqp_ctl->ConstantQP_InterPredictedFrame_BiDirectionalRef = fixed_qp_b; + + ctx->rc.ConfigParams.pConfiguration_CQP = cqp_ctl; + } + + // GOP + ctx->gop.DataSize = sizeof(D3D12_VIDEO_ENCODER_SEQUENCE_GOP_STRUCTURE_HEVC); + ctx->gop.pHEVCGroupOfPictures = av_mallocz(ctx->gop.DataSize); + if (!ctx->gop.pHEVCGroupOfPictures) + return AVERROR(ENOMEM); + + ctx->gop.pHEVCGroupOfPictures->GOPLength = base_ctx->gop_size; + ctx->gop.pHEVCGroupOfPictures->PPicturePeriod = base_ctx->b_per_p + 1; + // Power of 2 + if (base_ctx->gop_size & base_ctx->gop_size - 1 == 0) + ctx->gop.pHEVCGroupOfPictures->log2_max_pic_order_cnt_lsb_minus4 = + FFMAX(av_log2(base_ctx->gop_size) - 4, 0); + else + ctx->gop.pHEVCGroupOfPictures->log2_max_pic_order_cnt_lsb_minus4 = + FFMAX(av_log2(base_ctx->gop_size) - 3, 0); + + return 0; +} + +static int d3d12va_encode_hevc_set_level(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + int i; + + ctx->level.DataSize = sizeof(D3D12_VIDEO_ENCODER_LEVEL_TIER_CONSTRAINTS_HEVC); + ctx->level.pHEVCLevelSetting = av_mallocz(ctx->level.DataSize); + if (!ctx->level.pHEVCLevelSetting) + return AVERROR(ENOMEM); + + for (i = 0; i < FF_ARRAY_ELEMS(hevc_levels); i++) { + if (avctx->level == hevc_levels[i].level) { + ctx->level.pHEVCLevelSetting->Level = hevc_levels[i].d3d12_level; + break; + } + } + + if (i == FF_ARRAY_ELEMS(hevc_levels)) { + av_log(avctx, AV_LOG_ERROR, "Invalid level %d.\n", avctx->level); + return AVERROR(EINVAL); + } + + ctx->level.pHEVCLevelSetting->Tier = priv->raw_vps.profile_tier_level.general_tier_flag == 0 ? + D3D12_VIDEO_ENCODER_TIER_HEVC_MAIN : + D3D12_VIDEO_ENCODER_TIER_HEVC_HIGH; + + return 0; +} + +static void d3d12va_encode_hevc_free_picture_params(D3D12VAEncodePicture *pic) +{ + if (!pic->pic_ctl.pHEVCPicData) + return; + + av_freep(&pic->pic_ctl.pHEVCPicData->pList0ReferenceFrames); + av_freep(&pic->pic_ctl.pHEVCPicData->pList1ReferenceFrames); + av_freep(&pic->pic_ctl.pHEVCPicData->pReferenceFramesReconPictureDescriptors); + av_freep(&pic->pic_ctl.pHEVCPicData); +} + +static int d3d12va_encode_hevc_init_picture_params(AVCodecContext *avctx, + D3D12VAEncodePicture *pic) +{ + HWBaseEncodePicture *base_pic = &pic->base; + D3D12VAEncodeHEVCPicture *hpic = base_pic->priv_data; + HWBaseEncodePicture *prev = base_pic->prev; + D3D12VAEncodeHEVCPicture *hprev = prev ? prev->priv_data : NULL; + D3D12_VIDEO_ENCODER_REFERENCE_PICTURE_DESCRIPTOR_HEVC *pd = NULL; + UINT *ref_list0 = NULL, *ref_list1 = NULL; + int i, idx = 0; + + pic->pic_ctl.DataSize = sizeof(D3D12_VIDEO_ENCODER_PICTURE_CONTROL_CODEC_DATA_HEVC); + pic->pic_ctl.pHEVCPicData = av_mallocz(pic->pic_ctl.DataSize); + if (!pic->pic_ctl.pHEVCPicData) + return AVERROR(ENOMEM); + + if (base_pic->type == PICTURE_TYPE_IDR) { + av_assert0(base_pic->display_order == base_pic->encode_order); + hpic->last_idr_frame = base_pic->display_order; + } else { + av_assert0(prev); + hpic->last_idr_frame = hprev->last_idr_frame; + } + hpic->pic_order_cnt = base_pic->display_order - hpic->last_idr_frame; + + switch(base_pic->type) { + case PICTURE_TYPE_IDR: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_IDR_FRAME; + break; + case PICTURE_TYPE_I: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_I_FRAME; + break; + case PICTURE_TYPE_P: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_P_FRAME; + break; + case PICTURE_TYPE_B: + pic->pic_ctl.pHEVCPicData->FrameType = D3D12_VIDEO_ENCODER_FRAME_TYPE_HEVC_B_FRAME; + break; + default: + av_assert0(0 && "invalid picture type"); + } + + pic->pic_ctl.pHEVCPicData->slice_pic_parameter_set_id = 0; + pic->pic_ctl.pHEVCPicData->PictureOrderCountNumber = hpic->pic_order_cnt; + + if (base_pic->type == PICTURE_TYPE_P || base_pic->type == PICTURE_TYPE_B) { + pd = av_calloc(MAX_PICTURE_REFERENCES, sizeof(*pd)); + if (!pd) + return AVERROR(ENOMEM); + + ref_list0 = av_calloc(MAX_PICTURE_REFERENCES, sizeof(*ref_list0)); + if (!ref_list0) + return AVERROR(ENOMEM); + + pic->pic_ctl.pHEVCPicData->List0ReferenceFramesCount = base_pic->nb_refs[0]; + for (i = 0; i < base_pic->nb_refs[0]; i++) { + HWBaseEncodePicture *ref = base_pic->refs[0][i]; + D3D12VAEncodeHEVCPicture *href; + + av_assert0(ref && ref->encode_order < base_pic->encode_order); + href = ref->priv_data; + + ref_list0[i] = idx; + pd[idx].ReconstructedPictureResourceIndex = idx; + pd[idx].IsRefUsedByCurrentPic = TRUE; + pd[idx].PictureOrderCountNumber = href->pic_order_cnt; + idx++; + } + } + + if (base_pic->type == PICTURE_TYPE_B) { + ref_list1 = av_calloc(MAX_PICTURE_REFERENCES, sizeof(*ref_list1)); + if (!ref_list1) + return AVERROR(ENOMEM); + + pic->pic_ctl.pHEVCPicData->List1ReferenceFramesCount = base_pic->nb_refs[1]; + for (i = 0; i < base_pic->nb_refs[1]; i++) { + HWBaseEncodePicture *ref = base_pic->refs[1][i]; + D3D12VAEncodeHEVCPicture *href; + + av_assert0(ref && ref->encode_order < base_pic->encode_order); + href = ref->priv_data; + + ref_list1[i] = idx; + pd[idx].ReconstructedPictureResourceIndex = idx; + pd[idx].IsRefUsedByCurrentPic = TRUE; + pd[idx].PictureOrderCountNumber = href->pic_order_cnt; + idx++; + } + } + + pic->pic_ctl.pHEVCPicData->pList0ReferenceFrames = ref_list0; + pic->pic_ctl.pHEVCPicData->pList1ReferenceFrames = ref_list1; + pic->pic_ctl.pHEVCPicData->ReferenceFramesReconPictureDescriptorsCount = idx; + pic->pic_ctl.pHEVCPicData->pReferenceFramesReconPictureDescriptors = pd; + + return 0; +} + +static const D3D12VAEncodeType d3d12va_encode_type_hevc = { + .profiles = d3d12va_encode_hevc_profiles, + + .d3d12_codec = D3D12_VIDEO_ENCODER_CODEC_HEVC, + + .flags = FLAG_B_PICTURES | + FLAG_B_PICTURE_REFERENCES | + FLAG_NON_IDR_KEY_PICTURES, + + .default_quality = 25, + + .get_encoder_caps = &d3d12va_encode_hevc_get_encoder_caps, + + .configure = &d3d12va_encode_hevc_configure, + + .set_level = &d3d12va_encode_hevc_set_level, + + .picture_priv_data_size = sizeof(D3D12VAEncodeHEVCPicture), + + .init_sequence_params = &d3d12va_encode_hevc_init_sequence_params, + + .init_picture_params = &d3d12va_encode_hevc_init_picture_params, + + .free_picture_params = &d3d12va_encode_hevc_free_picture_params, + + .write_sequence_header = &d3d12va_encode_hevc_write_sequence_header, +}; + +static int d3d12va_encode_hevc_init(AVCodecContext *avctx) +{ + D3D12VAEncodeContext *ctx = avctx->priv_data; + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + + ctx->codec = &d3d12va_encode_type_hevc; + + if (avctx->profile == AV_PROFILE_UNKNOWN) + avctx->profile = priv->profile; + if (avctx->level == FF_LEVEL_UNKNOWN) + avctx->level = priv->level; + + if (avctx->level != FF_LEVEL_UNKNOWN && avctx->level & ~0xff) { + av_log(avctx, AV_LOG_ERROR, "Invalid level %d: must fit " + "in 8-bit unsigned integer.\n", avctx->level); + return AVERROR(EINVAL); + } + + if (priv->qp > 0) + ctx->explicit_qp = priv->qp; + + return ff_d3d12va_encode_init(avctx); +} + +static int d3d12va_encode_hevc_close(AVCodecContext *avctx) +{ + D3D12VAEncodeHEVCContext *priv = avctx->priv_data; + + ff_cbs_fragment_free(&priv->current_access_unit); + ff_cbs_close(&priv->cbc); + + av_freep(&priv->common.codec_conf.pHEVCConfig); + av_freep(&priv->common.gop.pHEVCGroupOfPictures); + av_freep(&priv->common.level.pHEVCLevelSetting); + + return ff_d3d12va_encode_close(avctx); +} + +#define OFFSET(x) offsetof(D3D12VAEncodeHEVCContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM) +static const AVOption d3d12va_encode_hevc_options[] = { + HW_BASE_ENCODE_COMMON_OPTIONS, + D3D12VA_ENCODE_RC_OPTIONS, + + { "qp", "Constant QP (for P-frames; scaled by qfactor/qoffset for I/B)", + OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 52, FLAGS }, + + { "profile", "Set profile (general_profile_idc)", + OFFSET(profile), AV_OPT_TYPE_INT, + { .i64 = AV_PROFILE_UNKNOWN }, AV_PROFILE_UNKNOWN, 0xff, FLAGS, "profile" }, + +#define PROFILE(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, "profile" + { PROFILE("main", AV_PROFILE_HEVC_MAIN) }, + { PROFILE("main10", AV_PROFILE_HEVC_MAIN_10) }, +#undef PROFILE + + { "tier", "Set tier (general_tier_flag)", + OFFSET(tier), AV_OPT_TYPE_INT, + { .i64 = 0 }, 0, 1, FLAGS, "tier" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = 0 }, 0, 0, FLAGS, "tier" }, + { "high", NULL, 0, AV_OPT_TYPE_CONST, + { .i64 = 1 }, 0, 0, FLAGS, "tier" }, + + { "level", "Set level (general_level_idc)", + OFFSET(level), AV_OPT_TYPE_INT, + { .i64 = FF_LEVEL_UNKNOWN }, FF_LEVEL_UNKNOWN, 0xff, FLAGS, "level" }, + +#define LEVEL(name, value) name, NULL, 0, AV_OPT_TYPE_CONST, \ + { .i64 = value }, 0, 0, FLAGS, "level" + { LEVEL("1", 30) }, + { LEVEL("2", 60) }, + { LEVEL("2.1", 63) }, + { LEVEL("3", 90) }, + { LEVEL("3.1", 93) }, + { LEVEL("4", 120) }, + { LEVEL("4.1", 123) }, + { LEVEL("5", 150) }, + { LEVEL("5.1", 153) }, + { LEVEL("5.2", 156) }, + { LEVEL("6", 180) }, + { LEVEL("6.1", 183) }, + { LEVEL("6.2", 186) }, +#undef LEVEL + + { NULL }, +}; + +static const FFCodecDefault d3d12va_encode_hevc_defaults[] = { + { "b", "0" }, + { "bf", "2" }, + { "g", "120" }, + { "i_qfactor", "1" }, + { "i_qoffset", "0" }, + { "b_qfactor", "1" }, + { "b_qoffset", "0" }, + { "qmin", "-1" }, + { "qmax", "-1" }, + { NULL }, +}; + +static const AVClass d3d12va_encode_hevc_class = { + .class_name = "hevc_d3d12va", + .item_name = av_default_item_name, + .option = d3d12va_encode_hevc_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_hevc_d3d12va_encoder = { + .p.name = "hevc_d3d12va", + CODEC_LONG_NAME("D3D12VA hevc encoder"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_HEVC, + .priv_data_size = sizeof(D3D12VAEncodeHEVCContext), + .init = &d3d12va_encode_hevc_init, + FF_CODEC_RECEIVE_PACKET_CB(&ff_hw_base_encode_receive_packet), + .close = &d3d12va_encode_hevc_close, + .p.priv_class = &d3d12va_encode_hevc_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE | + AV_CODEC_CAP_DR1 | AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | + FF_CODEC_CAP_INIT_CLEANUP, + .defaults = d3d12va_encode_hevc_defaults, + .p.pix_fmts = (const enum AVPixelFormat[]) { + AV_PIX_FMT_D3D12, + AV_PIX_FMT_NONE, + }, + .hw_configs = ff_d3d12va_encode_hw_configs, + .p.wrapper_name = "d3d12va", +}; From patchwork Wed May 22 01:26:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wu, Tong1" X-Patchwork-Id: 49128 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:9214:b0:1af:fc2d:ff5a with SMTP id tl20csp82374pzb; Tue, 21 May 2024 18:30:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXf7ndGmD+j52p/Q+/Uu8GR6DXYSJHJWQXBwIMozlJZiefCNXcNyV2ulpRL2mS2HcIJ7qx6yAgWoMMlQbVk3KyoSEQua9l3td2szw== X-Google-Smtp-Source: AGHT+IEPnKoqPf0qkrnExEOd+2Dce/kMLZ2ktSJO4H+RVjU+hNE6DM7yP47VPfmJ7/7T2g0/HVH/ X-Received: by 2002:a17:906:cf83:b0:a5a:5c8f:120d with SMTP id a640c23a62f3a-a5d5fd58885mr846706866b.36.1716341412150; Tue, 21 May 2024 18:30:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716341412; cv=none; d=google.com; s=arc-20160816; b=CQvLb19E6SApcPRJjlYOXNmDk9ov3nmIeRV2Hf15ewJy5XZ6gMEqN6s04WkYbUMRaj 9BUaWbp6zvXtMCaOcL0smq2pLTQ5XW8e7Ydc5U/tOn7ioFUQgNn1T9y7VwqjRgekJFGx NdT6rtV8Bc+qvSoU+TGW+Uwxlm2F3DzUP+QCQ+7FptV1UjomT8KoLB0QHfVqm9HFnmMD 9eBicHOW5o0ZPu3bywoV0kSZqOugTQ2KyxNt1lfNo6x2Bf+9VtbY6WxqO/5wU8sBuk/u SfkpxdCLn+hNUW+GY0Tk6hFhOdt5AocPZgaC7yEAllSkqNXPCxqre+JI6FsM6CNU5H1h Wm5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=yqPTFxhj67B8oPXMxhn8jPeeMILP8f2Kyl94Jo1CXIk=; fh=CX/m9qTuMDwrotqtm4RkAOJT6yXlKL2vcfWDitFPXZs=; b=IGQGcPhQw13zxLnFu0wfyhfvrzsDt6WUZx3vKufHFkAt9u7mW6YIu4MM9PyHgLi6D6 kDzKOjcEr8Q3OebRaNMHSbCKwh73lc8uKlrP86Awdg1lgSkRmf3/EYMh5LTwxIka0L1B HvN6Iy+cYexYt9OnENPtbE4HCymUZq8hpKWojp4AzW6AQef3XMwloef7p0Hmxnryau1b kbLW49zhlRTbPUqUAQUAifu96CGx4DS573YhBj1SfPOUQZ8+wrgr5OcLZZ9N3tRbMuCz HKYUCO6qbS043+nQgZRrsF8Rf+yjjwNNbY8jPaeMeDhZtHwvu63u64X3UhLVTs2sqyVw Po5g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=N2Zt8yNM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a5a179463basi1440731366b.126.2024.05.21.18.30.11; Tue, 21 May 2024 18:30:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=N2Zt8yNM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 38E9068D4A9; Wed, 22 May 2024 04:28:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6F68E68D432 for ; Wed, 22 May 2024 04:28:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1716341299; x=1747877299; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3eTf9yZQee05Owy7Ceb3MKzDKgoBSiNiFSUpGPTdeDg=; b=N2Zt8yNMF/obTHNKgvTm6/Uo2CW4QIR2NFUyRgFLszYYahnjIILvoOX1 3EGPem7hVQ/EWEJB2mBOa1U4nf7UqplxmAA7kM4lXSomSabVWpgDMXxnW Um2we3Qk+9crJUMPPnOsUU+VZt1TQxLD3U+cBS0S1JfULtYlGlnuGmb5e XJL+sy2Ts/69idyZ23P+weiKZaWbsMxQRCFrSeANIzsf+uuz1t755OtQt 8nRNnnehKbhbpScHtbITSgEj0pOoVYDX7p1zMid93fRxftPAX+qQOlqiS /XuSvrt7tQn33D5JunyPnXxaz2vS2Rqy0qyEKqfRnS/NLZuav4KJYln+x A==; X-CSE-ConnectionGUID: 0bE1e1s9Rz2LKzZDQsjDBw== X-CSE-MsgGUID: ST4lnSvmTYSR8lf5hLWb+w== X-IronPort-AV: E=McAfee;i="6600,9927,11079"; a="12797221" X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="12797221" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2024 18:28:14 -0700 X-CSE-ConnectionGUID: gO3QyN1fSvCe9YuzeS6RJg== X-CSE-MsgGUID: rw3GmkGzQruyqxVvhQad7g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,179,1712646000"; d="scan'208";a="33537516" Received: from unknown (HELO localhost.localdomain) ([10.239.160.66]) by orviesa006.jf.intel.com with ESMTP; 21 May 2024 18:28:13 -0700 From: tong1.wu-at-intel.com@ffmpeg.org To: ffmpeg-devel@ffmpeg.org Date: Wed, 22 May 2024 09:26:26 +0800 Message-ID: <20240522012627.1386-13-tong1.wu@intel.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20240522012627.1386-1-tong1.wu@intel.com> References: <20240522012627.1386-1-tong1.wu@intel.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v10 13/13] Changelog: add D3D12VA HEVC encoder changelog X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Tong Wu Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eZ2QzcAd7L+0 From: Tong Wu Signed-off-by: Tong Wu --- Changelog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Changelog b/Changelog index 12770e4296..c5d57b3813 100644 --- a/Changelog +++ b/Changelog @@ -11,7 +11,7 @@ version : - vf_scale2ref deprecated - qsv_params option added for QSV encoders - VVC decoder compatible with DVB test content - +- D3D12VA HEVC encoder version 7.0: - DXV DXT1 encoder