From patchwork Sat Aug 13 21:18:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 37261 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp520109pzi; Sat, 13 Aug 2022 14:18:49 -0700 (PDT) X-Google-Smtp-Source: AA6agR5JFmyqcSI4IQYJn6P4xW97iXPCQQ7cIGv+vo6IqcmtF0bIvI2rTUIyCtvp91vai/5HNh3L X-Received: by 2002:a17:907:7b95:b0:72f:9c64:4061 with SMTP id ne21-20020a1709077b9500b0072f9c644061mr6295629ejc.351.1660425529408; Sat, 13 Aug 2022 14:18:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660425529; cv=none; d=google.com; s=arc-20160816; b=xr42AWZhym3s3R3sZhIQ2nZArFyur48FG2RhOVQXVLiD1mVoLuZ6wf/YzSEdrNC2Xi xsyLNeISrnSrZkzfXC9+usg0NPYrIyqjt+cdCIx8AkHe71K83AREgxkHYO6rnvdewic+ qfONbk+MryGQ2h19nS8mNMhTEmfz68MU/nMu2e0kmAzhg6H7TE3TFfzdsmXpxCA1Y9YZ DMhGJq8LfQsnQTqxrr3sibQy4kRCe5wR+/LujrNQOuLYP8aEukvgCjUuCIkpz1leTVF4 /lEvV1lBfUIYcWhfAq4T2FFd8zuH26n2wZ9ASJXX8S1AnyI13+JPNeqFjsVJSourQyeR ixTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=2TqbVT7PLqpMddk5blfwHrFEKJ9cIz1F1oYFRWg92ek=; b=GodqY7rabTzA1qYs0j1IlyNDKp6GhEwsaCVlVJsdJghZGDmfNejnDfVaPSgc4PpmK8 Nn8VuU8uyZDaWcX4Z47AooqMceMKvXvWq5Ibq/72X49GZaZHwV1pALpzsuabYc6/ZXth aBSWQDhqVnyMCQDKQUfKI9A98vBZilbW/dLyAR29e+13qWL75X5ihJhnInemACmXCLmb FJCBmL1W6W0zKqxqWXXvjCTIYnOAaziiEGJpP9o6tgVzIIifo6r/rqfjUIANq8rhR+sg lgFMyIKR5O+GqDPpD6eHgo615XcfZWJpuBKsRseacLJYeIQ0y6MuiCEqgg8JLQktj4UD y/AA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=svjnD5vU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hc10-20020a170907168a00b0073065f0ddbdsi4929431ejc.599.2022.08.13.14.18.49; Sat, 13 Aug 2022 14:18:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=svjnD5vU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A9FE068B8FD; Sun, 14 Aug 2022 00:18:39 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.overt.org (mail.overt.org [157.230.92.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 341F168B7A7 for ; Sun, 14 Aug 2022 00:18:31 +0300 (EEST) Received: from authenticated-user (mail.overt.org [157.230.92.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id DF16B3F230; Sat, 13 Aug 2022 16:18:29 -0500 (CDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1660425510; bh=crve3e+RgQY3uIZwbX7oGmNnkJNNHGUGpaDrpfKXjmY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=svjnD5vUuCX5JdpZXNCMIrw3bk0qM/2ahrHCi2jU5BL8HVcAA4T9XPKud/WbgWZS/ IY5VIqAvaCbjTYR+7ofjWCqYjgsrNFXCgn/1//p6TCsPxuGKAAyODslKxuLbB/Q9bi 0Ksxgt0afmyoglnL/RQGlmgFFXWe+7r8fd4PUOgeT7ulMBDwF8XCV1eR9I1/u0F2ii KAwa13QiPROZCpYm4IygjNorvkxzjnFX+XHbrC9COAm1+Sh+nHBOPzfqdDFtg+kKqr 7t0UFrON/1iKGmQvW5+1DPcAY/ltTcX9NrtaMBGMgO4yQ/2jmRgj0DsHR4kIw91aPb oChVqwZk8zz2g== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Sat, 13 Aug 2022 14:18:20 -0700 Message-Id: <20220813211821.14916-2-philipl@overt.org> In-Reply-To: <20220813211821.14916-1-philipl@overt.org> References: <20220813211821.14916-1-philipl@overt.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavu/pixfmt: Add Y216, Y410, and Y416 formats X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: cuCvQfbAD97z These are the formats returned by the Intel VAAPI decoder for 12bit 4:2:2, 10bit 4:4:4, and 12bit 4:4:4 respectively. As with the already supported Y210 and YUVA (AVUY) formats, they are the formats Microsoft picked as their preferred 4:2:2 and 4:4:4 video formats, and Intel ran with it. Y216 is simply an extension of Y210 to say all 16bits will be used, and Y416 is a normal looking packed 4 channel format. Y410 is an annoying format that packs three 10bit channels into 32bits with 2bits of alpha. As a result, I had to define Y410 as a bitstream format, even though each pixel is byte-aligned. If it is in-fact possible to define as a normal byte-aligned format, please let me know how. As with VUYA, I have kept the formal definition of Y410 and Y416 as formats with alpha channels to maintain fidelity. The Intel folks say they prefer this and they would rather explicitly use a format defined as not having alpha than to silently drop it. The hardware decoder does at least ensure the alpha channel is set to full opacity. Signed-off-by: Philip Langdale --- libavutil/pixdesc.c | 77 +++++++++++++++++++++++++++++++- libavutil/pixfmt.h | 12 +++++ tests/ref/fate/imgutils | 6 +++ tests/ref/fate/sws-pixdesc-query | 25 +++++++++++ 4 files changed, 119 insertions(+), 1 deletion(-) diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index e078fd5320..12330fd5ea 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2504,6 +2504,81 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { }, .flags = AV_PIX_FMT_FLAG_ALPHA, }, + [AV_PIX_FMT_Y216LE] = { + .name = "y216le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 0, 16 }, /* Y */ + { 0, 8, 2, 0, 16 }, /* U */ + { 0, 8, 6, 0, 16 }, /* V */ + }, + }, + [AV_PIX_FMT_Y216BE] = { + .name = "y216be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 0, 16 }, /* Y */ + { 0, 8, 2, 0, 16 }, /* U */ + { 0, 8, 6, 0, 16 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_BE, + }, + [AV_PIX_FMT_Y410LE] = { + .name = "y410le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y410BE] = { + .name = "y410be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y416LE] = { + .name = "y416le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 0, 16 }, /* Y */ + { 0, 8, 0, 0, 16 }, /* U */ + { 0, 8, 4, 0, 16 }, /* V */ + { 0, 8, 6, 0, 16 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA, + }, + [AV_PIX_FMT_Y416BE] = { + .name = "y416be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 0, 16 }, /* Y */ + { 0, 8, 0, 0, 16 }, /* U */ + { 0, 8, 4, 0, 16 }, /* V */ + { 0, 8, 6, 0, 16 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA, + }, }; static const char * const color_range_names[] = { @@ -2739,7 +2814,7 @@ void ff_check_pixfmt_descriptors(void){ if (!d->name && !d->nb_components && !d->log2_chroma_w && !d->log2_chroma_h && !d->flags) continue; -// av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name); + av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name); av_assert0(d->log2_chroma_w <= 3); av_assert0(d->log2_chroma_h <= 3); av_assert0(d->nb_components <= 4); diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 9d1fdaf82d..aea25ddd02 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -369,6 +369,15 @@ enum AVPixelFormat { AV_PIX_FMT_VUYA, ///< packed VUYA 4:4:4, 32bpp, VUYAVUYA... + AV_PIX_FMT_Y216BE, ///< packed YUV 4:2:2 like YUYV422, 32bpp, big-endian + AV_PIX_FMT_Y216LE, ///< packed YUV 4:2:2 like YUYV422, 32bpp, big-endian + + AV_PIX_FMT_Y410BE, ///< packed AVYU 2:10:10:10, 32bpp, (msb)2A 10V 10Y 10U(lsb), big-endian + AV_PIX_FMT_Y410LE, ///< packed AVYU 2:10:10:10, 32bpp, (msb)2A 10V 10Y 10U(lsb), little-endian + + AV_PIX_FMT_Y416BE, ///< packed AVYU 16:16:16:16, 64bpp, big-endian + AV_PIX_FMT_Y416LE, ///< packed AVYU 16:16:16:16, 64bpp, little-endian + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; @@ -458,6 +467,9 @@ enum AVPixelFormat { #define AV_PIX_FMT_P016 AV_PIX_FMT_NE(P016BE, P016LE) #define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) +#define AV_PIX_FMT_Y216 AV_PIX_FMT_NE(Y216BE, Y216LE) +#define AV_PIX_FMT_Y410 AV_PIX_FMT_NE(Y410BE, Y410LE) +#define AV_PIX_FMT_Y416 AV_PIX_FMT_NE(Y416BE, Y416LE) #define AV_PIX_FMT_X2RGB10 AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE) #define AV_PIX_FMT_X2BGR10 AV_PIX_FMT_NE(X2BGR10BE, X2BGR10LE) diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils index 4ec66febb8..595a9897c6 100644 --- a/tests/ref/fate/imgutils +++ b/tests/ref/fate/imgutils @@ -247,3 +247,9 @@ p216le planes: 2, linesizes: 128 128 0 0, plane_sizes: 6144 6144 p416be planes: 2, linesizes: 128 256 0 0, plane_sizes: 6144 12288 0 0, plane_offsets: 6144 0 0, total_size: 18432 p416le planes: 2, linesizes: 128 256 0 0, plane_sizes: 6144 12288 0 0, plane_offsets: 6144 0 0, total_size: 18432 vuya planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y216be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y216le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y416be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +y416le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query index bd0f1fcb82..40592cfb5f 100644 --- a/tests/ref/fate/sws-pixdesc-query +++ b/tests/ref/fate/sws-pixdesc-query @@ -21,6 +21,10 @@ is16BPS: rgb48le rgba64be rgba64le + y216be + y216le + y416be + y416le ya16be ya16le yuv420p16be @@ -73,6 +77,8 @@ isNBPS: xyz12le y210be y210le + y410be + y410le yuv420p10be yuv420p10le yuv420p12be @@ -161,6 +167,9 @@ isBE: x2rgb10be xyz12be y210be + y216be + y410be + y416be ya16be yuv420p10be yuv420p12be @@ -220,6 +229,12 @@ isYUV: xyz12le y210be y210le + y216be + y216le + y410be + y410le + y416be + y416le ya16be ya16le ya8 @@ -656,6 +671,10 @@ ALPHA: rgba64be rgba64le vuya + y410be + y410le + y416be + y416le ya16be ya16le ya8 @@ -750,6 +769,12 @@ Packed: xyz12le y210be y210le + y216be + y216le + y410be + y410le + y416be + y416le ya16be ya16le ya8 From patchwork Sat Aug 13 21:18:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 37262 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp520146pzi; Sat, 13 Aug 2022 14:18:57 -0700 (PDT) X-Google-Smtp-Source: AA6agR4Kw6TVoaw1ADlgLdYMOkuKkM1ZaoIE358ijCrJ3/wqYHuxmq0tsR01lN7uOvlB8G2gcuef X-Received: by 2002:a17:907:3ea5:b0:730:8b95:dc69 with SMTP id hs37-20020a1709073ea500b007308b95dc69mr6384182ejc.34.1660425537048; Sat, 13 Aug 2022 14:18:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660425537; cv=none; d=google.com; s=arc-20160816; b=f7n7R1DeYu9Uh4fcS6wD8XhoWtWO+3On8ee/xEmAZGNITXZF507tQlk88dsSf0vijw mTLwNrcQfcnHhP3VMUhgvgkCziOx68Ll1r73e9FaGKf00s95Ov+naGVBjmkDp83/7UCP DvHS67cPlmiFiUGxG4Ic++sHvmqh2D7TXO5aHYYrrHF0suKiJkcE6AAmN9WciKTeZ051 MspjgeRti2N30kGKWT6Ij2w69iaZoE7uv0bWighVGmyeVNn1yQNd/fxCcqu1RU5oHWRA wp895vyUxVK1PjThKwbPJ139Z1O6Yt9Qodq1PmnrDzVNyOL1PVRrqqccJcZ9+3jPvTw0 a5IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=1lWMHKdYwe3vfxVT8JXqMV+u9ZhLi3+nmUWHsMpxQ2k=; b=wUyM9zDLWUDV9S/MPqFjyL1V9eJLqVWwwrmuaw1wtmI2yL8HpclRC5w1jOgrOF+44d dZqwkAv62IgSuGbf974zazW9e8DGCt25x9voR2fYh/Adpn2DWbkYRSGPn7X3uagz8eCt F8tj5KUuH/doQVQ/PDUGl0hLz31SRap0OmPHR7itPctv/v8cf9jTOo1oHB9Cuv6FN0h2 iQqtDtn2kfiy/c8M12GUdYNx+mlZFrYRPleFkNxF1q4HWhqb44Uof2dNGz2IJ/1HhwjR PLGC+dAnaUJAkjvnd8CdVr3TS2ePnrFSjgCiNAOjP1z5EaKooVqDF2Ya84nE1TqAhOub UhMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=LMrQBQgO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h19-20020a056402281300b0043def0e25c9si5623197ede.20.2022.08.13.14.18.56; Sat, 13 Aug 2022 14:18:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=LMrQBQgO; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D6B3F68B920; Sun, 14 Aug 2022 00:18:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.overt.org (mail.overt.org [157.230.92.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AF8C868B7A7 for ; Sun, 14 Aug 2022 00:18:31 +0300 (EEST) Received: from authenticated-user (mail.overt.org [157.230.92.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id 6CCFF3F3E3; Sat, 13 Aug 2022 16:18:30 -0500 (CDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1660425510; bh=jCpApjlMISORFy4n5GlFeW3Yw16tfC66dSYRImv6i8o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LMrQBQgOhCuwu7gyVnfyqh2cNsnqBGny3CAUQIbreB6b23z0/zbfOAXx+3OU3xzYR UiOdMa2sCggwkuVdyn03ZJhjSeuxDScepHd58/ML1LKYvdjsCnKA945UqJrJ38JSOK NBZuCAFubMzt+DZJbsgezC1AO1rVOg/Podhr1JBnyy1U/Dz39HUNTpgb4bqlbbIoI8 viYtjwGeqofA8yFLzeWbKok3ZVVkGM8IfQpX5gG9LbQgz8OJnQg2ymPWZ48bdEWeNS oWxcrACcM/3y1lk+hp7MGHgvhiyrk60G5PVV0WFxU32Q4VhABma6Gj2UErQtrfXx2p bTTNdu1ykdv6g== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Sat, 13 Aug 2022 14:18:21 -0700 Message-Id: <20220813211821.14916-3-philipl@overt.org> In-Reply-To: <20220813211821.14916-1-philipl@overt.org> References: <20220813211821.14916-1-philipl@overt.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] lavc/vaapi: Add support for remaining 10/12bit profiles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: k1R7Gxw6564j With the necessary pixel formats defined, we can now expose support for the remaining 10/12bit combinations that VAAPI can handle. Specifically, we are adding support for: * HEVC ** 12bit 420 ** 10bit 422 ** 12bit 422 ** 10bit 444 ** 12bit 444 * VP9 ** 10bit 422 ** 10bit 444 These obviously require actual hardware support to be usable, but where that exists, it is now enabled. I had to make some adjustments to the encode logic for matching bit depth as the existing code assumed that the picture depth and the pixel format depth were always the same, which is not true for 12bit content which uses 16bit pixel formats. Signed-off-by: Philip Langdale --- libavcodec/hevcdec.c | 8 ++++++++ libavcodec/vaapi_decode.c | 13 +++++++++++++ libavcodec/vaapi_encode.c | 17 +++++++++++++---- libavcodec/vaapi_encode_h265.c | 10 ++++++++-- libavcodec/vaapi_encode_vp9.c | 4 +++- libavcodec/vaapi_hevc.c | 11 ++++++++++- libavutil/hwcontext_vaapi.c | 12 ++++++++++++ 7 files changed, 67 insertions(+), 8 deletions(-) diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index f1be8af2cd..1a895800a6 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -481,11 +481,19 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) #endif case AV_PIX_FMT_YUV420P12: case AV_PIX_FMT_YUV444P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; +#endif #if CONFIG_HEVC_VDPAU_HWACCEL *fmt++ = AV_PIX_FMT_VDPAU; #endif #if CONFIG_HEVC_NVDEC_HWACCEL *fmt++ = AV_PIX_FMT_CUDA; +#endif + break; + case AV_PIX_FMT_YUV422P12: +#if CONFIG_HEVC_VAAPI_HWACCEL + *fmt++ = AV_PIX_FMT_VAAPI; #endif break; } diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index bc2d3ed803..9ba7dfd886 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -262,16 +262,28 @@ static const struct { MAP(YUY2, YUYV422), #ifdef VA_FOURCC_Y210 MAP(Y210, Y210), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, Y216), #endif // 4:4:0 MAP(422V, YUV440P), // 4:4:4 MAP(444P, YUV444P), MAP(AYUV, VUYA), +#ifdef VA_FOURCC_Y410 + MAP(Y410, Y410), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, Y416), +#endif // 4:2:0 10-bit #ifdef VA_FOURCC_P010 MAP(P010, P010), #endif +#ifdef VA_FOURCC_P016 + MAP(P016, P016), +#endif #ifdef VA_FOURCC_I010 MAP(I010, YUV420P10), #endif @@ -415,6 +427,7 @@ static const struct { #if VA_CHECK_VERSION(0, 39, 0) MAP(VP9, VP9_1, VP9Profile1 ), MAP(VP9, VP9_2, VP9Profile2 ), + MAP(VP9, VP9_3, VP9Profile3 ), #endif #if VA_CHECK_VERSION(1, 8, 0) MAP(AV1, AV1_MAIN, AV1Profile0), diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c index f13daa5cff..3e6252ad4a 100644 --- a/libavcodec/vaapi_encode.c +++ b/libavcodec/vaapi_encode.c @@ -1305,7 +1305,11 @@ static const VAAPIEncodeRTFormat vaapi_encode_rt_formats[] = { { "YUV420", VA_RT_FORMAT_YUV420, 8, 3, 1, 1 }, { "YUV422", VA_RT_FORMAT_YUV422, 8, 3, 1, 0 }, #if VA_CHECK_VERSION(1, 2, 0) + { "YUV420_12", VA_RT_FORMAT_YUV420_12, 12, 3, 1, 1 }, { "YUV422_10", VA_RT_FORMAT_YUV422_10, 10, 3, 1, 0 }, + { "YUV422_12", VA_RT_FORMAT_YUV422_12, 12, 3, 1, 0 }, + { "YUV444_10", VA_RT_FORMAT_YUV444_10, 10, 4, 0, 0 }, + { "YUV444_12", VA_RT_FORMAT_YUV444_12, 12, 4, 0, 0 }, #endif { "YUV444", VA_RT_FORMAT_YUV444, 8, 3, 0, 0 }, { "AYUV", VA_RT_FORMAT_YUV444, 8, 4, 0, 0 }, @@ -1342,7 +1346,7 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) VAConfigAttrib rt_format_attr; const VAAPIEncodeRTFormat *rt_format; const char *profile_string, *entrypoint_string; - int i, j, n, depth, err; + int i, j, n, depth, logical_depth, err; if (ctx->low_power) { @@ -1365,7 +1369,9 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) } depth = desc->comp[0].depth; for (i = 1; i < desc->nb_components; i++) { - if (desc->comp[i].depth != depth) { + // We do not apply this depth requirement to the fourth component as + // that will be the alpha channel when present, which can be smaller. + if (i < 3 && desc->comp[i].depth != depth) { av_log(avctx, AV_LOG_ERROR, "Invalid input pixfmt (%s).\n", desc->name); return AVERROR(EINVAL); @@ -1391,8 +1397,10 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) av_assert0(ctx->codec->profiles); for (i = 0; (ctx->codec->profiles[i].av_profile != FF_PROFILE_UNKNOWN); i++) { + profile = &ctx->codec->profiles[i]; - if (depth != profile->depth || + logical_depth = profile->depth > 10 ? 16 : profile->depth; + if (depth != logical_depth || desc->nb_components != profile->nb_components) continue; if (desc->nb_components > 1 && @@ -1476,7 +1484,8 @@ static av_cold int vaapi_encode_profile_entrypoint(AVCodecContext *avctx) for (i = 0; i < FF_ARRAY_ELEMS(vaapi_encode_rt_formats); i++) { rt_format = &vaapi_encode_rt_formats[i]; - if (rt_format->depth == depth && + logical_depth = rt_format->depth > 10 ? 16 : rt_format->depth; + if (logical_depth == depth && rt_format->nb_components == profile->nb_components && rt_format->log2_chroma_w == profile->log2_chroma_w && rt_format->log2_chroma_h == profile->log2_chroma_h) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 1de323af78..1a65fb9ebf 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -283,7 +283,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx) return AVERROR(EINVAL); } } - bit_depth = desc->comp[0].depth; + // Bit depth must be taken from the profile, as the pixel format will be + // 16bit for >= 12bit content. + bit_depth = ctx->profile->depth; // VPS @@ -1276,10 +1278,14 @@ static const VAAPIEncodeProfile vaapi_encode_h265_profiles[] = { { FF_PROFILE_HEVC_REXT, 10, 3, 1, 1, VAProfileHEVCMain10 }, #endif #if VA_CHECK_VERSION(1, 2, 0) + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 1, VAProfileHEVCMain12 }, { FF_PROFILE_HEVC_REXT, 8, 3, 1, 0, VAProfileHEVCMain422_10 }, { FF_PROFILE_HEVC_REXT, 10, 3, 1, 0, VAProfileHEVCMain422_10 }, - // Four channels because this uses the AYUV format which has Alpha + { FF_PROFILE_HEVC_REXT, 12, 3, 1, 0, VAProfileHEVCMain422_12 }, + // Four channels because these use formats which have Alpha { FF_PROFILE_HEVC_REXT, 8, 4, 0, 0, VAProfileHEVCMain444 }, + { FF_PROFILE_HEVC_REXT, 10, 4, 0, 0, VAProfileHEVCMain444_10 }, + { FF_PROFILE_HEVC_REXT, 12, 4, 0, 0, VAProfileHEVCMain444_12 }, #endif { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_encode_vp9.c b/libavcodec/vaapi_encode_vp9.c index 9b455e10c9..e6c7f01f11 100644 --- a/libavcodec/vaapi_encode_vp9.c +++ b/libavcodec/vaapi_encode_vp9.c @@ -228,9 +228,11 @@ static av_cold int vaapi_encode_vp9_configure(AVCodecContext *avctx) static const VAAPIEncodeProfile vaapi_encode_vp9_profiles[] = { { FF_PROFILE_VP9_0, 8, 3, 1, 1, VAProfileVP9Profile0 }, - // Four channels because this uses the AYUV format which has Alpha + // Four channels because this uses a format which has Alpha { FF_PROFILE_VP9_1, 8, 4, 0, 0, VAProfileVP9Profile1 }, { FF_PROFILE_VP9_2, 10, 3, 1, 1, VAProfileVP9Profile2 }, + // Four channels because this uses a format which has Alpha + { FF_PROFILE_VP9_3, 10, 4, 0, 0, VAProfileVP9Profile3 }, { FF_PROFILE_UNKNOWN } }; diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index d82975979a..20fb36adfa 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -567,15 +567,24 @@ VAProfile ff_vaapi_parse_hevc_rext_profile(AVCodecContext *avctx) } #if VA_CHECK_VERSION(1, 2, 0) - if (!strcmp(profile->name, "Main 4:2:2 10") || + if (!strcmp(profile->name, "Main 12") || + !strcmp(profile->name, "Main 12 Intra")) + return VAProfileHEVCMain12; + else if (!strcmp(profile->name, "Main 4:2:2 10") || !strcmp(profile->name, "Main 4:2:2 10 Intra")) return VAProfileHEVCMain422_10; + else if (!strcmp(profile->name, "Main 4:2:2 12") || + !strcmp(profile->name, "Main 4:2:2 12 Intra")) + return VAProfileHEVCMain422_12; else if (!strcmp(profile->name, "Main 4:4:4") || !strcmp(profile->name, "Main 4:4:4 Intra")) return VAProfileHEVCMain444; else if (!strcmp(profile->name, "Main 4:4:4 10") || !strcmp(profile->name, "Main 4:4:4 10 Intra")) return VAProfileHEVCMain444_10; + else if (!strcmp(profile->name, "Main 4:4:4 12") || + !strcmp(profile->name, "Main 4:4:4 12 Intra")) + return VAProfileHEVCMain444_12; #else av_log(avctx, AV_LOG_WARNING, "HEVC profile %s is " "not supported with this VA version.\n", profile->name); diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 2ee5145727..c3e79907fd 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -121,6 +121,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(YUY2, YUV422, YUYV422, 0), #ifdef VA_FOURCC_Y210 MAP(Y210, YUV422_10, Y210, 0), +#endif +#ifdef VA_FOURCC_Y216 + MAP(Y216, YUV422_12, Y216, 0), #endif MAP(411P, YUV411, YUV411P, 0), MAP(422V, YUV422, YUV440P, 0), @@ -129,6 +132,9 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { MAP(Y800, YUV400, GRAY8, 0), #ifdef VA_FOURCC_P010 MAP(P010, YUV420_10BPP, P010, 0), +#endif +#ifdef VA_FOURCC_P016 + MAP(P016, YUV420_12, P016, 0), #endif MAP(BGRA, RGB32, BGRA, 0), MAP(BGRX, RGB32, BGR0, 0), @@ -143,6 +149,12 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = { #ifdef VA_FOURCC_X2R10G10B10 MAP(X2R10G10B10, RGB32_10, X2RGB10, 0), #endif +#ifdef VA_FOURCC_Y410 + MAP(Y410, YUV444_10, Y410, 0), +#endif +#ifdef VA_FOURCC_Y416 + MAP(Y416, YUV444_12, Y416, 0), +#endif }; #undef MAP