From patchwork Tue Aug 16 04:36:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 37301 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp1652108pzi; Mon, 15 Aug 2022 21:37:28 -0700 (PDT) X-Google-Smtp-Source: AA6agR6qvOwOZJT9gLaazErEeCSgstn0oDU4ZzapLULaRC9LBNnajQ/vW8QjDhaTSIVPPh8vY75f X-Received: by 2002:a17:906:c154:b0:733:197:a8c with SMTP id dp20-20020a170906c15400b0073301970a8cmr12478328ejc.483.1660624648054; Mon, 15 Aug 2022 21:37:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660624648; cv=none; d=google.com; s=arc-20160816; b=oB+aAHLmeAiIXEvUTFD347ZrGNPvMHijioSW9+vrhwvJOchTejWyhI9sLMpngRsCiA BPppdjQztFIGLJ8NPaWmopQNbzQHM+Lu117TB/xoKtskj1PyfgjxREctUYFGX7n1YoEB BFmhoIrUaDFGxaK8HXwvZjKmGblOlOhTM5kd+h+KZqO5PleFXTeoJiDE2OSwNEd6Z2Og qxLoNCUgRbp/9zuzGVU1hVVo9IvTfPx2vokBmdsQ4wl8JyAzWYGBPInE7nVO/Cuj1cFN S5uBGrQiDS39zKpTQdnQr35jEScR1f+u/vC64thS6YPtgp9ZWeiRGFOr+SA3rZFIbAk4 FnZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=qKNoaDoBY/zEay5JkoGVzBoSPKKzUGK/6Vx6zLC4G9U=; b=sPa7MGCZYjpmwvK3oa9Qg9WFsPwOVKL7fYTiUXEhqD73GffuETzt2IZq/14GUWFSvt vNxtxOPtkugrYJ8jeyoRkcddHD2HLVD02ruk6EvvK0Tj/zz4o35iorS4/aDLgFBVFBPt kjmGHVwUg7rbk3tc823jPhhqxlUdthYkcWKbJRx4lU6/PR9AsIzamV9LisRrXB4nIW6Y ZEtRghz4I0dhREFFcQVpYWqFvLNBI+YVuFCkJci0/UbrwVy3G3zb92etCXoVem8EU3qM A70InEkWdlhgmMNp62N2vJNxF4RH8gaEPu/fuTdgqxFLveIZha/8+6DGIdxGXQKi48G1 aF3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=MevkJYYh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id eb14-20020a0564020d0e00b004359f471717si11903240edb.0.2022.08.15.21.37.27; Mon, 15 Aug 2022 21:37:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=MevkJYYh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1DA7568B945; Tue, 16 Aug 2022 07:37:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.overt.org (mail.overt.org [157.230.92.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CAB5E68B895 for ; Tue, 16 Aug 2022 07:36:59 +0300 (EEST) Received: from authenticated-user (mail.overt.org [157.230.92.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id 000263F739; Mon, 15 Aug 2022 23:36:57 -0500 (CDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1660624618; bh=N72hgCTuHFvx4OJ0Fjic3RGn88YHpycrHhD+AW3PdVE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MevkJYYhM5ycqyOo39WdEl208+FyldAmqjwS7IwVSwlaVk3TGAbh7KFA2EOCPaLbr 6odH9qnwn2rQCjcFyR1iYXDriuSVnZqB1WW2j1VHNh3A0XqPsq/qWFwOueEEAADTbr aoeOQ3uOLUXHRxERo0VY/RCpDm7J17duqBWOkYqTDKftzaG3IFZvf2ntv30c4lghPv VVmDE4MrtED0FOv2javBTs6CCD4UxfYFVsWmESr4+9Uk2yZyq77OicW72bUWIhFH0Z ropF6qrlwtyUDDFOn7xCX7pqF+/hfhMxQ0JASkeFxHPN1touYwHRIOjwkg/PTbWhLl VhXSj2q2w0CfQ== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Mon, 15 Aug 2022 21:36:39 -0700 Message-Id: <20220816043641.41521-2-philipl@overt.org> In-Reply-To: <20220816043641.41521-1-philipl@overt.org> References: <20220816043641.41521-1-philipl@overt.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add P012, Y212, Y410, and Y412 formats X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: moOG2pkMyfkZ These are the formats returned by the Intel VAAPI decoder for 12bit 4:2:0, 12bit 4:2:2, 10bit 4:4:4, and 12bit 4:4:4 respectively. As with the already supported Y210 and YUVA (AVUY) formats, they are based on formats Microsoft picked as their preferred 4:2:2 and 4:4:4 video formats, and Intel ran with it. P12 and Y212 are simply an extension of 10 bit formats to say 12 bits will be used, and Y412 is a normal looking packed 4 channel format where the 12 most significant bits are used. Y410 is an annoying format that packs three 10bit channels into 32bits with 2bits of alpha. As a result, I had to define Y410 as a bitstream format, even though each pixel is byte-aligned. If it is in-fact possible to define as a normal byte-aligned format, please let me know how. As with VUYA, I have kept the formal definition of Y410 and Y412 as formats with alpha channels to maintain fidelity. The Intel folks say they prefer this and they would rather explicitly use a format defined as not having alpha than to silently drop it. The hardware decoder does at least ensure the alpha channel is set to full opacity. Signed-off-by: Philip Langdale --- libavutil/pixdesc.c | 101 ++++++++++++++++++++++++++++++- libavutil/pixfmt.h | 16 +++++ tests/ref/fate/imgutils | 8 +++ tests/ref/fate/sws-pixdesc-query | 42 +++++++++++++ 4 files changed, 166 insertions(+), 1 deletion(-) diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index f7558ff8b9..0801927838 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2147,6 +2147,30 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { }, .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE, }, + [AV_PIX_FMT_P012LE] = { + .name = "p012le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 1, + .comp = { + { 0, 2, 0, 4, 12 }, /* Y */ + { 1, 4, 0, 4, 12 }, /* U */ + { 1, 4, 2, 4, 12 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR, + }, + [AV_PIX_FMT_P012BE] = { + .name = "p012be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 1, + .comp = { + { 0, 2, 0, 4, 12 }, /* Y */ + { 1, 4, 0, 4, 12 }, /* U */ + { 1, 4, 2, 4, 12 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE, + }, [AV_PIX_FMT_P016LE] = { .name = "p016le", .nb_components = 3, @@ -2532,6 +2556,81 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_FLOAT, }, + [AV_PIX_FMT_Y212LE] = { + .name = "y212le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 4, 12 }, /* Y */ + { 0, 8, 2, 4, 12 }, /* U */ + { 0, 8, 6, 4, 12 }, /* V */ + }, + }, + [AV_PIX_FMT_Y212BE] = { + .name = "y212be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 4, 12 }, /* Y */ + { 0, 8, 2, 4, 12 }, /* U */ + { 0, 8, 6, 4, 12 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_BE, + }, + [AV_PIX_FMT_Y410LE] = { + .name = "y410le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y410BE] = { + .name = "y410be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y412LE] = { + .name = "y412le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 4, 12 }, /* Y */ + { 0, 8, 0, 4, 12 }, /* U */ + { 0, 8, 4, 4, 12 }, /* V */ + { 0, 8, 6, 4, 12 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA, + }, + [AV_PIX_FMT_Y412BE] = { + .name = "y412be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 4, 12 }, /* Y */ + { 0, 8, 0, 4, 12 }, /* U */ + { 0, 8, 4, 4, 12 }, /* V */ + { 0, 8, 6, 4, 12 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA, + }, }; static const char * const color_range_names[] = { @@ -2767,7 +2866,7 @@ void ff_check_pixfmt_descriptors(void){ if (!d->name && !d->nb_components && !d->log2_chroma_w && !d->log2_chroma_h && !d->flags) continue; -// av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name); + av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name); av_assert0(d->log2_chroma_w <= 3); av_assert0(d->log2_chroma_h <= 3); av_assert0(d->nb_components <= 4); diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 86c9bdefeb..effaac29d3 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -372,6 +372,18 @@ enum AVPixelFormat { AV_PIX_FMT_RGBAF16BE, ///< IEEE-754 half precision packed RGBA 16:16:16:16, 64bpp, RGBARGBA..., big-endian AV_PIX_FMT_RGBAF16LE, ///< IEEE-754 half precision packed RGBA 16:16:16:16, 64bpp, RGBARGBA..., little-endian + AV_PIX_FMT_P012LE, ///< like NV12, with 12bpp per component, data in the high bits, zeros in the low bits, little-endian + AV_PIX_FMT_P012BE, ///< like NV12, with 12bpp per component, data in the high bits, zeros in the low bits, big-endian + + AV_PIX_FMT_Y212BE, ///< packed YUV 4:2:2 like YUYV422, 24bpp, data in the high bits, zeros in the low bits, big-endian + AV_PIX_FMT_Y212LE, ///< packed YUV 4:2:2 like YUYV422, 24bpp, data in the high bits, zeros in the low bits, little-endian + + AV_PIX_FMT_Y410BE, ///< packed AVYU 4:4:4:4, 32bpp, (msb)2A 10V 10Y 10U(lsb), big-endian + AV_PIX_FMT_Y410LE, ///< packed AVYU 4:4:4:4, 32bpp, (msb)2A 10V 10Y 10U(lsb), little-endian + + AV_PIX_FMT_Y412BE, ///< packed AVYU 4:4:4:4, 48bpp, data in the high bits, zeros in the low bits, big-endian + AV_PIX_FMT_Y412LE, ///< packed AVYU 4:4:4:4, 48bpp, data in the high bits, zeros in the low bits, little-endian + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; @@ -458,9 +470,13 @@ enum AVPixelFormat { #define AV_PIX_FMT_NV20 AV_PIX_FMT_NE(NV20BE, NV20LE) #define AV_PIX_FMT_AYUV64 AV_PIX_FMT_NE(AYUV64BE, AYUV64LE) #define AV_PIX_FMT_P010 AV_PIX_FMT_NE(P010BE, P010LE) +#define AV_PIX_FMT_P012 AV_PIX_FMT_NE(P012BE, P012LE) #define AV_PIX_FMT_P016 AV_PIX_FMT_NE(P016BE, P016LE) #define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) +#define AV_PIX_FMT_Y212 AV_PIX_FMT_NE(Y212BE, Y212LE) +#define AV_PIX_FMT_Y410 AV_PIX_FMT_NE(Y410BE, Y410LE) +#define AV_PIX_FMT_Y412 AV_PIX_FMT_NE(Y412BE, Y412LE) #define AV_PIX_FMT_X2RGB10 AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE) #define AV_PIX_FMT_X2BGR10 AV_PIX_FMT_NE(X2BGR10BE, X2BGR10LE) diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils index 01c9877de5..bc12d8dad4 100644 --- a/tests/ref/fate/imgutils +++ b/tests/ref/fate/imgutils @@ -249,3 +249,11 @@ p416le planes: 2, linesizes: 128 256 0 0, plane_sizes: 6144 12288 vuya planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 rgbaf16be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 rgbaf16le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +p012le planes: 2, linesizes: 128 128 0 0, plane_sizes: 6144 3072 0 0, plane_offsets: 6144 0 0, total_size: 9216 +p012be planes: 2, linesizes: 128 128 0 0, plane_sizes: 6144 3072 0 0, plane_offsets: 6144 0 0, total_size: 9216 +y212be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y212le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y412be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +y412le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query index f79d99e513..04baf6a3fa 100644 --- a/tests/ref/fate/sws-pixdesc-query +++ b/tests/ref/fate/sws-pixdesc-query @@ -63,6 +63,8 @@ isNBPS: nv20le p010be p010le + p012be + p012le p210be p210le p410be @@ -75,6 +77,12 @@ isNBPS: xyz12le y210be y210le + y212be + y212le + y410be + y410le + y412be + y412le yuv420p10be yuv420p10le yuv420p12be @@ -149,6 +157,7 @@ isBE: grayf32be nv20be p010be + p012be p016be p210be p216be @@ -164,6 +173,9 @@ isBE: x2rgb10be xyz12be y210be + y212be + y410be + y412be ya16be yuv420p10be yuv420p12be @@ -206,6 +218,8 @@ isYUV: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -223,6 +237,12 @@ isYUV: xyz12le y210be y210le + y212be + y212le + y410be + y410le + y412be + y412le ya16be ya16le ya8 @@ -309,6 +329,8 @@ isPlanarYUV: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -400,6 +422,8 @@ isSemiPlanarYUV: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -665,6 +689,10 @@ ALPHA: rgbaf16be rgbaf16le vuya + y410be + y410le + y412be + y412le ya16be ya16le ya8 @@ -761,6 +789,12 @@ Packed: xyz12le y210be y210le + y212be + y212le + y410be + y410le + y412be + y412le ya16be ya16le ya8 @@ -799,6 +833,8 @@ Planar: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -971,6 +1007,8 @@ usePal: DataInHighBits: p010be p010le + p012be + p012le p210be p210le p410be @@ -979,6 +1017,10 @@ DataInHighBits: xyz12le y210be y210le + y212be + y212le + y412be + y412le SwappedChroma: nv21