From patchwork Tue Aug 16 05:02:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 37304 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp1659895pzi; Mon, 15 Aug 2022 22:03:21 -0700 (PDT) X-Google-Smtp-Source: AA6agR7oQPKaVckXlSM3TnfnQTEzSnT9Mnf3VK1oOM2Y1OWVonaybj+zkmMgOhOYwcSEDMrdXUQJ X-Received: by 2002:a17:906:8445:b0:730:ba5a:ef59 with SMTP id e5-20020a170906844500b00730ba5aef59mr12829156ejy.182.1660626201717; Mon, 15 Aug 2022 22:03:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660626201; cv=none; d=google.com; s=arc-20160816; b=f27f2lkFZqHhJIlccsCbklojU5Gf56sbFgWF18wuK+N4Llld8BJzPx8jAPl1HmvmPr xdnmvxt/BArqGoi3lwTy1WUV50SEyCeWuioTW23+khH6rm1nshK24KSQCsrUFJwHNDCn ggVQZwakWToUDxndLkUm/c+GmHdHTE92c5wpeha1EUgAv6ECJT43TxFurlKBm07RO8v5 2fRJycdwngx7VoXwY3pxecLYXhwoL6EzFwesidXWPauerz+yekv8oyagXIHW1RGvd5KZ WbTuEJgCqEzrGl6G1TtaiTBBeVgkMWQTdqlwS1JaMaisPLUo3hNhE9obGNM2L3ekYI/c 7UeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=qKNoaDoBY/zEay5JkoGVzBoSPKKzUGK/6Vx6zLC4G9U=; b=hDjBLi3MPYHucGmVA9Wo1lhAqGNg4qS5mDZ8iTxAyRPjG8wcMcZvz88SIdLv51zGBN Clb4/4fDCxDahe5uwTjq3ERAIXN/Yr0MVogb2tujyuauljK/C8bjz+DxF1uAbt9ffC5x 6Fgx4/Nj1VnERTbmg89k+bds9M9j38zC+Z6cuXVKJB/nTq9GIcAPz2/J12Z4orrTRQw0 l5hyIiDAnScMGeEYahGA2MmkpnAJfeNSvfOcqouA/mQJCisd/ypn3a991HpQqcMVwE68 GqTkItNGXGNFpuZJwl4XP+ZFYakx5DPl3IkXaaQaCTnxz3I+4+Jq4rMUIEphUKFYp0oY sJvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=J7kj9+Sd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i14-20020a05640242ce00b0043d646c3ef9si10994414edc.259.2022.08.15.22.03.21; Mon, 15 Aug 2022 22:03:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=J7kj9+Sd; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=overt.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 37B1468B956; Tue, 16 Aug 2022 08:03:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail.overt.org (mail.overt.org [157.230.92.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 50F1568B43B for ; Tue, 16 Aug 2022 08:03:05 +0300 (EEST) Received: from authenticated-user (mail.overt.org [157.230.92.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id 039DE3F739; Tue, 16 Aug 2022 00:03:03 -0500 (CDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1660626184; bh=N72hgCTuHFvx4OJ0Fjic3RGn88YHpycrHhD+AW3PdVE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J7kj9+SdqweAOD1WNPchXpvKX5xS4h4NFF2LpXuO27Wc0WeqD5A/s7r2tQshqXmbZ UFPokb3Q/6C1PPu+Hv7kbtHv1pQMx50d9KUHcYiQs6JSWwqsBym1Tc+au18NY/VoI+ 85tpMzb99Xwglsmm7fV50LdmifUZU1FlYOloKSCk2RZRmOH/yhH7E+3B/sXzX4Yv6i yd9cMXb3BGwwmzKFls6cK2/Ki6aEmSh4j1/2Z56DqI5ZRVWoEVhj8L5faUFnsMiqLV oX0ldeJyAOoxCnoRLk5Q7B493T4g7ZgBkmZUalfQQyKu9lxz9H8kCS4RWFSi4f4ZWI Dtou0yROiY8lg== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Mon, 15 Aug 2022 22:02:54 -0700 Message-Id: <20220816050256.32149-2-philipl@overt.org> In-Reply-To: <20220816050256.32149-1-philipl@overt.org> References: <20220816050256.32149-1-philipl@overt.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] lavu/pixfmt: Add P012, Y212, Y410, and Y412 formats X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OTp4vjyBWpan These are the formats returned by the Intel VAAPI decoder for 12bit 4:2:0, 12bit 4:2:2, 10bit 4:4:4, and 12bit 4:4:4 respectively. As with the already supported Y210 and YUVA (AVUY) formats, they are based on formats Microsoft picked as their preferred 4:2:2 and 4:4:4 video formats, and Intel ran with it. P12 and Y212 are simply an extension of 10 bit formats to say 12 bits will be used, and Y412 is a normal looking packed 4 channel format where the 12 most significant bits are used. Y410 is an annoying format that packs three 10bit channels into 32bits with 2bits of alpha. As a result, I had to define Y410 as a bitstream format, even though each pixel is byte-aligned. If it is in-fact possible to define as a normal byte-aligned format, please let me know how. As with VUYA, I have kept the formal definition of Y410 and Y412 as formats with alpha channels to maintain fidelity. The Intel folks say they prefer this and they would rather explicitly use a format defined as not having alpha than to silently drop it. The hardware decoder does at least ensure the alpha channel is set to full opacity. Signed-off-by: Philip Langdale --- libavutil/pixdesc.c | 101 ++++++++++++++++++++++++++++++- libavutil/pixfmt.h | 16 +++++ tests/ref/fate/imgutils | 8 +++ tests/ref/fate/sws-pixdesc-query | 42 +++++++++++++ 4 files changed, 166 insertions(+), 1 deletion(-) diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index f7558ff8b9..0801927838 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2147,6 +2147,30 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { }, .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE, }, + [AV_PIX_FMT_P012LE] = { + .name = "p012le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 1, + .comp = { + { 0, 2, 0, 4, 12 }, /* Y */ + { 1, 4, 0, 4, 12 }, /* U */ + { 1, 4, 2, 4, 12 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR, + }, + [AV_PIX_FMT_P012BE] = { + .name = "p012be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 1, + .comp = { + { 0, 2, 0, 4, 12 }, /* Y */ + { 1, 4, 0, 4, 12 }, /* U */ + { 1, 4, 2, 4, 12 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_BE, + }, [AV_PIX_FMT_P016LE] = { .name = "p016le", .nb_components = 3, @@ -2532,6 +2556,81 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .flags = AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_FLOAT, }, + [AV_PIX_FMT_Y212LE] = { + .name = "y212le", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 4, 12 }, /* Y */ + { 0, 8, 2, 4, 12 }, /* U */ + { 0, 8, 6, 4, 12 }, /* V */ + }, + }, + [AV_PIX_FMT_Y212BE] = { + .name = "y212be", + .nb_components = 3, + .log2_chroma_w = 1, + .log2_chroma_h = 0, + .comp = { + { 0, 4, 0, 4, 12 }, /* Y */ + { 0, 8, 2, 4, 12 }, /* U */ + { 0, 8, 6, 4, 12 }, /* V */ + }, + .flags = AV_PIX_FMT_FLAG_BE, + }, + [AV_PIX_FMT_Y410LE] = { + .name = "y410le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y410BE] = { + .name = "y410be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 32, 10, 0, 10 }, /* Y */ + { 0, 32, 0, 0, 10 }, /* U */ + { 0, 32, 20, 0, 10 }, /* V */ + { 0, 32, 30, 0, 2 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA | AV_PIX_FMT_FLAG_BITSTREAM, + }, + [AV_PIX_FMT_Y412LE] = { + .name = "y412le", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 4, 12 }, /* Y */ + { 0, 8, 0, 4, 12 }, /* U */ + { 0, 8, 4, 4, 12 }, /* V */ + { 0, 8, 6, 4, 12 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_ALPHA, + }, + [AV_PIX_FMT_Y412BE] = { + .name = "y412be", + .nb_components= 4, + .log2_chroma_w= 0, + .log2_chroma_h= 0, + .comp = { + { 0, 8, 2, 4, 12 }, /* Y */ + { 0, 8, 0, 4, 12 }, /* U */ + { 0, 8, 4, 4, 12 }, /* V */ + { 0, 8, 6, 4, 12 }, /* A */ + }, + .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_ALPHA, + }, }; static const char * const color_range_names[] = { @@ -2767,7 +2866,7 @@ void ff_check_pixfmt_descriptors(void){ if (!d->name && !d->nb_components && !d->log2_chroma_w && !d->log2_chroma_h && !d->flags) continue; -// av_log(NULL, AV_LOG_DEBUG, "Checking: %s\n", d->name); + av_log(NULL, AV_LOG_INFO, "Checking: %s\n", d->name); av_assert0(d->log2_chroma_w <= 3); av_assert0(d->log2_chroma_h <= 3); av_assert0(d->nb_components <= 4); diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 86c9bdefeb..effaac29d3 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -372,6 +372,18 @@ enum AVPixelFormat { AV_PIX_FMT_RGBAF16BE, ///< IEEE-754 half precision packed RGBA 16:16:16:16, 64bpp, RGBARGBA..., big-endian AV_PIX_FMT_RGBAF16LE, ///< IEEE-754 half precision packed RGBA 16:16:16:16, 64bpp, RGBARGBA..., little-endian + AV_PIX_FMT_P012LE, ///< like NV12, with 12bpp per component, data in the high bits, zeros in the low bits, little-endian + AV_PIX_FMT_P012BE, ///< like NV12, with 12bpp per component, data in the high bits, zeros in the low bits, big-endian + + AV_PIX_FMT_Y212BE, ///< packed YUV 4:2:2 like YUYV422, 24bpp, data in the high bits, zeros in the low bits, big-endian + AV_PIX_FMT_Y212LE, ///< packed YUV 4:2:2 like YUYV422, 24bpp, data in the high bits, zeros in the low bits, little-endian + + AV_PIX_FMT_Y410BE, ///< packed AVYU 4:4:4:4, 32bpp, (msb)2A 10V 10Y 10U(lsb), big-endian + AV_PIX_FMT_Y410LE, ///< packed AVYU 4:4:4:4, 32bpp, (msb)2A 10V 10Y 10U(lsb), little-endian + + AV_PIX_FMT_Y412BE, ///< packed AVYU 4:4:4:4, 48bpp, data in the high bits, zeros in the low bits, big-endian + AV_PIX_FMT_Y412LE, ///< packed AVYU 4:4:4:4, 48bpp, data in the high bits, zeros in the low bits, little-endian + AV_PIX_FMT_NB ///< number of pixel formats, DO NOT USE THIS if you want to link with shared libav* because the number of formats might differ between versions }; @@ -458,9 +470,13 @@ enum AVPixelFormat { #define AV_PIX_FMT_NV20 AV_PIX_FMT_NE(NV20BE, NV20LE) #define AV_PIX_FMT_AYUV64 AV_PIX_FMT_NE(AYUV64BE, AYUV64LE) #define AV_PIX_FMT_P010 AV_PIX_FMT_NE(P010BE, P010LE) +#define AV_PIX_FMT_P012 AV_PIX_FMT_NE(P012BE, P012LE) #define AV_PIX_FMT_P016 AV_PIX_FMT_NE(P016BE, P016LE) #define AV_PIX_FMT_Y210 AV_PIX_FMT_NE(Y210BE, Y210LE) +#define AV_PIX_FMT_Y212 AV_PIX_FMT_NE(Y212BE, Y212LE) +#define AV_PIX_FMT_Y410 AV_PIX_FMT_NE(Y410BE, Y410LE) +#define AV_PIX_FMT_Y412 AV_PIX_FMT_NE(Y412BE, Y412LE) #define AV_PIX_FMT_X2RGB10 AV_PIX_FMT_NE(X2RGB10BE, X2RGB10LE) #define AV_PIX_FMT_X2BGR10 AV_PIX_FMT_NE(X2BGR10BE, X2BGR10LE) diff --git a/tests/ref/fate/imgutils b/tests/ref/fate/imgutils index 01c9877de5..bc12d8dad4 100644 --- a/tests/ref/fate/imgutils +++ b/tests/ref/fate/imgutils @@ -249,3 +249,11 @@ p416le planes: 2, linesizes: 128 256 0 0, plane_sizes: 6144 12288 vuya planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 rgbaf16be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 rgbaf16le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +p012le planes: 2, linesizes: 128 128 0 0, plane_sizes: 6144 3072 0 0, plane_offsets: 6144 0 0, total_size: 9216 +p012be planes: 2, linesizes: 128 128 0 0, plane_sizes: 6144 3072 0 0, plane_offsets: 6144 0 0, total_size: 9216 +y212be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y212le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410be planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y410le planes: 1, linesizes: 256 0 0 0, plane_sizes: 12288 0 0 0, plane_offsets: 0 0 0, total_size: 12288 +y412be planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 +y412le planes: 1, linesizes: 512 0 0 0, plane_sizes: 24576 0 0 0, plane_offsets: 0 0 0, total_size: 24576 diff --git a/tests/ref/fate/sws-pixdesc-query b/tests/ref/fate/sws-pixdesc-query index f79d99e513..04baf6a3fa 100644 --- a/tests/ref/fate/sws-pixdesc-query +++ b/tests/ref/fate/sws-pixdesc-query @@ -63,6 +63,8 @@ isNBPS: nv20le p010be p010le + p012be + p012le p210be p210le p410be @@ -75,6 +77,12 @@ isNBPS: xyz12le y210be y210le + y212be + y212le + y410be + y410le + y412be + y412le yuv420p10be yuv420p10le yuv420p12be @@ -149,6 +157,7 @@ isBE: grayf32be nv20be p010be + p012be p016be p210be p216be @@ -164,6 +173,9 @@ isBE: x2rgb10be xyz12be y210be + y212be + y410be + y412be ya16be yuv420p10be yuv420p12be @@ -206,6 +218,8 @@ isYUV: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -223,6 +237,12 @@ isYUV: xyz12le y210be y210le + y212be + y212le + y410be + y410le + y412be + y412le ya16be ya16le ya8 @@ -309,6 +329,8 @@ isPlanarYUV: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -400,6 +422,8 @@ isSemiPlanarYUV: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -665,6 +689,10 @@ ALPHA: rgbaf16be rgbaf16le vuya + y410be + y410le + y412be + y412le ya16be ya16le ya8 @@ -761,6 +789,12 @@ Packed: xyz12le y210be y210le + y212be + y212le + y410be + y410le + y412be + y412le ya16be ya16le ya8 @@ -799,6 +833,8 @@ Planar: nv42 p010be p010le + p012be + p012le p016be p016le p210be @@ -971,6 +1007,8 @@ usePal: DataInHighBits: p010be p010le + p012be + p012le p210be p210le p410be @@ -979,6 +1017,10 @@ DataInHighBits: xyz12le y210be y210le + y212be + y212le + y412be + y412le SwappedChroma: nv21