From patchwork Sat Sep 23 15:36:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chen Yufei X-Patchwork-Id: 43879 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:2a18:b0:15d:8365:d4b8 with SMTP id e24csp282905pzh; Sat, 23 Sep 2023 08:42:32 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGZYmCUUKeMIqI0eYynDvQi9oCFqhjdPNmPTIzuOCGFJzubxYk/Bpo/YZQV1dVOuSiwbbYj X-Received: by 2002:a17:906:9b8c:b0:9ad:8a96:ad55 with SMTP id dd12-20020a1709069b8c00b009ad8a96ad55mr8896371ejc.14.1695483752197; Sat, 23 Sep 2023 08:42:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695483752; cv=none; d=google.com; s=arc-20160816; b=gEJIpQOMsky9J3NQ9ngY2zWb9y+tujpyQH6Z2pjTYXCu0y+HRZiYFSMPDdbMjxL6Jk 5rte0RUsMTNmiODV5ithP1IOHGrzRtLEAzpMVSTW8WgEGQC2nuckzwXYsTeK7AoxakR6 DkDpJZsO6E9LSNKgZJH7rWthtu70h3M8nfXb5JVEiNzaadVaaZUh/I+tr6VbXWwPtoUw aJGg+sRBef+5QqOY0Hq/2+w8gZ77ni/1WW8xzE2X0uhoOtqv5hRvpcClwlGaAfYureXZ UkIseb/2U+4AYvuWZuCyyHudpZc6Oju5+tN5wV/NyYerWUuhEWzg6szO5GxsTUaFNAm8 gzbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=XEzsV6vwn/hrEa/AxWTS61oSbqn44H26hlyaFnF8zh4=; fh=WMNJqm9yn32VIGoZSiFxz00xdqNXh1DsfrkZBt4KBsg=; b=yy5y04Bc3FJ3nyN5AP0CikOY2ik8q/owm/i5mTM1SrvT4kxAzYfgWwhqqQls72bSe+ 5/VywuN3YzGjUeJNdBUOI1hcPLHc9zFDTVd5KuUt0JORuS92zgImMK3canzU/CHnoOAs ZUHF3conzt0KoImY91B6j1z2EaOHwRfhSCsACHAuERiKbvwxz39A8J6TTlvZthYidSes ggrXLvcvE4RAaY0oww4UiQ5dE2UyCKpHncI4GOEY1gPOYKCXFW/z81wGOhLK7kRIyT2M IOUgYzPzVMy4tm35MKPci7OcPSGVM+dZTczEamQdhzKh97T2MFvJ+n9GkVVW0BcJYqWu ICYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=c60xOS4W; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m12-20020a170906160c00b009ad84beca10si5358648ejd.765.2023.09.23.08.42.31; Sat, 23 Sep 2023 08:42:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=c60xOS4W; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4214368C9CD; Sat, 23 Sep 2023 18:42:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oo1-f43.google.com (mail-oo1-f43.google.com [209.85.161.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2677868C6D8 for ; Sat, 23 Sep 2023 18:42:20 +0300 (EEST) Received: by mail-oo1-f43.google.com with SMTP id 006d021491bc7-57bb0f5d00aso143261eaf.1 for ; Sat, 23 Sep 2023 08:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695483738; x=1696088538; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X/QQbblOLcCycbGj+7khz0dDx2yzqWdhaPjl4qN8Bs0=; b=c60xOS4WuMTgBnzBJDKcm2eleCDy5PMLc4sL+abUaI9vP0xoZ4oHea37fsgEXoTOQ0 2ZuOuda6COZakCOFSq8wWOkWk3Cfe+GhxFSAWUHg0e8CXCsAHWcvieBm1r/OiE70ZtP4 D2ZFYS9gh0F+3kKfmbOPwh/ddw+b1euD8tylDO7yyj49+3AjoYKhJnAFqlhrfMgENWyJ 1ju5rS19dAOVfOe005f3Fnh8jwbKDLlkfSMp+pslRUeVndX1QfXeHaXC2miL8qoYWCqh HPe/MjG8PlVlo/Cb11fCfkbVgWSS0EeOYg7yqoYzIrnTKitbasW0gbRbwTpCP78BSAjJ ZQMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695483738; x=1696088538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X/QQbblOLcCycbGj+7khz0dDx2yzqWdhaPjl4qN8Bs0=; b=R5E8v74HZxJhUsKNY8y+hkfXnDuuKiu8mmMUuBc0Y8Y2oKjNQOcy3Ub/jH7J79Ygaw tT1fuaPdr3dFvOLf2Sne2ZbnsUdClcjkc+oUnoFdWBaHhU/kZbSV+0jGPtU29G0RMfFB kmgW2GuNOh+8ZwWbtqKZWc+4Qtx6OKZPMURR14yKOy3BqrbLUdckfRudv9aPMHs+TNgR QkOqJo02ETGm/20fP55k2b3C6Y13eXwX7dEUjDMcgi7yZWEJ3F+cp01OiSvDRwN2ScQX 8+kkawMLlfw+GtL8ZpELCmAVS5P8O/hscROCHYWqd+ybavbcSJ7I2g8m/gZfma0hbrqn SPpg== X-Gm-Message-State: AOJu0YxoenC+KWu6TN9N2Py2+ib4F6TM76dO02l4paFNQB/+sGqOWZ/9 eZbcg57jkbODBmM8l+mV7V4Moq7hU9c= X-Received: by 2002:a05:6358:99a9:b0:142:e357:e777 with SMTP id j41-20020a05635899a900b00142e357e777mr3076247rwb.25.1695483737705; Sat, 23 Sep 2023 08:42:17 -0700 (PDT) Received: from archvm.home (hwsrv-1088392.hostwindsdns.com. [23.254.253.23]) by smtp.gmail.com with ESMTPSA id e2-20020a63ae42000000b0057c630d606asm3307178pgp.69.2023.09.23.08.42.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Sep 2023 08:42:17 -0700 (PDT) From: Chen Yufei To: ffmpeg-devel@ffmpeg.org Date: Sat, 23 Sep 2023 23:36:09 +0800 Message-ID: <20230923154125.31376-2-cyfdecyf@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230923154125.31376-1-cyfdecyf@gmail.com> References: <20230923154125.31376-1-cyfdecyf@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avfilter/vf_lut3d: expose 3D LUT file parse function. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Chen Yufei Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: P3TAV5XWuu3A Signed-off-by: Chen Yufei --- libavfilter/Makefile | 8 +- libavfilter/lut3d.c | 669 +++++++++++++++++++++++++++++++++++++++++ libavfilter/lut3d.h | 13 + libavfilter/vf_lut3d.c | 590 +----------------------------------- 4 files changed, 689 insertions(+), 591 deletions(-) create mode 100644 libavfilter/lut3d.c diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 2fe0033b21..c1cd797e5c 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -330,7 +330,7 @@ OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o OBJS-$(CONFIG_GUIDED_FILTER) += vf_guided.o -OBJS-$(CONFIG_HALDCLUT_FILTER) += vf_lut3d.o framesync.o +OBJS-$(CONFIG_HALDCLUT_FILTER) += vf_lut3d.o lut3d.o framesync.o OBJS-$(CONFIG_HFLIP_FILTER) += vf_hflip.o OBJS-$(CONFIG_HFLIP_VULKAN_FILTER) += vf_flip_vulkan.o vulkan.o OBJS-$(CONFIG_HISTEQ_FILTER) += vf_histeq.o @@ -367,10 +367,10 @@ OBJS-$(CONFIG_LIMITDIFF_FILTER) += vf_limitdiff.o framesync.o OBJS-$(CONFIG_LIMITER_FILTER) += vf_limiter.o OBJS-$(CONFIG_LOOP_FILTER) += f_loop.o OBJS-$(CONFIG_LUMAKEY_FILTER) += vf_lumakey.o -OBJS-$(CONFIG_LUT1D_FILTER) += vf_lut3d.o +OBJS-$(CONFIG_LUT1D_FILTER) += vf_lut3d.o lut3d.o OBJS-$(CONFIG_LUT_FILTER) += vf_lut.o OBJS-$(CONFIG_LUT2_FILTER) += vf_lut2.o framesync.o -OBJS-$(CONFIG_LUT3D_FILTER) += vf_lut3d.o framesync.o +OBJS-$(CONFIG_LUT3D_FILTER) += vf_lut3d.o lut3d.o framesync.o OBJS-$(CONFIG_LUTRGB_FILTER) += vf_lut.o OBJS-$(CONFIG_LUTYUV_FILTER) += vf_lut.o OBJS-$(CONFIG_MASKEDCLAMP_FILTER) += vf_maskedclamp.o framesync.o @@ -549,7 +549,7 @@ OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER) += vidstabutils.o vf_vidstabtransfo OBJS-$(CONFIG_VIF_FILTER) += vf_vif.o framesync.o OBJS-$(CONFIG_VIGNETTE_FILTER) += vf_vignette.o OBJS-$(CONFIG_VMAFMOTION_FILTER) += vf_vmafmotion.o framesync.o -OBJS-$(CONFIG_VPP_QSV_FILTER) += vf_vpp_qsv.o +OBJS-$(CONFIG_VPP_QSV_FILTER) += vf_vpp_qsv.o lut3d.o OBJS-$(CONFIG_VSTACK_FILTER) += vf_stack.o framesync.o OBJS-$(CONFIG_W3FDIF_FILTER) += vf_w3fdif.o OBJS-$(CONFIG_WAVEFORM_FILTER) += vf_waveform.o diff --git a/libavfilter/lut3d.c b/libavfilter/lut3d.c new file mode 100644 index 0000000000..173979adcc --- /dev/null +++ b/libavfilter/lut3d.c @@ -0,0 +1,669 @@ +/* + * Copyright (c) 2013 Clément Bœsch + * Copyright (c) 2018 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "lut3d.h" + +#include + +#include "libavutil/avstring.h" +#include "libavutil/file_open.h" + +#define EXPONENT_MASK 0x7F800000 +#define MANTISSA_MASK 0x007FFFFF +#define SIGN_MASK 0x80000000 + +static inline float sanitizef(float f) +{ + union av_intfloat32 t; + t.f = f; + + if ((t.i & EXPONENT_MASK) == EXPONENT_MASK) { + if ((t.i & MANTISSA_MASK) != 0) { + // NAN + return 0.0f; + } else if (t.i & SIGN_MASK) { + // -INF + return -FLT_MAX; + } else { + // +INF + return FLT_MAX; + } + } + return f; +} + +static inline float lerpf(float v0, float v1, float f) +{ + return v0 + (v1 - v0) * f; +} + +static inline struct rgbvec lerp(const struct rgbvec *v0, const struct rgbvec *v1, float f) +{ + struct rgbvec v = { + lerpf(v0->r, v1->r, f), lerpf(v0->g, v1->g, f), lerpf(v0->b, v1->b, f) + }; + return v; +} + +int ff_allocate_3dlut(AVFilterContext *ctx, LUT3DContext *lut3d, int lutsize, int prelut) +{ + int i; + if (lutsize < 2 || lutsize > MAX_LEVEL) { + av_log(ctx, AV_LOG_ERROR, "Too large or invalid 3D LUT size\n"); + return AVERROR(EINVAL); + } + + av_freep(&lut3d->lut); + lut3d->lut = av_malloc_array(lutsize * lutsize * lutsize, sizeof(*lut3d->lut)); + if (!lut3d->lut) + return AVERROR(ENOMEM); + + if (prelut) { + lut3d->prelut.size = PRELUT_SIZE; + for (i = 0; i < 3; i++) { + av_freep(&lut3d->prelut.lut[i]); + lut3d->prelut.lut[i] = av_malloc_array(PRELUT_SIZE, sizeof(*lut3d->prelut.lut[0])); + if (!lut3d->prelut.lut[i]) + return AVERROR(ENOMEM); + } + } else { + lut3d->prelut.size = 0; + for (i = 0; i < 3; i++) { + av_freep(&lut3d->prelut.lut[i]); + } + } + lut3d->lutsize = lutsize; + lut3d->lutsize2 = lutsize * lutsize; + return 0; +} + +static int set_identity_matrix(AVFilterContext *ctx, LUT3DContext *lut3d, int size) +{ + int ret, i, j, k; + const int size2 = size * size; + const float c = 1. / (size - 1); + + ret = ff_allocate_3dlut(ctx, lut3d, size, 0); + if (ret < 0) + return ret; + + for (k = 0; k < size; k++) { + for (j = 0; j < size; j++) { + for (i = 0; i < size; i++) { + struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; + vec->r = k * c; + vec->g = j * c; + vec->b = i * c; + } + } + } + + return 0; +} + +#define MAX_LINE_SIZE 512 + +static int skip_line(const char *p) +{ + while (*p && av_isspace(*p)) + p++; + return !*p || *p == '#'; +} + +static char* fget_next_word(char* dst, int max, FILE* f) +{ + int c; + char *p = dst; + + /* for null */ + max--; + /* skip until next non whitespace char */ + while ((c = fgetc(f)) != EOF) { + if (av_isspace(c)) + continue; + + *p++ = c; + max--; + break; + } + + /* get max bytes or up until next whitespace char */ + for (; max > 0; max--) { + if ((c = fgetc(f)) == EOF) + break; + + if (av_isspace(c)) + break; + + *p++ = c; + } + + *p = 0; + if (p == dst) + return NULL; + return p; +} + + +#define NEXT_LINE(loop_cond) do { \ + if (!fgets(line, sizeof(line), f)) { \ + av_log(ctx, AV_LOG_ERROR, "Unexpected EOF\n"); \ + return AVERROR_INVALIDDATA; \ + } \ +} while (loop_cond) + +#define NEXT_LINE_OR_GOTO(loop_cond, label) do { \ + if (!fgets(line, sizeof(line), f)) { \ + av_log(ctx, AV_LOG_ERROR, "Unexpected EOF\n"); \ + ret = AVERROR_INVALIDDATA; \ + goto label; \ + } \ +} while (loop_cond) + +/* Basically r g and b float values on each line, with a facultative 3DLUTSIZE + * directive; seems to be generated by Davinci */ +static int parse_dat(AVFilterContext *ctx, LUT3DContext *lut3d, FILE *f) +{ + char line[MAX_LINE_SIZE]; + int ret, i, j, k, size, size2; + + lut3d->lutsize = size = 33; + size2 = size * size; + + NEXT_LINE(skip_line(line)); + if (!strncmp(line, "3DLUTSIZE ", 10)) { + size = strtol(line + 10, NULL, 0); + + NEXT_LINE(skip_line(line)); + } + + ret = ff_allocate_3dlut(ctx, lut3d, size, 0); + if (ret < 0) + return ret; + + for (k = 0; k < size; k++) { + for (j = 0; j < size; j++) { + for (i = 0; i < size; i++) { + struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; + if (k != 0 || j != 0 || i != 0) + NEXT_LINE(skip_line(line)); + if (av_sscanf(line, "%f %f %f", &vec->r, &vec->g, &vec->b) != 3) + return AVERROR_INVALIDDATA; + } + } + } + return 0; +} + +/* Iridas format */ +static int parse_cube(AVFilterContext *ctx, LUT3DContext *lut3d, FILE *f) +{ + char line[MAX_LINE_SIZE]; + float min[3] = {0.0, 0.0, 0.0}; + float max[3] = {1.0, 1.0, 1.0}; + + while (fgets(line, sizeof(line), f)) { + if (!strncmp(line, "LUT_3D_SIZE", 11)) { + int ret, i, j, k; + const int size = strtol(line + 12, NULL, 0); + const int size2 = size * size; + + ret = ff_allocate_3dlut(ctx, lut3d, size, 0); + if (ret < 0) + return ret; + + for (k = 0; k < size; k++) { + for (j = 0; j < size; j++) { + for (i = 0; i < size; i++) { + struct rgbvec *vec = &lut3d->lut[i * size2 + j * size + k]; + + do { +try_again: + NEXT_LINE(0); + if (!strncmp(line, "DOMAIN_", 7)) { + float *vals = NULL; + if (!strncmp(line + 7, "MIN ", 4)) vals = min; + else if (!strncmp(line + 7, "MAX ", 4)) vals = max; + if (!vals) + return AVERROR_INVALIDDATA; + av_sscanf(line + 11, "%f %f %f", vals, vals + 1, vals + 2); + av_log(ctx, AV_LOG_DEBUG, "min: %f %f %f | max: %f %f %f\n", + min[0], min[1], min[2], max[0], max[1], max[2]); + goto try_again; + } else if (!strncmp(line, "TITLE", 5)) { + goto try_again; + } + } while (skip_line(line)); + if (av_sscanf(line, "%f %f %f", &vec->r, &vec->g, &vec->b) != 3) + return AVERROR_INVALIDDATA; + } + } + } + break; + } + } + + lut3d->scale.r = av_clipf(1. / (max[0] - min[0]), 0.f, 1.f); + lut3d->scale.g = av_clipf(1. / (max[1] - min[1]), 0.f, 1.f); + lut3d->scale.b = av_clipf(1. / (max[2] - min[2]), 0.f, 1.f); + + return 0; +} + +/* Assume 17x17x17 LUT with a 16-bit depth + * FIXME: it seems there are various 3dl formats */ +static int parse_3dl(AVFilterContext *ctx, LUT3DContext *lut3d, FILE *f) +{ + char line[MAX_LINE_SIZE]; + int ret, i, j, k; + const int size = 17; + const int size2 = 17 * 17; + const float scale = 16*16*16; + + lut3d->lutsize = size; + + ret = ff_allocate_3dlut(ctx, lut3d, size, 0); + if (ret < 0) + return ret; + + NEXT_LINE(skip_line(line)); + for (k = 0; k < size; k++) { + for (j = 0; j < size; j++) { + for (i = 0; i < size; i++) { + int r, g, b; + struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; + + NEXT_LINE(skip_line(line)); + if (av_sscanf(line, "%d %d %d", &r, &g, &b) != 3) + return AVERROR_INVALIDDATA; + vec->r = r / scale; + vec->g = g / scale; + vec->b = b / scale; + } + } + } + return 0; +} + +/* Pandora format */ +static int parse_m3d(AVFilterContext *ctx, LUT3DContext *lut3d, FILE *f) +{ + float scale; + int ret, i, j, k, size, size2, in = -1, out = -1; + char line[MAX_LINE_SIZE]; + uint8_t rgb_map[3] = {0, 1, 2}; + + while (fgets(line, sizeof(line), f)) { + if (!strncmp(line, "in", 2)) in = strtol(line + 2, NULL, 0); + else if (!strncmp(line, "out", 3)) out = strtol(line + 3, NULL, 0); + else if (!strncmp(line, "values", 6)) { + const char *p = line + 6; +#define SET_COLOR(id) do { \ + while (av_isspace(*p)) \ + p++; \ + switch (*p) { \ + case 'r': rgb_map[id] = 0; break; \ + case 'g': rgb_map[id] = 1; break; \ + case 'b': rgb_map[id] = 2; break; \ + } \ + while (*p && !av_isspace(*p)) \ + p++; \ +} while (0) + SET_COLOR(0); + SET_COLOR(1); + SET_COLOR(2); + break; + } + } + + if (in == -1 || out == -1) { + av_log(ctx, AV_LOG_ERROR, "in and out must be defined\n"); + return AVERROR_INVALIDDATA; + } + if (in < 2 || out < 2 || + in > MAX_LEVEL*MAX_LEVEL*MAX_LEVEL || + out > MAX_LEVEL*MAX_LEVEL*MAX_LEVEL) { + av_log(ctx, AV_LOG_ERROR, "invalid in (%d) or out (%d)\n", in, out); + return AVERROR_INVALIDDATA; + } + for (size = 1; size*size*size < in; size++); + lut3d->lutsize = size; + size2 = size * size; + + ret = ff_allocate_3dlut(ctx, lut3d, size, 0); + if (ret < 0) + return ret; + + scale = 1. / (out - 1); + + for (k = 0; k < size; k++) { + for (j = 0; j < size; j++) { + for (i = 0; i < size; i++) { + struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; + float val[3]; + + NEXT_LINE(0); + if (av_sscanf(line, "%f %f %f", val, val + 1, val + 2) != 3) + return AVERROR_INVALIDDATA; + vec->r = val[rgb_map[0]] * scale; + vec->g = val[rgb_map[1]] * scale; + vec->b = val[rgb_map[2]] * scale; + } + } + } + return 0; +} + +static int nearest_sample_index(float *data, float x, int low, int hi) +{ + int mid; + if (x < data[low]) + return low; + + if (x > data[hi]) + return hi; + + for (;;) { + av_assert0(x >= data[low]); + av_assert0(x <= data[hi]); + av_assert0((hi-low) > 0); + + if (hi - low == 1) + return low; + + mid = (low + hi) / 2; + + if (x < data[mid]) + hi = mid; + else + low = mid; + } + + return 0; +} + +#define NEXT_FLOAT_OR_GOTO(value, label) \ + if (!fget_next_word(line, sizeof(line) ,f)) { \ + ret = AVERROR_INVALIDDATA; \ + goto label; \ + } \ + if (av_sscanf(line, "%f", &value) != 1) { \ + ret = AVERROR_INVALIDDATA; \ + goto label; \ + } + +static int parse_cinespace(AVFilterContext *ctx, LUT3DContext *lut3d, FILE *f) +{ + char line[MAX_LINE_SIZE]; + float in_min[3] = {0.0, 0.0, 0.0}; + float in_max[3] = {1.0, 1.0, 1.0}; + float out_min[3] = {0.0, 0.0, 0.0}; + float out_max[3] = {1.0, 1.0, 1.0}; + int inside_metadata = 0, size, size2; + int prelut = 0; + int ret = 0; + + int prelut_sizes[3] = {0, 0, 0}; + float *in_prelut[3] = {NULL, NULL, NULL}; + float *out_prelut[3] = {NULL, NULL, NULL}; + + NEXT_LINE_OR_GOTO(skip_line(line), end); + if (strncmp(line, "CSPLUTV100", 10)) { + av_log(ctx, AV_LOG_ERROR, "Not cineSpace LUT format\n"); + ret = AVERROR(EINVAL); + goto end; + } + + NEXT_LINE_OR_GOTO(skip_line(line), end); + if (strncmp(line, "3D", 2)) { + av_log(ctx, AV_LOG_ERROR, "Not 3D LUT format\n"); + ret = AVERROR(EINVAL); + goto end; + } + + while (1) { + NEXT_LINE_OR_GOTO(skip_line(line), end); + + if (!strncmp(line, "BEGIN METADATA", 14)) { + inside_metadata = 1; + continue; + } + if (!strncmp(line, "END METADATA", 12)) { + inside_metadata = 0; + continue; + } + if (inside_metadata == 0) { + int size_r, size_g, size_b; + + for (int i = 0; i < 3; i++) { + int npoints = strtol(line, NULL, 0); + + if (npoints > 2) { + float v,last; + + if (npoints > PRELUT_SIZE) { + av_log(ctx, AV_LOG_ERROR, "Prelut size too large.\n"); + ret = AVERROR_INVALIDDATA; + goto end; + } + + if (in_prelut[i] || out_prelut[i]) { + av_log(ctx, AV_LOG_ERROR, "Invalid file has multiple preluts.\n"); + ret = AVERROR_INVALIDDATA; + goto end; + } + + in_prelut[i] = (float*)av_malloc(npoints * sizeof(float)); + out_prelut[i] = (float*)av_malloc(npoints * sizeof(float)); + if (!in_prelut[i] || !out_prelut[i]) { + ret = AVERROR(ENOMEM); + goto end; + } + + prelut_sizes[i] = npoints; + in_min[i] = FLT_MAX; + in_max[i] = -FLT_MAX; + out_min[i] = FLT_MAX; + out_max[i] = -FLT_MAX; + + for (int j = 0; j < npoints; j++) { + NEXT_FLOAT_OR_GOTO(v, end) + in_min[i] = FFMIN(in_min[i], v); + in_max[i] = FFMAX(in_max[i], v); + in_prelut[i][j] = v; + if (j > 0 && v < last) { + av_log(ctx, AV_LOG_ERROR, "Invalid file, non increasing prelut.\n"); + ret = AVERROR(ENOMEM); + goto end; + } + last = v; + } + + for (int j = 0; j < npoints; j++) { + NEXT_FLOAT_OR_GOTO(v, end) + out_min[i] = FFMIN(out_min[i], v); + out_max[i] = FFMAX(out_max[i], v); + out_prelut[i][j] = v; + } + + } else if (npoints == 2) { + NEXT_LINE_OR_GOTO(skip_line(line), end); + if (av_sscanf(line, "%f %f", &in_min[i], &in_max[i]) != 2) { + ret = AVERROR_INVALIDDATA; + goto end; + } + NEXT_LINE_OR_GOTO(skip_line(line), end); + if (av_sscanf(line, "%f %f", &out_min[i], &out_max[i]) != 2) { + ret = AVERROR_INVALIDDATA; + goto end; + } + + } else { + av_log(ctx, AV_LOG_ERROR, "Unsupported number of pre-lut points.\n"); + ret = AVERROR_PATCHWELCOME; + goto end; + } + + NEXT_LINE_OR_GOTO(skip_line(line), end); + } + + if (av_sscanf(line, "%d %d %d", &size_r, &size_g, &size_b) != 3) { + ret = AVERROR(EINVAL); + goto end; + } + if (size_r != size_g || size_r != size_b) { + av_log(ctx, AV_LOG_ERROR, "Unsupported size combination: %dx%dx%d.\n", size_r, size_g, size_b); + ret = AVERROR_PATCHWELCOME; + goto end; + } + + size = size_r; + size2 = size * size; + + if (prelut_sizes[0] && prelut_sizes[1] && prelut_sizes[2]) + prelut = 1; + + ret = ff_allocate_3dlut(ctx, lut3d, size, prelut); + if (ret < 0) + return ret; + + for (int k = 0; k < size; k++) { + for (int j = 0; j < size; j++) { + for (int i = 0; i < size; i++) { + struct rgbvec *vec = &lut3d->lut[i * size2 + j * size + k]; + + NEXT_LINE_OR_GOTO(skip_line(line), end); + if (av_sscanf(line, "%f %f %f", &vec->r, &vec->g, &vec->b) != 3) { + ret = AVERROR_INVALIDDATA; + goto end; + } + + vec->r *= out_max[0] - out_min[0]; + vec->g *= out_max[1] - out_min[1]; + vec->b *= out_max[2] - out_min[2]; + } + } + } + + break; + } + } + + if (prelut) { + for (int c = 0; c < 3; c++) { + + lut3d->prelut.min[c] = in_min[c]; + lut3d->prelut.max[c] = in_max[c]; + lut3d->prelut.scale[c] = (1.0f / (float)(in_max[c] - in_min[c])) * (lut3d->prelut.size - 1); + + for (int i = 0; i < lut3d->prelut.size; ++i) { + float mix = (float) i / (float)(lut3d->prelut.size - 1); + float x = lerpf(in_min[c], in_max[c], mix), a, b; + + int idx = nearest_sample_index(in_prelut[c], x, 0, prelut_sizes[c]-1); + av_assert0(idx + 1 < prelut_sizes[c]); + + a = out_prelut[c][idx + 0]; + b = out_prelut[c][idx + 1]; + mix = x - in_prelut[c][idx]; + + lut3d->prelut.lut[c][i] = sanitizef(lerpf(a, b, mix)); + } + } + lut3d->scale.r = 1.00f; + lut3d->scale.g = 1.00f; + lut3d->scale.b = 1.00f; + + } else { + lut3d->scale.r = av_clipf(1. / (in_max[0] - in_min[0]), 0.f, 1.f); + lut3d->scale.g = av_clipf(1. / (in_max[1] - in_min[1]), 0.f, 1.f); + lut3d->scale.b = av_clipf(1. / (in_max[2] - in_min[2]), 0.f, 1.f); + } + +end: + for (int c = 0; c < 3; c++) { + av_freep(&in_prelut[c]); + av_freep(&out_prelut[c]); + } + return ret; +} + +av_cold int ff_lut3d_init(AVFilterContext *ctx, LUT3DContext *lut3d) +{ + int ret; + FILE *f; + const char *ext; + + lut3d->scale.r = lut3d->scale.g = lut3d->scale.b = 1.f; + + if (!lut3d->file) { + return set_identity_matrix(ctx, lut3d, 32); + } + + f = avpriv_fopen_utf8(lut3d->file, "r"); + if (!f) { + ret = AVERROR(errno); + av_log(ctx, AV_LOG_ERROR, "%s: %s\n", lut3d->file, av_err2str(ret)); + return ret; + } + + ext = strrchr(lut3d->file, '.'); + if (!ext) { + av_log(ctx, AV_LOG_ERROR, "Unable to guess the format from the extension\n"); + ret = AVERROR_INVALIDDATA; + goto end; + } + ext++; + + if (!av_strcasecmp(ext, "dat")) { + ret = parse_dat(ctx, lut3d, f); + } else if (!av_strcasecmp(ext, "3dl")) { + ret = parse_3dl(ctx, lut3d, f); + } else if (!av_strcasecmp(ext, "cube")) { + ret = parse_cube(ctx, lut3d, f); + } else if (!av_strcasecmp(ext, "m3d")) { + ret = parse_m3d(ctx, lut3d, f); + } else if (!av_strcasecmp(ext, "csp")) { + ret = parse_cinespace(ctx, lut3d, f); + } else { + av_log(ctx, AV_LOG_ERROR, "Unrecognized '.%s' file type\n", ext); + ret = AVERROR(EINVAL); + } + + if (!ret && !lut3d->lutsize) { + av_log(ctx, AV_LOG_ERROR, "3D LUT is empty\n"); + ret = AVERROR_INVALIDDATA; + } + +end: + fclose(f); + return ret; +} + +av_cold void ff_lut3d_uninit(LUT3DContext *lut3d) +{ + int i; + av_freep(&lut3d->lut); + + for (i = 0; i < 3; i++) { + av_freep(&lut3d->prelut.lut[i]); + } +} diff --git a/libavfilter/lut3d.h b/libavfilter/lut3d.h index 14e3c7fea6..b6aaed85f1 100644 --- a/libavfilter/lut3d.h +++ b/libavfilter/lut3d.h @@ -84,4 +84,17 @@ typedef struct ThreadData { void ff_lut3d_init_x86(LUT3DContext *s, const AVPixFmtDescriptor *desc); +int ff_allocate_3dlut(AVFilterContext *ctx, LUT3DContext *lut3d, int lutsize, int prelut); + +/** + * Load 3D LUT from file. + * + * @param lut3d LUT3DContext Load 3D LUT from path specified by `lut3d->file`. + * If `lut3d->file` is NULL, initialize an identity 3D LUT. + */ +int ff_lut3d_init(AVFilterContext *ctx, LUT3DContext *lut3d); + +/** Release memory used to hold 3D LUT. */ +void ff_lut3d_uninit(LUT3DContext *lut3d); + #endif /* AVFILTER_LUT3D_H */ diff --git a/libavfilter/vf_lut3d.c b/libavfilter/vf_lut3d.c index 4edcc2c7a7..1da798e210 100644 --- a/libavfilter/vf_lut3d.c +++ b/libavfilter/vf_lut3d.c @@ -552,39 +552,6 @@ static int skip_line(const char *p) return !*p || *p == '#'; } -static char* fget_next_word(char* dst, int max, FILE* f) -{ - int c; - char *p = dst; - - /* for null */ - max--; - /* skip until next non whitespace char */ - while ((c = fgetc(f)) != EOF) { - if (av_isspace(c)) - continue; - - *p++ = c; - max--; - break; - } - - /* get max bytes or up until next whitespace char */ - for (; max > 0; max--) { - if ((c = fgetc(f)) == EOF) - break; - - if (av_isspace(c)) - break; - - *p++ = c; - } - - *p = 0; - if (p == dst) - return NULL; - return p; -} #define NEXT_LINE(loop_cond) do { \ if (!fgets(line, sizeof(line), f)) { \ @@ -593,505 +560,6 @@ static char* fget_next_word(char* dst, int max, FILE* f) } \ } while (loop_cond) -#define NEXT_LINE_OR_GOTO(loop_cond, label) do { \ - if (!fgets(line, sizeof(line), f)) { \ - av_log(ctx, AV_LOG_ERROR, "Unexpected EOF\n"); \ - ret = AVERROR_INVALIDDATA; \ - goto label; \ - } \ -} while (loop_cond) - -static int allocate_3dlut(AVFilterContext *ctx, int lutsize, int prelut) -{ - LUT3DContext *lut3d = ctx->priv; - int i; - if (lutsize < 2 || lutsize > MAX_LEVEL) { - av_log(ctx, AV_LOG_ERROR, "Too large or invalid 3D LUT size\n"); - return AVERROR(EINVAL); - } - - av_freep(&lut3d->lut); - lut3d->lut = av_malloc_array(lutsize * lutsize * lutsize, sizeof(*lut3d->lut)); - if (!lut3d->lut) - return AVERROR(ENOMEM); - - if (prelut) { - lut3d->prelut.size = PRELUT_SIZE; - for (i = 0; i < 3; i++) { - av_freep(&lut3d->prelut.lut[i]); - lut3d->prelut.lut[i] = av_malloc_array(PRELUT_SIZE, sizeof(*lut3d->prelut.lut[0])); - if (!lut3d->prelut.lut[i]) - return AVERROR(ENOMEM); - } - } else { - lut3d->prelut.size = 0; - for (i = 0; i < 3; i++) { - av_freep(&lut3d->prelut.lut[i]); - } - } - lut3d->lutsize = lutsize; - lut3d->lutsize2 = lutsize * lutsize; - return 0; -} - -/* Basically r g and b float values on each line, with a facultative 3DLUTSIZE - * directive; seems to be generated by Davinci */ -static int parse_dat(AVFilterContext *ctx, FILE *f) -{ - LUT3DContext *lut3d = ctx->priv; - char line[MAX_LINE_SIZE]; - int ret, i, j, k, size, size2; - - lut3d->lutsize = size = 33; - size2 = size * size; - - NEXT_LINE(skip_line(line)); - if (!strncmp(line, "3DLUTSIZE ", 10)) { - size = strtol(line + 10, NULL, 0); - - NEXT_LINE(skip_line(line)); - } - - ret = allocate_3dlut(ctx, size, 0); - if (ret < 0) - return ret; - - for (k = 0; k < size; k++) { - for (j = 0; j < size; j++) { - for (i = 0; i < size; i++) { - struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; - if (k != 0 || j != 0 || i != 0) - NEXT_LINE(skip_line(line)); - if (av_sscanf(line, "%f %f %f", &vec->r, &vec->g, &vec->b) != 3) - return AVERROR_INVALIDDATA; - } - } - } - return 0; -} - -/* Iridas format */ -static int parse_cube(AVFilterContext *ctx, FILE *f) -{ - LUT3DContext *lut3d = ctx->priv; - char line[MAX_LINE_SIZE]; - float min[3] = {0.0, 0.0, 0.0}; - float max[3] = {1.0, 1.0, 1.0}; - - while (fgets(line, sizeof(line), f)) { - if (!strncmp(line, "LUT_3D_SIZE", 11)) { - int ret, i, j, k; - const int size = strtol(line + 12, NULL, 0); - const int size2 = size * size; - - ret = allocate_3dlut(ctx, size, 0); - if (ret < 0) - return ret; - - for (k = 0; k < size; k++) { - for (j = 0; j < size; j++) { - for (i = 0; i < size; i++) { - struct rgbvec *vec = &lut3d->lut[i * size2 + j * size + k]; - - do { -try_again: - NEXT_LINE(0); - if (!strncmp(line, "DOMAIN_", 7)) { - float *vals = NULL; - if (!strncmp(line + 7, "MIN ", 4)) vals = min; - else if (!strncmp(line + 7, "MAX ", 4)) vals = max; - if (!vals) - return AVERROR_INVALIDDATA; - av_sscanf(line + 11, "%f %f %f", vals, vals + 1, vals + 2); - av_log(ctx, AV_LOG_DEBUG, "min: %f %f %f | max: %f %f %f\n", - min[0], min[1], min[2], max[0], max[1], max[2]); - goto try_again; - } else if (!strncmp(line, "TITLE", 5)) { - goto try_again; - } - } while (skip_line(line)); - if (av_sscanf(line, "%f %f %f", &vec->r, &vec->g, &vec->b) != 3) - return AVERROR_INVALIDDATA; - } - } - } - break; - } - } - - lut3d->scale.r = av_clipf(1. / (max[0] - min[0]), 0.f, 1.f); - lut3d->scale.g = av_clipf(1. / (max[1] - min[1]), 0.f, 1.f); - lut3d->scale.b = av_clipf(1. / (max[2] - min[2]), 0.f, 1.f); - - return 0; -} - -/* Assume 17x17x17 LUT with a 16-bit depth - * FIXME: it seems there are various 3dl formats */ -static int parse_3dl(AVFilterContext *ctx, FILE *f) -{ - char line[MAX_LINE_SIZE]; - LUT3DContext *lut3d = ctx->priv; - int ret, i, j, k; - const int size = 17; - const int size2 = 17 * 17; - const float scale = 16*16*16; - - lut3d->lutsize = size; - - ret = allocate_3dlut(ctx, size, 0); - if (ret < 0) - return ret; - - NEXT_LINE(skip_line(line)); - for (k = 0; k < size; k++) { - for (j = 0; j < size; j++) { - for (i = 0; i < size; i++) { - int r, g, b; - struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; - - NEXT_LINE(skip_line(line)); - if (av_sscanf(line, "%d %d %d", &r, &g, &b) != 3) - return AVERROR_INVALIDDATA; - vec->r = r / scale; - vec->g = g / scale; - vec->b = b / scale; - } - } - } - return 0; -} - -/* Pandora format */ -static int parse_m3d(AVFilterContext *ctx, FILE *f) -{ - LUT3DContext *lut3d = ctx->priv; - float scale; - int ret, i, j, k, size, size2, in = -1, out = -1; - char line[MAX_LINE_SIZE]; - uint8_t rgb_map[3] = {0, 1, 2}; - - while (fgets(line, sizeof(line), f)) { - if (!strncmp(line, "in", 2)) in = strtol(line + 2, NULL, 0); - else if (!strncmp(line, "out", 3)) out = strtol(line + 3, NULL, 0); - else if (!strncmp(line, "values", 6)) { - const char *p = line + 6; -#define SET_COLOR(id) do { \ - while (av_isspace(*p)) \ - p++; \ - switch (*p) { \ - case 'r': rgb_map[id] = 0; break; \ - case 'g': rgb_map[id] = 1; break; \ - case 'b': rgb_map[id] = 2; break; \ - } \ - while (*p && !av_isspace(*p)) \ - p++; \ -} while (0) - SET_COLOR(0); - SET_COLOR(1); - SET_COLOR(2); - break; - } - } - - if (in == -1 || out == -1) { - av_log(ctx, AV_LOG_ERROR, "in and out must be defined\n"); - return AVERROR_INVALIDDATA; - } - if (in < 2 || out < 2 || - in > MAX_LEVEL*MAX_LEVEL*MAX_LEVEL || - out > MAX_LEVEL*MAX_LEVEL*MAX_LEVEL) { - av_log(ctx, AV_LOG_ERROR, "invalid in (%d) or out (%d)\n", in, out); - return AVERROR_INVALIDDATA; - } - for (size = 1; size*size*size < in; size++); - lut3d->lutsize = size; - size2 = size * size; - - ret = allocate_3dlut(ctx, size, 0); - if (ret < 0) - return ret; - - scale = 1. / (out - 1); - - for (k = 0; k < size; k++) { - for (j = 0; j < size; j++) { - for (i = 0; i < size; i++) { - struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; - float val[3]; - - NEXT_LINE(0); - if (av_sscanf(line, "%f %f %f", val, val + 1, val + 2) != 3) - return AVERROR_INVALIDDATA; - vec->r = val[rgb_map[0]] * scale; - vec->g = val[rgb_map[1]] * scale; - vec->b = val[rgb_map[2]] * scale; - } - } - } - return 0; -} - -static int nearest_sample_index(float *data, float x, int low, int hi) -{ - int mid; - if (x < data[low]) - return low; - - if (x > data[hi]) - return hi; - - for (;;) { - av_assert0(x >= data[low]); - av_assert0(x <= data[hi]); - av_assert0((hi-low) > 0); - - if (hi - low == 1) - return low; - - mid = (low + hi) / 2; - - if (x < data[mid]) - hi = mid; - else - low = mid; - } - - return 0; -} - -#define NEXT_FLOAT_OR_GOTO(value, label) \ - if (!fget_next_word(line, sizeof(line) ,f)) { \ - ret = AVERROR_INVALIDDATA; \ - goto label; \ - } \ - if (av_sscanf(line, "%f", &value) != 1) { \ - ret = AVERROR_INVALIDDATA; \ - goto label; \ - } - -static int parse_cinespace(AVFilterContext *ctx, FILE *f) -{ - LUT3DContext *lut3d = ctx->priv; - char line[MAX_LINE_SIZE]; - float in_min[3] = {0.0, 0.0, 0.0}; - float in_max[3] = {1.0, 1.0, 1.0}; - float out_min[3] = {0.0, 0.0, 0.0}; - float out_max[3] = {1.0, 1.0, 1.0}; - int inside_metadata = 0, size, size2; - int prelut = 0; - int ret = 0; - - int prelut_sizes[3] = {0, 0, 0}; - float *in_prelut[3] = {NULL, NULL, NULL}; - float *out_prelut[3] = {NULL, NULL, NULL}; - - NEXT_LINE_OR_GOTO(skip_line(line), end); - if (strncmp(line, "CSPLUTV100", 10)) { - av_log(ctx, AV_LOG_ERROR, "Not cineSpace LUT format\n"); - ret = AVERROR(EINVAL); - goto end; - } - - NEXT_LINE_OR_GOTO(skip_line(line), end); - if (strncmp(line, "3D", 2)) { - av_log(ctx, AV_LOG_ERROR, "Not 3D LUT format\n"); - ret = AVERROR(EINVAL); - goto end; - } - - while (1) { - NEXT_LINE_OR_GOTO(skip_line(line), end); - - if (!strncmp(line, "BEGIN METADATA", 14)) { - inside_metadata = 1; - continue; - } - if (!strncmp(line, "END METADATA", 12)) { - inside_metadata = 0; - continue; - } - if (inside_metadata == 0) { - int size_r, size_g, size_b; - - for (int i = 0; i < 3; i++) { - int npoints = strtol(line, NULL, 0); - - if (npoints > 2) { - float v,last; - - if (npoints > PRELUT_SIZE) { - av_log(ctx, AV_LOG_ERROR, "Prelut size too large.\n"); - ret = AVERROR_INVALIDDATA; - goto end; - } - - if (in_prelut[i] || out_prelut[i]) { - av_log(ctx, AV_LOG_ERROR, "Invalid file has multiple preluts.\n"); - ret = AVERROR_INVALIDDATA; - goto end; - } - - in_prelut[i] = (float*)av_malloc(npoints * sizeof(float)); - out_prelut[i] = (float*)av_malloc(npoints * sizeof(float)); - if (!in_prelut[i] || !out_prelut[i]) { - ret = AVERROR(ENOMEM); - goto end; - } - - prelut_sizes[i] = npoints; - in_min[i] = FLT_MAX; - in_max[i] = -FLT_MAX; - out_min[i] = FLT_MAX; - out_max[i] = -FLT_MAX; - - for (int j = 0; j < npoints; j++) { - NEXT_FLOAT_OR_GOTO(v, end) - in_min[i] = FFMIN(in_min[i], v); - in_max[i] = FFMAX(in_max[i], v); - in_prelut[i][j] = v; - if (j > 0 && v < last) { - av_log(ctx, AV_LOG_ERROR, "Invalid file, non increasing prelut.\n"); - ret = AVERROR(ENOMEM); - goto end; - } - last = v; - } - - for (int j = 0; j < npoints; j++) { - NEXT_FLOAT_OR_GOTO(v, end) - out_min[i] = FFMIN(out_min[i], v); - out_max[i] = FFMAX(out_max[i], v); - out_prelut[i][j] = v; - } - - } else if (npoints == 2) { - NEXT_LINE_OR_GOTO(skip_line(line), end); - if (av_sscanf(line, "%f %f", &in_min[i], &in_max[i]) != 2) { - ret = AVERROR_INVALIDDATA; - goto end; - } - NEXT_LINE_OR_GOTO(skip_line(line), end); - if (av_sscanf(line, "%f %f", &out_min[i], &out_max[i]) != 2) { - ret = AVERROR_INVALIDDATA; - goto end; - } - - } else { - av_log(ctx, AV_LOG_ERROR, "Unsupported number of pre-lut points.\n"); - ret = AVERROR_PATCHWELCOME; - goto end; - } - - NEXT_LINE_OR_GOTO(skip_line(line), end); - } - - if (av_sscanf(line, "%d %d %d", &size_r, &size_g, &size_b) != 3) { - ret = AVERROR(EINVAL); - goto end; - } - if (size_r != size_g || size_r != size_b) { - av_log(ctx, AV_LOG_ERROR, "Unsupported size combination: %dx%dx%d.\n", size_r, size_g, size_b); - ret = AVERROR_PATCHWELCOME; - goto end; - } - - size = size_r; - size2 = size * size; - - if (prelut_sizes[0] && prelut_sizes[1] && prelut_sizes[2]) - prelut = 1; - - ret = allocate_3dlut(ctx, size, prelut); - if (ret < 0) - return ret; - - for (int k = 0; k < size; k++) { - for (int j = 0; j < size; j++) { - for (int i = 0; i < size; i++) { - struct rgbvec *vec = &lut3d->lut[i * size2 + j * size + k]; - - NEXT_LINE_OR_GOTO(skip_line(line), end); - if (av_sscanf(line, "%f %f %f", &vec->r, &vec->g, &vec->b) != 3) { - ret = AVERROR_INVALIDDATA; - goto end; - } - - vec->r *= out_max[0] - out_min[0]; - vec->g *= out_max[1] - out_min[1]; - vec->b *= out_max[2] - out_min[2]; - } - } - } - - break; - } - } - - if (prelut) { - for (int c = 0; c < 3; c++) { - - lut3d->prelut.min[c] = in_min[c]; - lut3d->prelut.max[c] = in_max[c]; - lut3d->prelut.scale[c] = (1.0f / (float)(in_max[c] - in_min[c])) * (lut3d->prelut.size - 1); - - for (int i = 0; i < lut3d->prelut.size; ++i) { - float mix = (float) i / (float)(lut3d->prelut.size - 1); - float x = lerpf(in_min[c], in_max[c], mix), a, b; - - int idx = nearest_sample_index(in_prelut[c], x, 0, prelut_sizes[c]-1); - av_assert0(idx + 1 < prelut_sizes[c]); - - a = out_prelut[c][idx + 0]; - b = out_prelut[c][idx + 1]; - mix = x - in_prelut[c][idx]; - - lut3d->prelut.lut[c][i] = sanitizef(lerpf(a, b, mix)); - } - } - lut3d->scale.r = 1.00f; - lut3d->scale.g = 1.00f; - lut3d->scale.b = 1.00f; - - } else { - lut3d->scale.r = av_clipf(1. / (in_max[0] - in_min[0]), 0.f, 1.f); - lut3d->scale.g = av_clipf(1. / (in_max[1] - in_min[1]), 0.f, 1.f); - lut3d->scale.b = av_clipf(1. / (in_max[2] - in_min[2]), 0.f, 1.f); - } - -end: - for (int c = 0; c < 3; c++) { - av_freep(&in_prelut[c]); - av_freep(&out_prelut[c]); - } - return ret; -} - -static int set_identity_matrix(AVFilterContext *ctx, int size) -{ - LUT3DContext *lut3d = ctx->priv; - int ret, i, j, k; - const int size2 = size * size; - const float c = 1. / (size - 1); - - ret = allocate_3dlut(ctx, size, 0); - if (ret < 0) - return ret; - - for (k = 0; k < size; k++) { - for (j = 0; j < size; j++) { - for (i = 0; i < size; i++) { - struct rgbvec *vec = &lut3d->lut[k * size2 + j * size + i]; - vec->r = k * c; - vec->g = j * c; - vec->b = i * c; - } - } - } - - return 0; -} - static const enum AVPixelFormat pix_fmts[] = { AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24, AV_PIX_FMT_RGBA, AV_PIX_FMT_BGRA, @@ -1230,66 +698,14 @@ AVFILTER_DEFINE_CLASS_EXT(lut3d, "lut3d", lut3d_haldclut_options); static av_cold int lut3d_init(AVFilterContext *ctx) { - int ret; - FILE *f; - const char *ext; LUT3DContext *lut3d = ctx->priv; - - lut3d->scale.r = lut3d->scale.g = lut3d->scale.b = 1.f; - - if (!lut3d->file) { - return set_identity_matrix(ctx, 32); - } - - f = avpriv_fopen_utf8(lut3d->file, "r"); - if (!f) { - ret = AVERROR(errno); - av_log(ctx, AV_LOG_ERROR, "%s: %s\n", lut3d->file, av_err2str(ret)); - return ret; - } - - ext = strrchr(lut3d->file, '.'); - if (!ext) { - av_log(ctx, AV_LOG_ERROR, "Unable to guess the format from the extension\n"); - ret = AVERROR_INVALIDDATA; - goto end; - } - ext++; - - if (!av_strcasecmp(ext, "dat")) { - ret = parse_dat(ctx, f); - } else if (!av_strcasecmp(ext, "3dl")) { - ret = parse_3dl(ctx, f); - } else if (!av_strcasecmp(ext, "cube")) { - ret = parse_cube(ctx, f); - } else if (!av_strcasecmp(ext, "m3d")) { - ret = parse_m3d(ctx, f); - } else if (!av_strcasecmp(ext, "csp")) { - ret = parse_cinespace(ctx, f); - } else { - av_log(ctx, AV_LOG_ERROR, "Unrecognized '.%s' file type\n", ext); - ret = AVERROR(EINVAL); - } - - if (!ret && !lut3d->lutsize) { - av_log(ctx, AV_LOG_ERROR, "3D LUT is empty\n"); - ret = AVERROR_INVALIDDATA; - } - -end: - fclose(f); - return ret; + return ff_lut3d_init(ctx, lut3d); } static av_cold void lut3d_uninit(AVFilterContext *ctx) { LUT3DContext *lut3d = ctx->priv; - int i; - av_freep(&lut3d->lut); - - for (i = 0; i < 3; i++) { - av_freep(&lut3d->prelut.lut[i]); - } + ff_lut3d_uninit(lut3d); } static const AVFilterPad lut3d_inputs[] = { @@ -1499,7 +915,7 @@ static int config_clut(AVFilterLink *inlink) return AVERROR(EINVAL); } - return allocate_3dlut(ctx, level, 0); + return ff_allocate_3dlut(ctx, lut3d, level, 0); } static int update_apply_clut(FFFrameSync *fs) From patchwork Sat Sep 23 15:36:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chen Yufei X-Patchwork-Id: 43880 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:2a18:b0:15d:8365:d4b8 with SMTP id e24csp282952pzh; Sat, 23 Sep 2023 08:42:42 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFPfy76t4ZPxGHJNVWgMRxRdGf00BOIsEBiKcxFOL5b4Lzd1libkxeYk6NHDc3oTYXaqVUA X-Received: by 2002:a17:907:3ea8:b0:9ad:e62c:4517 with SMTP id hs40-20020a1709073ea800b009ade62c4517mr8459093ejc.34.1695483762083; Sat, 23 Sep 2023 08:42:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695483762; cv=none; d=google.com; s=arc-20160816; b=y/cNsKl/ZQRFX5zwHnC8GerTgQVySvthRxlEmKbCfdlZZj7ozrgupUtyw5HtQZ4kbj yq3QrkcOx7nyRch8bagUAGNh/BhNGIeen2rmNbdI546kzwFYfC8VPGa7+BdHW4odRVwL uiEB3Q41rFUX9Z2KJFDuGc7p+WpWA1ZU854pma2zkag2m5fWN9WtqTCG3lPDL0xmJ62E QwPrCavylY9Tq1JW6Xr7qL/02cfSQl5l+wX+stvUEMb+xbJcyasaI2TNpIflH1qFg07n HTf/1kBDt7/NxicpCyCtQIv7XjPTH/ZyF0uoHVf3yMAOv4m7q/FXoFapoznbKiK3IByq dxtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=AYY15uyvj07CV0kj46tsK5gsap2Q2SQWRjuy3IPOZ0I=; fh=WMNJqm9yn32VIGoZSiFxz00xdqNXh1DsfrkZBt4KBsg=; b=vd0xsMJr5WFPVq6i3/w6s0ySvWikqbmEV2W+0uhhdxlEEFX6H0wVcQcnZS1ImhvOF/ m7yqFCgIsNwbXMb2AXazjNhfaMuz1obsJdWqMIPssHXaLgPglhmS4hkRio8lhg4ganCF o99R6Rzu3hlnO7M8lHZCNc+nsFsX0DGix0VNOCqE4ZyDxdS7GEUzxYU7B7InyMH0gt2q N3YoGaU+w8MyGEDygzg10yZjF+dNV/q0sYxJdbjz8nq6XEDhJNXKBK2z6bJtHODqGFY1 /XEm2pp1sgeY2eQ9HrmhLQT1K+ybAO/TNv4iXxhOT8s8VpbFtUr7d1Tl6XwKElxyjacz h3DQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=dIxJQnWr; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a24-20020a170906685800b00987d26a0998si5624580ejs.455.2023.09.23.08.42.41; Sat, 23 Sep 2023 08:42:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=dIxJQnWr; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5061768C9D5; Sat, 23 Sep 2023 18:42:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oa1-f41.google.com (mail-oa1-f41.google.com [209.85.160.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 587EC68C8B4 for ; Sat, 23 Sep 2023 18:42:27 +0300 (EEST) Received: by mail-oa1-f41.google.com with SMTP id 586e51a60fabf-1dce0c05171so925126fac.3 for ; Sat, 23 Sep 2023 08:42:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695483745; x=1696088545; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hUsW82VKYajesMKCas5NqDRFnHKGpntaP6Y7kAK9iUM=; b=dIxJQnWr83eVy/E2onz2QjmvqhT6auAiLwpJW1f5u1Leb7oj/HAo5qJBd5A7mqqwQr PiWMvkQ4KXil18f8oOvX1H2gR0IMqLd9N7bfJrJve2eR5EnO5cz7kAhVmI3wN+BoKC2O 8OnIONCgapqdzi8xRo/SSC//89Mlrmu5GKG+zc7CNWKIkkuXcpIAhz0GH2yZSblW0Wyx fMbPjSgf15q0x9mAhMHRNg79m668qw/10DMGsZpQqNkt0lhyx5uWnOEe8cSDWAapEL/z JTmW0h/OO5LMnfFpyIQ8dEsEN1TfiqcJTrQhdqa6RJLkaykBk5hOGvuNjHzfWXOBQtWo o4XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695483745; x=1696088545; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hUsW82VKYajesMKCas5NqDRFnHKGpntaP6Y7kAK9iUM=; b=XcR2XCLvKT+cEfpolxuMgCOgO1QEzO0xKvFbTU8X7EllxUYzVsdydKPXNAL8WILbzt RO5fiBaEBoEytYOvXLxPbuvKUZtNQv95UH4aVB6uNFUHLdvY0FQy1Z+7jdaPko4XqhOX m9V6k2g0r5QtSbGF5lZ3Q3FU9WW6C7JQAn6w/3orq8jWSJcF1Ras3FtXcV09aXh8NriU vPAyjl1cTPb+M/J9G0rbxWhlX7CzMDIww9LnB492At+L4Bscn5N1Mpj21RP2uGMwTvOo kbEDZSO68TXhauiF7fzhxuGVnD5cc2j7UL22JQvs10JgzEivZ0+B4m67GL0RVdOGj5vO ge9A== X-Gm-Message-State: AOJu0Yyxr8jft/qvtAph7tkxxxYar6Kxt7XR0Hk+fD41/qkXki2aVKyL ykEuuV9JiUUUwtzw3wmXihB36DlKq0k= X-Received: by 2002:a05:6870:a90e:b0:1d5:5d44:7404 with SMTP id eq14-20020a056870a90e00b001d55d447404mr3408257oab.43.1695483745500; Sat, 23 Sep 2023 08:42:25 -0700 (PDT) Received: from archvm.home (hwsrv-1088392.hostwindsdns.com. [23.254.253.23]) by smtp.gmail.com with ESMTPSA id e2-20020a63ae42000000b0057c630d606asm3307178pgp.69.2023.09.23.08.42.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 Sep 2023 08:42:25 -0700 (PDT) From: Chen Yufei To: ffmpeg-devel@ffmpeg.org Date: Sat, 23 Sep 2023 23:36:10 +0800 Message-ID: <20230923154125.31376-3-cyfdecyf@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230923154125.31376-1-cyfdecyf@gmail.com> References: <20230923154125.31376-1-cyfdecyf@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avfilter/vf_vpp_qsv: apply 3D LUT from file. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Chen Yufei Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7m9M+EJnQrYC Usage: "vpp_qsv=lut3d_file=" Only enabled with VAAPI because using VASurface to store 3D LUT. Signed-off-by: Chen Yufei --- libavfilter/vf_vpp_qsv.c | 241 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 236 insertions(+), 5 deletions(-) diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c index c07b45fedb..cd913d3c40 100644 --- a/libavfilter/vf_vpp_qsv.c +++ b/libavfilter/vf_vpp_qsv.c @@ -23,6 +23,7 @@ #include +#include "config.h" #include "config_components.h" #include "libavutil/opt.h" @@ -37,10 +38,15 @@ #include "internal.h" #include "avfilter.h" #include "filters.h" +#include "lut3d.h" #include "qsvvpp.h" #include "transpose.h" +#if QSV_ONEVPL && CONFIG_VAAPI +#include +#endif + #define OFFSET(x) offsetof(VPPContext, x) #define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM) @@ -67,6 +73,10 @@ typedef struct VPPContext{ /** HDR parameters attached on the input frame */ mfxExtMasteringDisplayColourVolume mdcv_conf; mfxExtContentLightLevelInfo clli_conf; + + /** LUT parameters attached on the input frame */ + mfxExtVPP3DLut lut3d_conf; + LUT3DContext lut3d; #endif /** @@ -260,6 +270,7 @@ static av_cold int vpp_preinit(AVFilterContext *ctx) static av_cold int vpp_init(AVFilterContext *ctx) { + int ret = 0; VPPContext *vpp = ctx->priv; if (!vpp->output_format_str || !strcmp(vpp->output_format_str, "same")) { @@ -288,9 +299,9 @@ static av_cold int vpp_init(AVFilterContext *ctx) STRING_OPTION(color_primaries, color_primaries, AVCOL_PRI_UNSPECIFIED); STRING_OPTION(color_transfer, color_transfer, AVCOL_TRC_UNSPECIFIED); STRING_OPTION(color_matrix, color_space, AVCOL_SPC_UNSPECIFIED); - #undef STRING_OPTION - return 0; + + return ret; } static int config_input(AVFilterLink *inlink) @@ -388,6 +399,194 @@ static mfxStatus get_mfx_version(const AVFilterContext *ctx, mfxVersion *mfx_ver return MFXQueryVersion(device_hwctx->session, mfx_version); } +#if QSV_ONEVPL && CONFIG_VAAPI +static mfxStatus get_va_display(AVFilterContext *ctx, VADisplay *va_display) +{ + VPPContext *vpp = ctx->priv; + QSVVPPContext *qsvvpp = &vpp->qsv; + mfxHDL handle; + mfxStatus ret; + + ret = MFXVideoCORE_GetHandle(qsvvpp->session, MFX_HANDLE_VA_DISPLAY, &handle); + if (ret != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "MFXVideoCORE_GetHandle failed, status: %d\n", ret); + *va_display = NULL; + return ret; + } + + *va_display = (VADisplay)handle; + return MFX_ERR_NONE; +} + +// Allocate memory on device and copy 3D LUT table. +// Reference https://spec.oneapi.io/onevpl/2.9.0/programming_guide/VPL_prg_vpp.html#video-processing-3dlut +static int init_3dlut_surface(AVFilterContext *ctx) +{ + VPPContext *vpp = ctx->priv; + LUT3DContext *lut3d = &vpp->lut3d; + mfxExtVPP3DLut *lut3d_conf = &vpp->lut3d_conf; + + VAStatus ret = 0; + VADisplay va_dpy = 0; + VASurfaceID surface_id = 0; + VASurfaceAttrib surface_attrib; + VAImage surface_image; + mfxU16 *surface_u16 = NULL; + mfx3DLutMemoryLayout mem_layout; + mfxMemId mem_id = 0; + + int lut_size = lut3d->lutsize; + int mul_size = 0; + + int r, g, b, lut_idx, sf_idx; + struct rgbvec *s = NULL; + + av_log(ctx, AV_LOG_VERBOSE, "create 3D LUT surface, size: %u.\n", lut_size); + + switch (lut_size) { + case 17: + mul_size = 32; + mem_layout = MFX_3DLUT_MEMORY_LAYOUT_INTEL_17LUT; + break; + case 33: + mul_size = 64; + mem_layout = MFX_3DLUT_MEMORY_LAYOUT_INTEL_33LUT; + break; + case 65: + mul_size = 128; + mem_layout = MFX_3DLUT_MEMORY_LAYOUT_INTEL_65LUT; + break; + default: + av_log(ctx, AV_LOG_ERROR, "3D LUT surface supports only LUT size: 17, 33, 65."); + return AVERROR(EINVAL); + } + + ret = get_va_display(ctx, &va_dpy); + if (ret != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "get VADisplay failed, unable to create 3D LUT surface.\n"); + return ret; + } + + memset(&surface_attrib, 0, sizeof(surface_attrib)); + surface_attrib.type = VASurfaceAttribPixelFormat; + surface_attrib.flags = VA_SURFACE_ATTRIB_SETTABLE; + surface_attrib.value.type = VAGenericValueTypeInteger; + surface_attrib.value.value.i = VA_FOURCC_RGBA; + + ret = vaCreateSurfaces(va_dpy, + VA_RT_FORMAT_RGB32, // 4 bytes + lut_size * mul_size, // width + lut_size * 2, // height + &surface_id, 1, + &surface_attrib, 1); + if (ret != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "vaCreateSurfaces for 3D LUT surface failed, status: %d %s\n", ret, vaErrorStr(ret)); + return AVERROR(ret); + } + av_log(ctx, AV_LOG_DEBUG, "3D LUT surface id %u\n", surface_id); + + ret = vaSyncSurface(va_dpy, surface_id); + if (ret != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "vaSyncSurface for 3D LUT surface failed, status: %d %s\n", ret, vaErrorStr(ret)); + goto err_destroy_surface; + } + + memset(&surface_image, 0, sizeof(surface_image)); + ret = vaDeriveImage(va_dpy, surface_id, &surface_image); + if (ret != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "vaDeriveImage for 3D LUT surface failed, status: %d %s\n", ret, vaErrorStr(ret)); + goto err_destroy_surface; + } + if (surface_image.format.fourcc != VA_FOURCC_RGBA) { + av_log(ctx, AV_LOG_ERROR, "vaDeriveImage format is not expected VA_FOURCC_RGBA, got 0x%x\n", surface_image.format.fourcc); + goto err_destroy_image; + } + + // Map surface to system memory for copy 3D LUT table. + ret = vaMapBuffer(va_dpy, surface_image.buf, (void **)&surface_u16); + if (ret != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "vaMapBuffer for 3D LUT surface failed, status: %d %s\n", ret, vaErrorStr(ret)); + goto err_destroy_image; + } + + // Copy 3D LUT to surface. + memset(surface_u16, 0, surface_image.width * surface_image.height * 4); +#define INTEL_3DLUT_SCALE (UINT16_MAX - 1) + for (r = 0; r < lut_size; ++r) { + for (g = 0; g < lut_size; ++g) { + for (b = 0; b < lut_size; ++b) { + lut_idx = r * lut_size * lut_size + g * lut_size + b; + s = &lut3d->lut[lut_idx]; + + sf_idx = (r * lut_size * mul_size + g * mul_size + b) * 4; + surface_u16[sf_idx + 0] = (mfxU16)(s->r * INTEL_3DLUT_SCALE); + surface_u16[sf_idx + 1] = (mfxU16)(s->g * INTEL_3DLUT_SCALE); + surface_u16[sf_idx + 2] = (mfxU16)(s->b * INTEL_3DLUT_SCALE); + // surface_u16[sf_idx + 4] is reserved channel. + } + } + } +#undef INTEL_3DLUT_SCALE + + if (vaUnmapBuffer(va_dpy, surface_image.buf)) { + av_log(ctx, AV_LOG_ERROR, "vaUnmapBuffer for 3D LUT surface failed, status: %d %s\n", ret, vaErrorStr(ret)); + goto err_destroy_image; + } + vaDestroyImage(va_dpy, surface_image.image_id); + + mem_id = av_malloc(sizeof(VASurfaceID)); + if (mem_id == 0) { + ret = AVERROR(ENOMEM); + goto err_destroy_surface; + } + + av_log(ctx, AV_LOG_DEBUG, + "upload 3D LUT surface width %d, height %d\n", + (int)surface_image.width, (int)surface_image.height); + + memset(lut3d_conf, 0, sizeof(*lut3d_conf)); + lut3d_conf->Header.BufferId = MFX_EXTBUFF_VPP_3DLUT; + lut3d_conf->Header.BufferSz = sizeof(*lut3d_conf); + lut3d_conf->ChannelMapping = MFX_3DLUT_CHANNEL_MAPPING_RGB_RGB; + lut3d_conf->BufferType = MFX_RESOURCE_VA_SURFACE; + lut3d_conf->VideoBuffer.DataType = MFX_DATA_TYPE_U16; + lut3d_conf->VideoBuffer.MemLayout = mem_layout; + lut3d_conf->VideoBuffer.MemId = mem_id; + *((VASurfaceID*)lut3d_conf->VideoBuffer.MemId) = surface_id; + + return 0; + +err_destroy_image: + vaDestroyImage(va_dpy, surface_image.image_id); +err_destroy_surface: + vaDestroySurfaces(va_dpy, &surface_id, 1); + return ret; +} + +static int uninit_3dlut_surface(AVFilterContext *ctx) { + VPPContext *vpp = ctx->priv; + mfxExtVPP3DLut *lut3d_conf = &vpp->lut3d_conf; + VADisplay va_dpy = 0; + int ret; + + if (lut3d_conf->Header.BufferId == MFX_EXTBUFF_VPP_3DLUT) { + ret = get_va_display(ctx, &va_dpy); + if (!va_dpy) { + return ret; + } + ret = vaDestroySurfaces(va_dpy, (VASurfaceID*)lut3d_conf->VideoBuffer.MemId, 1); + if (ret != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "vaDestroySurfaces failed, status: %d %s\n", ret, vaErrorStr(ret) ); + return ret; + } + av_free(lut3d_conf->VideoBuffer.MemId); + } + memset(lut3d_conf, 0, sizeof(*lut3d_conf)); + + return 0; +} +#endif // QSV_ONEVPL && CONFIG_VAAPI + static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVFrame *out, QSVVPPFrameParam *fp) { #if QSV_ONEVPL @@ -401,6 +600,7 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF fp->num_ext_buf = 0; + av_log(ctx, AV_LOG_DEBUG, "vpp_set_frame_ext_params QSV_ONEVPL\n"); if (!in || !out || !QSV_RUNTIME_VERSION_ATLEAST(qsvvpp->ver, 2, 0)) return 0; @@ -499,6 +699,13 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF outvsi_conf.MatrixCoefficients = (out->colorspace == AVCOL_SPC_UNSPECIFIED) ? AVCOL_SPC_BT709 : out->colorspace; outvsi_conf.ColourDescriptionPresent = 1; +#if CONFIG_VAAPI + if (vpp->lut3d.file && (vpp->lut3d_conf.Header.BufferId == 0)) { + // 3D LUT does not depend on in/out frame, so initialize just once. + init_3dlut_surface(ctx); + } +#endif + if (memcmp(&vpp->invsi_conf, &invsi_conf, sizeof(mfxExtVideoSignalInfo)) || memcmp(&vpp->mdcv_conf, &mdcv_conf, sizeof(mfxExtMasteringDisplayColourVolume)) || memcmp(&vpp->clli_conf, &clli_conf, sizeof(mfxExtContentLightLevelInfo)) || @@ -516,6 +723,10 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF vpp->clli_conf = clli_conf; if (clli_conf.Header.BufferId) fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer*)&vpp->clli_conf; + + if (vpp->lut3d_conf.Header.BufferId) { + fp->ext_buf[fp->num_ext_buf++] = (mfxExtBuffer *)&vpp->lut3d_conf; + } } #endif @@ -524,6 +735,7 @@ static int vpp_set_frame_ext_params(AVFilterContext *ctx, const AVFrame *in, AVF static int config_output(AVFilterLink *outlink) { + int ret; AVFilterContext *ctx = outlink->src; VPPContext *vpp = ctx->priv; QSVVPPParam param = { NULL }; @@ -711,9 +923,17 @@ static int config_output(AVFilterLink *outlink) vpp->color_transfer != AVCOL_TRC_UNSPECIFIED || vpp->color_matrix != AVCOL_SPC_UNSPECIFIED || vpp->tonemap || - !vpp->has_passthrough) + vpp->lut3d.file || + !vpp->has_passthrough) { + if (vpp->lut3d.file) { + av_log(ctx, AV_LOG_INFO, "load 3D LUT from file: %s\n", vpp->lut3d.file); + ret = ff_lut3d_init(ctx, &vpp->lut3d); + if (ret != 0) { + return ret; + } + } return ff_qsvvpp_init(ctx, ¶m); - else { + } else { /* No MFX session is created in this case */ av_log(ctx, AV_LOG_VERBOSE, "qsv vpp pass through mode.\n"); if (inlink->hw_frames_ctx) @@ -801,6 +1021,15 @@ eof: static av_cold void vpp_uninit(AVFilterContext *ctx) { + VPPContext *vpp = ctx->priv; + +#if QSV_ONEVPL && CONFIG_VAAPI + uninit_3dlut_surface(ctx); +#endif + + if (vpp->lut3d.file) { + ff_lut3d_uninit(&vpp->lut3d); + } ff_qsvvpp_close(ctx); } @@ -924,7 +1153,9 @@ static const AVOption vpp_options[] = { OFFSET(color_transfer_str), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, {"tonemap", "Perform tonemapping (0=disable tonemapping, 1=perform tonemapping if the input has HDR metadata)", OFFSET(tonemap), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 1, .flags = FLAGS}, - +#if QSV_ONEVPL && CONFIG_VAAPI + { "lut3d_file", "Load and apply 3D LUT file", OFFSET(lut3d) + offsetof(LUT3DContext, file), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = FLAGS }, +#endif { NULL } };