From patchwork Fri Mar 15 11:58:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47100 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp193639pzh; Fri, 15 Mar 2024 05:05:21 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW022ANVdXI6ToGzWCo9MSZW4PwUaTG8ppuma4T2sh5ccFbrOJtavskrh7qR1Fys362rPmfb9urBbkFr50tiB7VZ/nee0MhBhgpag== X-Google-Smtp-Source: AGHT+IEwEQGtswxwhjgCoIY5YERx/TeWZgofHWeZBp1sql0wL7y0tH9Os8Dz9YgpfdGSDEBbloI+ X-Received: by 2002:a2e:7015:0:b0:2d4:71e8:5312 with SMTP id l21-20020a2e7015000000b002d471e85312mr3109818ljc.23.1710504321006; Fri, 15 Mar 2024 05:05:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504320; cv=none; d=google.com; s=arc-20160816; b=qB1bix/mNmCP6A2poYwsg7rLvfI2GvxRIX2msDElAsnNXplB6mZ5GI3ujb2SJlQwTX W9C1Al/j9Oh5Ehosf3jaLDkAFlPZFxg7k0nj5RyZ2th6w0WA3vrV80pJOWw/myM+1eEK WooPlOoQAgjbKOlbSER26fuHg9Ls9JuzKE4XmLuk/9RkWFWncBy4pVNofGJiDQ2TpLdk lcschaHIHH7Lr2qhS3GHzNPiMd/tJfnJMrLlKu6Fc8qS8eN5xZ7/w1EKIbF5jlzS/qPM m0D+M6oMM9+dJe8K6T9rwjBBlS0tskNeSr16/lhJq0MlaPRhu3YrDrjAzjzdvGTtHIi9 QjJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=eAwA+kQ+pojPWilQmfF709Bos843hR40KSHZ7oAi5Q4=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=fXM759stOjDRP/zHbYhVki7Wf7fi1O8OTfF9pBzOarL+7k5rxU/oxA4YjxOAW4iwgj J93TuEq/x5y+n+1jLkaGyNEjw5qdCKoxBJwWEVzEW0hUOddPQkyeYboA1jBJ8XuLv/0/ vuieb1vbvSqr9n+s/euMPM/ZEzDB7YUbQKBTOirxl1Oq6vXICjnrIq4WGXF2yVNqGsrr sWOIQvsO6sKmwU60rQcI6qLPpbWP4N3oKkRHCbRsisEse9kNuI7UTbbhug35PzO3iDGZ QcwgqeWKmZJ2EjO/eJMJB8583FBOLgiLQDgSchwmHWFCP7fmqmdjiBaNXgB1smxWLbyI SZyA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="hDluH/E6"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h10-20020a0564020e0a00b005689f107675si1775511edh.475.2024.03.15.05.05.07; Fri, 15 Mar 2024 05:05:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="hDluH/E6"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5345C68D170; Fri, 15 Mar 2024 14:04:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8910F68CF9D for ; Fri, 15 Mar 2024 14:04:46 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504286; bh=0PnNFfKMNC4Nbvt4v+GtVkE40zQYNcZ7xn2v2wPKMRA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hDluH/E6rsLGh9+V8EIJAgmPsT0tRKNHMZsrhA37G3QfguNTnRkuJec3eziYdK/vk f7k45+NSJRPf9vtkc1t49F+2KbpNlsjAMr5K2FTD23qEe9C4NElf46q/7EIXIC998I eVSQRX4CWbnQtPjZJ3PeX7BwxUyUTN+xJkCu0cKg= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 4A5954223F; Fri, 15 Mar 2024 13:04:46 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:58:54 +0100 Message-ID: <20240315120442.73754-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 1/9] avutil/film_grain_params: add extra AFGS1 metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: y27dgWvlS0J5 From: Niklas Haas In hindsight, it may have been better for this to be part of the common film grain struct, to avoid duplication of these fields between it and H274 (and to allow initialization by av_film_grain_params_alloc), but I can't add new fields before the union without breaking ABI, and having the union not be the last parameter will make future extensibility similarly difficult. --- doc/APIchanges | 4 ++++ libavutil/film_grain_params.h | 15 +++++++++++++++ libavutil/version.h | 2 +- 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index cf58c8c5f0a..5a192b600af 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,10 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-03-08 - xxxxxxxxxx - lavu 59.2.100 - film_grain_params.h + Add AVFilmGrainAOMParams.color_range, color_primaries, color_trc, + color_space, width, height, subx, suby and bit_depth. + 2024-03-08 - xxxxxxxxxx - lavc 61.1.100 - avcodec.h Add AVCodecContext.[nb_]side_data_prefer_packet. diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h index f3bd0a4a6a3..17fefeb06c3 100644 --- a/libavutil/film_grain_params.h +++ b/libavutil/film_grain_params.h @@ -120,6 +120,21 @@ typedef struct AVFilmGrainAOMParams { * Signals to clip to limited color levels after film grain application. */ int limit_output_range; + + /** + * Video characteristics metadata, only for AFGS1 streams. + */ + enum AVColorRange color_range; + enum AVColorPrimaries color_primaries; + enum AVColorTransferCharacteristic color_trc; + enum AVColorSpace color_space; + + /** + * Intended display resolution and format, only for AFGS1 streams. + */ + int width, height; + int subx, suby; + int bit_depth; /* or 0 if unknown */ } AVFilmGrainAOMParams; /** diff --git a/libavutil/version.h b/libavutil/version.h index 09f8cdc2926..57cad02ec0a 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 1 +#define LIBAVUTIL_VERSION_MINOR 2 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Fri Mar 15 11:58:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47101 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp193807pzh; Fri, 15 Mar 2024 05:05:31 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVw+dIi7ihWKP0QsYNAZJek+uCHcMq+bCHcp1mmFg0A8opw+Ps4MewfVGI1ew8iUjoTwfG+I2gJcY9OHTXWFEmjTPcjvi4XOg1fdA== X-Google-Smtp-Source: AGHT+IEm3dihXV2YMwVCQpSNkM/k0ZP1BoclFl2otNEMn9DTsmzzLvuR8Dvg2rcFICQHr2I1FmXc X-Received: by 2002:a17:906:3299:b0:a46:53e7:999f with SMTP id 25-20020a170906329900b00a4653e7999fmr3344042ejw.4.1710504331268; Fri, 15 Mar 2024 05:05:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504331; cv=none; d=google.com; s=arc-20160816; b=Av/fLciu4QpmJ7teVRkFvYkrwwEEnk/4p/Ol9phlS0hwOVjf19kIJh4NYmNGOzjwEW 18VaMZ1jOCAwHOsjtHbMD/P5GV8EhrUBVGW4F79MY8UuOQQNsx8m9F58qzcAY1aGXDg7 Vh/Mi6HxsoS1t8Rx7fY74c89dfPh+XvCnHDbIsdw97jaTAm5xSoRFI+ptl0awLkM9t55 F1OGyMcofikDPkZSMA1sNHIuLalz9EpqJsayufEKmx6GnnyfFzBh3VMNObRAYuFWklpq zfLdoRuWQ/q6FzytW+aQx72gsyaV5X7NcPUGDZ5gEoMmmtMiZ2LLugwcFaPEVMUNQ4La 2Tqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=EnpgT5DrQ6SCirRpqHbR4ZzxETrAYkuW9tI4IstzauQ=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=iVwAuIAp7DjQCKxD42jCN63sGTIr8kX80+1jmIpHsf/EB6u5GPFeM236yE0/WroaJU 1KyR9qjS8uuoaNkxxaiEKvWn8p5sLhpNaqplHSmX5H1i7UEevrGWR1njTk7F+b9J2wXr 3MzFBcYfgg6NYtFXsVdBim9GPUPnUkYzJbR4Qd5QKvSz/QvcQjHoMoeOznf6jMrI8MsW j27jaWbj8TnztoQr+KLPqqSeJJ5wZ14xxu26KbSH/NV6jsqJO6CmnkVlpbvN/Rwx6S8O Ni6PU5bOH+PYFp2X9H0jeprbh82Pg/EpZ4GHvmaEAyJB9jpwE5Cv+V7yyWbJQ9gqAs1C zMPg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=ftERFgq3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ht17-20020a170907609100b00a465d48539dsi1710279ejc.242.2024.03.15.05.05.19; Fri, 15 Mar 2024 05:05:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=ftERFgq3; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2780668D083; Fri, 15 Mar 2024 14:04:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C742068CF9D for ; Fri, 15 Mar 2024 14:04:46 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504286; bh=yks3L25oFVBI7mdPqjzuXrM286X52Xqw0SVp19lhlv0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ftERFgq3LOz02zG/25ifFSTRAfJkNDDFZh6sijgpQqt8QxevxE47WyciIZ3g40Nws hK3veWEtthMYV5NRhIxM7f+amBGdNTavG7YGpfSv7Euhei+fHYBvf+pD9fLU9fGKh4 MI40P6DZszAvjsdJZitQgSqZdHHN7XbvAdv7dELc= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 83677424EE; Fri, 15 Mar 2024 13:04:46 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:58:55 +0100 Message-ID: <20240315120442.73754-3-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 2/9] avcodec/av1dec: initialize AFGS1 VSC metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: aLXt2ZwXMrDk From: Niklas Haas Unused by AV1, but should still be set properly. --- libavcodec/av1dec.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index e6346b51dbe..8a5796c757f 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -1124,6 +1124,10 @@ static int export_film_grain(AVCodecContext *avctx, AVFrame *frame) aom->uv_mult_luma[1] = film_grain->cr_luma_mult; aom->uv_offset[0] = film_grain->cb_offset; aom->uv_offset[1] = film_grain->cr_offset; + aom->color_range = AVCOL_RANGE_UNSPECIFIED; + aom->color_primaries = AVCOL_PRI_UNSPECIFIED; + aom->color_trc = AVCOL_TRC_UNSPECIFIED; + aom->color_space = AVCOL_SPC_UNSPECIFIED; return 0; } From patchwork Fri Mar 15 11:58:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47102 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp193932pzh; Fri, 15 Mar 2024 05:05:40 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUGTjo3+66zNSEI+XdTkwz1WVrdaWQ7aLN2kwr19I4JY1WXoiJB4NyHPZ7XgCkwnXB/6AEsMd6bMU/iI1khV3t+7eeQ8HDPRHYgzg== X-Google-Smtp-Source: AGHT+IGaWjiH1KfT9w3ses6bNgVofxwXN3VFFslA+XEgEaMLQ34QFJMHpnYujD3VRjMg0eV8hZep X-Received: by 2002:a17:906:fa8d:b0:a46:8799:bddf with SMTP id lt13-20020a170906fa8d00b00a468799bddfmr915478ejb.43.1710504340119; Fri, 15 Mar 2024 05:05:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504340; cv=none; d=google.com; s=arc-20160816; b=UdxBe1OE7ky6+5H5KgN/gCzRGx3K3jyvD4+JUn65SZ6Lq5kwBsWD0f7Yv+OibWRuLo lNXjkccL+IxR0xy/Fy7tNFPs04Qdpiv809CaP0v0YSr2CfvtROEEECqtaWPNHqiAwnW4 rEQBouP5BCNC3TAk+YAdoF5AwrPrKwy5ituDlmdO4Z+c7u+EfKYzwS3yliSdOvjrpfyP 59SQLUOjaOSByCNvX/yOB6DyONaTi08DVSeBqZmn79z/ceIYfIOeu4yG5GCWYnw5qkjQ IpVCzKAueLEoppWI3LRF/37IP33cb5kx+Il0CgFT2tPKI5Z8OJT8pceTmR1lhvnp79a7 o1sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=bsar5DbFR0tqLJpzHJvcUMGfALqb5mUSISxSjPojPFA=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=GH87I1T7lF4vpskFQotJjr0iathbncWns9IBjTKxovvjoUWlEgNMI79peCyORe3q1z lQYxoN60wTqIeEKp1nGRNL+HTcKmIZdXFG6Ur7irlFQT0lL84z4Wa732VfozsPQjnIwj UdbR49pgYnI0767xNet/GzqPNXMj6veI+KOtegYXnBSLaYSnM4CnqRGv9oc40haE0Iqj +sr1+JTgOMjlI0tWYnG/2GMPGrTCb1sqQxSO19q1MlXMBUtPJ2ycZ1mwiwIIN8YkIbEG jvj0mteZ13voKNHrVsthNm0EICAfl403bl5Kk19BcIiByVwnHuHuhaVRDcOgG3ruWeQ5 JqqA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=JbjXj+sg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id oz40-20020a1709077da800b00a468c2be77fsi395848ejc.495.2024.03.15.05.05.29; Fri, 15 Mar 2024 05:05:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=JbjXj+sg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3B12868D191; Fri, 15 Mar 2024 14:04:57 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0639268CF9D for ; Fri, 15 Mar 2024 14:04:47 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504286; bh=1hWHoLtjopLZgAtevzyiDpkRiLac2qhRQQmOlM8l83M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JbjXj+sgH/tfZurioZE4cCmvqZl8J7ILU7OtfhVL8AwjeXRLcGPvhP3wkfWZkfZka PB5eaMddnWxULW89qfA7b61ucxTDyZvMe+iU23vRjzSQ/7xk78sPdMeuZo5iSO/BfC 2eL6EnKTVzzREUye5LCilteTn5qGywvzcW5jssWU= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id C3D854290F; Fri, 15 Mar 2024 13:04:46 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:58:56 +0100 Message-ID: <20240315120442.73754-4-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 3/9] avcodec/libdav1d: initialize AFGS1 VSC metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BJLZJF7XYxWQ From: Niklas Haas Unused by AV1, but should still be set properly. --- libavcodec/libdav1d.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c index 1aa2d1f3436..6251ea79ba8 100644 --- a/libavcodec/libdav1d.c +++ b/libavcodec/libdav1d.c @@ -628,6 +628,10 @@ static int libdav1d_receive_frame(AVCodecContext *c, AVFrame *frame) fgp->codec.aom.grain_scale_shift = p->frame_hdr->film_grain.data.grain_scale_shift; fgp->codec.aom.overlap_flag = p->frame_hdr->film_grain.data.overlap_flag; fgp->codec.aom.limit_output_range = p->frame_hdr->film_grain.data.clip_to_restricted_range; + fgp->codec.aom.color_range = AVCOL_RANGE_UNSPECIFIED; + fgp->codec.aom.color_primaries = AVCOL_PRI_UNSPECIFIED; + fgp->codec.aom.color_trc = AVCOL_TRC_UNSPECIFIED; + fgp->codec.aom.color_space = AVCOL_SPC_UNSPECIFIED; memcpy(&fgp->codec.aom.y_points, &p->frame_hdr->film_grain.data.y_points, sizeof(fgp->codec.aom.y_points)); From patchwork Fri Mar 15 11:58:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47103 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp193956pzh; Fri, 15 Mar 2024 05:05:41 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVGiS4JH8nghZKHDePwWROm+oAFXNB79NVIZLm1abA8w4Uy3fxJUti6WJH+e+pvm0pjW7dSLEJe3Vp92I88xM1yqFASToJV6x3lnA== X-Google-Smtp-Source: AGHT+IFRyaQ7zhPsHFIDVthiVWCf+Id1CoByxGfL5CDG3xXik0jxLX1aytvHHZroNhe6Jh1Me/Im X-Received: by 2002:a17:906:190c:b0:a46:464c:7e8b with SMTP id a12-20020a170906190c00b00a46464c7e8bmr3156072eje.28.1710504341262; Fri, 15 Mar 2024 05:05:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504341; cv=none; d=google.com; s=arc-20160816; b=njbJxTOg14q6aIx5Zqg8XpD3zJvfKtTS1Imue6ITrmnuVcrNYHU0zzqDbAEJkLGc6W g45dy9eLeb8PSJTZwSDu8SjvUBvtceaKHIBo6KwNd5ES43UBPXXpshUqlM2zjSPyxOk/ jDFemHhRqu+XvQcbfBXKoz+4tzDuDjn9Vl/7g9V5eCZdUw1zismAk2oOIOWvxOpY6Siz 4dC8nzVLbk52bqsdZa83Zr0vcYFDZwHa+m+nRKTqXx2HPSLzElfVa9HeThNpHmzHEfs6 I+0+tmf/7KBDTewhn9Tf22aMywCCE8YnIs+wV0p0IuLIELk+eKVP6SS+vM54bbng+lFt ozEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=98PMhzvtN9aUTvKjgYEeCjr2iydtYGxd9njOBYw+Tas=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=i2JF3XIhJRBzKNNZ53qsM3pNCNpHo+kjjCKr4+KNFJ9piiyG3HJR3HE16HEM2D156W J6niSf1tLZpDNPctrgi0Byh5F3skRh68W4JNhRF53Q8sd8S2uSoYnbobWb6WhiUcQtjP GNcAgd8w7dh7wTJmkl+1QBtCvNQk/W73DWwX/nhuUVzn1CS0b31fCvNf64rcupJE7WOI yvxitiEZ52zFnGCJ1z70xbEYT9VmivUnsdtLfWcFb/uhoGzRQNk0DWBXPmLqrMinGpGC gp6Mlp9NO/ZK4/LEgPUYgPqlPVutGXkrtm83onDeRpS9r20A855CfPj/W2up2dAsctvj dhqg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Ow3qd8hR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lx9-20020a170906af0900b00a45f6c9a4c8si1613230ejb.369.2024.03.15.05.05.40; Fri, 15 Mar 2024 05:05:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Ow3qd8hR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3FA9368D1A4; Fri, 15 Mar 2024 14:04:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 50A3C68D159 for ; Fri, 15 Mar 2024 14:04:47 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504287; bh=BANHU47C/dfd6/5pbk9YXHO+dD892i3PWNvAM+kxlPY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ow3qd8hRA8x7b8yf0Hp6UpucjyKSaQ4ahUIcDC/e1J0uG1duIlM9hxq1Dp+YgzBFB nsuGoUs0RBpQXYlqQitF022EMby2D4bCRduve940BEBdmxeTUGSFvPjz8fKcxXNATJ zyUmLd6zSzE4twF0XFWmHar0K+EBxQa+PPeP2Nbc= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 055FE42922; Fri, 15 Mar 2024 13:04:46 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:58:57 +0100 Message-ID: <20240315120442.73754-5-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 4/9] avutil/frame: clarify AV_FRAME_DATA_FILM_GRAIN_PARAMS usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IOK7wVPr/0aN From: Niklas Haas To allow for AFGS1 usage, which can expose multiple parameter sets for a single frame. --- libavutil/frame.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavutil/frame.h b/libavutil/frame.h index b94687941db..7f616488be8 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -180,6 +180,10 @@ enum AVFrameSideDataType { /** * Film grain parameters for a frame, described by AVFilmGrainParams. * Must be present for every frame which should have film grain applied. + * + * May be present multiple times, for example when there are multiple + * alternative parameter sets for different video signal characteristics. + * The user should select the most appropriate set for the application. */ AV_FRAME_DATA_FILM_GRAIN_PARAMS, From patchwork Fri Mar 15 11:58:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47105 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp194180pzh; Fri, 15 Mar 2024 05:06:00 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXaQgoLzsdzQOUE5JsMT2AwuU0fSkqZRk1Eog3j0buOvS/X6vrZ24I4sC+lvtxsLAZvmTBNW7/qYrKPB5XXAhlYIC3TniLne7ZvVg== X-Google-Smtp-Source: AGHT+IFxWnr93t972PEMFAUt7J6/dAN640pTkMeu++LgUlCJ+/9hnOgeWiFr8e4Kle43ce+SaHR0 X-Received: by 2002:a17:906:2c48:b0:a46:8a9b:7fb9 with SMTP id f8-20020a1709062c4800b00a468a9b7fb9mr918044ejh.51.1710504359966; Fri, 15 Mar 2024 05:05:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504359; cv=none; d=google.com; s=arc-20160816; b=EJx9yYcx2r3rhFPC5USKRsxBv/nXNC/VCOsjLcxLwvCoTwcMRR+mrU80m3MPdlvk6v gD5VyJ9Fgg6VlTvH1a40SqSdtdbclFA1bJMgQo6VqFpIXzH5bQuz7Kz4OOox/3p01zgo 1KPk+6o9XQNau5n0gL6CzxIMbKa70GPh+YO177yE+eA4nmIO8Td8HTAMNesx5GzQYyoZ SiUuyYUXkrebZ6XX952FkfxqJFeoFiG2vOE95vxhspppKDDwJxJFm3MCZwtyuJ6uxX9v wV44QOTGFY3ftKzERg+zDdViGlHyYutUF7c0WoccSp8LvrSr4V8hciooMTdCnshNHz8c y2Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=Xc6rF+cEJXizix4Er9QZwu51eiwC1dV9KFcOwAhObws=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=Elp2yLjZBYOsjubPMfwTa6phNSjCozP4Ck0f5YYG1XRp+TNC4j53xR5VNtx0MQdawz z7Syw+1sJbYpmxmwiUPOSflmRxuUdQg74hOZKZO0KS+FPV7MxIQBFFcbv0Z3hEcnE5o+ wKg+CUSpDIGTmPdbu1TI/zPCGS4vLCO01XyS+AqvrrP1s6mrYr8cRujl2Qeg9oP2BvA6 f9+6oVRPp27RBPiWxCpVdqHP/iho8/ZOkbXbG4662mk3vvNTfxm6kKUxjwxv+x/TRsSA b52jzBDsamNZwKNKUKMFY7X+TPdakCMKenrdnSaypvnttrYGZqbWLT4I7J08zMfj1M9u hZHQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=UAxo6rZr; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o12-20020a17090611cc00b00a469469a02bsi196909eja.650.2024.03.15.05.05.51; Fri, 15 Mar 2024 05:05:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=UAxo6rZr; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5D86D68D1B4; Fri, 15 Mar 2024 14:04:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7E1E368D180 for ; Fri, 15 Mar 2024 14:04:51 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504287; bh=knzCmHEnRLvBgs+TTcnWOcBNGxlEc2tvlF2516FukH4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UAxo6rZrt0PsmajwrQzHoLl5sYNBQj+XjiLIS7UBuVhEaNHVrt57VvaSUHzrtkXSp UXW6suXND2NedjVwcFzxYJcHa9rZqDB/5onmSTc88PkK+8uOjidDbs2JAnFnpb90AE RST3rBvKGqHK83pyg6W3SweaVwMeCQlfwwPmzgLE= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 44B76429BE; Fri, 15 Mar 2024 13:04:47 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:58:58 +0100 Message-ID: <20240315120442.73754-6-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 5/9] avutil/film_grain_params: add av_film_grain_params_select() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: LOBm2UuM0J+s From: Niklas Haas Common utility function that can be used by all codecs to select the right (any valid) film grain parameter set. In particular, this is useful for AFGS1, which has support for multiple parameters. However, it also performs parameter validation for H274. --- doc/APIchanges | 3 ++ libavutil/film_grain_params.c | 57 +++++++++++++++++++++++++++++++++++ libavutil/film_grain_params.h | 7 +++++ libavutil/version.h | 2 +- 4 files changed, 68 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index 5a192b600af..34245c8b708 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-03-08 - xxxxxxxxxx - lavu 59.3.100 - film_grain_params.h + Add av_film_grain_params_select(). + 2024-03-08 - xxxxxxxxxx - lavu 59.2.100 - film_grain_params.h Add AVFilmGrainAOMParams.color_range, color_primaries, color_trc, color_space, width, height, subx, suby and bit_depth. diff --git a/libavutil/film_grain_params.c b/libavutil/film_grain_params.c index 930d23c7fe9..8c80adc66a7 100644 --- a/libavutil/film_grain_params.c +++ b/libavutil/film_grain_params.c @@ -17,6 +17,7 @@ */ #include "film_grain_params.h" +#include "pixdesc.h" AVFilmGrainParams *av_film_grain_params_alloc(size_t *size) { @@ -40,3 +41,59 @@ AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame) return (AVFilmGrainParams *)side_data->data; } + +const AVFilmGrainParams *av_film_grain_params_select(const AVFrame *frame) +{ + const AVFilmGrainParams *fgp, *best = NULL; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format); + const AVFilmGrainAOMParams *aom; + const AVFilmGrainH274Params *h274; + if (!desc) + return NULL; + +#define CHECK(a, b, unspec) \ + do { \ + if ((a) != (unspec) && (b) != (unspec) && (a) != (b)) \ + continue; \ + } while (0) + + for (int i = 0; i < frame->nb_side_data; i++) { + if (frame->side_data[i]->type != AV_FRAME_DATA_FILM_GRAIN_PARAMS) + continue; + fgp = (const AVFilmGrainParams*)frame->side_data[i]->data; + switch (fgp->type) { + case AV_FILM_GRAIN_PARAMS_NONE: + continue; + case AV_FILM_GRAIN_PARAMS_AV1: + aom = &fgp->codec.aom; + if (aom->subx != desc->log2_chroma_w || aom->suby != desc->log2_chroma_h) + continue; + CHECK(aom->bit_depth, desc->comp[0].depth, 0); + CHECK(aom->width, frame->width, 0); + CHECK(aom->height, frame->height, 0); + CHECK(aom->color_range, frame->color_range, AVCOL_RANGE_UNSPECIFIED); + CHECK(aom->color_primaries, frame->color_primaries, AVCOL_PRI_UNSPECIFIED); + CHECK(aom->color_trc, frame->color_trc, AVCOL_TRC_UNSPECIFIED); + CHECK(aom->color_space, frame->colorspace, AVCOL_SPC_UNSPECIFIED); + + if (!best || + best->codec.aom.width < aom->width || + best->codec.aom.height < aom->height) + best = fgp; + break; + case AV_FILM_GRAIN_PARAMS_H274: + h274 = &fgp->codec.h274; + /* There are no YUV formats with different bit depth per component, + * so just check both against the first component for simplicity */ + CHECK(h274->bit_depth_luma, desc->comp[0].depth, 0); + CHECK(h274->bit_depth_chroma, desc->comp[0].depth, 0); + CHECK(h274->color_range, frame->color_range, AVCOL_RANGE_UNSPECIFIED); + CHECK(h274->color_primaries, frame->color_primaries, AVCOL_PRI_UNSPECIFIED); + CHECK(h274->color_trc, frame->color_trc, AVCOL_TRC_UNSPECIFIED); + CHECK(h274->color_space, frame->colorspace, AVCOL_SPC_UNSPECIFIED); + return fgp; /* H274 can't have multiple resolutions */ + } + } + + return best; +} diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h index 17fefeb06c3..75c020ceb92 100644 --- a/libavutil/film_grain_params.h +++ b/libavutil/film_grain_params.h @@ -272,4 +272,11 @@ AVFilmGrainParams *av_film_grain_params_alloc(size_t *size); */ AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame); +/** + * Select the most appropriate film grain parameters set for the frame, + * taking into account the frame's format, resolution and video signal + * characteristics. + */ +const AVFilmGrainParams *av_film_grain_params_select(const AVFrame *frame); + #endif /* AVUTIL_FILM_GRAIN_PARAMS_H */ diff --git a/libavutil/version.h b/libavutil/version.h index 57cad02ec0a..5027b025be4 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 2 +#define LIBAVUTIL_VERSION_MINOR 3 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ From patchwork Fri Mar 15 11:58:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47104 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp194212pzh; Fri, 15 Mar 2024 05:06:03 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCV2KmdJ3knnYoSsAjZcuzpzKlEzZJSGivTqYGo9i+/kuFZfWZ48ZUIBxYkTrRWMB8RoVZlutML1/O4GkQ2cO9GIamAuiggkls9zkA== X-Google-Smtp-Source: AGHT+IGDrabmfsNZsoNGPeOp99Glc02Gqvdy2briock6mt9EJnSwddE62vhWm1yNd8Jw9fMTs8Bu X-Received: by 2002:a17:906:f845:b0:a3e:8300:1af3 with SMTP id ks5-20020a170906f84500b00a3e83001af3mr2978999ejb.30.1710504362763; Fri, 15 Mar 2024 05:06:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504362; cv=none; d=google.com; s=arc-20160816; b=gBuedLGzhKdljfYwDFsM6Y9VbtiG9q/E4N5YWbR+SLXPcTfXA6zgAdWJz/lnA20w+w 48MY/Q2lZ96fm0MPu626ao+WHC8z3HjkKuKvxrigzY7o9unWfTlK8WULtahevjpxdyPU 37/P/9UwpKUENeM1IMtZ1UpfOsSaEvxozNTrm0p0kW80NKW7Jmp2M017JOMcCiOWMSJ4 ASH6SQXpxxy2BLg/do4ADhl2m2u3/IbSKlpCcbSFgNshSFNq/VcsEdR7jKCYUm2FTCgl ae4PJzLY0E1wkYK4ma0svw9OjIImA9f72g90XmqU6L5d864/flbKsq9M6q+8IsmD/Nk5 B52g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=uI/i95UXE/UNU7SJivw0LFQBtQznSXB4+q/rQJO5DJw=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=P+Z9eliPW3C7l5y2AuPYUDKKGsrIggSWp7rguKysWvmYdqW7X6xsMqSvI01N5Huv5p +TqjELDNcBSHJC0w+WNaIb8Z8sQSGehn1k2rsrHcFc6Fyp0Yid1471+X7bWyGst5ujxe zRIAfXH1EjlXMxCysL+A+Re+Oniiy8lDAUh/nCr6UUA4FFuzUSdld7qy7O23HKG+tDgA XqetuAHoncEi5BV4X5+bY8p/uU8+BkCD2T5ji9N19O5w0LBg6/rCGwMszNMWs9eXCw6W QUSVgzZkMC55WnTtSbJWjut26IK+8E3LMnzqNE/l3Qb2w/ozloDnjw+H+G5BivYZY3kN uj6A==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Fw9RAIM5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j20-20020a170906255400b00a46605ce5fcsi1711509ejb.715.2024.03.15.05.06.02; Fri, 15 Mar 2024 05:06:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Fw9RAIM5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5635768D1A6; Fri, 15 Mar 2024 14:05:00 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8632068D181 for ; Fri, 15 Mar 2024 14:04:51 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504287; bh=MHGOEF9qGOjgiHwV6HsxQVnsLPml0q0vLMEAz1yBR8A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fw9RAIM5RxhJhMKuVbeEPbO7Cwn1/bBJrBNh4cckQTM72Zqjj/vLwoXdPe/GazvtW Sx9PZJPiGw+vls6b3dpxaeSyQ2rC41bZrDL6O55GY9Q/OdC+1mor5V6fjHCHoB7AT+ XHWyHi2PBi1UNSTrlv/bpNOO6QJYV3GMHkVcgbdE= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 79B58429DE; Fri, 15 Mar 2024 13:04:47 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:58:59 +0100 Message-ID: <20240315120442.73754-7-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 6/9] avcodec/aom_film_grain: add AOM film grain synthesis X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3O9WKSHdMRbj From: Niklas Haas Implementation copied wholesale from dav1d, sans SIMD, under permissive license. This implementation was extensively verified to be bit-exact, so it serves as a much better starting point than trying to re-engineer this from scratch for no reason. (I also authored the original implementation in dav1d, so any "clean room" implementation would end up looking much the same, anyway) The notable changes I had to make while adapting this from the dav1d code-base to the FFmpeg codebase include: - reordering variable declarations to avoid triggering warnings - replacing several inline helpers by avutil equivalents - changing code that accesses frame metadata - replacing raw plane copying logic by av_image_copy_plane Apart from this, the implementation is basically unmodified. --- libavcodec/aom_film_grain.c | 310 ++++++++++++++ libavcodec/aom_film_grain.h | 38 ++ libavcodec/aom_film_grain_template.c | 577 +++++++++++++++++++++++++++ 3 files changed, 925 insertions(+) create mode 100644 libavcodec/aom_film_grain.c create mode 100644 libavcodec/aom_film_grain.h create mode 100644 libavcodec/aom_film_grain_template.c diff --git a/libavcodec/aom_film_grain.c b/libavcodec/aom_film_grain.c new file mode 100644 index 00000000000..ffcd71b584b --- /dev/null +++ b/libavcodec/aom_film_grain.c @@ -0,0 +1,310 @@ +/* + * AOM film grain synthesis + * Copyright (c) 2023 Niklas Haas + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * AOM film grain synthesis. + * @author Niklas Haas + */ + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" + +#include "aom_film_grain.h" + +// Common/shared helpers (not dependent on BIT_DEPTH) +static inline int get_random_number(const int bits, unsigned *const state) { + const int r = *state; + unsigned bit = ((r >> 0) ^ (r >> 1) ^ (r >> 3) ^ (r >> 12)) & 1; + *state = (r >> 1) | (bit << 15); + + return (*state >> (16 - bits)) & ((1 << bits) - 1); +} + +static inline int round2(const int x, const uint64_t shift) { + return (x + ((1 << shift) >> 1)) >> shift; +} + +enum { + GRAIN_WIDTH = 82, + GRAIN_HEIGHT = 73, + SUB_GRAIN_WIDTH = 44, + SUB_GRAIN_HEIGHT = 38, + FG_BLOCK_SIZE = 32, +}; + +static const int16_t gaussian_sequence[2048]; + +#define BIT_DEPTH 16 +#include "aom_film_grain_template.c" +#undef BIT_DEPTH + +#define BIT_DEPTH 8 +#include "aom_film_grain_template.c" +#undef BIT_DEPTH + + +int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in, + const AVFilmGrainParams *params) +{ + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(out->format); + const int subx = desc->log2_chroma_w, suby = desc->log2_chroma_h; + const int pxstep = desc->comp[0].step; + + av_assert0(out->format == in->format); + av_assert0(params->type == AV_FILM_GRAIN_PARAMS_AV1); + + // Copy over the non-modified planes + if (!params->codec.aom.num_y_points) { + av_image_copy_plane(out->data[0], out->linesize[0], + in->data[0], in->linesize[0], + out->width * pxstep, out->height); + } + for (int uv = 0; uv < 2; uv++) { + if (!data->num_uv_points[uv]) { + av_image_copy_plane(out->data[1+uv], out->linesize[1+uv], + in->data[1+uv], in->linesize[1+uv], + AV_CEIL_RSHIFT(out->width, subx) * pxstep, + AV_CEIL_RSHIFT(out->height, suby)); + } + } + + switch (in->format) { + case AV_PIX_FMT_GRAY8: + case AV_PIX_FMT_YUV420P: + case AV_PIX_FMT_YUV422P: + case AV_PIX_FMT_YUV444P: + case AV_PIX_FMT_YUVJ420P: + case AV_PIX_FMT_YUVJ422P: + case AV_PIX_FMT_YUVJ444P: + return apply_film_grain_8(out, in, params); + case AV_PIX_FMT_GRAY9: + case AV_PIX_FMT_YUV420P9: + case AV_PIX_FMT_YUV422P9: + case AV_PIX_FMT_YUV444P9: + return apply_film_grain_16(out, in, params, 9); + case AV_PIX_FMT_GRAY10: + case AV_PIX_FMT_YUV420P10: + case AV_PIX_FMT_YUV422P10: + case AV_PIX_FMT_YUV444P10: + return apply_film_grain_16(out, in, params, 10); + case AV_PIX_FMT_GRAY12: + case AV_PIX_FMT_YUV420P12: + case AV_PIX_FMT_YUV422P12: + case AV_PIX_FMT_YUV444P12: + return apply_film_grain_16(out, in, params, 12); + } + + /* The AV1 spec only defines film grain synthesis for these formats */ + return AVERROR_INVALIDDATA; +} + +// Taken from the AV1 spec. Range is [-2048, 2047], mean is 0 and stddev is 512 +static const int16_t gaussian_sequence[2048] = { + 56, 568, -180, 172, 124, -84, 172, -64, -900, 24, 820, + 224, 1248, 996, 272, -8, -916, -388, -732, -104, -188, 800, + 112, -652, -320, -376, 140, -252, 492, -168, 44, -788, 588, + -584, 500, -228, 12, 680, 272, -476, 972, -100, 652, 368, + 432, -196, -720, -192, 1000, -332, 652, -136, -552, -604, -4, + 192, -220, -136, 1000, -52, 372, -96, -624, 124, -24, 396, + 540, -12, -104, 640, 464, 244, -208, -84, 368, -528, -740, + 248, -968, -848, 608, 376, -60, -292, -40, -156, 252, -292, + 248, 224, -280, 400, -244, 244, -60, 76, -80, 212, 532, + 340, 128, -36, 824, -352, -60, -264, -96, -612, 416, -704, + 220, -204, 640, -160, 1220, -408, 900, 336, 20, -336, -96, + -792, 304, 48, -28, -1232, -1172, -448, 104, -292, -520, 244, + 60, -948, 0, -708, 268, 108, 356, -548, 488, -344, -136, + 488, -196, -224, 656, -236, -1128, 60, 4, 140, 276, -676, + -376, 168, -108, 464, 8, 564, 64, 240, 308, -300, -400, + -456, -136, 56, 120, -408, -116, 436, 504, -232, 328, 844, + -164, -84, 784, -168, 232, -224, 348, -376, 128, 568, 96, + -1244, -288, 276, 848, 832, -360, 656, 464, -384, -332, -356, + 728, -388, 160, -192, 468, 296, 224, 140, -776, -100, 280, + 4, 196, 44, -36, -648, 932, 16, 1428, 28, 528, 808, + 772, 20, 268, 88, -332, -284, 124, -384, -448, 208, -228, + -1044, -328, 660, 380, -148, -300, 588, 240, 540, 28, 136, + -88, -436, 256, 296, -1000, 1400, 0, -48, 1056, -136, 264, + -528, -1108, 632, -484, -592, -344, 796, 124, -668, -768, 388, + 1296, -232, -188, -200, -288, -4, 308, 100, -168, 256, -500, + 204, -508, 648, -136, 372, -272, -120, -1004, -552, -548, -384, + 548, -296, 428, -108, -8, -912, -324, -224, -88, -112, -220, + -100, 996, -796, 548, 360, -216, 180, 428, -200, -212, 148, + 96, 148, 284, 216, -412, -320, 120, -300, -384, -604, -572, + -332, -8, -180, -176, 696, 116, -88, 628, 76, 44, -516, + 240, -208, -40, 100, -592, 344, -308, -452, -228, 20, 916, + -1752, -136, -340, -804, 140, 40, 512, 340, 248, 184, -492, + 896, -156, 932, -628, 328, -688, -448, -616, -752, -100, 560, + -1020, 180, -800, -64, 76, 576, 1068, 396, 660, 552, -108, + -28, 320, -628, 312, -92, -92, -472, 268, 16, 560, 516, + -672, -52, 492, -100, 260, 384, 284, 292, 304, -148, 88, + -152, 1012, 1064, -228, 164, -376, -684, 592, -392, 156, 196, + -524, -64, -884, 160, -176, 636, 648, 404, -396, -436, 864, + 424, -728, 988, -604, 904, -592, 296, -224, 536, -176, -920, + 436, -48, 1176, -884, 416, -776, -824, -884, 524, -548, -564, + -68, -164, -96, 692, 364, -692, -1012, -68, 260, -480, 876, + -1116, 452, -332, -352, 892, -1088, 1220, -676, 12, -292, 244, + 496, 372, -32, 280, 200, 112, -440, -96, 24, -644, -184, + 56, -432, 224, -980, 272, -260, 144, -436, 420, 356, 364, + -528, 76, 172, -744, -368, 404, -752, -416, 684, -688, 72, + 540, 416, 92, 444, 480, -72, -1416, 164, -1172, -68, 24, + 424, 264, 1040, 128, -912, -524, -356, 64, 876, -12, 4, + -88, 532, 272, -524, 320, 276, -508, 940, 24, -400, -120, + 756, 60, 236, -412, 100, 376, -484, 400, -100, -740, -108, + -260, 328, -268, 224, -200, -416, 184, -604, -564, -20, 296, + 60, 892, -888, 60, 164, 68, -760, 216, -296, 904, -336, + -28, 404, -356, -568, -208, -1480, -512, 296, 328, -360, -164, + -1560, -776, 1156, -428, 164, -504, -112, 120, -216, -148, -264, + 308, 32, 64, -72, 72, 116, 176, -64, -272, 460, -536, + -784, -280, 348, 108, -752, -132, 524, -540, -776, 116, -296, + -1196, -288, -560, 1040, -472, 116, -848, -1116, 116, 636, 696, + 284, -176, 1016, 204, -864, -648, -248, 356, 972, -584, -204, + 264, 880, 528, -24, -184, 116, 448, -144, 828, 524, 212, + -212, 52, 12, 200, 268, -488, -404, -880, 824, -672, -40, + 908, -248, 500, 716, -576, 492, -576, 16, 720, -108, 384, + 124, 344, 280, 576, -500, 252, 104, -308, 196, -188, -8, + 1268, 296, 1032, -1196, 436, 316, 372, -432, -200, -660, 704, + -224, 596, -132, 268, 32, -452, 884, 104, -1008, 424, -1348, + -280, 4, -1168, 368, 476, 696, 300, -8, 24, 180, -592, + -196, 388, 304, 500, 724, -160, 244, -84, 272, -256, -420, + 320, 208, -144, -156, 156, 364, 452, 28, 540, 316, 220, + -644, -248, 464, 72, 360, 32, -388, 496, -680, -48, 208, + -116, -408, 60, -604, -392, 548, -840, 784, -460, 656, -544, + -388, -264, 908, -800, -628, -612, -568, 572, -220, 164, 288, + -16, -308, 308, -112, -636, -760, 280, -668, 432, 364, 240, + -196, 604, 340, 384, 196, 592, -44, -500, 432, -580, -132, + 636, -76, 392, 4, -412, 540, 508, 328, -356, -36, 16, + -220, -64, -248, -60, 24, -192, 368, 1040, 92, -24, -1044, + -32, 40, 104, 148, 192, -136, -520, 56, -816, -224, 732, + 392, 356, 212, -80, -424, -1008, -324, 588, -1496, 576, 460, + -816, -848, 56, -580, -92, -1372, -112, -496, 200, 364, 52, + -140, 48, -48, -60, 84, 72, 40, 132, -356, -268, -104, + -284, -404, 732, -520, 164, -304, -540, 120, 328, -76, -460, + 756, 388, 588, 236, -436, -72, -176, -404, -316, -148, 716, + -604, 404, -72, -88, -888, -68, 944, 88, -220, -344, 960, + 472, 460, -232, 704, 120, 832, -228, 692, -508, 132, -476, + 844, -748, -364, -44, 1116, -1104, -1056, 76, 428, 552, -692, + 60, 356, 96, -384, -188, -612, -576, 736, 508, 892, 352, + -1132, 504, -24, -352, 324, 332, -600, -312, 292, 508, -144, + -8, 484, 48, 284, -260, -240, 256, -100, -292, -204, -44, + 472, -204, 908, -188, -1000, -256, 92, 1164, -392, 564, 356, + 652, -28, -884, 256, 484, -192, 760, -176, 376, -524, -452, + -436, 860, -736, 212, 124, 504, -476, 468, 76, -472, 552, + -692, -944, -620, 740, -240, 400, 132, 20, 192, -196, 264, + -668, -1012, -60, 296, -316, -828, 76, -156, 284, -768, -448, + -832, 148, 248, 652, 616, 1236, 288, -328, -400, -124, 588, + 220, 520, -696, 1032, 768, -740, -92, -272, 296, 448, -464, + 412, -200, 392, 440, -200, 264, -152, -260, 320, 1032, 216, + 320, -8, -64, 156, -1016, 1084, 1172, 536, 484, -432, 132, + 372, -52, -256, 84, 116, -352, 48, 116, 304, -384, 412, + 924, -300, 528, 628, 180, 648, 44, -980, -220, 1320, 48, + 332, 748, 524, -268, -720, 540, -276, 564, -344, -208, -196, + 436, 896, 88, -392, 132, 80, -964, -288, 568, 56, -48, + -456, 888, 8, 552, -156, -292, 948, 288, 128, -716, -292, + 1192, -152, 876, 352, -600, -260, -812, -468, -28, -120, -32, + -44, 1284, 496, 192, 464, 312, -76, -516, -380, -456, -1012, + -48, 308, -156, 36, 492, -156, -808, 188, 1652, 68, -120, + -116, 316, 160, -140, 352, 808, -416, 592, 316, -480, 56, + 528, -204, -568, 372, -232, 752, -344, 744, -4, 324, -416, + -600, 768, 268, -248, -88, -132, -420, -432, 80, -288, 404, + -316, -1216, -588, 520, -108, 92, -320, 368, -480, -216, -92, + 1688, -300, 180, 1020, -176, 820, -68, -228, -260, 436, -904, + 20, 40, -508, 440, -736, 312, 332, 204, 760, -372, 728, + 96, -20, -632, -520, -560, 336, 1076, -64, -532, 776, 584, + 192, 396, -728, -520, 276, -188, 80, -52, -612, -252, -48, + 648, 212, -688, 228, -52, -260, 428, -412, -272, -404, 180, + 816, -796, 48, 152, 484, -88, -216, 988, 696, 188, -528, + 648, -116, -180, 316, 476, 12, -564, 96, 476, -252, -364, + -376, -392, 556, -256, -576, 260, -352, 120, -16, -136, -260, + -492, 72, 556, 660, 580, 616, 772, 436, 424, -32, -324, + -1268, 416, -324, -80, 920, 160, 228, 724, 32, -516, 64, + 384, 68, -128, 136, 240, 248, -204, -68, 252, -932, -120, + -480, -628, -84, 192, 852, -404, -288, -132, 204, 100, 168, + -68, -196, -868, 460, 1080, 380, -80, 244, 0, 484, -888, + 64, 184, 352, 600, 460, 164, 604, -196, 320, -64, 588, + -184, 228, 12, 372, 48, -848, -344, 224, 208, -200, 484, + 128, -20, 272, -468, -840, 384, 256, -720, -520, -464, -580, + 112, -120, 644, -356, -208, -608, -528, 704, 560, -424, 392, + 828, 40, 84, 200, -152, 0, -144, 584, 280, -120, 80, + -556, -972, -196, -472, 724, 80, 168, -32, 88, 160, -688, + 0, 160, 356, 372, -776, 740, -128, 676, -248, -480, 4, + -364, 96, 544, 232, -1032, 956, 236, 356, 20, -40, 300, + 24, -676, -596, 132, 1120, -104, 532, -1096, 568, 648, 444, + 508, 380, 188, -376, -604, 1488, 424, 24, 756, -220, -192, + 716, 120, 920, 688, 168, 44, -460, 568, 284, 1144, 1160, + 600, 424, 888, 656, -356, -320, 220, 316, -176, -724, -188, + -816, -628, -348, -228, -380, 1012, -452, -660, 736, 928, 404, + -696, -72, -268, -892, 128, 184, -344, -780, 360, 336, 400, + 344, 428, 548, -112, 136, -228, -216, -820, -516, 340, 92, + -136, 116, -300, 376, -244, 100, -316, -520, -284, -12, 824, + 164, -548, -180, -128, 116, -924, -828, 268, -368, -580, 620, + 192, 160, 0, -1676, 1068, 424, -56, -360, 468, -156, 720, + 288, -528, 556, -364, 548, -148, 504, 316, 152, -648, -620, + -684, -24, -376, -384, -108, -920, -1032, 768, 180, -264, -508, + -1268, -260, -60, 300, -240, 988, 724, -376, -576, -212, -736, + 556, 192, 1092, -620, -880, 376, -56, -4, -216, -32, 836, + 268, 396, 1332, 864, -600, 100, 56, -412, -92, 356, 180, + 884, -468, -436, 292, -388, -804, -704, -840, 368, -348, 140, + -724, 1536, 940, 372, 112, -372, 436, -480, 1136, 296, -32, + -228, 132, -48, -220, 868, -1016, -60, -1044, -464, 328, 916, + 244, 12, -736, -296, 360, 468, -376, -108, -92, 788, 368, + -56, 544, 400, -672, -420, 728, 16, 320, 44, -284, -380, + -796, 488, 132, 204, -596, -372, 88, -152, -908, -636, -572, + -624, -116, -692, -200, -56, 276, -88, 484, -324, 948, 864, + 1000, -456, -184, -276, 292, -296, 156, 676, 320, 160, 908, + -84, -1236, -288, -116, 260, -372, -644, 732, -756, -96, 84, + 344, -520, 348, -688, 240, -84, 216, -1044, -136, -676, -396, + -1500, 960, -40, 176, 168, 1516, 420, -504, -344, -364, -360, + 1216, -940, -380, -212, 252, -660, -708, 484, -444, -152, 928, + -120, 1112, 476, -260, 560, -148, -344, 108, -196, 228, -288, + 504, 560, -328, -88, 288, -1008, 460, -228, 468, -836, -196, + 76, 388, 232, 412, -1168, -716, -644, 756, -172, -356, -504, + 116, 432, 528, 48, 476, -168, -608, 448, 160, -532, -272, + 28, -676, -12, 828, 980, 456, 520, 104, -104, 256, -344, + -4, -28, -368, -52, -524, -572, -556, -200, 768, 1124, -208, + -512, 176, 232, 248, -148, -888, 604, -600, -304, 804, -156, + -212, 488, -192, -804, -256, 368, -360, -916, -328, 228, -240, + -448, -472, 856, -556, -364, 572, -12, -156, -368, -340, 432, + 252, -752, -152, 288, 268, -580, -848, -592, 108, -76, 244, + 312, -716, 592, -80, 436, 360, 4, -248, 160, 516, 584, + 732, 44, -468, -280, -292, -156, -588, 28, 308, 912, 24, + 124, 156, 180, -252, 944, -924, -772, -520, -428, -624, 300, + -212, -1144, 32, -724, 800, -1128, -212, -1288, -848, 180, -416, + 440, 192, -576, -792, -76, -1080, 80, -532, -352, -132, 380, + -820, 148, 1112, 128, 164, 456, 700, -924, 144, -668, -384, + 648, -832, 508, 552, -52, -100, -656, 208, -568, 748, -88, + 680, 232, 300, 192, -408, -1012, -152, -252, -268, 272, -876, + -664, -648, -332, -136, 16, 12, 1152, -28, 332, -536, 320, + -672, -460, -316, 532, -260, 228, -40, 1052, -816, 180, 88, + -496, -556, -672, -368, 428, 92, 356, 404, -408, 252, 196, + -176, -556, 792, 268, 32, 372, 40, 96, -332, 328, 120, + 372, -900, -40, 472, -264, -592, 952, 128, 656, 112, 664, + -232, 420, 4, -344, -464, 556, 244, -416, -32, 252, 0, + -412, 188, -696, 508, -476, 324, -1096, 656, -312, 560, 264, + -136, 304, 160, -64, -580, 248, 336, -720, 560, -348, -288, + -276, -196, -500, 852, -544, -236, -1128, -992, -776, 116, 56, + 52, 860, 884, 212, -12, 168, 1020, 512, -552, 924, -148, + 716, 188, 164, -340, -520, -184, 880, -152, -680, -208, -1156, + -300, -528, -472, 364, 100, -744, -1056, -32, 540, 280, 144, + -676, -32, -232, -280, -224, 96, 568, -76, 172, 148, 148, + 104, 32, -296, -32, 788, -80, 32, -16, 280, 288, 944, + 428, -484 +}; diff --git a/libavcodec/aom_film_grain.h b/libavcodec/aom_film_grain.h new file mode 100644 index 00000000000..5d772bd7d17 --- /dev/null +++ b/libavcodec/aom_film_grain.h @@ -0,0 +1,38 @@ +/* + * AOM film grain synthesis + * Copyright (c) 2021 Niklas Haas + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * AOM film grain synthesis. + * @author Niklas Haas + */ + +#ifndef AVCODEC_AOM_FILM_GRAIN_H +#define AVCODEC_AOM_FILM_GRAIN_H + +#include "libavutil/film_grain_params.h" + +// Synthesizes film grain on top of `in` and stores the result to `out`. `out` +// must already have been allocated and set to the same size and format as `in`. +int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in, + const AVFilmGrainParams *params); + +#endif /* AVCODEC_AOM_FILM_GRAIN_H */ diff --git a/libavcodec/aom_film_grain_template.c b/libavcodec/aom_film_grain_template.c new file mode 100644 index 00000000000..5f9f29f1fab --- /dev/null +++ b/libavcodec/aom_film_grain_template.c @@ -0,0 +1,577 @@ +/* + * AOM film grain synthesis + * Copyright (c) 2023 Niklas Haas + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/* + * Copyright © 2018, Niklas Haas + * Copyright © 2018, VideoLAN and dav1d authors + * Copyright © 2018, Two Orioles, LLC + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * + * 1. Redistributions of source code must retain the above copyright notice, this + * list of conditions and the following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above copyright notice, + * this list of conditions and the following disclaimer in the documentation + * and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR + * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include "bit_depth_template.c" + +#undef entry +#undef bitdepth +#undef bitdepth_max +#undef HBD_DECL +#undef HBD_CALL +#undef SCALING_SIZE + +#if BIT_DEPTH > 8 +# define entry int16_t +# define bitdepth_max ((1 << bitdepth) - 1) +# define HBD_DECL , const int bitdepth +# define HBD_CALL , bitdepth +# define SCALING_SIZE 4096 +#else +# define entry int8_t +# define bitdepth 8 +# define bitdepth_max UINT8_MAX +# define HBD_DECL +# define HBD_CALL +# define SCALING_SIZE 256 +#endif + +static void FUNC(generate_grain_y_c)(entry buf[][GRAIN_WIDTH], + const AVFilmGrainParams *const params + HBD_DECL) +{ + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const int bitdepth_min_8 = bitdepth - 8; + unsigned seed = params->seed; + const int shift = 4 - bitdepth_min_8 + data->grain_scale_shift; + const int grain_ctr = 128 << bitdepth_min_8; + const int grain_min = -grain_ctr, grain_max = grain_ctr - 1; + + const int ar_pad = 3; + const int ar_lag = data->ar_coeff_lag; + + for (int y = 0; y < GRAIN_HEIGHT; y++) { + for (int x = 0; x < GRAIN_WIDTH; x++) { + const int value = get_random_number(11, &seed); + buf[y][x] = round2(gaussian_sequence[ value ], shift); + } + } + + for (int y = ar_pad; y < GRAIN_HEIGHT; y++) { + for (int x = ar_pad; x < GRAIN_WIDTH - ar_pad; x++) { + const int8_t *coeff = data->ar_coeffs_y; + int sum = 0, grain; + for (int dy = -ar_lag; dy <= 0; dy++) { + for (int dx = -ar_lag; dx <= ar_lag; dx++) { + if (!dx && !dy) + break; + sum += *(coeff++) * buf[y + dy][x + dx]; + } + } + + grain = buf[y][x] + round2(sum, data->ar_coeff_shift); + buf[y][x] = av_clip(grain, grain_min, grain_max); + } + } +} + +static void +FUNC(generate_grain_uv_c)(entry buf[][GRAIN_WIDTH], + const entry buf_y[][GRAIN_WIDTH], + const AVFilmGrainParams *const params, const intptr_t uv, + const int subx, const int suby HBD_DECL) +{ + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const int bitdepth_min_8 = bitdepth - 8; + unsigned seed = params->seed ^ (uv ? 0x49d8 : 0xb524); + const int shift = 4 - bitdepth_min_8 + data->grain_scale_shift; + const int grain_ctr = 128 << bitdepth_min_8; + const int grain_min = -grain_ctr, grain_max = grain_ctr - 1; + + const int chromaW = subx ? SUB_GRAIN_WIDTH : GRAIN_WIDTH; + const int chromaH = suby ? SUB_GRAIN_HEIGHT : GRAIN_HEIGHT; + + const int ar_pad = 3; + const int ar_lag = data->ar_coeff_lag; + + for (int y = 0; y < chromaH; y++) { + for (int x = 0; x < chromaW; x++) { + const int value = get_random_number(11, &seed); + buf[y][x] = round2(gaussian_sequence[ value ], shift); + } + } + + for (int y = ar_pad; y < chromaH; y++) { + for (int x = ar_pad; x < chromaW - ar_pad; x++) { + const int8_t *coeff = data->ar_coeffs_uv[uv]; + int sum = 0, grain; + for (int dy = -ar_lag; dy <= 0; dy++) { + for (int dx = -ar_lag; dx <= ar_lag; dx++) { + // For the final (current) pixel, we need to add in the + // contribution from the luma grain texture + if (!dx && !dy) { + const int lumaX = ((x - ar_pad) << subx) + ar_pad; + const int lumaY = ((y - ar_pad) << suby) + ar_pad; + int luma = 0; + if (!data->num_y_points) + break; + for (int i = 0; i <= suby; i++) { + for (int j = 0; j <= subx; j++) { + luma += buf_y[lumaY + i][lumaX + j]; + } + } + luma = round2(luma, subx + suby); + sum += luma * (*coeff); + break; + } + + sum += *(coeff++) * buf[y + dy][x + dx]; + } + } + + grain = buf[y][x] + round2(sum, data->ar_coeff_shift); + buf[y][x] = av_clip(grain, grain_min, grain_max); + } + } +} + +// samples from the correct block of a grain LUT, while taking into account the +// offsets provided by the offsets cache +static inline entry FUNC(sample_lut)(const entry grain_lut[][GRAIN_WIDTH], + const int offsets[2][2], + const int subx, const int suby, + const int bx, const int by, + const int x, const int y) +{ + const int randval = offsets[bx][by]; + const int offx = 3 + (2 >> subx) * (3 + (randval >> 4)); + const int offy = 3 + (2 >> suby) * (3 + (randval & 0xF)); + return grain_lut[offy + y + (FG_BLOCK_SIZE >> suby) * by] + [offx + x + (FG_BLOCK_SIZE >> subx) * bx]; +} + +static void FUNC(fgy_32x32xn_c)(pixel *const dst_row, const pixel *const src_row, + const ptrdiff_t stride, + const AVFilmGrainParams *const params, const size_t pw, + const uint8_t scaling[SCALING_SIZE], + const entry grain_lut[][GRAIN_WIDTH], + const int bh, const int row_num HBD_DECL) +{ + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const int rows = 1 + (data->overlap_flag && row_num > 0); + const int bitdepth_min_8 = bitdepth - 8; + const int grain_ctr = 128 << bitdepth_min_8; + const int grain_min = -grain_ctr, grain_max = grain_ctr - 1; + unsigned seed[2]; + int offsets[2 /* col offset */][2 /* row offset */]; + + int min_value, max_value; + if (data->limit_output_range) { + min_value = 16 << bitdepth_min_8; + max_value = 235 << bitdepth_min_8; + } else { + min_value = 0; + max_value = bitdepth_max; + } + + // seed[0] contains the current row, seed[1] contains the previous + for (int i = 0; i < rows; i++) { + seed[i] = params->seed; + seed[i] ^= (((row_num - i) * 37 + 178) & 0xFF) << 8; + seed[i] ^= (((row_num - i) * 173 + 105) & 0xFF); + } + + av_assert1(stride % (FG_BLOCK_SIZE * sizeof(pixel)) == 0); + + // process this row in FG_BLOCK_SIZE^2 blocks + for (unsigned bx = 0; bx < pw; bx += FG_BLOCK_SIZE) { + const int bw = FFMIN(FG_BLOCK_SIZE, (int) pw - bx); + const pixel *src; + pixel *dst; + int noise; + + // x/y block offsets to compensate for overlapped regions + const int ystart = data->overlap_flag && row_num ? FFMIN(2, bh) : 0; + const int xstart = data->overlap_flag && bx ? FFMIN(2, bw) : 0; + + static const int w[2][2] = { { 27, 17 }, { 17, 27 } }; + + if (data->overlap_flag && bx) { + // shift previous offsets left + for (int i = 0; i < rows; i++) + offsets[1][i] = offsets[0][i]; + } + + // update current offsets + for (int i = 0; i < rows; i++) + offsets[0][i] = get_random_number(8, &seed[i]); + +#define add_noise_y(x, y, grain) \ + src = (const pixel*)((const char*)src_row + (y) * stride) + (x) + bx; \ + dst = (pixel*)((char*)dst_row + (y) * stride) + (x) + bx; \ + noise = round2(scaling[ *src ] * (grain), data->scaling_shift); \ + *dst = av_clip(*src + noise, min_value, max_value); + + for (int y = ystart; y < bh; y++) { + // Non-overlapped image region (straightforward) + for (int x = xstart; x < bw; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y); + add_noise_y(x, y, grain); + } + + // Special case for overlapped column + for (int x = 0; x < xstart; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y); + int old = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 1, 0, x, y); + grain = round2(old * w[x][0] + grain * w[x][1], 5); + grain = av_clip(grain, grain_min, grain_max); + add_noise_y(x, y, grain); + } + } + + for (int y = 0; y < ystart; y++) { + // Special case for overlapped row (sans corner) + for (int x = xstart; x < bw; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y); + int old = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 1, x, y); + grain = round2(old * w[y][0] + grain * w[y][1], 5); + grain = av_clip(grain, grain_min, grain_max); + add_noise_y(x, y, grain); + } + + // Special case for doubly-overlapped corner + for (int x = 0; x < xstart; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y); + int top = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 1, x, y); + int old = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 1, 1, x, y); + + // Blend the top pixel with the top left block + top = round2(old * w[x][0] + top * w[x][1], 5); + top = av_clip(top, grain_min, grain_max); + + // Blend the current pixel with the left block + old = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 1, 0, x, y); + grain = round2(old * w[x][0] + grain * w[x][1], 5); + grain = av_clip(grain, grain_min, grain_max); + + // Mix the row rows together and apply grain + grain = round2(top * w[y][0] + grain * w[y][1], 5); + grain = av_clip(grain, grain_min, grain_max); + add_noise_y(x, y, grain); + } + } + } +} + +static void +FUNC(fguv_32x32xn_c)(pixel *const dst_row, const pixel *const src_row, + const ptrdiff_t stride, const AVFilmGrainParams *const params, + const size_t pw, const uint8_t scaling[SCALING_SIZE], + const entry grain_lut[][GRAIN_WIDTH], const int bh, + const int row_num, const pixel *const luma_row, + const ptrdiff_t luma_stride, const int uv, const int is_id, + const int sx, const int sy HBD_DECL) +{ + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const int rows = 1 + (data->overlap_flag && row_num > 0); + const int bitdepth_min_8 = bitdepth - 8; + const int grain_ctr = 128 << bitdepth_min_8; + const int grain_min = -grain_ctr, grain_max = grain_ctr - 1; + unsigned seed[2]; + int offsets[2 /* col offset */][2 /* row offset */]; + + int min_value, max_value; + if (data->limit_output_range) { + min_value = 16 << bitdepth_min_8; + max_value = (is_id ? 235 : 240) << bitdepth_min_8; + } else { + min_value = 0; + max_value = bitdepth_max; + } + + // seed[0] contains the current row, seed[1] contains the previous + for (int i = 0; i < rows; i++) { + seed[i] = params->seed; + seed[i] ^= (((row_num - i) * 37 + 178) & 0xFF) << 8; + seed[i] ^= (((row_num - i) * 173 + 105) & 0xFF); + } + + av_assert1(stride % (FG_BLOCK_SIZE * sizeof(pixel)) == 0); + + // process this row in FG_BLOCK_SIZE^2 blocks (subsampled) + for (unsigned bx = 0; bx < pw; bx += FG_BLOCK_SIZE >> sx) { + const int bw = FFMIN(FG_BLOCK_SIZE >> sx, (int)(pw - bx)); + int val, lx, ly, noise; + const pixel *src, *luma; + pixel *dst, avg; + + // x/y block offsets to compensate for overlapped regions + const int ystart = data->overlap_flag && row_num ? FFMIN(2 >> sy, bh) : 0; + const int xstart = data->overlap_flag && bx ? FFMIN(2 >> sx, bw) : 0; + + static const int w[2 /* sub */][2 /* off */][2] = { + { { 27, 17 }, { 17, 27 } }, + { { 23, 22 } }, + }; + + if (data->overlap_flag && bx) { + // shift previous offsets left + for (int i = 0; i < rows; i++) + offsets[1][i] = offsets[0][i]; + } + + // update current offsets + for (int i = 0; i < rows; i++) + offsets[0][i] = get_random_number(8, &seed[i]); + +#define add_noise_uv(x, y, grain) \ + lx = (bx + x) << sx; \ + ly = y << sy; \ + luma = (const pixel*)((const char*)luma_row + ly * luma_stride) + lx;\ + avg = luma[0]; \ + if (sx) \ + avg = (avg + luma[1] + 1) >> 1; \ + src = (const pixel*)((const char *)src_row + (y) * stride) + bx + (x);\ + dst = (pixel *) ((char *) dst_row + (y) * stride) + bx + (x); \ + val = avg; \ + if (!data->chroma_scaling_from_luma) { \ + const int combined = avg * data->uv_mult_luma[uv] + \ + *src * data->uv_mult[uv]; \ + val = av_clip( (combined >> 6) + \ + (data->uv_offset[uv] * (1 << bitdepth_min_8)), \ + 0, bitdepth_max ); \ + } \ + noise = round2(scaling[ val ] * (grain), data->scaling_shift); \ + *dst = av_clip(*src + noise, min_value, max_value); + + for (int y = ystart; y < bh; y++) { + // Non-overlapped image region (straightforward) + for (int x = xstart; x < bw; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y); + add_noise_uv(x, y, grain); + } + + // Special case for overlapped column + for (int x = 0; x < xstart; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y); + int old = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 1, 0, x, y); + grain = round2(old * w[sx][x][0] + grain * w[sx][x][1], 5); + grain = av_clip(grain, grain_min, grain_max); + add_noise_uv(x, y, grain); + } + } + + for (int y = 0; y < ystart; y++) { + // Special case for overlapped row (sans corner) + for (int x = xstart; x < bw; x++) { + int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y); + int old = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 1, x, y); + grain = round2(old * w[sy][y][0] + grain * w[sy][y][1], 5); + grain = av_clip(grain, grain_min, grain_max); + add_noise_uv(x, y, grain); + } + + // Special case for doubly-overlapped corner + for (int x = 0; x < xstart; x++) { + int top = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 1, x, y); + int old = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 1, 1, x, y); + int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y); + + // Blend the top pixel with the top left block + top = round2(old * w[sx][x][0] + top * w[sx][x][1], 5); + top = av_clip(top, grain_min, grain_max); + + // Blend the current pixel with the left block + old = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 1, 0, x, y); + grain = round2(old * w[sx][x][0] + grain * w[sx][x][1], 5); + grain = av_clip(grain, grain_min, grain_max); + + // Mix the row rows together and apply to image + grain = round2(top * w[sy][y][0] + grain * w[sy][y][1], 5); + grain = av_clip(grain, grain_min, grain_max); + add_noise_uv(x, y, grain); + } + } + } +} + +static void FUNC(generate_scaling)(const uint8_t points[][2], const int num, + uint8_t scaling[SCALING_SIZE] HBD_DECL) +{ + const int shift_x = bitdepth - 8; + const int scaling_size = 1 << bitdepth; + const int max_value = points[num - 1][0] << shift_x; + av_assert0(scaling_size <= SCALING_SIZE); + + if (num == 0) { + memset(scaling, 0, scaling_size); + return; + } + + // Fill up the preceding entries with the initial value + memset(scaling, points[0][1], points[0][0] << shift_x); + + // Linearly interpolate the values in the middle + for (int i = 0; i < num - 1; i++) { + const int bx = points[i][0]; + const int by = points[i][1]; + const int ex = points[i+1][0]; + const int ey = points[i+1][1]; + const int dx = ex - bx; + const int dy = ey - by; + const int delta = dy * ((0x10000 + (dx >> 1)) / dx); + av_assert1(dx > 0); + for (int x = 0, d = 0x8000; x < dx; x++) { + scaling[(bx + x) << shift_x] = by + (d >> 16); + d += delta; + } + } + + // Fill up the remaining entries with the final value + memset(&scaling[max_value], points[num - 1][1], scaling_size - max_value); + +#if BIT_DEPTH != 8 + for (int i = 0; i < num - 1; i++) { + const int pad = 1 << shift_x, rnd = pad >> 1; + const int bx = points[i][0] << shift_x; + const int ex = points[i+1][0] << shift_x; + const int dx = ex - bx; + for (int x = 0; x < dx; x += pad) { + const int range = scaling[bx + x + pad] - scaling[bx + x]; + for (int n = 1, r = rnd; n < pad; n++) { + r += range; + scaling[bx + x + n] = scaling[bx + x] + (r >> shift_x); + } + } + } +#endif +} + +static av_always_inline void +FUNC(apply_grain_row)(AVFrame *out, const AVFrame *in, + const int ss_x, const int ss_y, + const uint8_t scaling[3][SCALING_SIZE], + const entry grain_lut[3][GRAIN_HEIGHT+1][GRAIN_WIDTH], + const AVFilmGrainParams *params, + const int row HBD_DECL) +{ + // Synthesize grain for the affected planes + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const int cpw = (out->width + ss_x) >> ss_x; + const int is_id = out->colorspace == AVCOL_SPC_RGB; + const int bh = (FFMIN(out->height - row * FG_BLOCK_SIZE, FG_BLOCK_SIZE) + ss_y) >> ss_y; + const ptrdiff_t uv_off = row * FG_BLOCK_SIZE * out->linesize[1] >> ss_y; + pixel *const luma_src = (pixel *) + ((char *) in->data[0] + row * FG_BLOCK_SIZE * in->linesize[0]); + + if (data->num_y_points) { + const int bh = FFMIN(out->height - row * FG_BLOCK_SIZE, FG_BLOCK_SIZE); + const ptrdiff_t off = row * FG_BLOCK_SIZE * out->linesize[0]; + FUNC(fgy_32x32xn_c)((pixel *) ((char *) out->data[0] + off), luma_src, + out->linesize[0], params, out->width, scaling[0], + grain_lut[0], bh, row HBD_CALL); + } + + if (!data->num_uv_points[0] && !data->num_uv_points[1] && + !data->chroma_scaling_from_luma) + { + return; + } + + // extend padding pixels + if (out->width & ss_x) { + pixel *ptr = luma_src; + for (int y = 0; y < bh; y++) { + ptr[out->width] = ptr[out->width - 1]; + ptr = (pixel *) ((char *) ptr + (in->linesize[0] << ss_y)); + } + } + + if (data->chroma_scaling_from_luma) { + for (int pl = 0; pl < 2; pl++) + FUNC(fguv_32x32xn_c)((pixel *) ((char *) out->data[1 + pl] + uv_off), + (const pixel *) ((const char *) in->data[1 + pl] + uv_off), + in->linesize[1], params, cpw, scaling[0], + grain_lut[1 + pl], bh, row, luma_src, + in->linesize[0], pl, is_id, ss_x, ss_y HBD_CALL); + } else { + for (int pl = 0; pl < 2; pl++) { + if (data->num_uv_points[pl]) { + FUNC(fguv_32x32xn_c)((pixel *) ((char *) out->data[1 + pl] + uv_off), + (const pixel *) ((const char *) in->data[1 + pl] + uv_off), + in->linesize[1], params, cpw, scaling[1 + pl], + grain_lut[1 + pl], bh, row, luma_src, + in->linesize[0], pl, is_id, ss_x, ss_y HBD_CALL); + } + } + } +} + +static int FUNC(apply_film_grain)(AVFrame *out_frame, const AVFrame *in_frame, + const AVFilmGrainParams *params HBD_DECL) +{ + entry grain_lut[3][GRAIN_HEIGHT + 1][GRAIN_WIDTH]; + uint8_t scaling[3][SCALING_SIZE]; + + const AVFilmGrainAOMParams *const data = ¶ms->codec.aom; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(out_frame->format); + const int rows = AV_CEIL_RSHIFT(out_frame->height, 5); /* log2(FG_BLOCK_SIZE) */ + const int subx = desc->log2_chroma_w, suby = desc->log2_chroma_h; + + // Generate grain LUTs as needed + FUNC(generate_grain_y_c)(grain_lut[0], params HBD_CALL); + if (data->num_uv_points[0] || data->chroma_scaling_from_luma) + FUNC(generate_grain_uv_c)(grain_lut[1], grain_lut[0], params, 0, subx, suby HBD_CALL); + if (data->num_uv_points[1] || data->chroma_scaling_from_luma) + FUNC(generate_grain_uv_c)(grain_lut[2], grain_lut[0], params, 1, subx, suby HBD_CALL); + + // Generate scaling LUTs as needed + if (data->num_y_points || data->chroma_scaling_from_luma) + FUNC(generate_scaling)(data->y_points, data->num_y_points, scaling[0] HBD_CALL); + if (data->num_uv_points[0]) + FUNC(generate_scaling)(data->uv_points[0], data->num_uv_points[0], scaling[1] HBD_CALL); + if (data->num_uv_points[1]) + FUNC(generate_scaling)(data->uv_points[1], data->num_uv_points[1], scaling[2] HBD_CALL); + + for (int row = 0; row < rows; row++) { + FUNC(apply_grain_row)(out_frame, in_frame, subx, suby, scaling, grain_lut, + params, row HBD_CALL); + } + + return 0; +} From patchwork Fri Mar 15 11:59:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47107 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp194515pzh; Fri, 15 Mar 2024 05:06:26 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXvcVPKpDUZ0HyspIxE5qS/JbJdtSDEkK8W0QUh/+1BcEphtBAW0SBW97UQyrSvgswLKXWQNhKFDNeq4FE5l1h7EkeMZGwFuXZxJg== X-Google-Smtp-Source: AGHT+IE9ybU8jq/l5soFo2angwoQVIN1ASDuGidkQ9IH0ETRjPEGcz4izzGFgQ/kZkE+8I9QOogr X-Received: by 2002:a05:6402:501d:b0:566:6640:10e5 with SMTP id p29-20020a056402501d00b00566664010e5mr3773865eda.12.1710504386355; Fri, 15 Mar 2024 05:06:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504386; cv=none; d=google.com; s=arc-20160816; b=V2+CQ8ZCF9uukEL3Z5Nq0DzsEqXQj6wNlXB9E7jOXbQ9BFTNFIsuMXQGsyJ01+AANg h/typrWbeMSLml1qkIjmm8S53lCWiN9pl2zSqV/2MVoF7yv/dxxuBEHl0sSTOwgtLHSQ 4R3RAVnx6UioXdASoH5UwH2T/mkn8Vt7otbF28aAFYG1oDDTNErQto8cPima9bokm8AP CSOyCs/r6Sb6HNwC47voqfHfSXDIKnMra/P1CLTmM4AF2tDl5ZFkRP+4QSHuKfXC7OqF i86UyjKb2Zcas5fL022XF5kbjnYRxvy4mj1Om2SB6RBZCJJsnP8tG8145OGpKuqu0Ry4 Ix0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=4jqV77sb1DUXK7uTkdIt4iwAnVWHMncAmH/tAKppbVg=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=sbZ8MZzrjND9c2hyN/ygi8kQ2NmMoWpH7VB+BufiuWjRlJdolLrwBIHn8zniQPDNjy E9iP1n9RVpuqpzl9QDJMuRv0TRnq+jYvFPfY/tFsK/UYtuR82YBojxAGAal199guWYur bezInI2F7j+cDLDdwxfwRxeLMADJTTYPDva+aCI8yiFm6g8JTcftlT+U20SDghR8eLxv IiJFFqxrq1nCJJQqMrPgVrTZ2VO6JIP2Q9li2jLDexlQc6LwaVaNfxSyEvo6ZIfK7kfu sNOnwaayKgeeokkGzR1SpIziUnhf9RfFj4l6py1LnNhTE0PQq3+MwevNTQTnSeX4u51t e5cw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=GpZTDOgU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y18-20020a056402441200b0056899ce83a0si1814189eda.293.2024.03.15.05.06.13; Fri, 15 Mar 2024 05:06:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=GpZTDOgU; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 41AC768D18B; Fri, 15 Mar 2024 14:05:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B4E7768D18B for ; Fri, 15 Mar 2024 14:04:51 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504287; bh=ZZWcpfPYxIKHZ8Q15jBrm2g1GRu+wxk7UMgnwPhvk1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GpZTDOgUAQrjKbS05SxoixTUTncdooWh1N97n1Vw5IgGWEDc4gK1S2dC6SBIB2DDT FMg8+sGSZva7+t2NgsAz7cDuk2t0VfHGG/HolWv3/TiUIYajf52FRaBXEt47wcszdx 08PHbZi8RhitCmJ7lqR+wPJ1CY3i9uzg6iQnzOpM= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id B4E4842A8B; Fri, 15 Mar 2024 13:04:47 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:59:00 +0100 Message-ID: <20240315120442.73754-8-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 7/9] avcodec/aom_film_grain: implement AFGS1 parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 8zQLxMA2cBn5 From: Niklas Haas Based on the AOMedia Film Grain Synthesis 1 (AFGS1) spec: https://aomediacodec.github.io/afgs1-spec/ The parsing has been changed substantially relative to the AV1 film grain OBU. In particular: 1. There is the possibility of maintaining multiple independent film grain parameter sets, and decoders/players are recommended to pick the one most appropriate for the intended display resolution. This could also be used to e.g. switch between different grain profiles without having to re-signal the appropriate coefficients. 2. Supporting this, it's possible to *predict* the grain coefficients from previously signalled parameter sets, transmitting only the residual. 3. When not predicting, the parameter sets are now stored as a series of increments, rather than being directly transmitted. 4. There are several new AFGS1-exclusive fields. I placed this parser in its own file, rather than h2645_sei.c, since nothing in the generic AFGS1 film grain payload is specific to T.35, and to compartmentalize the code base. --- libavcodec/aom_film_grain.c | 236 ++++++++++++++++++++++++++++++++++++ libavcodec/aom_film_grain.h | 13 ++ 2 files changed, 249 insertions(+) diff --git a/libavcodec/aom_film_grain.c b/libavcodec/aom_film_grain.c index ffcd71b584b..ea38935ca98 100644 --- a/libavcodec/aom_film_grain.c +++ b/libavcodec/aom_film_grain.c @@ -29,6 +29,7 @@ #include "libavutil/imgutils.h" #include "aom_film_grain.h" +#include "get_bits.h" // Common/shared helpers (not dependent on BIT_DEPTH) static inline int get_random_number(const int bits, unsigned *const state) { @@ -118,6 +119,241 @@ int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in, return AVERROR_INVALIDDATA; } +int ff_aom_parse_film_grain_sets(AVFilmGrainAFGS1Params *s, + const uint8_t *payload, int payload_size) +{ + GetBitContext gbc, *gb = &gbc; + AVFilmGrainAOMParams *aom; + AVFilmGrainParams *fgp, *ref = NULL; + int ret, num_sets, n, i, uv, num_y_coeffs, update_grain, luma_only; + + ret = init_get_bits8(gb, payload, payload_size); + if (ret < 0) + return ret; + + s->enable = get_bits1(gb); + if (!s->enable) + return 0; + + skip_bits(gb, 4); // reserved + num_sets = get_bits(gb, 3) + 1; + for (n = 0; n < num_sets; n++) { + int payload_4byte, payload_size, set_idx, apply_units_log2, vsc_flag; + int predict_scaling, predict_y_scaling, predict_uv_scaling[2]; + int payload_bits, start_position; + + start_position = get_bits_count(gb); + payload_4byte = get_bits1(gb); + payload_size = get_bits(gb, payload_4byte ? 2 : 8); + set_idx = get_bits(gb, 3); + fgp = &s->sets[set_idx]; + aom = &fgp->codec.aom; + + fgp->type = get_bits1(gb) ? AV_FILM_GRAIN_PARAMS_AV1 : AV_FILM_GRAIN_PARAMS_NONE; + if (!fgp->type) + continue; + + fgp->seed = get_bits(gb, 16); + update_grain = get_bits1(gb); + if (!update_grain) + continue; + + apply_units_log2 = get_bits(gb, 4); + aom->width = get_bits(gb, 12) << apply_units_log2; + aom->height = get_bits(gb, 12) << apply_units_log2; + luma_only = get_bits1(gb); + if (luma_only) { + aom->subx = aom->suby = 0; + } else { + aom->subx = get_bits1(gb); + aom->suby = get_bits1(gb); + } + + aom->bit_depth = 0; + aom->color_primaries = AVCOL_PRI_UNSPECIFIED; + aom->color_trc = AVCOL_TRC_UNSPECIFIED; + aom->color_space = AVCOL_SPC_UNSPECIFIED; + aom->color_range = AVCOL_RANGE_UNSPECIFIED; + + vsc_flag = get_bits1(gb); // video_signal_characteristics_flag + if (vsc_flag) { + int cicp_flag; + aom->bit_depth = get_bits(gb, 3) + 8; + cicp_flag = get_bits1(gb); + if (cicp_flag) { + aom->color_primaries = get_bits(gb, 8); + aom->color_trc = get_bits(gb, 8); + aom->color_space = get_bits(gb, 8); + aom->color_range = get_bits1(gb) ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + if (aom->color_primaries > AVCOL_PRI_NB || + aom->color_primaries == AVCOL_PRI_RESERVED || + aom->color_primaries == AVCOL_PRI_RESERVED0 || + aom->color_trc > AVCOL_TRC_NB || + aom->color_trc == AVCOL_TRC_RESERVED || + aom->color_trc == AVCOL_TRC_RESERVED0 || + aom->color_space > AVCOL_SPC_NB || + aom->color_space == AVCOL_SPC_RESERVED) + goto error; + } + } + + predict_scaling = get_bits1(gb); + if (predict_scaling && (!ref || ref == fgp)) + goto error; // prediction must be from valid, different set + + predict_y_scaling = predict_scaling ? get_bits1(gb) : 0; + if (predict_y_scaling) { + int y_scale, y_offset, bits_res; + y_scale = get_bits(gb, 9) - 256; + y_offset = get_bits(gb, 9) - 256; + bits_res = get_bits(gb, 3); + if (bits_res) { + int res[14], pred, granularity; + aom->num_y_points = ref->codec.aom.num_y_points; + for (i = 0; i < aom->num_y_points; i++) + res[i] = get_bits(gb, bits_res); + granularity = get_bits(gb, 3); + for (i = 0; i < aom->num_y_points; i++) { + pred = ref->codec.aom.y_points[i][1]; + pred = ((pred * y_scale + 8) >> 4) + y_offset; + pred += (res[i] - (1 << (bits_res - 1))) * granularity; + aom->y_points[i][0] = ref->codec.aom.y_points[i][0]; + aom->y_points[i][1] = av_clip_uint8(pred); + } + } + } else { + aom->num_y_points = get_bits(gb, 4); + if (aom->num_y_points > 14) { + goto error; + } else if (aom->num_y_points) { + int bits_inc, bits_scaling; + int y_value = 0; + bits_inc = get_bits(gb, 3) + 1; + bits_scaling = get_bits(gb, 2) + 5; + for (i = 0; i < aom->num_y_points; i++) { + y_value += get_bits(gb, bits_inc); + if (y_value > UINT8_MAX) + goto error; + aom->y_points[i][0] = y_value; + aom->y_points[i][1] = get_bits(gb, bits_scaling); + } + } + } + + if (luma_only) { + aom->chroma_scaling_from_luma = 0; + aom->num_uv_points[0] = aom->num_uv_points[1] = 0; + } else { + aom->chroma_scaling_from_luma = get_bits1(gb); + if (aom->chroma_scaling_from_luma) { + aom->num_uv_points[0] = aom->num_uv_points[1] = 0; + } else { + for (uv = 0; uv < 2; uv++) { + predict_uv_scaling[uv] = predict_scaling ? get_bits1(gb) : 0; + if (predict_uv_scaling[uv]) { + int uv_scale, uv_offset, bits_res; + uv_scale = get_bits(gb, 9) - 256; + uv_offset = get_bits(gb, 9) - 256; + bits_res = get_bits(gb, 3); + aom->uv_mult[uv] = ref->codec.aom.uv_mult[uv]; + aom->uv_mult_luma[uv] = ref->codec.aom.uv_mult_luma[uv]; + aom->uv_offset[uv] = ref->codec.aom.uv_offset[uv]; + if (bits_res) { + int res[10], pred, granularity; + aom->num_uv_points[uv] = ref->codec.aom.num_uv_points[uv]; + for (i = 0; i < aom->num_uv_points[uv]; i++) + res[i] = get_bits(gb, bits_res); + granularity = get_bits(gb, 3); + for (i = 0; i < aom->num_uv_points[uv]; i++) { + pred = ref->codec.aom.uv_points[uv][i][1]; + pred = ((pred * uv_scale + 8) >> 4) + uv_offset; + pred += (res[i] - (1 << (bits_res - 1))) * granularity; + aom->uv_points[uv][i][0] = ref->codec.aom.uv_points[uv][i][0]; + aom->uv_points[uv][i][1] = av_clip_uint8(pred); + } + } + } else { + int bits_inc, bits_scaling, uv_offset; + int uv_value = 0; + aom->num_uv_points[uv] = get_bits(gb, 4); + if (aom->num_uv_points[uv] > 10) + goto error; + bits_inc = get_bits(gb, 3) + 1; + bits_scaling = get_bits(gb, 2) + 5; + uv_offset = get_bits(gb, 8); + for (i = 0; i < aom->num_uv_points[uv]; i++) { + uv_value += get_bits(gb, bits_inc); + if (uv_value > UINT8_MAX) + goto error; + aom->uv_points[uv][i][0] = uv_value; + aom->uv_points[uv][i][1] = get_bits(gb, bits_scaling) + uv_offset; + } + } + } + } + } + + aom->scaling_shift = get_bits(gb, 2) + 8; + aom->ar_coeff_lag = get_bits(gb, 2); + num_y_coeffs = 2 * aom->ar_coeff_lag * (aom->ar_coeff_lag + 1); + if (aom->num_y_points) { + int ar_bits = get_bits(gb, 2) + 5; + for (i = 0; i < num_y_coeffs; i++) + aom->ar_coeffs_y[i] = get_bits(gb, ar_bits) - (1 << (ar_bits - 1)); + } + for (uv = 0; uv < 2; uv++) { + if (aom->chroma_scaling_from_luma || aom->num_uv_points[uv]) { + int ar_bits = get_bits(gb, 2) + 5; + for (i = 0; i < num_y_coeffs + !!aom->num_y_points; i++) + aom->ar_coeffs_uv[uv][i] = get_bits(gb, ar_bits) - (1 << (ar_bits - 1)); + } + } + aom->ar_coeff_shift = get_bits(gb, 2) + 6; + aom->grain_scale_shift = get_bits(gb, 2); + for (uv = 0; uv < 2; uv++) { + if (aom->num_uv_points[uv] && !predict_uv_scaling[uv]) { + aom->uv_mult[uv] = get_bits(gb, 8) - 128; + aom->uv_mult_luma[uv] = get_bits(gb, 8) - 128; + aom->uv_offset[uv] = get_bits(gb, 9) - 256; + } + } + aom->overlap_flag = get_bits1(gb); + aom->limit_output_range = get_bits1(gb); + + // use first set as reference only if it was fully transmitted + if (n == 0) + ref = fgp; + + payload_bits = get_bits_count(gb) - start_position; + if (payload_bits > payload_size * 8) + goto error; + skip_bits(gb, payload_size * 8 - payload_bits); + } + return 0; + +error: + memset(s, 0, sizeof(*s)); + return AVERROR_INVALIDDATA; +} + +int ff_aom_attach_film_grain_sets(const AVFilmGrainAFGS1Params *s, AVFrame *frame) +{ + AVFilmGrainParams *fgp; + if (!s->enable) + return 0; + + for (int i = 0; i < FF_ARRAY_ELEMS(s->sets); i++) { + if (s->sets[i].type != AV_FILM_GRAIN_PARAMS_AV1) + continue; + fgp = av_film_grain_params_create_side_data(frame); + if (!fgp) + return AVERROR(ENOMEM); + memcpy(fgp, &s->sets[i], sizeof(*fgp)); + } + + return 0; +} + // Taken from the AV1 spec. Range is [-2048, 2047], mean is 0 and stddev is 512 static const int16_t gaussian_sequence[2048] = { 56, 568, -180, 172, 124, -84, 172, -64, -900, 24, 820, diff --git a/libavcodec/aom_film_grain.h b/libavcodec/aom_film_grain.h index 5d772bd7d17..1f8c78f657a 100644 --- a/libavcodec/aom_film_grain.h +++ b/libavcodec/aom_film_grain.h @@ -30,9 +30,22 @@ #include "libavutil/film_grain_params.h" +typedef struct AVFilmGrainAFGS1Params { + int enable; + AVFilmGrainParams sets[8]; +} AVFilmGrainAFGS1Params; + // Synthesizes film grain on top of `in` and stores the result to `out`. `out` // must already have been allocated and set to the same size and format as `in`. int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in, const AVFilmGrainParams *params); +// Parse AFGS1 parameter sets from an ITU-T T.35 payload. Returns 0 on success, +// or a negative error code. +int ff_aom_parse_film_grain_sets(AVFilmGrainAFGS1Params *s, + const uint8_t *payload, int payload_size); + +// Attach all valid film grain param sets to `frame`. +int ff_aom_attach_film_grain_sets(const AVFilmGrainAFGS1Params *s, AVFrame *frame); + #endif /* AVCODEC_AOM_FILM_GRAIN_H */ From patchwork Fri Mar 15 11:59:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47106 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp194499pzh; Fri, 15 Mar 2024 05:06:25 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWF2yf5MspyX/v9DFskJFZLYdVREbjB76BX2lfW8zc2rw5YOMJC3vfaVvoCOWclFbEBu/L9IS9SqW5WybZCiLmB9kLrG9ueHG978A== X-Google-Smtp-Source: AGHT+IHP1NoZU9hi8YcCqc+75iGWFBhpr2smLfTgu+LOxDGL6A7WZTalpBwmZ96JwhOVEsgCSUwz X-Received: by 2002:a05:6402:380b:b0:568:9b13:77fc with SMTP id es11-20020a056402380b00b005689b1377fcmr2463635edb.36.1710504385612; Fri, 15 Mar 2024 05:06:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504385; cv=none; d=google.com; s=arc-20160816; b=O2wwOtxFjjiGhTQfrwkDVEODT5XRplb8oEpfaWuHai7jIf2dVd/6K+RAOqdgqpxt15 R/HoDFRYlDuQEtkpOjkjR7ICl4jfjvJJ8jMCuFOQQWSTUtf2Z/3j+oPQQLRG70ObK00s +uzZ36Q2dV9WBJE9b613SAzxxYeQoRtaPL+XHxIMWo9oA55WXVTuVWmdPoxTOjmMZkvK PlCDjxnKCHlQs3wuT4w08s5UsWrAZM5JV9yNqTUDclRo/yB0xz5z7pxrsIOE24wYJo4m yVGTWVCmuoXT63rH3iESBtdyFq6h2WyVixHsqHYQm857K3fQDNWUm93iJFzHpUKRp5Ce RojA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=RKozTO2tYKD3y75ruqnvaRa6+NC3JKXhl0y5HYgVd7w=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=I2ltPbnwtOUOEkcvZ0arITMS80v/Q0dDI6U/8qXyJIfq5txflpPM3bfXr5U/XidlVF JmqwkhZP9wSRWtr2d5qgELOtvN9BN//oiHRWq1QBC5Ehb4k3n07Uupq9p7bgm/xtOFiX wGZbTzH3ST8DKYwe6w3ytOKcTGx8QItacTR3CzguvFnrRajb1Wd+yDDUl4u7O2KUIcsE XXh4ysQ4LDFwGMekosFIy2A5CRhkE1QfVLF26XQoIeTmcoYZ0vhJ2OLV8OQwwn8JyhvH YnDa+6M5z6sZhEnjmKrbQsTFiDrX0FypGowjFxPJa3GAaiiyIyO6AqpY5LGpzgaafcAg znVg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=finyzGAQ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r17-20020a05640251d100b0056869ae4414si1719101edd.55.2024.03.15.05.06.25; Fri, 15 Mar 2024 05:06:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=finyzGAQ; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 36E6668D1CD; Fri, 15 Mar 2024 14:05:02 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C571A68D171 for ; Fri, 15 Mar 2024 14:04:51 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504288; bh=gkDZKZ+Msg9vVA2YIzUC6pmquae0OqLBh/b3c8bm6Xc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=finyzGAQQyzksOa2UVJv6TX8WG7ZhxhBkwDrLNMHlZWaYgPq9OGTmEYXC0UY2sigr KpT9Kwgb+2iv8Z5F0gaN4IAh7sQEYuGlYhrl5TtHtFuXZZCTz9oZEGM7NYMRpSnYCb s2PaAeWHcFzgx3ObRNZQhgNZSlUGa9ZMm1Aw8Wj8= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id EB35842A98; Fri, 15 Mar 2024 13:04:47 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:59:01 +0100 Message-ID: <20240315120442.73754-9-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 8/9] avcodec/h2645_sei: decode AFGS1 T.35 SEI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IJZoRsviajsN From: Niklas Haas I restricted this SEI to HEVC for now, until I see a H.264 sample. --- libavcodec/Makefile | 2 +- libavcodec/h2645_sei.c | 25 +++++++++++++++++++++++++ libavcodec/h2645_sei.h | 3 +++ 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 708434ac76c..824845276ae 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -105,7 +105,7 @@ OBJS-$(CONFIG_H264_SEI) += h264_sei.o h2645_sei.o OBJS-$(CONFIG_HEVCPARSE) += hevc_parse.o hevc_ps.o hevc_data.o \ h2645data.o h2645_parse.o h2645_vui.o OBJS-$(CONFIG_HEVC_SEI) += hevc_sei.o h2645_sei.o \ - dynamic_hdr_vivid.o + dynamic_hdr_vivid.o aom_film_grain.o OBJS-$(CONFIG_HPELDSP) += hpeldsp.o OBJS-$(CONFIG_HUFFMAN) += huffman.o OBJS-$(CONFIG_HUFFYUVDSP) += huffyuvdsp.o diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c index e60606f43f9..2e79b9ea78b 100644 --- a/libavcodec/h2645_sei.c +++ b/libavcodec/h2645_sei.c @@ -209,6 +209,24 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } break; } + case 0x5890: { // aom_provider_code + const uint16_t aom_grain_provider_oriented_code = 0x0001; + uint16_t provider_oriented_code; + + if (!IS_HEVC(codec_id)) + goto unsupported_provider_code; + + if (bytestream2_get_bytes_left(gb) < 2) + return AVERROR_INVALIDDATA; + + provider_oriented_code = bytestream2_get_byteu(gb); + if (provider_oriented_code == aom_grain_provider_oriented_code) { + return ff_aom_parse_film_grain_sets(&h->aom_film_grain, + gb->buffer, + bytestream2_get_bytes_left(gb)); + } + break; + } unsupported_provider_code: #endif default: @@ -692,6 +710,12 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, avctx->properties |= FF_CODEC_PROPERTY_FILM_GRAIN; } +#if CONFIG_HEVC_SEI + ret = ff_aom_attach_film_grain_sets(&sei->aom_film_grain, frame); + if (ret < 0) + return ret; +#endif + if (sei->ambient_viewing_environment.present) { H2645SEIAmbientViewingEnvironment *env = &sei->ambient_viewing_environment; @@ -788,4 +812,5 @@ void ff_h2645_sei_reset(H2645SEI *s) s->ambient_viewing_environment.present = 0; s->mastering_display.present = 0; s->content_light.present = 0; + s->aom_film_grain.enable = 0; } diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h index 0ebf48011af..b9a6c7587b1 100644 --- a/libavcodec/h2645_sei.h +++ b/libavcodec/h2645_sei.h @@ -23,7 +23,9 @@ #include "libavutil/buffer.h" #include "libavutil/frame.h" +#include "libavutil/film_grain_params.h" +#include "aom_film_grain.h" #include "avcodec.h" #include "bytestream.h" #include "codec_id.h" @@ -132,6 +134,7 @@ typedef struct H2645SEI { H2645SEIAmbientViewingEnvironment ambient_viewing_environment; H2645SEIMasteringDisplay mastering_display; H2645SEIContentLight content_light; + AVFilmGrainAFGS1Params aom_film_grain; } H2645SEI; enum { From patchwork Fri Mar 15 11:59:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 47099 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8c91:b0:1a3:31a3:7958 with SMTP id k17csp194768pzh; Fri, 15 Mar 2024 05:06:44 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCU4DYIQ9Q6hIocfZnZVCQdlWkdoseY3/HqthTwZiP5rwXQQh/ueaiPM7/f/oWx+fOd6Kxk+4PiHWsjlnzulmCpTA/pLhawiPf8U+g== X-Google-Smtp-Source: AGHT+IHrZvNJuGAl1h/kbzl0fqAAT1GWLiq15vOGupZVCChTdAmHrI2CcP8677eRFNt7qlBCJfVc X-Received: by 2002:a17:906:d104:b0:a44:2ba0:8200 with SMTP id b4-20020a170906d10400b00a442ba08200mr3102527ejz.26.1710504403850; Fri, 15 Mar 2024 05:06:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710504403; cv=none; d=google.com; s=arc-20160816; b=R70V9VS67tjii0tgk13jAzydObGS0X9b1kxPANvazUVJEZIWUIV9/ChXMGnGMxWjAi IJo3fdiV04rD5G66QEdm3Z12j17GbpPaiU8/RPFJ4X7C5+fELqFCcgfQQBW2c3tndFFf rZT4Q1IdGHT2x5gQ7isULPhq1kNGMUzJcGpUVqvMxRn7gp2TUDLs6bRt/ZgX2BoImX+m ih1T1eCwdLrjzoies9HaaITo6xqgSXYG5fg5YM91ZD8NX+moMe0E+umKYXs/Q00Llu3w C4+r0+ZFgpcNiRo6j/AtJP1F047YOtrpx62CNXZM8lvhfWdxGRW78aHOyOioEe9nOZr7 1vMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=4HNbHGvUeEMkys0guomKXtyESAVfhxJ3mxxgiIpynzI=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=ti6y26pfu+wbLGSmeTwSZFj93+fn0nvAb6V0p2/7h6TRvDG4lGpWeCFfoKv670WwKu gvfqveQYkrQsBQePUHuCHRpnaQQNpcIdUujl3LzQZbKmiXUcewnSyyPz+m/F4tYkD27l lm68VQ76x/gEEmwhxl49qGefdA1eZiSS2SDiFHhu734L5AebyXfKmWRikHdszpFFLqnt R4OKB20CP1eKMZwW4YOHIsHivacvwnFKrnRzKR3wJjGyLDR6hLyz4Rgamacryl9OGqLw 9XiiM4TUQ/dcuDteiiJo8jCE4YdKhjZaBPW0BL+g778KYRvgkprzCIotfog0X3twD66Q hXxw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="g/maXeM9"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f23-20020a170906049700b00a4633aa3431si1673370eja.458.2024.03.15.05.06.34; Fri, 15 Mar 2024 05:06:43 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b="g/maXeM9"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3831D68D1D5; Fri, 15 Mar 2024 14:05:03 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 01B2268D191 for ; Fri, 15 Mar 2024 14:04:52 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1710504288; bh=Brdg+HXEf+ZdoVNYcTOZPkoyTxPUvpzR8Nmylznw8LI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g/maXeM9ciieY8OMTeY7N3CqfEfAzGVB30GLozyJZHHibpYhltav55gS2q+DM4LU0 gj6Gj8DNFPDQRtqiKTzLOuHWj3nXyHXsc5H2TMP1M4vsOoKRxf+vxCNTa/JE1uslVd eon/mhv4JOM143CcE90/NgP1SJBOfG+mcgwdAbD0= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 2CEF243065; Fri, 15 Mar 2024 13:04:48 +0100 (CET) From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Mar 2024 12:59:02 +0100 Message-ID: <20240315120442.73754-10-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240315120442.73754-1-ffmpeg@haasn.xyz> References: <20240315120442.73754-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 9/9] avcodec/hevcdec: apply AOM film grain synthesis X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: m20YHzOg8+B+ From: Niklas Haas Following the usual logic for H.274 film grain. --- libavcodec/Makefile | 2 +- libavcodec/hevcdec.c | 29 ++++++++++++++++++++++------- 2 files changed, 23 insertions(+), 8 deletions(-) diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 824845276ae..7ef2e03ca6a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -432,7 +432,7 @@ OBJS-$(CONFIG_HDR_ENCODER) += hdrenc.o OBJS-$(CONFIG_HEVC_DECODER) += hevcdec.o hevc_mvs.o \ hevc_cabac.o hevc_refs.o hevcpred.o \ hevcdsp.o hevc_filter.o hevc_data.o \ - h274.o + h274.o aom_film_grain.o OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index 76aa6b45882..575836e340c 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -35,6 +35,7 @@ #include "libavutil/pixdesc.h" #include "libavutil/timecode.h" +#include "aom_film_grain.h" #include "bswapdsp.h" #include "cabac_functions.h" #include "codec_internal.h" @@ -388,7 +389,8 @@ static int export_stream_params_from_sei(HEVCContext *s) avctx->color_trc = s->sei.common.alternative_transfer.preferred_transfer_characteristics; } - if (s->sei.common.film_grain_characteristics.present) + if (s->sei.common.film_grain_characteristics.present || + s->sei.common.aom_film_grain.enable) avctx->properties |= FF_CODEC_PROPERTY_FILM_GRAIN; return 0; @@ -2885,11 +2887,13 @@ static int hevc_frame_start(HEVCContext *s) else s->ref->frame->flags &= ~AV_FRAME_FLAG_KEY; - s->ref->needs_fg = s->sei.common.film_grain_characteristics.present && + s->ref->needs_fg = (s->sei.common.film_grain_characteristics.present || + s->sei.common.aom_film_grain.enable) && !(s->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) && !s->avctx->hwaccel; if (s->ref->needs_fg && + s->sei.common.film_grain_characteristics.present && !ff_h274_film_grain_params_supported(s->sei.common.film_grain_characteristics.model_id, s->ref->frame->format)) { av_log_once(s->avctx, AV_LOG_WARNING, AV_LOG_DEBUG, &s->film_grain_warning_shown, @@ -2934,14 +2938,24 @@ fail: static int hevc_frame_end(HEVCContext *s) { HEVCFrame *out = s->ref; - const AVFrameSideData *sd; + const AVFilmGrainParams *fgp; av_unused int ret; if (out->needs_fg) { - sd = av_frame_get_side_data(out->frame, AV_FRAME_DATA_FILM_GRAIN_PARAMS); - av_assert0(out->frame_grain->buf[0] && sd); - ret = ff_h274_apply_film_grain(out->frame_grain, out->frame, &s->h274db, - (AVFilmGrainParams *) sd->data); + av_assert0(out->frame_grain->buf[0]); + fgp = av_film_grain_params_select(out->frame); + switch (fgp->type) { + case AV_FILM_GRAIN_PARAMS_NONE: + av_assert0(0); + return AVERROR_BUG; + case AV_FILM_GRAIN_PARAMS_H274: + ret = ff_h274_apply_film_grain(out->frame_grain, out->frame, + &s->h274db, fgp); + break; + case AV_FILM_GRAIN_PARAMS_AV1: + ret = ff_aom_apply_film_grain(out->frame_grain, out->frame, fgp); + break; + } av_assert1(ret >= 0); } @@ -3596,6 +3610,7 @@ static int hevc_update_thread_context(AVCodecContext *dst, s->sei.common.alternative_transfer = s0->sei.common.alternative_transfer; s->sei.common.mastering_display = s0->sei.common.mastering_display; s->sei.common.content_light = s0->sei.common.content_light; + s->sei.common.aom_film_grain = s0->sei.common.aom_film_grain; ret = export_stream_params_from_sei(s); if (ret < 0)