From patchwork Wed Aug 10 20:47:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 37229 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp1142868pzi; Wed, 10 Aug 2022 13:49:06 -0700 (PDT) X-Google-Smtp-Source: AA6agR7k4PEOOySPo6k0EFX5IvytqTiLdMGWeurg54kqYBdKT4OH+/OlmUSmMzfN/SBjmJKxX6bJ X-Received: by 2002:a17:907:94c7:b0:730:d5bc:14c with SMTP id dn7-20020a17090794c700b00730d5bc014cmr22015938ejc.68.1660164545869; Wed, 10 Aug 2022 13:49:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660164545; cv=none; d=google.com; s=arc-20160816; b=Y5wiBEag9c2/PMTMvrvLur4kXq89mTuCiqHn0BKkx4V6kAW6MhoXVQD+MnY3O+MCHp 8mITVgOAk3E25R3cCBdc/jzEiUeGF5mkuNeRriUYdpN762xPDiaPFj6IG5Ef/sifrxSm 74pClWGize/8jcgAO2imsk/EA+uhnt6MjQDtLC4q4MYWSnzC337kX0VC0EJqntXkJc47 U+TSTBDBWhpKnOXMAoLInknBngyuh9tldBvLnKU+cw+KcOoClt3EaqUBu8sXog8zKLwg 0FbBcCm5vUDtFsYgalPStRyLk1R9Dy+2J63SZF58REf35gzgIJsN8DMZC5YOSY13zqrf YarQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=tTi/43BreJYp/zvBD7SDdVv4Gvwy3AH8gmSa/2rSAr4=; b=y+rhN7el1ikfMX5xnP1Bv7sx89k10ycXpLPa4mSdxG58Rwg5fBPesO4/pFGY5K8jNG Wufj6bAMqZcfvNNFXmRSpFi27ieXWunzxpXZ1yb8TyQ+yBoKo9KzO0rs3e1mJexXDHZR zVg7EczwX57MxbHP/G2MYVH6f7x8uDaS7nQ7l20BhPp7D8patElNcDCqL0KWOlCbD0VE c36GtJetf9qQ3iNIukVCS57NK49f0lbpsbv8+4PqRJQq5Ukx2U22xxiVD5UfZ16zYqT3 QW1vpAG1fSCLZUxI2LtW8h4aU9SLkG0g3lF/kERjyv5zQTvGR6o0z59JbCcw33zxYki+ g9xA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=azJMuu5n; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hr10-20020a1709073f8a00b0073069573e2esi5257762ejc.667.2022.08.10.13.49.05; Wed, 10 Aug 2022 13:49:05 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=azJMuu5n; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 239F368B93E; Wed, 10 Aug 2022 23:47:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 238B268B89B for ; Wed, 10 Aug 2022 23:47:31 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 0E3182F260C; Wed, 10 Aug 2022 22:47:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1660164445; bh=H2csBUjFdQ2Aipfsh/Ovz+x8Wry9D0rysCl51yVY7JY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=azJMuu5n+i/JHawR/wGptM9rWi4Eq0znBGWWPVxDPGD4OD71SPqzrgmBAvVqKfpjQ J1+5lDXBbb4xsFZWgj+aQQY17rBxpf5ZtCM960bChQ12xD3dE3gAxhFr4Vr4mLu00R wZhwHt33ZKahh2vuifPpCr+HxbHsyNZ6cp19/v0KeYTt8oXExg0JUWTP7xrvzB5tHE c7vWyG9r2gj1bSYLGrjfxwLCtxJ0C2O0+EHu+gsbxbZfed0aZObvWl+ZXl0FSVqjqw KPFyS4qdN91Tcac0G3+CDp8jwSzEl/wQJUAUq7XLTuCHV37KWPDwiEXl3EkfeVWt/G c2f4yyOqqjLUw== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Aug 2022 22:47:07 +0200 Message-Id: <20220810204712.3123-6-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220810204712.3123-1-timo@rothenpieler.org> References: <20220810204712.3123-1-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/11] avutil/half2float: adjust conversion of NaN X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rJ9fBeAzk25e IEEE-754 differentiates two different kind of NaNs. Quiet and Signaling ones. They are differentiated by the MSB of the mantissa. For whatever reason, actual hardware conversion of half to single always sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0. So our code has to follow suite or fate-testing hardware float16 will be impossible. --- libavcodec/exr.c | 2 +- libavcodec/pnm.h | 2 +- libavutil/half2float.h | 5 +++++ tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-) diff --git a/libavcodec/exr.c b/libavcodec/exr.c index 5c6ca9adbf..47f4786491 100644 --- a/libavcodec/exr.c +++ b/libavcodec/exr.c @@ -191,7 +191,7 @@ typedef struct EXRContext { float gamma; union av_intfloat32 gamma_table[65536]; - uint32_t mantissatable[2048]; + uint32_t mantissatable[3072]; uint32_t exponenttable[64]; uint16_t offsettable[64]; } EXRContext; diff --git a/libavcodec/pnm.h b/libavcodec/pnm.h index 5bf2eaa4d9..7e5445f529 100644 --- a/libavcodec/pnm.h +++ b/libavcodec/pnm.h @@ -34,7 +34,7 @@ typedef struct PNMContext { int half; float scale; - uint32_t mantissatable[2048]; + uint32_t mantissatable[3072]; uint32_t exponenttable[64]; uint16_t offsettable[64]; } PNMContext; diff --git a/libavutil/half2float.h b/libavutil/half2float.h index 1f6deade07..5af4690cfe 100644 --- a/libavutil/half2float.h +++ b/libavutil/half2float.h @@ -45,6 +45,9 @@ static void half2float_table(uint32_t *mantissatable, uint32_t *exponenttable, mantissatable[i] = convertmantissa(i); for (int i = 1024; i < 2048; i++) mantissatable[i] = 0x38000000UL + ((i - 1024) << 13UL); + for (int i = 2048; i < 3072; i++) + mantissatable[i] = mantissatable[i - 1024] | 0x400000UL; + mantissatable[2048] = mantissatable[1024]; exponenttable[0] = 0; for (int i = 1; i < 31; i++) @@ -58,7 +61,9 @@ static void half2float_table(uint32_t *mantissatable, uint32_t *exponenttable, offsettable[0] = 0; for (int i = 1; i < 64; i++) offsettable[i] = 1024; + offsettable[31] = 2048; offsettable[32] = 0; + offsettable[63] = 2048; } static uint32_t half2float(uint16_t h, const uint32_t *mantissatable, const uint32_t *exponenttable, diff --git a/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF b/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF index b6201116fe..e45a40b498 100644 --- a/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF +++ b/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 256x256 #sar 0: 1/1 -0, 0, 0, 1, 786432, 0x1445e411 +0, 0, 0, 1, 786432, 0xce9be2be