From patchwork Sun Aug 14 16:48:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timo Rothenpieler X-Patchwork-Id: 37270 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3d0d:b0:8d:a68e:8a0e with SMTP id y13csp879913pzi; Sun, 14 Aug 2022 09:49:10 -0700 (PDT) X-Google-Smtp-Source: AA6agR44v63xz0G4ATYVpNWF+tyLuuK6FHQkvl0gxIeRXXohYiaVEVlVTV7wHVEgz22NAQw5i3kW X-Received: by 2002:a05:6402:424e:b0:43d:9d9f:38f9 with SMTP id g14-20020a056402424e00b0043d9d9f38f9mr11209768edb.411.1660495749910; Sun, 14 Aug 2022 09:49:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660495749; cv=none; d=google.com; s=arc-20160816; b=iixPQChHHITcz3z2LDeCxXtQSNamMlOEkVTjUnbOjXe3rbDHfLp8VxY2oLUpWq1OsT 4OAMiftbGZ1ksThQLzU+G8GHbszrNjoXw/p6/VVg9hpScXZ9DrIuct2T0zuR5vUuQRA4 cyKai9+p22Rv68uQS1a5DlvaYEdzX/KpdGYpmNUoX+gsqaOL6ajA1W5PvwHHWVmoTvBC dX3hESXKPawmtGCAye8CqxK1yxAlx2PAUOViwGPqVfJxSlC6magCSVBsKqmDHBrIoq9B 1nJrfp0Aa2FrNeZnBSBk8/twuPVOEcmbUPUNA9R1N17yiRxMiPjR1ru/3nHEmJXxR34z r3dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=tTi/43BreJYp/zvBD7SDdVv4Gvwy3AH8gmSa/2rSAr4=; b=T32xibOC+btXlY9U7L32aZpFDCHUIzzj8LsB5B8Sa3J97G6ShPDku5pEKNAlk2txtW /kFi5DQ3ZXerv5+rX5T+RQRidvTQobxoTV6eJ51Zi1RzDWs6ta6C8G97HB2t/WMiZL6z LllRjmW2j3KT2E54ctkXsYXyFz3BdOeFAfQ/cAYEeHzTD7A0F3yaYVNmowxhnyBrookh Pvbm3LGu4dFTwH4sVEFWjBXg3XoxdNA68H1g4dWhvzLExn5L9WDUOk7/JcuQKWmgFLm1 96aWOGNjIfe+sQJbg8KjlMxjRwesBawW1MbYevjbapkfCcjHV2lsrgd37hamJzYXJd8Q 8L3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=KyI2RjbD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f21-20020a170906495500b0072b0d25e317si4588763ejt.230.2022.08.14.09.49.09; Sun, 14 Aug 2022 09:49:09 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rothenpieler.org header.s=mail header.b=KyI2RjbD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=rothenpieler.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2B5B068B8E5; Sun, 14 Aug 2022 19:48:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AFBCD68B7AD for ; Sun, 14 Aug 2022 19:48:46 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 3CE192AEB84; Sun, 14 Aug 2022 18:48:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1660495726; bh=H2csBUjFdQ2Aipfsh/Ovz+x8Wry9D0rysCl51yVY7JY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=KyI2RjbDedP2m/12LACOvM1QzDWxfQcTmhUUWjUnTdlbaxzwC+oO5RquIgmgi2szB 74IqruxdW8TWLUIvz6TS+dcIQYMKB0fD1Sx0I3BREUcaI5yMad86c+HrXQXsZ4yLz9 VjJ11wejvIMYsn7nX70scpivN0kudAiReRAshHP9FkYwKbZZEDNvlLRObjI01aWHb+ YLGPEQULsjW2LIL/fZZJuthbXmD44t+w/FmQPh4+26IUm3ncODLpjEyASTgsDuBIFV Gcr5RzYthy4IiN1/Wce0ADPJmSkL38dAn8Ml53CI4NiRN5PqTXjszVWEZwxwwQDleY E5cClXJ0WkfWQ== From: Timo Rothenpieler To: ffmpeg-devel@ffmpeg.org Date: Sun, 14 Aug 2022 18:48:28 +0200 Message-Id: <20220814164833.19965-2-timo@rothenpieler.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220814164833.19965-1-timo@rothenpieler.org> References: <20220814164833.19965-1-timo@rothenpieler.org> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/7] avutil/half2float: adjust conversion of NaN X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Timo Rothenpieler Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: rb0iSnn0AXGS IEEE-754 differentiates two different kind of NaNs. Quiet and Signaling ones. They are differentiated by the MSB of the mantissa. For whatever reason, actual hardware conversion of half to single always sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0. So our code has to follow suite or fate-testing hardware float16 will be impossible. --- libavcodec/exr.c | 2 +- libavcodec/pnm.h | 2 +- libavutil/half2float.h | 5 +++++ tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-) diff --git a/libavcodec/exr.c b/libavcodec/exr.c index 5c6ca9adbf..47f4786491 100644 --- a/libavcodec/exr.c +++ b/libavcodec/exr.c @@ -191,7 +191,7 @@ typedef struct EXRContext { float gamma; union av_intfloat32 gamma_table[65536]; - uint32_t mantissatable[2048]; + uint32_t mantissatable[3072]; uint32_t exponenttable[64]; uint16_t offsettable[64]; } EXRContext; diff --git a/libavcodec/pnm.h b/libavcodec/pnm.h index 5bf2eaa4d9..7e5445f529 100644 --- a/libavcodec/pnm.h +++ b/libavcodec/pnm.h @@ -34,7 +34,7 @@ typedef struct PNMContext { int half; float scale; - uint32_t mantissatable[2048]; + uint32_t mantissatable[3072]; uint32_t exponenttable[64]; uint16_t offsettable[64]; } PNMContext; diff --git a/libavutil/half2float.h b/libavutil/half2float.h index 1f6deade07..5af4690cfe 100644 --- a/libavutil/half2float.h +++ b/libavutil/half2float.h @@ -45,6 +45,9 @@ static void half2float_table(uint32_t *mantissatable, uint32_t *exponenttable, mantissatable[i] = convertmantissa(i); for (int i = 1024; i < 2048; i++) mantissatable[i] = 0x38000000UL + ((i - 1024) << 13UL); + for (int i = 2048; i < 3072; i++) + mantissatable[i] = mantissatable[i - 1024] | 0x400000UL; + mantissatable[2048] = mantissatable[1024]; exponenttable[0] = 0; for (int i = 1; i < 31; i++) @@ -58,7 +61,9 @@ static void half2float_table(uint32_t *mantissatable, uint32_t *exponenttable, offsettable[0] = 0; for (int i = 1; i < 64; i++) offsettable[i] = 1024; + offsettable[31] = 2048; offsettable[32] = 0; + offsettable[63] = 2048; } static uint32_t half2float(uint16_t h, const uint32_t *mantissatable, const uint32_t *exponenttable, diff --git a/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF b/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF index b6201116fe..e45a40b498 100644 --- a/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF +++ b/tests/ref/fate/exr-rgb-scanline-zip-half-0x0-0xFFFF @@ -3,4 +3,4 @@ #codec_id 0: rawvideo #dimensions 0: 256x256 #sar 0: 1/1 -0, 0, 0, 1, 786432, 0x1445e411 +0, 0, 0, 1, 786432, 0xce9be2be