From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 05F8D42EDA for ; Wed, 10 Aug 2022 21:36:24 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9993468B881; Thu, 11 Aug 2022 00:36:21 +0300 (EEST) Received: from btbn.de (btbn.de [136.243.74.85]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4CE4B68B0A6 for ; Thu, 11 Aug 2022 00:36:15 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id EFB102F46A7 for ; Wed, 10 Aug 2022 23:36:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1660167375; bh=AXxFj6XYi7Y9k6tYIZB9xLZ01nEKDXHBnbjCbGF/JoI=; h=Date:Subject:To:References:From:In-Reply-To; b=XVfXVsPERwLYrAf/fnf0SXYAKFzd1hPolG5hBml7Grw9NvkINqmvV/lNOU0VqW/b3 UN8JtBWUtKvbqiiAlw1OAHNvQSJ0AWrLJb1wFNDAKTAOTP4A29Nb0AifYzyMJvkRkr ZUrtZQwlEdMT8FjKbbTJ1C35oO/6/kXwvUvHI8XTgUwv+e4uER45JBmiqGxYlTMY5B pVy0N2kDmHss/yaVzTtGcygqAHaWbTW1x+A9rY2uvYjG+XK7J15ppCEgqXR0tUFDOJ GfECRzZsadqE9ZD5hdcuE43OUIIU6cmckjgAp0nnVQ+ISDnUUE9YFn/e0T8u+7nUAn 1TZMeQmkuIKCA== Message-ID: <423b2beb-9ad6-003e-be31-e3237f9f9788@rothenpieler.org> Date: Wed, 10 Aug 2022 23:36:14 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20220810204712.3123-1-timo@rothenpieler.org> <20220810204712.3123-6-timo@rothenpieler.org> From: Timo Rothenpieler In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH 06/11] avutil/half2float: adjust conversion of NaN X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 10.08.2022 23:24, Andreas Rheinhardt wrote: > Timo Rothenpieler: >> IEEE-754 differentiates two different kind of NaNs. >> Quiet and Signaling ones. They are differentiated by the MSB of the >> mantissa. >> >> For whatever reason, actual hardware conversion of half to single always >> sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0. >> So our code has to follow suite or fate-testing hardware float16 will be >> impossible. > > What does the exr spec say about quiet and signaling nans? Not sure how exr would be involved here. But I tested this on both aarch64, x86 with sse2 emulation and x86 f16c on alderlake and zen2. They all perfectly agree and match 100% what this changed code produces for the entire range of 65k possible values. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".