lua-users home
lua-l archive

Re: [ANN] Lua 5.2.0 (work1) now available

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


Rene Rebe <rene@exactcode.de> writes:
> Actually your bitop lib has very strange rounding characteristics due to the exotic number conversion:
>
>> return bit.bor(1.4, 0)
> 1
>> return bit.bor(1.5, 0)
> 2
>> return bit.bor(1000.5, 0)
> 1000
>> return bit.bor(1000.6, 0)
> 1001
Round to even is standard IEEE rounding behavior. Causes the smallest
mean error for cumulative rounding operations and bias of 0.
-- 
David Kastrup

AltStyle によって変換されたページ (->オリジナル) /