Re: Formatting numbers without precision loss
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: Re: Formatting numbers without precision loss
- From: Thijs Schreijer <thijs@...>
- Date: 2023年6月21日 11:00:34 +0200
>
> In short, the options so far;
>
> - use “%a”, but it is ugly and not always available
> - determine the number of decimals for each number (Pallene approach)
> - determine once the number of significant decimals for the current system, and use every time with a "%g” modifier
So here's what I ended up with;
local num2string do
-- will not handle NaN, infinity and the likes.
local format = string.format
if pcall(function() format("%a", 1.1) end) then
-- the "%a" specifier is available
num2string = function(value) return format("%a", value) end
else
-- the "%a" specifier is unavailable
-- dynamically test the available number of decimals for the %g specifier
local max_dec = 1
while true do -- testing showed 99 locally...
if not pcall(function() string.format("%."..max_dec.."g", 1.1) end) then
max_dec = max_dec - 1
break
end
max_dec = max_dec + 1
end
local MAX_FLOAT_FMT = "%."..tostring(max_dec).."g"
--print(MAX_FLOAT_FMT)
-- 17 digits should be enough to represent a double
-- See https://stackoverflow.com/a/21162120 and DBL_DECIMAL_DIG in float.h
-- But we don't know how the Lua engine was compiled, so let's test it dynamically
-- use a known value 1/9; 0.1111111111... to detect relevant number of decimals
local relevant_size = #(MAX_FLOAT_FMT):format(1/9):match("%.(1+)") + 1
local FLOAT_FMT = "%."..tostring(relevant_size).."g" -- local test resulted in 17
--print(FLOAT_FMT)
num2string = function(value) return format(FLOAT_FMT, value) end
end
end