musl - musl - an implementation of the standard library for Linux-based systems

fix incorrect rounding in printf floating point corner cases

the printf floating point formatting code contains an optimization to avoid computing digits that will be thrown away by rounding at the specified (or default) precision. while it was correctly retaining all places up to the last decimal place to be printed, it was not retaining enough precision to see the next nonzero decimal place in all cases. this could cause incorrect rounding down in round-to-even (default) rounding mode, for example, when printing 0.5+DBL_EPSILON with "%.0f". in the fix, LDBL_MANT_DIG/3 is a lazy (non-sharp) upper bound on the number of zeros between any two nonzero decimal digits.

Diffstat

1 files changed, 2 insertions, 2 deletions