musl - musl - an implementation of the standard library for Linux-based systems

fix undefined behavior in strto* via FILE buffer pointer abuse

in order to produce FILE objects to pass to the intscan/floatscan backends without any (prohibitively costly) extra buffering layer, the strto* functions set the FILE's rend (read end) buffer pointer to an invalid value at the end of the address space, or SIZE_MAX/2 past the beginning of the string. this led to undefined behavior comparing and subtracting the end pointer with the buffer position pointer (rpos). the comparison issue is easily eliminated by using != instead of <. however the subtractions require nontrivial changes: previously, f->shcnt stored the count that would have been read if consuming the whole buffer, which required an end pointer for the buffer. the purpose for this was that it allowed reading it and adding rpos-rend at any time to get the actual count so far, and required no adjustment at the time of __shgetc (actual function call) since the call would only happen when reaching the end of the buffer. to get rid of the dependency on rend, instead offset shcnt by buf-rpos (start of buffer) at the time of last __shlim/__shgetc call. this makes for slightly more work in __shgetc the function, but for the inline macro it's still just as easy to compute the current count. since the scan helper interfaces used here are a big hack, comments are added to document their contracts and what's going on with their implementations.

Diffstat (limited to 'src/stdlib/strtol.c')

1 files changed, 1 insertions, 8 deletions