Category Archives: Floating Point

This is second and final part of the story of how my career as a software developer unfolded (part 1 is here). In this half I work at four different companies in the Seattle area, make my mark, and then … Continue reading →

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading →

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading →

A few years ago I did a lot of thinking and writing about floating-point math. It was good fun, and I learned a lot in the process, but sometimes I go a long time without actually using that hard-earned knowledge. … Continue reading →

I write a lot about investigations into tricky bugs – CPU defects, kernel bugs, transient 4-GB memory allocations – but most bugs are not that esoteric. Sometimes tracking down a bug is as simple as paying attention to server dashboards, … Continue reading →

Years ago I worked in the Xbox 360 group at Microsoft. We were thinking about releasing a new console, and we thought it would be nice if that console could run the games of the previous console. Emulation is always … Continue reading →

I’ve written in the past about how to compare floating-point numbers for the common scenario where two results should be similar but may not be identical. In that scenario it is reasonable to use an AlmostEqual function for comparisons. But … Continue reading →

“What’s an EXCEPTION_FLT_STACK_CHECK exception?” one of my coworkers said. I said “It’s a weird and rare crash. Why do you ask?” It turns out that one of these weird and rare crashes had started showing up in Chrome (M54 branch, … Continue reading →

Intel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sine) has a maximum error, in round-to-nearest mode, of one unit in the last place. This is not true. It’s not even close. The worst-case … Continue reading →

"Please write a C++ function that takes a circle’s diameter as a float and returns the circumference as a float." It sounds like the sort of question you might get in the first week of a C++ programming class. And … Continue reading →