Why can less precise data like float be faster than larger, more precise data like double?

Question

I am currently reading a chapter in a textbook on Processor Architecture and saw the following statement:

The less precision there is, the less space is occupied by a program variable in memory. Further, there is often a time advantage, both in ferrying the operands back and forth between the processor and memory, and for arithmetic and logic operations that need less precision. This is particularly true for floating-point arithmetic operations.

Why are less precise data like float sometimes faster than larger, more precise data like double? Can somebody develop on this explanation and maybe give an example?

• amon: That quote doesn't say "floating point is slower than fixed point" which is true for other reasons, but it says "smaller, less precise data like float is noticeably faster than larger, more precise data like double or long double".
• forgetaboutme: I have edited my question accordingly.
• user949300: It must be noted that in many cases, depending on hardware and CPU, doubles are actually faster. So take this textbook with a big grain of salt.
• dan04: Some FPU architectures don't even have single-precision operations, but internally convert everything to double or extended-precision numbers.

Answer 1

For intuitively the same reason why it's faster to calculate 2 + 2 by hand than it is to calculate 3685 + 2193: there's simply less data to work your way through.

• Frank Hileman: Less data might be more precisely defined as a fewer number of bytes.
• dan04: Yep. And even on an architecture where individual single-precision operations are not faster than double-precision, using half as much memory can still be a great performance advantage (provided that you're OK with the reduced accuracy).

Answer 2

Single precision floating point format compared to double precision:

1. uses less memory, so can be transferred into register faster (in one machine instruction, usually)
2. has less accuracy, so some approximations can be used for faster calculations (on software level this means less machine instructions per call, on hardware level this means less CPU clocks per instruction)

The size of double word types (double, long), is also influenced higher level languages specifications, for example, Java does not guarantee access to variable of such type to be atomic (done in one step for external observer).

Answer 3

An FPU or GPU can (sometimes) parallelize more 32-bit (float) FP operations than 64-bit (double) FP operations. That is, if it can add 2 doubles in parallel, it can add 4 floats in parallel.

For highly-optimized tight loops this can have a dramatic effect, especially on GPU where the processing units are less constrained with memory bandwidth.

• 8bittree: GPUs tend to be worse than a 1/2 ratio. Only professional-oriented GPUs (Quadro and FirePro) get anywhere close, typically around 1/3 if they're marketed towards those needing FP64. Though, Nvidia's recently announced Quadro GP100 is a rare sample that does have a 1/2 ratio. Typical consumer GPUs these days tend to have ratios like 1/32 to 1/24.