154 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
2
votes
0
answers
226
views
Why does floating point division take less than 50% of the latency of integer division and also 10x more latency than usual when underflow occurs?
I am measuring the latency of instructions.
For 64-bit primitives, integer division takes about 25 cycles each, usually on my 2.3GHz Digital Ocean vCPU, while floating point division takes about 10 ...
1
vote
1
answer
100
views
Unsigned array subtraction with scalar bias to avoid underflow
I want to subtract two vectors, represented by arrays of unsigned shorts, into a resulting vector also represented by unsigned shorts, while avoiding underflow. The subtraction is performed ...
1
vote
3
answers
173
views
With IEEE 754 floating point numbers, is underflow still better than overflow? [closed]
In his classic works from the 1970s, Richard Hamming advises setting up floating point calculations to be more likely to underflow than overflow.
For example, if you want to calculate 1 - 1/(exp(x)+1),...
0
votes
3
answers
88
views
How to create an underflow in this situation?
So I've got this situation:
a=b*39216/100000
Both a and b are uint8 variables
Is there a value I could assign to b so that a is>100 afterwards?
And then a has to be >100
I've thought about ...
0
votes
2
answers
269
views
Dealing with overflow and underflow in integer and floating-point multiplication
I'm dealing with a problem where I have a huge collection of 16-bit signed integers. I must multiply each by a constant factor of type double and promote convert it back to the integer type. I need to ...
-1
votes
1
answer
108
views
Java Integer underflow / overflow not happen during math operation?
Could someone explain me, why a is equals 2'147'483'647 ?
int a = (int)(3434 * 2353253.0)
Why is this "sticking to Integer.MAX_VALUE" happening?
I expected some integer overflow behavior ...
-1
votes
1
answer
83
views
Compare overflow(underflow) number
I am trying to understand the following code:
#include <stdio.h>
int main()
{
int a=2147483647,b=1;
printf("%d\n",a+b);
printf("%d\n",a+b>a);
return 0;
}
...
1
vote
2
answers
178
views
Casting inside a ternary operator causes underflow
I have the following snippet of code
#include <iostream>
int main() {
int64_t res;
int some_val = 5;
if(false)
{
res = static_cast<uint32_t>(some_val);
}
...
0
votes
0
answers
138
views
Does x86 support "before rounding" tininess detection?
From experience I know that x86 has "after rounding" tininess detection (as recommended by Annex U of 754r).
However, does x86 support "before rounding" tininess detection as well? ...
1
vote
1
answer
74
views
Why does expanding this C assignment into 2 lines produce a different result?
This code produces the (incorrect) result of 255
#include <stdio.h>
int main()
{
unsigned char x = 0;
x = (x-1)%16;
printf("%d", x);
return 0;
}
While this code ...
0
votes
1
answer
565
views
Difference between machine precision and underflow
I don't get the difference between machine precision and underflow. Take for example the single precision system: there the machine precision is 10^-7 while the underflow is 1.18 *10^-38.
That means ...
0
votes
0
answers
190
views
How do I make the value of integer underflow to zero instead of it wrapping around?
I want to make sure that the user inputs unsigned integers. Is it possible to implement a simple solution so that when the user inputs negative values, it will always return zero instead of it ...
3
votes
2
answers
166
views
numbers near underflow limit display as `Inf.e-324` in tibble?
Per the docs, "on a typical R platform the smallest positive double is about 5e-324." Given a double vector with values above, near, and below this limit:
library(tibble)
small_vec <- c(...
0
votes
1
answer
211
views
How do I get the results from argmax of [0., 1e-8]?
When I ran below code, I got x=1, y=0
a = np.array([0., 1e-8]).astype('float32')
x = a.argmax()
y = (a+1).argmax()
I already know floating point expression. But, I don't know why I can get x=1, y=0.
...
0
votes
1
answer
321
views
IEEE 754: Underflow: is inexact flag required to be raised?
IEEE 754-2008, 7.5 Underflow:
The underflow exception shall be signaled ...
... the underflow flag shall be raised and the inexact (see 7.6) exception shall be signaled.
Here we see that:
the ...