Search code examples
Why do certain floating point calculations turn the way they do? (e.g. 123456789f +1 = 123456792)...

floating-pointprecisionfloating-accuracy

Read More
Why is the float value changing in my C program?...

carraysloopsfloating-accuracy

Read More
Errors in Directly vs Recursively Calculating a given Fibonacci Number...

pythonfibonaccifloating-accuracy

Read More
How to avoid roundoff errors in numpy.random.choice?...

pythonnumpyrandomfloating-pointfloating-accuracy

Read More
numpy.vstack losing precision float16...

pythonnumpyfloating-pointprecisionfloating-accuracy

Read More
C - ceil/float rounding to int guarantees...

cfloating-accuracy

Read More
Do FMA instructions of different CPUs have different intermediate accuracy? If yes, then how does a ...

c++gccfloating-pointprecisionfloating-accuracy

Read More
For loop in which the counter goes up to a value generated by Math.floor()...

javascriptfor-loopmathprecisionfloating-accuracy

Read More
Would float point format be affected by big-endian and little endian?...

c++cfloating-accuracy

Read More
For a given precision, what is the maximum value for which a float32 will give the same result as a ...

pythonnumpyfloating-accuracy

Read More
How can i translate these bytes to float?...

floating-pointhexasciifloating-accuracybytestream

Read More
define double constant as hexadecimal?...

c++floating-pointfloating-accuracyieee-754notation

Read More
0.1 + 0.7 != 0.8...

floating-pointfloating-accuracyboolean-expression

Read More
What is the standard solution in JavaScript for handling big numbers (BigNum)?...

javascriptfloating-accuracybignum

Read More
Getting the decimal part of a double in Swift...

swiftmathfloating-accuracy

Read More
remove percentage sign from scale::percent() in R...

rggplot2percentagefloating-accuracy

Read More
JavaScript seems to be doing floating point wrong (compared to C)...

javascriptcfloating-pointfloating-accuracy

Read More
Numerical Stability - does Multiply/Divide give a more precise value than Divide/Multiply?...

floating-pointprecisionfloating-accuracynumerical-stability

Read More
Why doesn't the same generated assembler code lead to the same output?...

cfloating-pointx86-64floating-accuracyicc

Read More
Unexpected integer division vs. floating-point division result in Python...

pythonfloating-accuracyfloor-division

Read More
Why does a subtype of AbstractArray result in imprecise matrix operations in Julia?...

arraysjuliafloating-accuracy

Read More
Why do I need 17 significant digits (and not 16) to represent a double?...

cfloating-pointprecisionieee-754floating-accuracy

Read More
Can a subtraction between two exactly represented floating point numbers with the same exponent be i...

floating-pointfloating-accuracy

Read More
Float precision when dealing with logarithms in musical pitch calculations...

pythonfloating-accuracylogarithmpitch

Read More
Is there any way to not lose the precision and still get the value?...

cfloating-pointdoublefloating-accuracy

Read More
Python replace integers in a Counter with their permillages of the total...

pythonpython-3.xfloating-pointprecisionfloating-accuracy

Read More
python floating number...

pythonfloating-pointnumericalfloating-accuracy

Read More
Calculating the angle between two vectors using a needle-like triangle...

pythonnumpygeometrynumericfloating-accuracy

Read More
Convert double number to digits and vice versa?...

c#mathnumbersdoublefloating-accuracy

Read More
Convert a continuous range of doubles to a discrete int range...

dartmappingroundingprecisionfloating-accuracy

Read More
BackNext