I'm trying to use python to investigate the effect of C++ truncating doubles to floats.
In C++ I have relativistic energies and momenta which are cast to floats, and I'm trying to work out whether at these energies saving them as doubles would actually result in any improved precision in the difference between energy and momentum.
I chose python because it seemed like a quick and easy way to look at this with some test energies, but now I realise that the difference between C++ 32 bit floats and 64 bit doubles isn't reflect in python.
I haven't yet found a simple way of taking a number and reducing the number of bits used to store it in python. Please can someone enlighten me?
I'd suggest using Numpy as well. It exposes various data types including C style floats and doubles.
Other useful tools are the C++17 style hex encoding and the decimal module for getting accurate decimal expansions.
For example:
import numpy as np
from decimal import Decimal
for ftype in (np.float32, np.float64):
v = np.exp(ftype(1))
pyf = float(v)
print(f"{v.dtype} {v:20} {pyf.hex()} {Decimal(pyf)}")
giving
float32 2.7182819843292236 0x1.5bf0aa0000000p+1 2.7182819843292236328125
float64 2.718281828459045 0x1.5bf0a8b145769p+1 2.718281828459045090795598298427648842334747314453125
Unfortunately the hex-encoding is a bit verbose for float32s (i.e. the zeros are redundant), and the decimal module doesn't know about Numpy floats, so you need to convert it to a Python-native float first. But given that binary32's can be directly converted to binary64's this doesn't seem too bad.
Just thought, that it sounds like you might want these written out to a file. If so, Numpy scalars (and ndarrays) support the buffer protocol, which means you can just write them out or use bytes(v) to get the underlying bytes.
Related
I am attempting to replicate a DSP algorithm in Python that was originally written in C. The trick is I also need to retain the same behavior of the 32 bit fixed point variables from the C version, including any numerical errors that the limited precision would introduce.
The current options I think are available are:
I know the python Decimal type can be used for fixed-point arithmetic, however from what I can tell there is no way to adjust the size of a Decimal variable. To my knowledge numpy does not support doing fixed point operations.
I did a quick experiment to see how fiddling with the Decimal precision affected things:
>>> a = dc.Decimal(1.1)
>>> a
Decimal('1.100000000000000088817841970012523233890533447265625')
>>> sys.getsizeof(a)
104
>>> dc.getcontext().prec = 16
>>> a = dc.Decimal(1.1)
>>> a
Decimal('1.1999999999999999555910790149937383830547332763671875')
>>> sys.getsizeof(a)
104
There is a change before/after the precision change, however there are still a large number of decimal places. The variable is still the same size, and has quite a few decimal places after it.
How can I best go about achieving the original objective? I do know that Python ctypes has the C language float, but I do not know if that will be useful in this case. I do not know if there is even a way to accurately mimic C type fixed point math in Python.
Thanks!
I recommend fxpmath module for fixed-point operations in python. Maybe with that you can emulate the fixed point arithmetic defining precision in bits (fractional part). It supports arrays and some arithmetic operations.
Repo at: https://github.com/francof2a/fxpmath
Here an example:
from fxpmath import Fxp
x = Fxp(1.1, True, 32, 16) # (val, signed, n_word, n_frac)
print(x)
print(x.precision)
results in:
1.0999908447265625
1.52587890625e-05
I have used one python code in PyCharm in Linux and the format of number was
-91.35357. When I used the same code in PyCharm in Windows format was
-91.35356999999999. The problem is that value is consisted in the file name which I need to open (and the list of files to open is long).
Anyone knows possible explanation and hot to fix it?
Floats
Always remember that float numbers have a limited precision. If you think about it, there must be a limit to how exactly you represent a number if you limit storage to 32 or 64 bits (or any other number).
in Python
Python provides just one float type. Float numbers are usually implemented using 64 bits, but yet they might be 64 bit in one Python binary, 32 bit on another, so you can't really rely on that (however, see #Mark Dickinson comment below).
Let's test this. But note that, because Python does not provide float32 and float64 alternatives, we will use a different library, numpy, to provide us with those types and operations:
>>> n = 1.23456789012345678901234567890
>>> n
1.2345678901234567
>>> numpy.float64(n)
1.2345678901234567
>>> numpy.float32(n)
1.2345679
Here we can see that Python, in my computer, handles the variable as a float64. This already truncates the number we introduced (because a float64 can only handle so much precision).
When we use a float32, precision is further reduced and, because of truncation, the closest number we can represent is slightly different.
Conclusion
Float resolution is limited. Furthermore, some operations behave differently across different architectures.
Even if you are using a consistent float size, not all numbers can be represented, and operations will accumulate truncation errors.
Comparing a float to another float shall be done considering a possible error margin. Do not use float_a == float_b, instead use abs(float_a - float_b) < error_margin.
Relying on float representations is always a bad idea. Python sometimes uses scientific notation:
>>> a = 0.0000000001
>>> str(a)
'1e-10'
You can get consistent rounding approximation (ie, to use in file names), but remember that storage and representation are different things. This other thread may assist you: Limiting floats to two decimal points
In general, I'd advise against using float numbers in file names or as any other kind of identifier.
Latitude / Longitude
float32 numbers have not enough precision to represent the 5th and 6th decimal numbers in latitude/longitude pairs (depending on whether the integer part has one, two or three digits).
If you want to learn what's really happening, check this page and test some of your numbers: https://www.h-schmidt.net/FloatConverter/IEEE754.html
Representing
Note that Python rounds float values when representing them:
>>> lat = 123.456789
>>> "{0:.6f}".format(lat)
'123.456789'
>>> "{0:.5f}".format(lat)
'123.45679'
And as stated above, latitude/longitude cannot be correctly represented by a float32 down to the 6th decimal, and furthermore, the truncated float values are rounded when presented by Python:
>>> lat = 123.456789
>>> lat
123.456789
>>> "{0:.5f}".format(numpy.float64(lat))
'123.45679'
>>> "{0:.5f}".format(numpy.float32(lat))
'123.45679'
>>> "{0:.6f}".format(numpy.float32(lat))
'123.456787'
As you can see, the rounded version of that float32 number fails to match the original number from the 5th decimal. But also does the rounded version to the 5th decimal of the float64 number.
Your PyCharm on Linux is simply rounding of your large floating point number. Rounding it off to the nearest 6 or 7 can resolve your issue but DONT USE THESE AS FILE NAMES.
Keeping your code constant in both cases then, their can be many explanations:
1) 32-bit Processors handles float differently than 64-Bit Processors.
2) PyCharm for both Linux and Windows behaves differently for floating points which we cannot determine exactly, may be PyCharm for Windows is better optimised.
edit 1
Explanation for Point 1
on 32-Bit processors everything is really done in 80-bit precision internally. The precision really just determines how many of those bits are stored in memory. This is part of the reason why different optimisation settings can change results slightly: They change the amount of rounding from 80-bit to 32- or 64-bit.
edit 2
You can use hashmapping for saving your data in files and then mapping them onto the co-ordinates.
Example:
# variable = {(long,lat):"<random_file_name>"}
cordinates_and_file ={(-92.45453534,-87.2123123):"AxdwaWAsdAwdz"}
I am trying to work with the np.longdouble dtype in my Python code, and am trying to use NumPy to manipulate long doubles that I get from a C module compiled with Cython.
Suppose I do this:
import numpy as np
print np.finfo(np.longdouble)
Machine parameters for float128
---------------------------------------------------------------------
precision= 18 resolution= 1e-18
machep= -63 eps= 1.08420217249e-19
negep = -64 epsneg= 5.42101086243e-20
minexp=-16382 tiny= 3.36210314311e-4932
maxexp= 16384 max= 1.18973149536e+4932
nexp = 15 min= -max
---------------------------------------------------------------------
a = np.longdouble(1e+346)
a
Out[4]: inf
b = np.longdouble(1e+347)
b
Out[6]: inf
c = a/b
/usr/lib/python2.7/site-packages/spyderlib/widgets/externalshell/start_ipython_kernel.py:1:
RuntimeWarning: invalid value encountered in longdouble_scalars
# -*- coding: utf-8 -*-
c
Out[8]: nan
a.dtype, b.dtype, c.dtype
Out[9]: (dtype('float128'), dtype('float128'), dtype('float128'))
In essence, it is linked to the same issue as in this question and I understand that Python first converts the 1e+346 into a float, whose represntation would be inf. However, can someone suggest a workaround? Is there a way to create NumPy longdoubles that are not converted to floats first?
I have a C module that can output long doubles, which I want to use in a numpy array of dtype np.longdouble.
Even if the solution involves re-compiling Python/NumPy, I am willing to try it.
There are a few things you might want to take into account.
First, this is a mess. NumPy knows longdouble and float128. Unfortunately, the names are misleading, the underlying implementation is C long double, which is usually (but not necessarily always) an 80-bit float. (Actually you can see it here by looking at "precision"; 18 digits is approximately 60 bits, and the 80-bit float has 64 bits in its mantissa. The precision would be around 34 digits if real 128-bit floats were used.)
There may not be any direct way to pass long doubles as arguments to a C function, but if you pass pointers instead, then you can avoid the problem. For example, you may pass your array data as uint8 (by using myarray.view(dtype='uint8')) and the cast the pointer to the buffer into long double * in your C program. At least then Python has nothing to do with type conversions. (Most probably you do not need to take the view because you are, after all, only exporting a pointer to the array buffer.)
Just beware that this trick relies on the compiler having the same kind of type settings when compiling Python and your C program. In addition to precision differences, there may be byte order differences (rarely if the programs run in the same machine) and alignment differences. My Python seems to align the longdouble items at 16 byte borders (i.e. there is always 6 bytes of zeros per each element), but the C compiler may use 10/12/16 byte alignment.
As far as I know, the details are implementation specific. So, this is doable but requires some extra care and there may be portability issues.
I am trying to do some arithmetic with gmpy2 in python. Unfortunately I don't
know what are the types of returned values of this arithmetics. For example:
x=float(gmpy2.comb(100,50))/gmpy2.comb(200,100)
print x
print isinstance(x, (int, long, float, complex))
gives me:
1.114224180581451e-30
False
I couldn't find any helpful information when I Googled a bit.
Is there a way that I can get an object type in python in general?
Otherwise, what is the exact type of this value? Is it an mpz?
And the last question, when I do arithmetic with mpz values and for
example float, will it always cast the type to mpz?
P.S. I don't know if mpz is a correct term I am using here! I would be also happy
if somebody with high reputation adds gmpy to tags in stackoverflow to make them
questions more accessible to the people who know gmpy.
I don't know about gmpy2 but you can find the type of an object in Python using x.__class__.
With new-style classes you can also use type(x).
gmpy2 introduces several new data types: mpz, mpq, mpfr, and mpc. They are analogous to Python's int/long, fractions.Fraction, float and complex types. So division involving an mpz will normally result in an mpfr.
In your example, you create an mpz, convert it to a float, and then divide it by an mpz. When performing the division, gmpy2 converts the float to an mpfr and then performs the division. If you want a float result, you should apply float() to the entire result and not just gmpy2.comb(100,50). Note the differences in the parenthesis.
>>> float(gmpy2.comb(100,50))/gmpy2.comb(200,100)
mpfr('1.114224180581451e-30')
>>> float(gmpy2.comb(100,50)/gmpy2.comb(200,100))
1.114224180581451e-30
Why the conversion from float to mpfr? The mpfr data type can support much higher precision and has a significantly wider range that a float. As long as the precision of mpfr is greater than or equal to the precision of a float (i.e. 53), then the conversion is lossless.
Disclaimer: I maintain gmpy and gmpy2.
I don't think this is possible, hence I decided to ask here to see as googling around hasn't returned any results that hint that I can do so.
Especially after reading this:
Can doubles be used to represent a 64 bit number without loss of precision
Though my numbers can be held in 32bit as the example below shows.
But is there any way in MATLAB to convert a double precision value to single without loosing information?
e.g. in MATLAB
> a = 103364148
a =
103364148
> single(a)
ans =
103364144
Or maybe there is another way in another language, e.g. Python?
I'm working with GPUMat where I can only use GPUSingle, so I'm trying to find a way to work with stuff that is double to MATLAB in single to the GPU.
Thanks,
A single can hold integer numbers up to 2^24 (16,777,216) without loss of precision - some bits are required for the sign and the exponent .
In other words, no, there is no way that you can make a number larger than 2^24 fit into a single without error (note that it can hold some larger numbers, as long as they can be written as the product of a number smaller 2^24 and some power of 2).
However, are you sure you need that kind of precision for your calculations? As long as all your integers are less than 2^24, you should be fine.
When you're doing these kinds of experiments, you should turn on
format long
so you can see more decimal values. For example,
>> pi
ans =
3.1416
>> format long
>> ans
ans =
3.141592653589793
If your only concern are integers, you could use int32 instead