I've spent countless hours researching, reading, testing, and ultimately confused and dismayed at Python's Decimal object's lack of the most fundamental concept: Formatting a Decimal's output to a string.
Let's assume we have some strings or Decimal objects with the following values:
0.0008
11.1111
222.2222
3333.3333
1234.5678
The goal is to simply set the Decimal's precision to the second decimal place. Eg, 11.1111 would be formatted as 11.11, and 1234.5678 as 1234.57.
I envision code similar to the following:
import decimal
decimals = [
decimal.Decimal('0.0008'),
decimal.Decimal('11.1111'),
decimal.Decimal('222.2222'),
decimal.Decimal('3333.3333'),
decimal.Decimal('1234.5678'),
]
for dec in decimals:
print dec.as_string(precision=2, rounding=ROUND_HALF_UP)
The resulting output would be:
0.00
11.11
222.22
3333.33
1234.57
Obviously we cannot make use of the Decimal's context's precision, because this takes into consideration the TOTAL number of digits, not just decimal precision.
I'm also not interested in converting the Decimal to a float to output its value. The ENTIRE reason behind Decimal is to avoid storing and running calculations on floats.
What other solutions are there? I understand there are many other similar questions on stack overflow, but none of them have I found to resolve the underlying issue I am inquiring of.
Thanks much!
Just use string formatting or the format() function:
>>> for dec in decimals:
... print format(dec, '7.2f')
...
0.00
11.11
222.22
3333.33
1234.57
decimal.Decimal supports the same format specifications as floats do, so you can use exponent, fixed point, general, number or percentage formatting as needed.
This is the official and pythonic method of formatting decimals; the Decimal class implements the .__format__() method to handle such formatting efficiently.
def d(_in, decimal_places = 3):
''' Convert number to Decimal and do rounding, for doing calculations
Examples:
46.18271 to 46.183 rounded up
46.18749 to 46.187 rounded down
117.34999999999999 to 117.350
_rescale is a private function, bad practice yet works for now.
'''
return Decimal(_in)._rescale(-decimal_places, 'ROUND_HALF_EVEN')
Edit: Again, _rescale() is not meant to be used by us regular bipeds, it works in Python 2.7, is not available in 3.4.
Related
I want to convert some floats to Decimal retaining 5 digits after decimal place regardless of how many digits before the decimal place. Is using string formatting the most efficient way to do this?
I see in the docs:
The significance of a new Decimal is determined solely by the number of digits input. Context precision and rounding only come into play during arithmetic operations.
So that means I need to add 0 to force it to use the specified prec but the prec is total digits not after decimal so it doesn't actually help.
The best thing I can come up with is
a=[1.132434, 22.2334,99.33999434]
[Decimal("%.5f" % round(x,5)) for x in a]
to get [Decimal('1.13243'), Decimal('22.23340'), Decimal('99.33999')]
Is there a better way? It feels like turning floats into strings just to convert them back to a number format isn't very good although I can't articulate why.
Do all the formatting on the way out from your code, inside the print and write statements. There is no reason I can think of to lose precision (and convert the numbers to some fixed format) while doing numeric calculations inside the code.
I don't understand why, by formatting a string containing a float value, the precision of this last one is not respected. Ie:
'%f' % 38.2551994324
returns:
'38.255199'
(4 signs lost!)
At the moment I solved specifying:
'%.10f' % 38.2551994324
which returns '38.2551994324' as expected… but should I really force manually how many decimal numbers I want? Is there a way to simply tell to python to keep all of them?! (what should I do for example if I don't know how many decimals my number has?)
but should I really force manually how many decimal numbers I want? Yes.
And even with specifying 10 decimal digits, you are still not printing all of them. Floating point numbers don't have that kind of precision anyway, they are mostly approximations of decimal numbers (they are really binary fractions added up). Try this:
>>> format(38.2551994324, '.32f')
'38.25519943239999776096738060005009'
there are many more decimals there that you didn't even specify.
When formatting a floating point number (be it with '%f' % number, '{:f}'.format(number) or format(number, 'f')), a default number of decimal places is displayed. This is no different from when using str() (or '%s' % number, '{}'.format(number) or format(number), which essentially use str() under the hood), only the number of decimals included by default differs; Python versions prior to 3.2 use 12 digits for the whole number when using str().
If you expect your rational number calculations to work with a specific, precise number of digits, then don't use floating point numbers. Use the decimal.Decimal type instead:
Decimal “is based on a floating-point model which was designed with people in mind, and necessarily has a paramount guiding principle – computers must provide an arithmetic that works in the same way as the arithmetic that people learn at school.” – excerpt from the decimal arithmetic specification.
Decimal numbers can be represented exactly. In contrast, numbers like 1.1 and 2.2 do not have exact representations in binary floating point. End users typically would not expect 1.1 + 2.2 to display as 3.3000000000000003 as it does with binary floating point.
I would use the modern str.format() method:
>>> '{}'.format(38.2551994324)
'38.2551994324'
The modulo method for string formatting is now deprecated as per PEP-3101
When checking if a floor is an int, the recommend method would be is_integer:
However, I get a weird behaviour with the results of the log function:
print(log(9,3)); #2.0
print((log(9,3)).is_integer()); #True
print((log(243,3))); #5.0
print((log(243,3)).is_integer()); #False
Furthermore:
print((int) (log(9,3))); #2
print((int) (log(243,3))); #4
Is this normal?
log(243,3) simply doesn't give you exactly 5:
>>> '%.60f' % log(243,3)
'4.999999999999999111821580299874767661094665527343750000000000'
As the docs say, log(x, base) is "calculated as log(x)/log(base)". And neither log(243) nor log(3) can be represented exactly, and you get rounding errors. Sometimes you're lucky, sometimes you're not. Don't count on it.
When you want to compare float numbers, use math.isclose().
When you want to convert a float number that is close to an integer, use round().
Float numbers are too subject to error for "conventional" methods to be used. Their precision (and the precision of functions like log) is too limited, unfortunately. What looks like a 5 may not be an exact 5.
And yes: it is normal. This is not a problem with Python, but with every language I'm aware of (they all use the same underlying representation). Python offers some ways to work around float problems: decimal and fractions. Both have their own drawbacks, but sometimes they help. For example, with fractions, you can represent 1/3 without loss of precision. Similarly, with decimal, you can represent 0.1 exactly. However, you'll still have problems with log, sqrt, irrational numbers, numbers that require many digits to be represented and so on.
I am depending on some code that uses the Decimal class because it needs precision to a certain number of decimal places. Some of the functions allow inputs to be floats because of the way that it interfaces with other parts of the codebase. To convert them to decimal objects, it uses things like
mydec = decimal.Decimal(str(x))
where x is the float taken as input. My question is, does anyone know what the standard is for the 'str' method as applied to floats?
For example, take the number 2.1234512. It is stored internally as 2.12345119999999999 because of how floats are represented.
>>> x = 2.12345119999999999
>>> x
2.1234511999999999
>>> str(x)
'2.1234512'
Ok, str(x) in this case is doing something like '%.6f' % x. This is a problem with the way my code converts to decimals. Take the following:
>>> d = decimal.Decimal('2.12345119999999999')
>>> ds = decimal.Decimal(str(2.12345119999999999))
>>> d - ds
Decimal('-1E-17')
So if I have the float, 2.12345119999999999, and I want to pass it to Decimal, converting it to a string using str() gets me the wrong answer. I need to know what are the rules for str(x) that determine what the formatting will be, because I need to determine whether this code needs to be re-written to avoid this error (note that it might be OK, because, for example, the code might round to the 10th decimal place once we have a decimal object)
There must be some set of rules in python's docs that hopefully someone here can point me to. Thanks!
In the Python source, look in "Include/floatobject.h". The precision for the string conversion is set a few lines from the top after an comment with some explanation of the choice:
/* The str() precision PyFloat_STR_PRECISION is chosen so that in most cases,
the rounding noise created by various operations is suppressed, while
giving plenty of precision for practical use. */
#define PyFloat_STR_PRECISION 12
You have the option of rebuilding, if you need something different. Any changes will change formatting of floats and complex numbers. See ./Objects/complexobject.c and ./Objects/floatobject.c. Also, you can compare the difference between how repr and str convert doubles in these two files.
There's a couple of issues worth discussing here, but the summary is: you cannot extract information that is not stored on your system already.
If you've taken a decimal number and stored it as a floating point, you'll have lost information, since most decimal (base 10) numbers with a finite number of digits cannot be stored using a finite number of digits in base 2 (binary).
As was mentioned, str(a_float) will really call a_float.__str__(). As the documentation states, the purpose of that method is to
return a string containing a nicely printable representation of an object
There's no particular definition for the float case. My opinion is that, for your purposes, you should consider __str__'s behavior to be undefined, since there's no official documentation on it - the current implementation can change anytime.
If you don't have the original strings, there's no way to extract the missing digits of the decimal representation from the float objects. All you can do is round predictably, using string formatting (which you mention):
Decimal( "{0:.5f}".format(a_float) )
You can also remove 0s on the right with resulting_string.rstrip("0").
Again, this method does not recover the information that has been lost.
>>> num = 4.123456
>>> round(num, 3) # expecting 4.123
4.1230000000000002
I'm expecting 4.123 as a result, Am I wrong?
This is not a mistake. You need to read What Every computer Scientist Should Know About Floating Point Arithmetic:
http://docs.sun.com/source/806-3568/ncg_goldberg.html
Yep, your expectations don't match the design intent of your tools.
Check out this section of the Python tutorial.
Using math.round is actually pretty rare. if you're trying to display a number as a string to a certain precision, you might want something more like
>>> num = 4.123456
>>> print "%.3f" % num
4.123
You might be interested in the documentation on string formatting.
Why do you care? (That's a serious question.)
The answer that you're getting is so close to 4.123 as to make no difference. It can't be exactly 4.123, since there are only finitely many numbers (around 2**64 on a typical machine) that Python can represent exactly, and without going into detail about floating-point representations, it just so happens that 4.123 isn't one of those numbers. By the way, 4.1230000000000002 isn't one of the numbers that can be exactly represented, either; the actual number stored is 4.12300000000000022026824808563105762004852294921875, but Python truncates the decimal representation to 17 significant digits for display purposes. So:
If you're doing mathematics with the result, then the difference between 4.123 and what you're getting is so tiny as to make no real difference. Just don't worry about it.
If you just care about the output looking pretty (i.e., what you're after here is a string rather than a number) then use str, or string formatting.
In the unlikely case that the difference really does matter, e.g., because you're doing financial work and this affects the direction that something rounds later on, use the decimal module.
Final note: In Python 3.x and Python 2.7, the repr of a float has changed so that you will actually get 4.123 as you expect here.
If you want to have an exact representation of your floating point number, you have to use decimal.