I am reading a big array of JSONS from a file with ijson.items , but numbers are converted to type Decimal.
Example Decimal('14.2')
The documentation says that there is an option use-float = true. But I don't know where to put this option.
Is it possible to use ijson.itemsand obtain floats?
ijson.items(file, 'your.prefix', use_float=True)
Related
I'm trying to use read_sql_query() to read a query from MySQL database, one of the field in the database, its type is double(24, 8), I want to use dtype= parameter to have full control of the datatypes and read it to decimal, but seems like pandas can't recognize decimal type so I had to read it to Float64
In the database, the values for this field look like this:
Value
100.96000000
77.17000000
1.00000000
0.12340000
Then I'm trying to read it from Python code:
from decimal import *
dtypes = {
'id': 'Int64',
'date': 'datetime64',
'value': 'Float64'
}
df = pd.read_sql_query(sql_query, mysql_engine, dtype=dtypes)
but after reading the data from the code above, it looks like this:
Value
100.96
77.17
1.0
0.1234
How can I read this column to decimal and keep all the digits? Thanks.
What "the data looks like in the database" is tricky. This is because the act of printing it out feeds the bits through a formatting algorithm. In this case it removes trailing zeros. To see what is "in the database", one needs to get a hex dump of the file and then decipher it; this is non-trivial.
I believe that DECIMAL numbers hold all the digits specified, packed 2 digits per byte. No, I don't know how they are packed (0..99 versus 2 hex digits; what to do if the number of digits is odd; where is the sign?)
I believe that FLOAT and DOUBLE exactly conform to IEEE-764 encoding format. No, I don't know how the bytes are stored (big-endian vs little-endian). I suspect Python's Float64 is IEEE DOUBLE.
For DECIMAL(10,6), I would expect to see "1.234" to be stored as +, 0001, and 234000, but never displayed with leading zeros and optionally displayed with trailing zeros -- depending on the output formatting package.
For DOUBLE, I would expect to find hex 3ff3be76c8b43958 after adjusting for endianism, and I would not be surprised to see the output be 1.23399999999999999e+0. (Yes, I actually got that, given a suitable formatting in PHP, which I am using.) I would hope to see 1.234 since that is presumably the intent of the number.
Do not use DOUBLE(m,n). The (m,n) leads to extra rounding and it is deprecated syntax. Float and Double are not intended for exact number of decimal places; use DECIMAL for such.
For FLOAT: 1.234 becomes hex 3f9df3b6 and displays something like 1.2339999675751 assuming the output method works in DOUBLE and is asked to show lots of decimal places.
Bottom line: The output method you are using is causing the problem.
i have a string like '0.00008400' and i just want to turn it 0.00008400 (float). But python changes it to scientific notation type and it is looking like 8.4e-05,
float('0.00008400') = 8.4e-05 #as float
I saw some formatting answers but they are turning it to string instead of float.
format(8.4e-05, '.8f') = 0.00008400 #as string
and of course I can't turn this string value into float again...
#stupid alert
float(format(8.4e-05, '.8f')) = 8.4e-05
I just want to my string type input to turn float exactly...
'0.00008400' => 0.00008400 #string to float
Thanks in advance...
Edit:
there is a function that I want to use gets float value but can't understand scientific notion format of float. So this is why I want to show this float normal.
SORRY:
string format is okay too. Sorry.
The scientific representation is "just" a representation of your float, that does not change the real value of the float, whatever how python display it.
So I don't understand what you want to do. Could you send more informations ?
I've managed to get 2dp format for a number, however, it returns a string. Is there any way to keep float and always have 2dp regardless if there are instances of .00?
average = "{:.2f}".format(10.0)
print(average)
Outputs
'10.00'
However, if I try to convert this to a float using the following
average = float("{:.2f}".format(10.0))
It outputs the following float:
10.0
I've seen other threads that say to use Decimal, but i'm uncertain of how to do this without importing library.
Is there anyway to achieve this without importing any libraries?
Keep it as a float the entire time, at the last step when you want to show it covert it to string with two decimal points. This is not the perfect answer but it will work.
I have the following number of type float:
1.019e-05
To convert it to a decimal number, i did this:
print("%.10f" % float(1.019e-05))
Which gives the following output: 0.0000101900
The problem is that 0.0000101900 is of type str, but i need to make some calculations with that number. How can i convert 0.0000101900 from string to a number again? if i use float()the decimal will be converted again to an exponential. Am i forced to use the exponential number here?
The number is a float and has the same value no matter how it looks - the scientific notation used is just a way of representing how the data looks in a more concise way. By doing your formatting string, you are formatting the number to look a certain way for output, but that creates a string representation of the number rather than "converting it to a decimal number". You can't change the way Python natively represents a float, but I'm not understanding why this matters to you because it doesn't change the actual value of the number, just how it looks when printed out
Is there any format specifier in Python format() function which can be used in order to get a specified number of digits from the decimal part without the dot?
For example I would like to get only 86 from the float 0.86.
EDIT:
The workaround I do right now is this:
"{:.0f}".format(num*100)
The answers so far suggest workarounds like this one or using string manipulations. I am aware of these solutions. What I am asking for, is if there is a way to do this only with a format specifier.
I'm not aware of any specific stuff in str.format that we could use to accomplish this, but this is how I would go about it.
You can cast the float to an integer which will chop off the decimal to the right of the float. Then just subtract them and you should get just the decimal part.
my_float = 12.12121200
print("Just the decimal: {0}".format(
int(str(my_float - int(my_float)).split(".")[1])
))
You could .replace?
such as
decimal = .85
int(str(decimal).replace('.', ''))
This is not a .format thought, Hope it helps anyways.