Unfortunately the printing instruction of a code was written without an end-of-the-line character and one every 26 numbers consists of two numbers joined together. The following is a code that shows an example of such behaviour; at the end there is a fragment of the original database.
import numpy as np
for _ in range(2):
A=np.random.rand()+np.random.randint(0,100)
B=np.random.rand()+np.random.randint(0,100)
C=np.random.rand()+np.random.randint(0,100)
D=np.random.rand()+np.random.randint(0,100)
with open('file.txt','a') as f:
f.write(f'{A},{B},{C},{D}')
And thus the output example file looks very similar to what follows:
40.63358599010553,53.86722741700399,21.800795158561158,13.95828176311762557.217562728494684,2.626308403991772,4.840593988487278,32.401778122213486
With the issue being that there are two numbers 'printed together', in the example they were as follows:
13.95828176311762557.217562728494684
So you cannot know if they should be
13.958281763117625, 57.217562728494684
or
13.9582817631176255, 7.217562728494684
Please understand that in this case they are only two options, but the problem that I want to address considers 'unbounded numbers' which are type Python's "float" (where 'unbounded' means in a range we don't know e.g. in the range +- 1E4)
Can the original numbers be reconstructed based on "some" python internal behavior I'm missing?
Actual data with periodicity 27 (i.e. the 26th number consists of 2 joined together):
0.9221878978925224, 0.9331311610066017,0.8600582424784715,0.8754578588852764,0.8738648974725404, 0.8897837559800233,0.6773502027673041,0.736325377603136,0.7956454122424133, 0.8083168444596229,0.7089031184165164, 0.7475306242508357,0.9702361286847581, 0.9900689384633811,0.7453878225174624, 0.7749000030576826,0.7743879170108678, 0.8032590543649807,0.002434,0.003673,0.004194,0.327903,11.357262,13.782266,20.14374,31.828905,33.9260060.9215201173775437, 0.9349343132442707,0.8605282244327555,0.8741626682026793,0.8742163597524663, 0.8874673376386358,0.7109322043854609,0.7376362393985332,0.796158275345
To expand my comment into an actual answer:
We do have some information - An IEEE-754 standard float only has 32 bits of precision, some of which is taken up by the mantissa (not all numbers can be represented by a float). For datasets like yours, they're brushing up against the edge of that precision.
We can make that work for us - we just need to test whether the number can, in fact, be represented by a float, at each possible split point. We can abuse strings for this, by testing num_str == str(float(num_str)) (i.e. a string remains the same after being converted to a float and back to a string)
If your number is able to be represented exactly by the IEEE float standard, then the before and after will be equal
If the number cannot be represented exactly by the IEEE float standard, it will be coerced into the nearest number that the float can represent. Obviously, if we then convert this back to a string, will not be identical to the original.
Here's a snippet, for example, that you can play around with
def parse_number(s: str) -> List[float]:
if s.count('.') == 2:
first_decimal = s.index('.')
second_decimal = s[first_decimal + 1:].index('.') + first_decimal + 1
split_idx = second_decimal - 1
for i in range(second_decimal - 1, first_decimal + 1, -1):
a, b = s[:split_idx], s[split_idx:]
if str(float(a)) == a and str(float(b)) == b:
return [float(a), float(b)]
# default to returning as large an a as possible
return [float(s[:second_decimal - 1]), float(s[second_decimal - 1:])]
else:
return [float(s)]
parse_number('33.9260060.9215201173775437')
# [33.926006, 0.9215201173775437]
# this is the only possible combination that actually works for this particular input
Obviously this isn't foolproof, and for some numbers there may not be enough information to differentiate the first number from the second. Additionally, for this to work, the tool that generated your data needs to have worked with IEEE standards-compliant floats (which does appear to be the case in this example, but may not be if the results were generated using a class like Decimal (python) or BigDecimal (java) or something else).
Some inputs might also have multiple possibilities. In the above snippet I've biased it to take the longest possible [first number], but you could modify it to go in the opposite order and instead take the shortest possible [first number].
Yes, you have one available weapon: you're using the default precision to display the numbers. In the example you cite, there are 15 digits after the decimal point, making it easy to reconstruct the original numbers.
Let's take a simple case, where you have only 3 digits after the decimal point. It's trivial to separate
13.95857.217
The formatting requires a maximum of 2 digits before the decimal point, and three after.
Any case that has five digits between the points, is trivial to split.
13.958 57.217
However, you run into the "trailing zero" problem in some cases. If you see, instead
13.9557.217
This could be either
13.950 57.217
or
13.955 07.217
Your data do not contain enough information to differentiate the two cases.
if i have a number that is too big to be represented with 64 bits so i receive a string that contains it.
what happens if i use:
num = int(num_str)
i am asking because it looks like it works accurately and i dont understand how, does is allocate more memory for that?
i was required to check if a huge number is a power of 2. someone suggested:
def power(self, A):
A = int(A)
if A == 1:
return 0
x =bin(A)
if x.count('1')>1:
return 0
else:
return 1
while i understand why under regular circumstances it would work, the fact that the numbers are much larger than 2^64 and it still works baffles me.
According to the Python manual's description on the representation of integers:
These represent numbers in an unlimited range, subject to available (virtual) memory only. For the purpose of shift and mask operations, a binary representation is assumed, and negative numbers are represented in a variant of 2’s complement which gives the illusion of an infinite string of sign bits extending to the left.
I am performing calculation in python that results in very large numbers. The smallest of them is 2^10^6, this number is extremely long so I attempted to use format() to convert it to scientific notation. I keep getting an error message stating that the number is too large to convert to a float.
this is the error I keep getting:
print(format(2**10**6, "E"))
OverflowError: int too large to convert to float
I would like to print the result of 2^10^6 in a way that is concise and readable
You calculated 2 raised to the 10th then raised to the 6th power. If your aim is "2 times 10 to the sixth", then 2*10**6 is what you want. In python that can also be expressed by 2E6 where E means "to the 10th power". This is confusing when you are thinking in terms of natural logs and Euler's Number e.
You can also use the decimal.Decimal package if you want to side step decimal to binary float problems. In python, floats expressed in decimal are rounded to the nearest binary float. If you really did want the huge number, Decimal can handle it.
>>> Decimal("2E6")
Decimal('2E+6')
>>> Decimal("2")*10**6
Decimal('2000000')
>>> Decimal("2")**10**6
Decimal('9.900656229295898250697923616E+301029')
For printing, use the "g" format
>>> d = Decimal('2')**10**6
>>> format(d,'g')
'9.900656229295898250697923616e+301029'
>>> format(d,'.6g')
'9.90066e+301029'
>>> "{:g}".format(d)
'9.900656229295898250697923616e+301029'
>>> "{:.6g}".format(d)
'9.90066e+301029'
Why is python inserting strange negative numbers into my array?
I am reading some numerical data from a plain text file like below:
fp=np.genfromtxt("mytextfile.txt", dtype=('int32'), delimiter='\r\n')
The information contained in the file are all positive numbers, file is formatted like below and there are 300000 of these numbers:
12345
45678
1056789
232323
6789010001
1023242556
When I print out the fp read in array, the first half of the array is correctly read in but the last half is strange negative numbers that aren't in my file at all.
How can I get it to read correctly what is in the file?
You told it the numbers are int32s, but at least some of your numbers, e.g. 6789010001 are larger than a signed 32 bit quantity can represent (6789010001 is larger than an unsigned 32 bit quantity can represent).
If all the numbers are positive, I'd suggested using uint64 as your data type (you should check that all numbers in the file are in fact less than 2**64 though).
You have data larger then int32 can save, this causes overflow and makes it negative.
Numpy's Integers doesn't act like python's integer it's like a C integer
Try change dtype from int32 to int64 or object may help
I am getting a large value as a string as follows
s='1234567'
d='12345678912'
I want to do arithmetic as (100/d)*s
To do this, I need to convert the strings to appropriate large values. What would be the way to represent them as a number?
Just convert them using float. Python takes care of creating appropriately large representation. You can read more about Numerals here.
s='1234567'
d='12345678912'
(100/float(d))*float(s)
You could convert them using int, but as #GamesBrainiac pointed, that will work only in Python3; in Python2 it will most of the time give you 0 as result.
(100/int(d))*int(s)
If s and d are large e.g., thousands of digits then you could use fractions module to find the fraction:
from fractions import Fraction
s = int('1234567')
d = int('12345678912')
result = Fraction(100, d) * s
print(result)
# -> 30864175/3086419728
float has finite precision; It won't work for very large/small numbers.