I have this code in python:
from numpy import *
import itertools
m, n = 6, 10
set_m = [i + 1 for i in range(m + 1)]
comb = zeros(((m + 1) ** n, n), dtype=int)
k = 0
for i in itertools.product(set_m, repeat=n):
comb[k][:] = i
k += 1
But when I run it, I got this error:
Traceback (most recent call last):
File "main.py", line 33, in <module>
comb = zeros(((m + 1) ** n, n), dtype=int)
ValueError: array is too big.
If you are sure you MUST have a billion element array, and there is no way around it whatsoever (sometimes happens, but not every day), you can use memmap to create the array in the hard drive instead of the RAM memory. But I think it would be wise to search for ways to re-write your code to avoid doing such a slow thing.
Related
I'm trying to build out a project where I use exponential smoothing to predict the prices of commodities. I'm starting with basic exponential smoothing and am going to work my way up to triple exponential smoothing. However, when I use this code
with open('HistoricalData_1617379638571.csv', newline='') as f:
reader = csv.reader(f)
data = list(reader)
df = pd.read_csv('HistoricalData_1617379638571.csv')
df.columns = ['Date','Close/Last','Volume','Open','High','Low']
df.set_index('Date', inplace=True)
print(df)
x=df.loc[:,'Close/Last'].values
y=list(x)
z=[int(i) for i in y]
print(z)
alpha=0.9
def exp_smooth(alpha,z):
for i in z:
new_res=z.append((alpha * z[i] + (1 - alpha * z[i-1])))
print(new_res)
exp_smooth(alpha,z)
I keep getting the error message
Traceback (most recent call last):
File "main.py", line 38, in <module>
exp_smooth(alpha,z)
File "main.py", line 34, in exp_smooth
new_res=z.append((alpha * z[i] + (1 - alpha * z[i-1])))
IndexError: list index out of range
What am I doing wrong? How do I fix this so that my exponential smoothing function returns something other than an error?
When you try to index the list with values of the list, you may get a value which is not a valid index of the list. For example, if the list were z=[100000], the for loop would try to index z at index 1000000, which clearly does not exist.
You're probably looking to use range(len()):
def exp_smooth(alpha, z):
for i in range(len(z)):
new_res=z.append((alpha * z[i] + (1 - alpha * z[i-1])))
or, even better, enumerate():
def exp_smooth(alpha, z):
for index, value in enumerate(z):
new_res = z.append((alpha * value + (1 - alpha * z[index-1])))
First of all: I am by no means a Python expert, so this question is probably rather easy. Secondly, I worked over more than 2 hours on this and now I've decided I needed help. I want to implement a distance in Python. The distance is given as follows:
Where ui and uj are vectors that are given. d is the dimension of that vector. For example: if ui = (1,2,3), then ui_0= 1.
Now, this is what I've come up with so far: (here, xi = ui and xj = uj)
def dist(xi, xj, k):
distances = np.zeros(len(xi))
min1 = (0, 0)
min2 = (0, 0)
for dim in [0, len(xi)]:
for s in [-k, k]:
min1 = abs(xi[dim] - xj[dim + s])
min1[-k, k].min()
min2 = min(abs(xj[dim] - xi[dim + s]))
min2[-k, k].min()
distances = max(min1,min2)
but it doesn't work. Does anybody know where I've gone wrong?
Traceback:
Traceback (most recent call last): File "<input>", line 1, in
<module> File "<input>", line 8, in dist IndexError: invalid index
to scalar variable.
EDIT:
Ok, I tried to look at the case where k is maximal and I've taken the comments of Riley and Wouda into account. I came up with this piece of code:
def dist1(xi, xj):
for dim in range(len(xi)):
for s in range(-dim, len(xi) - dim):
return max(min(abs(xi[dim] - xj[dim + s])), min(abs(xj[dim] - xi[dim + s])))
and I still get the error:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 4, in dist1
TypeError: 'numpy.int64' object is not iterable
What is going on?
You appear to forget d is a parameter also (since you compute Dist_d^{ij}, where d is clearly a passed parameter). Together with k, I think the following represents the snippet of math you posted.
def dist(xi, xj, d, k):
min1 = min(abs(xi[d] - xj[d + s]) for s in range(-k, k + 1))
min2 = min(abs(xj[d] - xi[d + s]) for s in range(-k, k + 1))
return max(min1, min2)
Notice that Python will index negative values from the end of the list, which may or may not be what you want (in math, generally, not). So getting the arguments to this function right is a bit tricky, and you should build-in some checks to make sure all cases are handled correctly.
I am working on Google's "Doomsday Fuel" problem in Python 2.7 (it needs to be done in Python 2.7, hence the from __future__ import division line) that uses NumPy, which admittedly I am not too familiar with.
The WIP code (with a lot of comments added for your convenience):
from __future__ import division
from fractions import Fraction
import numpy as np
from numpy import linalg as LA
def gcd(m,n):
'''
function for finding the greatest common divisor of m and n
used mostly for the LCM function
'''
if m < n:
return gcd(n,m)
return gcd(n,m%n)
def lcm(m,n):
'''
function for finding the least common multiple of m and n
using the fact that m*n = gcd(m,n)*lcm(m,n)
'''
return (m*n)/(gcd(m,n))
def answer(m):
'''
m is an square matrix of nonnegative integers
dimensions guaranteed to be at most 10x10
'''
tbd = [] #stands for To Be Deleted
l = len(m)
for i in range(l):
'''
Checks each row
If row i is empty, add i to tbd, then make m[i][i] = 1
Otherwise, divide row i by the sum of row i to "normalize" it
e.g. [[3,2],[0,0]] would become [[0.6,0.4],[0,1]]
'''
s = sum(m[i])
if s == 0:
tbd.append(i)
m[i][i] = 1
else:
for j in range(l):
m[i][j] /= s
tbd.sort(reverse=True)
a = np.array(m)
r = np.diag([1 for x in range(l)]) #set initial matrix r which is just the identity matrix with same dimensions as a
for i in range(100):
r *= a #with each row adding up to just 1, r should stay stable
initial = [0 for x in range(l)]
initial[0] = 1
final = initial * r
for i in tbd:
del final[i]
dens = [] #denominators
for i in range(len(final)):
final[i] = final[i].limit_denominator()
dens.append(final[i].denominator) #collect all denominators
lc = dens[0]
for j in range(1,len(dens)):
lc = lcm(lc,dens[j]) #find LCM of all the denominators
for i in range(len(final)):
final[i] = int(final[i] * lc) #multiply the final array (which uses Fractions) by the LCM, then convert elements to int
final.append(lc)
return final
def main():
print answer([[1,2],[2,1]])
print answer([[0,1,0,0,0,1],[4,0,0,3,2,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0],[0,0,0,0,0,0]])
main()
Code in Ideone: http://ideone.com/DO1otS
The error message:
Traceback (most recent call last):
File "prog.py", line 51, in <module>
File "prog.py", line 48, in main
File "prog.py", line 29, in answer
TypeError: Cannot cast ufunc multiply output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
Why can't the program multiply int and float together? Or is there another part in this error message that I'm missing?
I have a simple for loop to calculate RMS(root mean square) which is defined in sigma summation form:
for i in range(int(N-(n*periyot/delta)), N+1):
sum = np.sqrt((1 / N) * (sum((Cl[i]**2))))
Then I got this error:
TypeError: 'numpy.float64' object is not iterable
Here are some information about my definitons:
N=40000, n=10.0, periyot=6.451290, delta=0.005
Cl=[-21.91969 -12.452671 -7.928303 ..., -0.0833991 -0.0579686
-0.0823822]
Remove that sum, each element of Cl is a float so you can't possibly call sum on them:
>>> sum(2.4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'float' object is not iterable
If you intend to invoke numpy's broadcasting to perform the power operation then you don't need to index the array.
The problem is that you overwrite sum function with sum variable. Try something like this:
my_sum = 0
for i in range(int(N-(n*periyot/delta)), N+1):
my_sum += np.sqrt((1 / N) * (sum((Cl[i]**2))))
Replicating your calculation, a bit simplified:
In [1]: Cl = np.array([-21.91969 , -12.452671 , -7.928303 , -0.0833991,-0.0579686,-0.0823822])
To calculate a sum in a loop, initial a value, and add to it at each iteration:
In [2]: res = 0
In [3]: for i in range(len(Cl)):
...: res += np.sqrt((1/3)*Cl[i]**2)
...:
In [4]: res
Out[4]: 24.551481812296061
Letting numpy calculate everything (slightly different)
In [5]: np.sqrt((1/3)*Cl**2).sum()
Out[5]: 24.551481812296064
Your range is a little more complicated, but I think that can be accommodated with:
s, e = int(N-(n*periyot/delta)), N+1 # start, end of range
for i in range(s, e): ....
or
np.sqrt((1/N) * Cl[s:e]**2).sum()
But I wonder why you started with that sum((Cl[i]**2))). Where you hoping to square a range of Cl values and then sum them? And repeat that for multiple ranges?
=============
There's a np.sum and a Python sum. Python sum works nicely with a list of numbers, such as those generated by a list comprehension:
In [6]: [np.sqrt((1/3)*Cl[i]**2) for i in range(len(Cl))]
Out[6]:
[12.655338922053147,
7.1895529539798462,
4.5774078712669173,
0.048150492835172518,
0.03346818681454574,
0.047563385346433583]
In [7]: sum([np.sqrt((1/3)*Cl[i]**2) for i in range(len(Cl))])
Out[7]: 24.551481812296061
The errors that result from trying to apply sum to a single value:
In [9]: sum(Cl[0])
....
TypeError: 'numpy.float64' object is not iterable
In [10]: sum(12.234)
...
TypeError: 'float' object is not iterable
In [11]: sum(Cl[:3]) # sum of several items
Out[11]: -42.300663999999998
==========
RMS = ( (1 / N ) * (Cl[1]^2 + Cl[2]^2 + Cl[3]^2 + ... Cl[N]^2) ) ^0.5
is expressed, for lists as:
rms = (1/n) * math.sqrt(sum([Cl[1]**2, Cl[2]**2, ....]))
rms = (1/n) * math.sqrt(sum([Cl[i]**2 for i in range(len(Cl))]))
rms = (1/n) * math.sqrt(sum([c**2 for c in Cl])) # iterate on Cl directly
rms = (1/n) * np.sqrt(np.sum(Cl**2)) # for array Cl
I am trying to write some code for an assignment in python. What I am not finding anywhere is what is wrong and why it will not run. It is sense and move robotic localization function. I do not understand why this line will not work.
q.append(p[i] * (hit * sensor_right + (1 - hit) * (1-sensor_right)))
hit = a comparison between two strings. That evaluates to true or false which is 1 or 0, right?
sensor_right = 0.7
Traceback (most recent call last):
File "vm_main.py", line 26, in <module> import main
File "/tmp/sbdxfjuois/main.py", line 50, in <module> p = sense(p, measurements[k])
File "/tmp/sbdxfjuois/main.py", line 34, in sense q.append(p[i] * (hit * sensor_right + (1 - hit) * (1-sensor_right)))
TypeError: can't multiply sequence by non-int of type 'float'
Can you suggest anything for what I have here posted?
def sense(p, Z):
q = [ ]
for i in range(len(p)):
hit = (Z == colors[i])
q.append(p[i] * (hit * sensor_right + (1 - hit) * (1-sensor_right)))
s = sum(q)
for i in range(len(q)):
q[i] = q[i]/s
return q
As others have pointed out, this p variable is apparently a sequence of sequences. You can verify this by putting
print(type(p))
print(type(p[i]))
before the append statement. You'll probably see something like
tuple
tuple
If that's what you expected, then you'll need to loop over the other index of the array. Also, does your q need to be returned with the same shape? I suspect you want something more like this.
def sense(p, Z):
q = p[:]
for i in range(len(p)):
for j in range(len(p[i])):
hit = (Z == colors[i])
q[i][j] = (p[i][j] * (hit * sensor_right + (1 - hit) * (1-sensor_right)))
s = sum(q)
for i in range(len(q)):
q[i] = q[i]/s
return q
Note that you also might want to look into numpy arrays.
If you're correct about the other variables, it is telling you that p[i] is a sequence (most likely a list), which can't be multiplied by a float. Perhaps p[i] is not what you're expecting it to be?
Try printing p[i] before the line that throws an error.
The problem here is what others have said. You can only multiply a sequence by an int, not a float.
For example
>>> [1] * 3
[1, 1, 1]
>>> "f" * 6
ffffff
>>> [1] * 0.7
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't multiply sequence by non-int of type 'float'
Double check your data type for p, to make sure it is supposed to be a sequence. If that is truly what p should be, then make sure to convert the following to an int before multiplying
(hit * sensor_right + (1 - hit) * (1-sensor_right))
Related to what #Mike said, you could also do:
q = []
sum = 0
for i in p:
sub_q = []
for val in i:
computed_val = val * (hit * sensor_right + (1 - hit) * (1-sensor_right))
sum += computed_val
sub_q.append(computed_val)
q.append(sub_q)
I like that because it is more concise and doesn't have to build a bunch of ranges every time you call it, and it also reduces the number of times you iterate over the data, but to each their own.