I warn in advance: I may be utterly confused at the moment. I tell a short story about what I actually try to achieve because that may clear things up. Say I have f(a,b,c,d,e), and I want to find arg max (d,e) f(a,b,c,d,e). Consider a (trivial example of a ) discretized grid F of f:
F = np.tile(np.arange(0,10,0.1)[newaxis,newaxis,:,newaxis,newaxis], [10, 10, 1, 10, 10])
maxE = F.max(axis=-1)
argmaxD = maxE.argmax(axis=-1)
maxD = F.max(axis=-2)
argmaxE = maxD.argmax(axis=-1)
This is the case how I typically solve the discretized version. But now assume instead, that I want to solve arg max d f(a,b,c,d,e=X): Instead of optimally chosen e for every other input, e is a fixed and given (of size AxBxCxD, which in this example would be 10x10x100x10). I have troubles solving this.
My naive approach was
X = np.tile(np.arange(0,10)[newaxis,newaxis,:,newaxis], [10,10,1,10])
maxX = F[X]
argmaxD = maxX.argmax(axis=-1)
However, the huge surge of memory that crashes my IDE implies that F[X] is apparently not what I was looking for.
Performance is key.
I believe you can do it like this, but maybe there's a better way..
n = 10
F = np.tile(np.arange(0,n,0.1)[None,None,:,None,None], [n, n, 1, n, n])
X = np.tile(np.arange(0,n)[None,None,:,None], [n, n, 1, n])
a,b,c,d = np.ogrid[:n,:n,:n,:n]
argmaxD = F[a,b,c,d,X].argmax(axis=-1)
Above X doesn't occupy the whole space, as we discussed in the comments. If you would like to choose e for all a,b,c and d you could do e.g.:
X = np.tile(np.arange(0,n,0.1).astype(int)[None,None,:,None], [n, n, 1, n])
a,b,c,d = np.ogrid[:n,:n,:100,:n]
argmaxD = F[a,b,c,d,X].argmax(axis=-1)
Also, notice that instead of tile you could make use of broadcasting. But then F[a,b,c,d,X] has a singular dimension so you should provide something like axis=3:
X = np.arange(0,n,0.1).astype(int)[None,None,:,None]
a,b,c,d = np.ogrid[:n,:n,:100,:n]
argmaxD = F[a,b,c,d,X].argmax(axis=3)
This would be my idea to solve this.
from itertools import product, starmap
f = lambda a,b,c,d,e : d / e
args_iterable = product([1],[2],[3],range(1,1000),range(1,1000))
max_val, max_args = max(starmap(lambda *args: (f(*args), args) , args_iterable))
print max_args
Related
Everybody. I wrote the following code. Please help me, to optimize this, when I submit in some test cases compiler writing time-limit-exceeded 2.069s / 13.33Mb.
import math
N = int(input())
arr = [None]*N; new_list = []
stepen = 0; res = .0;
arr = input().split(" ")
arr = [float(h) for h in arr]
Q = int(input())
for j in range(Q):
x, y = input().split()
new_list.extend([int(x), int(y)])
for i, j in zip(new_list[0::2], new_list[1::2]):
stepen = (j - i)+ 1
res = math.prod(arr[i:j+1])
print(pow(res, 1./stepen))
The slowest thing in your algorithm is the math.prod(arr[i:j+1]). If all the x and y inputs denote the entire range, you will surely TLE, as the calls to prod must loop over the entire range.
In order to avoid this, you must do a prefix product on your array. The idea is this: Keep a second array pref, with the property that pref[i] = arr[i] * pref[i-1]. As a result, pref[i] will be the product of everything at the ith position and before in arr.
Then to find the product between positions i and j, you want pref[j] / pref[i-1]. See if you can figure out why this gives the correct answer.
I have a strong background in Matlab, and I am trying to switch to python. I am trying to write a nested for loop with numpy array and storing output values.
My code reads like:
import numpy as np
import math
# T parameter
kk = np.arange(0, 20, 0.1)
print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for T in kk:
print(T)
for V in pp:
print(V)
P = math.exp(-T*V/10)
print(P)
Explanation/Question
kk, pp are the vectors. In for loop(s) correct values of T and V parameters are being called. However, values of P are not being stored.
I tried the following change P[T][V] = math.exp(-T*V/10), I get the following error: IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices
Any help will be appreciated. Thank you in advance.
In this code you define P as a 2d array. But the loop you assign the scalar result of the math.exp expression to that variable. That replaces the original P value, and also replaces the value calculated in the previous loop. This kind of loop doesn't work in MATLAB does it? Don't you have to assign the scalar value to some 'slot' in P?
P = np.zeros((a,b))
for T in kk:
print(T)
for V in pp:
print(V)
P = math.exp(-T*V/10)
A better way:
In [301]: kk = np.arange(0,20,0.1)
In [302]: kk.shape
Out[302]: (200,)
In [303]: pp = np.arange(1, 5,1)
In [304]: pp.shape
Out[304]: (4,)
In numpy we prefer to use fast whole-array methods. Here I use broadcasting to perform an outer like calculation of kk with pp.
In [305]: P = np.exp(-kk[:,None]*pp/10)
In [306]: P.shape
Out[306]: (200, 4)
(I believe MATLAB added broadcasting in recent years; numpy has had it from the beginning.)
Comparing this with the iterative version:
In [309]: P1 = np.zeros((200,4))
...: for i in range(0,len(kk)):
...: for j in range(0,len(pp)):
...: T = kk[i]
...: V = pp[j]
...: P1[i,j] = math.exp(-T*V/10)
...:
In [310]: P1.shape
Out[310]: (200, 4)
In [311]: np.allclose(P,P1)
Out[311]: True
A cleaner way of writing indexed iteration in Python is with enumerate:
In [312]: P1 = np.zeros((200,4))
...: for i,T in enumerate(kk):
...: for j,V in enumerate(pp):
...: P1[i,j] = math.exp(-T*V/10)
Based on the line where you mentioned trying P[T][V] = math.exp(-T*V/10), you might also be interested in this option:
import numpy as np
import math
# T parameter
kk = np.arange(0, 20, 0.1)
print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for i in range(0,len(kk)):
for j in range(0,len(pp)):
T = kk[i]
V = pp[j]
P[i][j] = math.exp(-T*V/10)
# you can also simply do this:
#P[i][j] = math.exp(-kk[i]*pp[j]/10)
Although it's straightforward, it's not particularly clean. Since you mentioned that you're switching to python, I'd take a look at hpaulj's answer for a more thorough explanation and as well as a nice alternative to iterating through arrays.
You can make a dictionary if you want to see the keys and values per your comment. This might make more sense actually. I would recommend against a plethora of dynamically created variables, as with a dictionary, you can call the entire dictionary OR specific values, which you could store as variables later anyway. Obviously, it depends on the scope of your project and what solution makes sense, but you could also turn the dictionary into a pandas dataframe with pd.DataFrame() for analysis, so it gives you flexibility. You said you are new to python, so you might want to check out pandas if you haven't heard of it, but you probably have as it is one of or the most popular library.
import numpy as np
import math
P_dict = {}
# T parameter
kk = np.arange(0, 20, 0.1)
# print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
# print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for T in kk:
# print(T)
for V in pp:
# print(V)
P = math.exp(-T*V/10)
key = f'{T},{V}'
value = P
P_dict[key] = value
print(P_dict)
This is how you would call a value in the dict based on the key.
P_dict['19.900000000000002,3']
You can also edit this line of code to whatever format you want: key = f'{T},{V}' and call the key acording to the format as I have done in my example.
Output:
0.002554241418992996
Either way, a list or a dict prints some interesting python abstract art!
I wrote a code that works great for solving equations numerically, but there is I a specific equation that when I get in there and try running the code - it will run and no output will ever come out!
Equation I got an output for: x^3−3*x+2−a*(np.sin(x))
Equation I didn't get an output for: (x-1)(x-2)(x-3)-a*(np.cos(x))
I also tried writing the second equation without brackets, like this: x^3-6*x^2+11*x-6-a*np.cos(x)
and it didn't help. where is the problem?!
this is my code:
import math
import numpy as np
h =1e-5
eps =1e-8
#function of the equation
def nf(x,a,c):
c=math.cos(x)
solu=(x-1)*(x-2)*(x-3)-a*c
return(solu)
#numerical method
def sl(a,x):
c=math.cos(x)
f = nf(x,a,c)
while abs(f)>eps:
x = x - h*f/(nf(x+h,a,c)-f)
f = nf(x,a,c)
return(x)
N = 101
mya = np.linspace(0.0,1.0,N)
myb = np.zeros(mya.shape)
myc = np.zeros(mya.shape)
myd = np.zeros(mya.shape)
for i in range(0,N):
myb[i] = sl(mya[i],1.0)
myc[i] = sl(mya[i],2.0)
myd[i] = sl(mya[i],3.0)
print(myb[i])
print(myc[i])
print(myd[i])
The problem is that for some input to sl, abs(f)>eps might not ever become False, creating an infinite loop. I havn't investigated the mathematical problem of yours, so I can't solve this problem "for real". What I can provide is automatic detection of when this happens, so that the code returns without a result rather than looping forever.
def sl(a,x):
c=math.cos(x)
f = nf(x,a,c)
count, maxcount = 0, 1000
while abs(f)>eps:
x = x - h*f/(nf(x+h,a,c)-f)
f = nf(x,a,c)
count += 1
if count > maxcount:
return
return(x)
Here, a maximum of 1000 iterations is allowed before a solution is deemed unreachable. In such a case, sl returns None, which when inserted into your NumPy float arrays becomes np.nan.
Upon investigating the output, only myc[60] fails in this way.
Your nf function was a little weird. You were passing c = math.cos(x) into nf() but whithin nf() you tried to assign c to math.cos(x) again. Just use the value c what you passed. Commenting it out fixes your code. As for the mathematical correctness, I cannot determine that unless you provide a better explanation what youre trying to do.
import math
import numpy as np
h =1e-5
eps =1e-8
#function of the equation
def nf(x,a,c):
# this line is not needed. Commenting allows your code to run
# c=math.cos(x)
solu=(x-1)*(x-2)*(x-3)-a*c
return(solu)
#numerical method
def sl(a,x):
c = math.cos(x)
f = nf(x,a,c)
while abs(f)>eps:
x = x - h*f/(nf(x+h,a,c)-f)
f = nf(x,a,c)
return(x)
N = 101
mya = np.linspace(0.0,1.0,N)
myb = np.zeros(mya.shape)
myc = np.zeros(mya.shape)
myd = np.zeros(mya.shape)
for i in range(0,N):
myb[i] = sl(mya[i],1.0)
myc[i] = sl(mya[i],2.0)
myd[i] = sl(mya[i],3.0)
print(myb[i])
print(myc[i])
print(myd[i])
Output:
3.2036907284
0.835006605064
0.677633820877
I'm having some troubles trying to use four lists with the zip function.
In particular, I'm getting the following error at line 36:
TypeError: zip argument #3 must support iteration
I've already read that it happens with not iterable objects, but I'm using it on two lists! And if I try use the zip only on the first 2 lists it works perfectly: I have problems only with the last two.
Someone has ideas on how to solve that? Many thanks!
import numpy
#setting initial values
R = 330
C = 0.1
f_T = 1/(2*numpy.pi*R*C)
w_T = 2*numpy.pi*f_T
n = 10
T = 1
w = (2*numpy.pi)/T
t = numpy.linspace(-2, 2, 100)
#making the lists c_k, w_k, a_k, phi_k
c_karray = []
w_karray = []
A_karray = []
phi_karray = []
#populating the lists
for k in range(1, n, 2):
c_k = 2/(k*numpy.pi)
w_k = k*w
A_k = 1/(numpy.sqrt(1+(w_k)**2))
phi_k = numpy.arctan(-w_k)
c_karray.append(c_k)
w_karray.append(w_k)
A_karray.append(A_k)
phi_karray.append(phi_k)
#making the function w(t)
w = []
#doing the sum for each t and populate w(t)
for i in t:
w_i = ([(A_k*c_k*numpy.sin(w_k*i+phi_k)) for c_k, w_k, A_k, phi_k in zip(c_karray, w_karray, A_k, phi_k)])
w.append(sum(w_i)
Probably you mistyped the last 2 elements in zip. They should be A_karray and phi_karray, because phi_k and A_k are single values.
My result for w is:
[-0.11741034896740517,
-0.099189027720991918,
-0.073206290274556718,
...
-0.089754003567358978,
-0.10828235682188027,
-0.1174103489674052]
HTH,
Germán.
I believe you want zip(c_karray, w_karray, A_karray, phi_karray). Additionally, you should produce this once, not each iteration of the for the loop.
Furthermore, you are not really making use of numpy. Try this instead of your loops.
d = numpy.arange(1, n, 2)
c_karray = 2/(d*numpy.pi)
w_karray = d*w
A_karray = 1/(numpy.sqrt(1+(w_karray)**2))
phi_karray = numpy.arctan(-w_karray)
w = (A_karray*c_karray*numpy.sin(w_karray*t[:,None]+phi_karray)).sum(axis=-1)
I'm rather new to NumPy. Anyone have an idea for making this code, especially the nested loops, more compact/efficient? BTW, dist and data are three-dimensional numpy arrays.
def interpolate_to_distance(self,distance):
interpolated_data=np.ndarray(self.dist.shape[1:])
for j in range(interpolated_data.shape[1]):
for i in range(interpolated_data.shape[0]):
interpolated_data[i,j]=np.interp(
distance,self.dist[:,i,j],self.data[:,i,j])
return(interpolated_data)
Thanks!
Alright, I'll take a swag with this:
def interpolate_to_distance(self, distance):
dshape = self.dist.shape
dist = self.dist.T.reshape(-1, dshape[-1])
data = self.data.T.reshape(-1, dshape[-1])
intdata = np.array([np.interp(distance, di, da)
for di, da in zip(dist, data)])
return intdata.reshape(dshape[0:2]).T
It at least removes one loop (and those nested indices), but it's not much faster than the original, ~20% faster according to %timeit in IPython. On the other hand, there's a lot of (probably unnecessary, ultimately) transposing and reshaping going on.
For the record, I wrapped it up in a dummy class and filled some 3 x 3 x 3 arrays with random numbers to test:
import numpy as np
class TestClass(object):
def interpolate_to_distance(self, distance):
dshape = self.dist.shape
dist = self.dist.T.reshape(-1, dshape[-1])
data = self.data.T.reshape(-1, dshape[-1])
intdata = np.array([np.interp(distance, di, da)
for di, da in zip(dist, data)])
return intdata.reshape(dshape[0:2]).T
def interpolate_to_distance_old(self, distance):
interpolated_data=np.ndarray(self.dist.shape[1:])
for j in range(interpolated_data.shape[1]):
for i in range(interpolated_data.shape[0]):
interpolated_data[i,j]=np.interp(
distance,self.dist[:,i,j],self.data[:,i,j])
return(interpolated_data)
if __name__ == '__main__':
testobj = TestClass()
testobj.dist = np.random.randn(3, 3, 3)
testobj.data = np.random.randn(3, 3, 3)
distance = 0
print 'Old:\n', testobj.interpolate_to_distance_old(distance)
print 'New:\n', testobj.interpolate_to_distance(distance)
Which prints (for my particular set of randoms):
Old:
[[-0.59557042 -0.42706077 0.94629049]
[ 0.55509032 -0.67808257 -0.74214045]
[ 1.03779189 -1.17605275 0.00317679]]
New:
[[-0.59557042 -0.42706077 0.94629049]
[ 0.55509032 -0.67808257 -0.74214045]
[ 1.03779189 -1.17605275 0.00317679]]
I also tried np.vectorize(np.interp) but couldn't get that to work. I suspect that would be much faster if it did work.
I couldn't get np.fromfunction to work either, as it passed (2) 3 x 3 (in this case) arrays of indices to np.interp, the same arrays you get from np.mgrid.
One other note: according the the docs for np.interp,
np.interp does not check that the x-coordinate sequence xp is increasing. If
xp is not increasing, the results are nonsense. A simple check for
increasingness is::
np.all(np.diff(xp) > 0)
Obviously, my random numbers violate the 'always increasing' rule, but you'll have to be more careful.