I want to execute a function with different parameter values. I have the following snippet of code which works perfectly well:
tau = np.arange(2,4.01,0.1)
R = []
P = []
T = []
L = []
D = []
E = []
Obj = []
for i, tenum in enumerate(tau):
[r, p, t, l, d, e, obj] = (foo.cvxEDA(edaN, 1./fs, tenum, 0.7, 10.0, 0.0008, 0.01))
R.append(r)
P.append(p)
T.append(t)
L.append(l)
D.append(d)
E.append(e)
Obj.append(obj)
However, I was wondering though: Is there an easier way to accomplish this?
I have tried using
res.append(foo.cvxEDA(edaN, 1./fs, tenum, 0.7, 10.0, 0.0008, 0.01) but res[1] returns <generator object <genexpr> at 0x046E7698>.
You can turn a generator object into a list object by just passing it to the list() function so maybe this will do what you want:
res = []
for i, tenum in enumerate(tau):
res.append(list(foo.cvxEDA(edaN, 1./fs, tenum, 0.7, 10.0, 0.0008, 0.01)))
Even shorter with a list comprehension:
res = [list(foo.cvxEDA(edaN, 1./fs, tenum, 0.7, 10.0, 0.0008, 0.01)) for i, tenum in enumerate(tau)]
Either way, this leaves res transposed compared to what you want (thinking of it as a matrix). You can fix that with a call to zip:
res_tr = zip(*res)
R, P, T, L, D, E, Obj = res_tr
Edit: Shortest of all, you can avoid building the intermediate list with a generator expression passed directly to zip():
R, P, T, L, D, E, Obj = zip(*(list(foo.cvxEDA(edaN, 1./fs, tenum, 0.7, 10.0, 0.0008, 0.01)) for tenum in tau))
One final note: In all of these, you can replace "for i, tenum in enumerate(tau)" with "for tenum in tau" since you don't seem to be using i.
tau = np.arange(2,4.01,0.1)
results = [[] for _ in range(7)]
for i, tenum in enumerate(tau):
data = foo.cvxEDA(edaN, 1./fs, tenum, 0.7, 10.0, 0.0008, 0.01)
for r,d in zip(results, data):
r.append(d)
r, p, t, l, d, e, _obj = results
Related
I want to run an integral for different values of "e" and "m" and put the results in a list.
m =[0.14, 0.14, 0.14, 1.30, 4.50]
e = [2/3, -1/3, -1/3, 2/3, -1/3]
def f(z, r):
return ((e)**2)*(alpha_elm*Nc/2*np.pi**2)*(4)*(Q2)*(z**2)*((1-z)**2)*((scipy.special.k0(r*(z*(1-z)*Q2 + (m**2))))**2)
integrate.nquad(f, [[0, 1],[0, np.inf]])
how can i do that?
You can define a partially applied version of your function where you set the values for e and m. Then iterate over their ranges of values and compute the specific results:
from functools import partial
def f(m, e, z, r):
return ((e)**2)*(alpha_elm*Nc/2*np.pi**2)*(4)*(Q2)*(z**2)*((1-z)**2)*((scipy.special.k0(r*(z*(1-z)*Q2 + (m**2))))**2)
results = []
for mm, ee in zip(m, e):
partial_f = partial(f, mm, ee)
result = integrate.nquad(partial_f, [[0, 1], [0, np.inf]])
results.append(result)
I would however strongly suggest to reformat and break down the overly complex definition of your function f.
I am going through my coordinates data and I see some duplicate coordinates with different parameters due to certain preprocessing. I want to be able to merge the attributes corresponding to the matched coordinates and get the simplified results. To clarify what I mean here is an example:
X = [1.0, 2.0, 3.0, 2.0]
Y = [8.0, 3.0, 4.0, 3.0]
A = [13, 16, 20, 8]
The above data is read as follows: point (1.0, 8.0) has a value of 13 and (2.0, 3.0) has a value of 16. Notice that the second point and fourth point have the same coordinates but different attribute values. I want to be able to remove the duplicates from the lists of coordinates and sum the attributes so the results would be new lists:
New_X = [1.0, 2.0, 3.0]
New_Y = [8.0, 3.0, 4.0]
New_A = [13, 24, 20]
24 is the sum of 16 and 8 from the second and fourth points with the same coordinates, therefore one point is kept and the values are summed.
I am not sure how to do this, I thought of using nested for loops of zips of the coordinates but I am not sure how to formulate it to sum the attributes.
Any help is appreciated!
I think that maintaining 3 lists is a bit awkward. Something like:
D = dict()
for x,y,a in zip(X,Y,A):
D[(x,y)] = D.get((x,y),0) + a
would put everything together in one place.
If you'd prefer to decompose it back into 3 lists:
for (x,y),a in D.items():
newX.append(x)
newY.append(y)
newA.append(a)
Another option here is to use itertools.groupby. But since this only groups consecutive keys, you'll have to first sort your coordinates.
First we can zip them together to create tuples of the form (x, y, a). Then sort these by the (x, y) coordinates:
sc = sorted(zip(X, Y, A), key=lambda P: (P[0], P[1])) # sorted coordinates
print(sc)
#[(1.0, 8.0, 13), (2.0, 3.0, 16), (2.0, 3.0, 8), (3.0, 4.0, 20)]
Now we can groupby the coordinates and sum the values:
from itertools import groupby
print([(*a, sum(c[2] for c in b)) for a, b in groupby(sc, key=lambda P: (P[0], P[1]))])
#[(1.0, 8.0, 13), (2.0, 3.0, 24), (3.0, 4.0, 20)]
And since zip is its own inverse, you can get New_X, New_Y, and New_A via:
New_X, New_Y, New_A = zip(
*((*a, sum(c[2] for c in b)) for a, b in groupby(sc, key=lambda P: (P[0], P[1])))
)
print(New_X)
print(New_Y)
print(New_A)
#(1.0, 2.0, 3.0)
#(8.0, 3.0, 4.0)
#(13, 24, 20)
Of course, you can do this all in one line but I broke it up into pieces so that it's easier to understand.
you could put the (x,y) coords in a dictionary:
dict = {}
for i in range(len(X)) # len(X) = len(Y)
if (X[i], Y[i]) not in dict.keys():
dict[(X[i], Y[i])] = A[i]
else:
dict[(X[i], Y[i])] += A[i]
Can use a dictionary
d = {}
for i in range(len(X)):
tup = (X[i], Y[i])
if tup in d:
d[tup] += A[i]
else:
d[tup] = A[i]
New_X = []
New_Y = []
New_A = []
for key in d.keys():
New_X.append(key[0])
New_Y.append(key[1])
New_A.append(d[key])
Try this list comprehension:
y,x,a=zip(*[e for c,e in enumerate(zip(Y,X,A)) if not e[0:1] in [x[0:1] for x in zip(X,Y,A)][c:]])
A dict seems like a more appropriate data structure here. This will build one.
from collections import Counter
D = sum((Counter({(x, y): a}) for x, y, a in zip(X, Y, A)), Counter())
print(D)
#Counter({(2.0, 3.0): 24, (3.0, 4.0): 20, (1.0, 8.0): 13})
You can unpack these back into separate lists using:
New_X, New_Y, New_A = map(list, zip(*[(x,y,a) for (x,y),a in D.items()]))
print(New_X)
print(New_Y)
print(New_A)
#[1.0, 2.0, 3.0]
#[8.0, 3.0, 4.0]
#[13, 24, 20]
I have a function like this:
def foo(v, w):
return sum(np.exp(v/w))
Where v in the beginning is a numpy array and w a number. Now I want to plot the value of this function for more values of w, so I need a function that works for different sizes of vectors.
My solution for now is the obvious one
r = []
for e in w:
r.append(foo(v, e))
but I wonder if there is a better way to do it. Also, I want to stay low on memory, so I need to avoid create a big matrix, then applying the function to every value and sum over the columns (the length of v is more than 5e+4 and the length of w is 1e+3).
Thanks
If you cannot determine an upper bound for the length of v and ensure that you don't exceed the memory requirements, I think you will have to stay with your solution.
If you can determine an upper bound for length of v and meet your memory requirements using a Mx1000 array, you can do this.
import numpy as np
v = np.array([1,2,3,4,5])
w = np.array([10.,5.])
c = v / w[:, np.newaxis]
d = np.exp(c)
e = d.sum(axis = 1)
>>>
>>> v
array([1, 2, 3, 4, 5])
>>> w
array([ 10., 5.])
>>> c
array([[ 0.1, 0.2, 0.3, 0.4, 0.5],
[ 0.2, 0.4, 0.6, 0.8, 1. ]])
>>> d
array([[ 1.10517092, 1.22140276, 1.34985881, 1.4918247 , 1.64872127],
[ 1.22140276, 1.4918247 , 1.8221188 , 2.22554093, 2.71828183]])
>>> e
array([ 6.81697845, 9.47916901])
>>>
I need to split a sorted list of probabilities into groups. The first group contains probabilities from (0.5,1), the second (0.25,0.5) etc.
I've produced some code that splits a list containing powers of two less than 1 into two lists: one of list members greater than 0.5, the other containing (original) list members less than 0.5.
from itertools import groupby
from operator import itemgetter
import doctest
N= 10
twos = [2**(-(i+1)) for i in range(0,N)]
def split_by_prob(items,cutoff):
"""
(list of double) -> list of (lists) of double
Splits a set into subsets based on probability
>>> split_by_prob(twos, 0.5)
[[0.5], [ 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]]
"""
groups = []
keys = []
for k,g in it.groupby(enumerate(items), lambda (j, x): x<cutoff):
groups.append((map(itemgetter(1),g)))
return groups
Calling this code from the command line does exactly this:
>>> g = split_into_groups(twos,0.5)
>>> g
[[0.5], [0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]]
My question: how can I change the cutoff on each iteration? I.e. if I passed the function a list of cutoffs (e.g. cutoffs = [0.5, 0.125, 0.0625], I'd get a list of lists each with the respective members of the original list grouped into the correct category. In this case the groups returned will be something like [[0.5],[0.25,0125],[0.0625],[0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]]
If I understand you correctly you can just iterate over a list of cutoffs using x < i for each i in cutoffs.
cutoffs = [0.5, 0.125, 0.0625]
def split_by_prob(items,cutoffs):
"""
(list of double) -> list of (lists) of double
Splits a set into subsets based on probability
# >>> split_by_prob(twos, 0.5)
[[0.5], [ 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]]
"""
groups = []
keys = []
for i in cutoffs:
for k,g in groupby(enumerate(items), lambda (j, x): x < i):
groups.append((map(itemgetter(1),g)))
return groups
print split_by_prob(twos, cutoffs)
[0.5], [0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625], [0.5, 0.25, 0.125], [0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625], [0.5, 0.25, 0.125, 0.0625], [0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]
I've figured out what I needed to do, and the full code is below. I'm not sure how efficient or pythonic it is however:
import numpy as np
from itertools import groupby
from operator import itemgetter
import doctest
N= 10
twos = [2**(-(i+1)) for i in range(0,N)]
cutoffs = [0.5, 0.125, 0.03125]
def split_by_prob(items,cutoff,groups):
"""
(list of double) -> list of (lists) of double
Splits a set into subsets based on probability
>>> split_by_prob(twos, 0.5)
[[0.5], [ 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]]
"""
for k,g in groupby(enumerate(items), lambda (j, x): x<cutoff):
groups.append((map(itemgetter(1),g)))
return groups
def split_into_groups(items, cutoffs):
"""
(list of double) -> list of (lists) of double
Splits a set into subsets based on probability
>>> split_by_prob(twos, cutoffs)
[[0.5], [0.25, 0.125], [0.0625, 0.03125], [0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625]]
"""
groups = items
final = []
for i in cutoffs:
groups = split_by_prob(groups,i,[])
final.append(groups[0])
groups = groups.pop()
final.append(groups)
return final
I have 3 lists x, y, z and I plot them with:
ax.plot3D(x, y, z, linestyle = 'None', marker = 'o').
What is the easiest way to only plot the points where x > 0.5?
(my problem is how to define a sublist under a condition without making a for loop on that list).
I'm not sure why you're avoiding looping over a list and I'm assuming that you want the related points in the other lists also removing.
>>> x = [0.0, 0.4, 0.6, 1.0]
>>> y = [0.0, 2.2, 1.5, 1.6]
>>> z = [0.0, 9.1, 1.0, 0.9]
>>> zip(x,y,z)
[(0.0, 0.0, 0.0), (0.4, 2.2, 9.1), (0.6, 1.5, 1.0), (1.0, 1.6, 0.9)]
>>> [item for item in zip(x,y,z) if item[0] > 0.5]
[(0.6, 1.5, 1.0), (1.0, 1.6, 0.9)]
Separating the list into it's constituent lists will require looping over the list somehow.
It's impossible to verify a condition on every element of a list without iterating over it at least once. You could use numpy here for easy access to the elements after condition is passsed and do:
import numpy
x = [0.0, 0.4, 0.6, 1.0]
y = [0.0, 2.2, 1.5, 1.6]
z = [0.0, 9.1, 1.0, 0.9]
res = numpy.array([[x[i], y[i], z[i]] for i in xrange(len(x)) if x[i] > 0.5])
ax.plot3D(res[:,0], res[:,1], res[:,2], linestyle="None, marker='o'")
A simple list comprehension won't be enough to remove the (x,y,z) tuples if x <= 0.5, you'll have to do a little more, I use operator.itemgetter for the second part :
from operator import itemgetter
result = [(a, b, c) for a,b,c in zip(x,y,z) if a > 0.5] # first, remove the triplet
x = itemgetter(0)(result) # then grab from the new list the x,y,z parts
y = itemgetter(1)(result)
z = itemgetter(2)(result)
ax.plot3D(x, y, z, linestyle="None, marker='o')
EDIT:
Following and upgrading #shenshei advice we can achieve it with a one-line:
ax.plot3D(
*zip(*[(a, b, c) for a,b,c in zip(x,y,z) if a > 0.5]),
linestyle="None,
marker='o'
)
Reposting my comment as an answer as suggested by #StephenPaulger . You can do this with a generator expression and a couple of calls to the built-in zip():
x = [0.0, 0.4, 0.6, 1.0]
y = [0.0, 2.2, 1.5, 1.6]
z = [0.0, 9.1, 1.0, 0.9]
points = (point for point in zip(x, y, z) if point[0] > 0.5)
x, y, z = zip(*points)
You could also use a list comprehension for points if you want to, but - assuming Python 3, where zip() no longer precomputes a full list when called - that might hurt your memory usage and speed, especially if the number of points is large.
Probably using numpy would provide the cleanest approach. However, you will need to have lists/arrays x, y, and z as numpy arrays. So, first convert these lists to numpy arrays:
import numpy as np
x = np.asarray(x)
y = np.asarray(y)
z = np.asarray(z)
Now compute an array of indices of elements that satisfy your condition:
idx = np.where(x > 0.5)
NOTE: Alternatively, you could compute a boolean mask: idx=x>0.5 (this will not change the use of idx in the next ax.plot3D statement).
Use these indices to select only those specific points in x, y, and z that satisfy desired condition:
ax.plot3D(x[idx], y[idx], z[idx], linestyle = 'None', marker = 'o')
I don't want to steal lvc's thunder, but here's a variant on their answer:
>>> x = [0.1, 0.6, 0.2, 0.8, 0.9]
>>> y = [0.3, 0.1, 0.9, 0.5, 0.8]
>>> z = [0.9, 0.2, 0.7, 0.4, 0.3]
>>>
>>> a, b, c = zip(*filter(lambda t: t[0] > 0.5, zip(x, y, z)))
>>> print a, "\n", b, "\n", c
(0.6, 0.8, 0.9)
(0.1, 0.5, 0.8)
(0.2, 0.4, 0.3)
>>> ax.plot3D(a, b, c, linestyle = 'None', marker = 'o')