Develop Reference

How to make a function that outputs an array with a predefined size?

I want to make a function that outputs an array of predefined size. The function is dependent on variables x, y, rx, and ry, and d. Variables x and y are directly related to the cartesian values. rx and ry are the radius of the blobs generated from the function.
I have already converted array indiies from the traditional 0, 0 on the upper left corner to the middle most pixel. This array should always be odd in herms of length and width.
The blob below gives me one array at a time. I need a stack of arrays I can add together. When I change x and y, the size of the array changes, but I need the array to be the same size or zeroed out at on the borders.
I have some code that follows:
def equation(x, y):
return 1 * np.exp(-(x**2 + y**2))
def make_matrix(xs, ys, x_min=nxs, y_min=nys): #makes cartesian values
out = [] #outputs array
for i in range(x_min, xs - center_x):
row = []
for j in range(y_min, ys - center_y):
row.append(equation(i, j))
out.append(row)
return out
blob = np.asarray(list(np.float_(make_matrix(x, y))))

Vectorization is your friend here. No need to compute lists element-by element when your function can be evaluated element-wise for an entire array.
def make_matrix(xs, ys, x_min=nxs, y_min=nys):
x = np.arange(x_min, xs - center_x).reshape(1, -1)
y = np.arange(y_min, ys - center_y).reshape(-1, 1)
out = np.exp(-(x**2 + y**2))
return out
The reshape operations create a 1xN row vector for x and a Nx1 column vector for y. These shapes ensure that broadcasting in the add (+) operation expands x**2 + y**2 to an array of the final size you want.

ask for help for a sum (sigma) function

need help to calculate this:
so, the total number of y is equal to number of x, and each y is calculated with one x and several a.
My code list below, it gives the correct results for a0. what is a simple way to calculate this? maybe a different version can also verify the results.
Thanks a lot.
import numpy as np
import matplotlib.pyplot as plt
a = np.array([1,2,3,4],float) # here we can give several a
b = np.asarray(list(enumerate(a)))
x = np.linspace(0.0,1.0,10)
y1 = []
for r in x:
y1.append(np.exp(np.sum((1-r)**2*a*((2*b[:,0]+1)*r-1+r)*(r-1+r)**(b[:,0]-1))))
y1=np.asarray(y1)

You can write almost literally the same in numpy:
def f(x, a):
x, a = np.asanyarray(x), np.asanyarray(a)
x = x[:, None] # create new dimension to sum along
i = np.arange(len(a)) # create counter
return np.sum((1-x)**2 * a * ((2*i + 1) * x - (1-x)) * (x - (1-x))**(i-1), axis=-1)
As a side note: there are obvious algebraic simplifications you could take advantage of.

Combining different elements of array in Numpy while doing vector operations

I have a function f which I am using to evolve a Numpy array z repeatedly. Thus my code looks like this:
for t in range(100):
z = f(z)
However, now I want to combine elements of array while evolving. For example, in simple Python, I would like to do something like this:
N = len(z)
for t in range(100):
for i in range(len(z)):
z_new[i] = f(z[i]) + z[(i-1)%N] + z[(i+1)%N]
z = z_new
How can achieve the same thing in Numpy vector operations so that I wouldn't have to compromise with the great speed that Numpy gives me?
Thanks in advance

You can roll the data back and forth to achieve the same result.
Z = f(z)
Z = np.roll(Z, 1) + z
Z = np.roll(Z, -2) + z
z = np.roll(Z, 1)
I had also first thought about slicing but went with np.roll when I found it.
Prompted by #hpaulj's comment I came up with a slice solution:
q = np.array([1,2,3,4,5])
Q = f(q)
# operate on the middle
Q[1:] += q[:-1]
Q[:-1] += q[1:]
# operate on the ends
Q[0] += q[-1]
Q[-1] += q[0]
q = Q.copy()

Assigning input markers to function output in python

Sorry for the seemingly elementary question. What I'm trying to implement is summarized in the following steps:
Generate input variables: x, y.
Let z = F(x,y).
Plot z's for particular combinations of x and y.
For example:
zlist = []
for _ in range(100):
x = np.random.random()*1.
y = np.random.random()*.5
if x < .5:
z = y / 2
else:
z = y * 2
zlist.append(z)
Now if I want to plot z for all the x between (0, 0.3), I presumably would need some marker on each element in zlist indicating its inputs variables. How would I attach such marker and then access it from the list when plotting?

I don't actually know anything about Numpy, so someone please comment and tell me if I'm making a fool out of myself. It seems like vanilla python behavior, though.
Rather than appending z, let's append (z,x) instead. Now zlist is a list of tuples, and you can loop through and plot by checking zlist[i][1].
zlist = []
for _ in range(100):
x = np.random.random()*1.
y = np.random.random()*.5
if x < .5:
z = y / 2
else:
z = y * 2
zlist.append((z,x))
for value in zlist:
if value[1] > 0 and value[1] < 0.3:
# Plot value[0]
# Or if you prefer list comprehensions:
# [value[0] for value in zlist if value[1] >0 and value[1] < 0.3]
# that will return a list with only the z values in zlist.

With numpy it's almost always much more efficient to perform operations on vectors and
arrays rather than on built-in Python sequence types such as lists. Here's one
way you can quickly find F(x, y) for every combination of two sets of random x
and y values without looping in Python. The result is going to be an nx-by-ny
array Z, where Z[i, j] = F(x[i], y[j]).
First of all, you can generate all of your x, y inputs as vectors:
nx = 100
ny = 200
x = np.random.random(size=nx) * 1.
y = np.random.random(size=ny) * 5.
For the result to be an nx-by-ny array, you could take these two vectors and
multiply them by ones to get two 2D nx-by-ny arrays containing the x and y
values in the rows and columns respectively. You can do this by taking advantage of numpy's
broadcasting rules:
x_arr = x[:,np.newaxis] * np.ones((nx,ny))
y_arr = y[np.newaxis,:] * np.ones((nx,ny))
The function you will apply to each x,y pair depends on the x value.
Fortunately, you can use np.where(<condition>, <do_this>, <do_that>) to apply
different operations to the values in your input depending on some condition:
Z = np.where(x_arr < 0.5, y_arr / 2., y_arr * 2.)
We can check that all the results are correct:
for i in xrange(nx):
for j in xrange(ny):
if x[i] < 0.5:
assert Z[i, j] == y[j] / 2.
else:
assert Z[i, j] == y[j] * 2
There's actually an even cleaner way to compute Z without expanding x and y into 2D arrays. Using the same broadcasting trick we used to get
x_arr and y_arr, you can pass x and y directly to np.where():
x2 = x[:,np.newaxis]
y2 = y[np.newaxis,:]
Z2 = np.where(x2 < 0.5, y2 / 2., y2 * 2.)
assert np.all(Z == Z2)

Using combinations or another trick to iterate though 3 different arrays?

consider my code
a,b,c = np.loadtxt ('test.dat', dtype='double', unpack=True)
a,b, and c are the same array length.
for i in range(len(a)):
q[i] = 3*10**5*c[i]/100
x[i] = q[i]*math.sin(a)*math.cos(b)
y[i] = q[i]*math.sin(a)*math.sin(b)
z[i] = q[i]*math.cos(a)
I am trying to find all the combinations for the difference between 2 points in x,y,z to iterate this equation (xi-xj)+(yi-yj)+(zi-zj) = r
I use this combination code
for combinations in it.combinations(x,2):
xdist = (combinations[0] - combinations[1])
for combinations in it.combinations(y,2):
ydist = (combinations[0] - combinations[1])
for combinations in it.combinations(z,2):
zdist = (combinations[0] - combinations[1])
r = (xdist + ydist +zdist)
This takes a long time for python for a large file I have and I am wondering if there is a faster way to get my array for r preferably using a nested loop?
Such as
if i in range(?):
if j in range(?):

Since you're apparently using numpy, let's actually use numpy; it'll be much faster. It's almost always faster and usually easier to read if you avoid python loops entirely when working with numpy, and use its vectorized array operations instead.
a, b, c = np.loadtxt('test.dat', dtype='double', unpack=True)
q = 3e5 * c / 100 # why not just 3e3 * c?
x = q * np.sin(a) * np.cos(b)
y = q * np.sin(a) * np.sin(b)
z = q * np.cos(a)
Now, your example code after this doesn't do what you probably want it to do - notice how you just say xdist = ... each time? You're overwriting that variable and not doing anything with it. I'm going to assume you want the squared euclidean distance between each pair of points, though, and make a matrix dists with dists[i, j] equal to the distance between the ith and jth points.
The easy way, if you have scipy available:
# stack the points into a num_pts x 3 matrix
pts = np.hstack([thing.reshape((-1, 1)) for thing in (x, y, z)])
# get squared euclidean distances in a matrix
dists = scipy.spatial.squareform(scipy.spatial.pdist(pts, 'sqeuclidean'))
If your list is enormous, it's more memory-efficient to not use squareform, but then it's in a condensed format that's a little harder to find specific pairs of distances with.
Slightly harder, if you can't / don't want to use scipy:
pts = np.hstack([thing.reshape((-1, 1)) for thing in (x, y, z)])
sqnorms = np.sum(pts ** 2, axis=1)
dists = sqnorms.reshape((-1, 1)) - 2 * np.dot(pts, pts.T) + sqnorms
which basically implements the formula (a - b)^2 = a^2 - 2 a b + b^2, but all vector-like.

Apologies for not posting a full solution, but you should avoid nesting calls to range(), as it will create a new tuple every time it gets called. You are better off either calling range() once and storing the result, or using a loop counter instead.
For example, instead of:
max = 50
for number in range (0, 50):
doSomething(number)
...you would do:
max = 50
current = 0
while current < max:
doSomething(number)
current += 1

Well, the complexity of your calculation is pretty high. Also, you need to have huge amounts of memory if you want to store all r values in a single list. Often, you don't need a list and a generator might be enough for what you want to do with the values.
Consider this code:
def calculate(x, y, z):
for xi, xj in combinations(x, 2):
for yi, yj in combinations(y, 2):
for zi, zj in combinations(z, 2):
yield (xi - xj) + (yi - yj) + (zi - zj)
This returns a generator that computes only one value each time you call the generator's next() method.
gen = calculate(xrange(10), xrange(10, 20), xrange(20, 30))
gen.next() # returns -3
gen.next() # returns -4 and so on