Related
I am trying to write a code where I do a summation over a function with two variables, E and n,
where n is a range of 0-5, and plot the result which should be a multiple step function.
import matplotlib.pyplot as plt
import numpy as np
E = np.linspace(0, 10, 10000)
for n in range(0, 6):
h= []
h.append(1/(1 + np.exp(-2*np.pi * (E-(n-0.5)*3))))
print(h)
plt.plot(E,h)
This is the code I have been using. It produces multiple arrays for h:
[array([0.99991931, 0.99991981, 0.99992031, ..., 1. , 1. ,
1. ])]
[array([8.06930057e-05, 8.12016207e-05, 8.17134412e-05, ...,
1.00000000e+00, 1.00000000e+00, 1.00000000e+00])]
[array([5.25548518e-13, 5.28861364e-13, 5.32195094e-13, ...,
1.00000000e+00, 1.00000000e+00, 1.00000000e+00])]
[array([3.42258854e-21, 3.44416317e-21, 3.46587379e-21, ...,
9.99999847e-01, 9.99999848e-01, 9.99999849e-01])]
[array([2.22893072e-29, 2.24298100e-29, 2.25711985e-29, ...,
4.09276641e-02, 4.11750329e-02, 4.14238322e-02])]
[array([1.45157155e-37, 1.46072167e-37, 1.46992947e-37, ...,
2.77912110e-10, 2.79663956e-10, 2.81426846e-10])]
but when I try to plot I get the following error:
ValueError: x and y must have same first dimension, but have shapes (10000,) and (1, 10000)
I don't undestand what is causing this and any help would be appreciated.
I assume what you want is the following:
E = np.expand_dims(np.linspace(0, 10, 10000), 1)
n = np.arange(0, 6)
h = 1/(1 + np.exp(-2*np.pi * (E-(n-0.5)*3)))
plt.plot(E, h)
Note that this vectorized calculation does not require a loop, as Numpy broadcasting figures out automatically how to combine the E and n vectors to a 2D h array, if the input dimension are consistent (note the np.expand_dims). In general, whenever you see the need to use a loop in Numpy, it's good advice to take a step back and think about vectorization.
Fixed version of your original code:
import matplotlib.pyplot as plt
import numpy as np
E = np.linspace(0, 10, 10000)
h= []
for n in range(0, 6):
h.append(1/(1 + np.exp(-2*np.pi * (E-(n-0.5)*3))).T)
plt.plot(E, np.stack(h).T)
Define h before the loop, such that it is not defined as the empty list on each iteration.
Combine the list of 1D arrays into a 6x10_000 2D array using np.stack and transpose (.T) in order to plot all curves.
Given a numpy array of the form below:
x = [[4.,3.,2.,1.,8.],[1.2,3.1,0.,9.2,5.5],[0.2,7.0,4.4,0.2,1.3]]
is there a way to retain the top-3 values in each row and set others to zero in python (without an explicit loop). The result in the case of the example above would be
x = [[4.,3.,0.,0.,8.],[0.,3.1,0.,9.2,5.5],[0.0,7.0,4.4,0.0,1.3]]
Code for one example
import numpy as np
arr = np.array([1.2,3.1,0.,9.2,5.5,3.2])
indexes=arr.argsort()[-3:][::-1]
a = list(range(6))
A=set(indexes); B=set(a)
zero_ind=(B.difference(A))
arr[list(zero_ind)]=0
The output:
array([0. , 0. , 0. , 9.2, 5.5, 3.2])
Above is my sample code (with many lines) for a 1-D numpy array. Looping through each row of a numpy array and performing this same computation repeatedly would be quite expensive. Is there a simpler way?
Here is a fully vectorized code without third party outside numpy. It is using numpy's argpartition to efficiently find the k-th values. See for instance this answer for other use cases.
def truncate_top_k(x, k, inplace=False):
m, n = x.shape
# get (unsorted) indices of top-k values
topk_indices = numpy.argpartition(x, -k, axis=1)[:, -k:]
# get k-th value
rows, _ = numpy.indices((m, k))
kth_vals = x[rows, topk_indices].min(axis=1)
# get boolean mask of values smaller than k-th
is_smaller_than_kth = x < kth_vals[:, None]
# replace mask by 0
if not inplace:
return numpy.where(is_smaller_than_kth, 0, x)
x[is_smaller_than_kth] = 0
return x
Use np.apply_along_axis to apply a function to 1-D slices along a given axis
import numpy as np
def top_k_values(array):
indexes = array.argsort()[-3:][::-1]
A = set(indexes)
B = set(list(range(array.shape[0])))
array[list(B.difference(A))]=0
return array
arr = np.array([[4.,3.,2.,1.,8.],[1.2,3.1,0.,9.2,5.5],[0.2,7.0,4.4,0.2,1.3]])
result = np.apply_along_axis(top_k_values, 1, arr)
print(result)
Output
[[4. 3. 0. 0. 8. ]
[0. 3.1 0. 9.2 5.5]
[0. 7. 4.4 0. 1.3]]
def top_k(arr, k, axis = 0):
top_k_idx = = np.take_along_axis(np.argpartition(arr, -k, axis = axis),
np.arange(-k,-1),
axis = axis) # indices of top k values in axis
out = np.zeros.like(arr) # create zero array
np.put_along_axis(out, top_k_idx, # put idx values of arr in out
np.take_along_axis(arr, top_k_idx, axis = axis),
axis = axis)
return out
This should work for arbitrary axis and k, but does not work in-place. If you want in-place it's a bit simpler:
def top_k(arr, k, axis = 0):
remove_idx = = np.take_along_axis(np.argpartition(arr, -k, axis = axis),
np.arange(arr.shape[axis] - k),
axis = axis) # indices to remove
np.put_along_axis(out, remove_idx, 0, axis = axis) # put 0 in indices
Here is an alternative that use a list comprehension to look thru your array and applying the keep_top_3 function
import numpy as np
import heapq
def keep_top_3(arr):
smallest = heapq.nlargest(3, arr)[-1] # find the top 3 and use the smallest as cut off
arr[arr < smallest] = 0 # replace anything lower than the cut off with 0
return arr
x = [[4.,3.,2.,1.,8.],[1.2,3.1,0.,9.2,5.5],[0.2,7.0,4.4,0.2,1.3]]
result = [keep_top_3(np.array(arr)) for arr in x]
I hope this helps :)
I want to use Scipy-> interpolate->interp2d. Result of interpolation is correct, but why result is saved in list three times? I used next script:
from scipy import interpolate
import numpy as np
def main():
x = np.array([1, 2, 3, 4])
y = np.array([1, 2])
z = np.array([[4,4,6,6],[6, 6,12, 12]])
f = interpolate.interp2d(x, y, z, kind='linear', copy = False)
xi = np.array([1.5, 2.5, 3.5])
yi = np.array([1.5,1.5,1.
Result is:
zi =[[ 5. 7. 9.][ 5. 7. 9.] [ 5. 7. 9.]]
Please can you help me?
Your input arrays x,y,z need to be of identical shape. In your case one has four entries, one has two entries, and the other one is two dimensional with four entries. This will trigger numpy broadcasting. https://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html
The code is:
import numpy as np
def Mahalanobis(x, covariance_matrix, mean):
x = np.array(x)
mean = np.array(mean)
covariance_matrix = np.array(covariance_matrix)
return (x-mean)*np.linalg.inv(covariance_matrix)*(x.transpose()-mean.transpose())
#variables x and mean are 1xd arrays; covariance_matrix is a dxd matrix
#the 1xd array passed to x should be multiplied by the (inverted) dxd array
#that was passed into the second argument
#the resulting 1xd matrix is to be multiplied by a dx1 matrix, the transpose of
#[x-mean], which should result in a 1x1 array (a number)
But for some reason I get a matrix for my output when I enter the parameters
Mahalanobis([2,5], [[.5,0],[0,2]], [3,6])
output:
out[]: array([[ 2. , 0. ],
[ 0. , 0.5]])
It seems my function is just giving me the inverse of the 2x2 matrix that I input in the 2nd argument.
You've made the classic mistake of assuming that the * operator is doing matrix multiplication. This is not true in Python/numpy (see http://www.scipy-lectures.org/intro/numpy/operations.html and https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab-users.html). I broke it down into intermediate steps and used the dot function
import numpy as np
def Mahalanobis(x, covariance_matrix, mean):
x = np.array(x)
mean = np.array(mean)
covariance_matrix = np.array(covariance_matrix)
t1 = (x-mean)
print(f'Term 1 {t1}')
icov = np.linalg.inv(covariance_matrix)
print(f'Inverse covariance {icov}')
t2 = (x.transpose()-mean.transpose())
print(f'Term 2 {t2}')
mahal = t1.dot(icov.dot(t2))
#return (x-mean)*np.linalg.inv(covariance_matrix).dot(x.transpose()-mean.transpose())
return mahal
#variables x and mean are 1xd arrays; covariance_matrix is a dxd matrix
#the 1xd array passed to x should be multiplied by the (inverted) dxd array
#that was passed into the second argument
#the resulting 1xd matrix is to be multiplied by a dx1 matrix, the transpose of
#[x-mean], which should result in a 1x1 array (a number)
Mahalanobis([2,5], [[.5,0],[0,2]], [3,6])
produces
Term 1 [-1 -1]
Inverse covariance [[2. 0. ]
[0. 0.5]]
Term 2 [-1 -1]
Out[9]: 2.5
One can use scipy's mahalanobis() function to verify:
import scipy.spatial, numpy as np
scipy.spatial.distance.mahalanobis([2,5], [3,6], np.linalg.inv([[.5,0],[0,2]]))
# 1.5811388300841898
1.5811388300841898**2 # squared Mahalanobis distance
# 2.5000000000000004
def Mahalanobis(x, covariance_matrix, mean):
x, m, C = np.array(x), np.array(mean), np.array(covariance_matrix)
return (x-m)#np.linalg.inv(C)#(x-m).T
Mahalanobis([2,5], [[.5,0],[0,2]], [3,6])
# 2.5
np.isclose(
scipy.spatial.distance.mahalanobis([2,5], [3,6], np.linalg.inv([[.5,0],[0,2]]))**2,
Mahalanobis([2,5], [[.5,0],[0,2]], [3,6])
)
# True
I have used a function to calculate the difference between 2 values. From printing the output of the function below, the answer ranges from -5 to 4. However I only want the function to display positive answers only (i.e: 1 to 4).
Is it possible to disregard the negative values without changing the boundaries of x and nor change the value of a?
import numpy as np
L = 10
a = 5
def position(x,a):
return x-a
x = np.arange(0.0, L, 1)
print (position(x,a))
[-5. -4. -3. -2. -1. 0. 1. 2. 3. 4.]
import numpy as np
L = 10
a = 5
def position(x,a):
return x-a
x = np.arange(0.0, L, 1)
tmp = position(x,a)
print (tmp[tmp>=0])
This may help, an example of filtering a numpy array
import numpy
arr = numpy.array([-1.1, 0.0, 1.1])
print(arr)
bools = arr >= 0.0 # define selection
print(bools)
# filter by "bools"
print(arr[bools])
based on what I can take from your question this should work:
result = list(filter(lambda x:x>=0, position(x,a))