I want to achieve the same result with least complexity in python as min(Ar(Ar~=0)) in MATLAB where Ar is a 2D numpy array.
For those who are not familiar with MATLAB, ~= means != or not equal to.
Is there a function in python which returns the indexes of the elements:
1. Whose values fulfill a condition (elements which are != 0 in this case)
2.
Which can directly be used as list index input for another array? (As (Ar~=0)'s result is being used as an input like this Ar(Ar~=0)
Here Ar~=0 has been used as list index input like this Ar(Ar~=0) and then min of the array Ar(Ar~=0) is being found out. In other words minimum value of the array is found out excluding the elements whose value is 0.
The python syntax for a numpy array A would be:
A[A!=0].min()
you can also set the array elements:
B = A.copy()
B[A==0] = A[A!=0].min()
just as an example setting a cutoff
Related
I can't find an efficient way to conduct Matlab's "ismember(a,b,'rows')" with Python where a and b are arrays of size (ma,2) and (mb,2) respectively and m is the number of couples.
The ismember module (https://pypi.org/project/ismember/) crashes because at some point i.e. when doing np.all(a[:, None] == b, axis=2).any(axis=1) it needs to create an array of size (ma,mb,2) and it is too big. Moreover, even when the function works (because arrays are small enough), it is about a 100times slower than in Matlab. I guess it is because Matlab uses a built-in mex function. Why python does not have what I would think to be such an important function ? I use it countless times in my calculations...
ps : the solution proposed here Python version of ismember with 'rows' and index does not correspond to the true matlab's ismember function since it does not work element by element i.e. it does not verify that a couple of values of 'a' exists in 'b' but only if values of each column of 'a' exist in each columns of 'b'.
You can use np.unique(array,axis=0) in order to find the identical row of an array. So with this function you can simplify your 2D problem to a 1D problem which can be easily solve with np.isin():
import numpy as np
# Dummy example array:
a = np.array([[1,2],[3,4]])
b = np.array([[3,5],[2,3],[3,4]])
# ismember_row function, which rows of a are in b:
def ismember_row(a,b):
# Get the unique row index
_, rev = np.unique(np.concatenate((b,a)),axis=0,return_inverse=True)
# Split the index
a_rev = rev[len(b):]
b_rev = rev[:len(b)]
# Return the result:
return np.isin(a_rev,b_rev)
res = ismember_row(a,b)
# res = array([False, True])
Assume M(i,j,k) is a 3-D array, with some non-0 values and many zero values. I want to keep all of the at least three-consecutive-non-0 values, all other non-0 values that do not satisfy this condition must be replaced with 0. i.e:
M(:,1,1)=[8,0,1,2,0,3,3,2,0,0,4,7,6,4,0,1,0,2]
should be changed to:
M(:,1,1)=[0,0,0,0,0,3,3,2,0,0,4,7,6,4,0,0,0,0]
This process should be done across j and k without a loop since M is very large.
`import pandas as pd
df_test=pd.DataFrame([8,0,1,2,0,3,3,2,0,0,4,7,6,4,0,1,0,2])
df_mask=df_test > 0
df_pass1=df_mask.rolling(3,center=True).sum().mask(~df_mask,0).fillna(0)
df_pass2=df_pass1.rolling(3,center=True).sum().fillna(0)
df_mask2=df_pass2 >= 5
df_test.mask(~df_mask2,0)`
If the sequences of three or more non-zero values include the first or the last value in the array we could add a 0 both at the beginning and at the end of the array, and then remove them once everything is done.
I am trying to translate a piece of 'for' loop code from Matlab to Python. And there is one statement in this block: A[B]=C. All those three A, B and C are matrices. In python, I need to write as A[B-1]=C, because of the difference of index criteria between Matlab and Python.
When B is non-empty, this statement goes well in python. However, if B is empty, this statement goes like this:
A11 = np.copy(A[:,B-1]) #Remind that B is an empty matrix, like B=np.array([0])
IndexError:arrays used as indices must be of integer (or boolean) type
Actually, if B is empty, what I want to have for matrix A11 is just another empty matrix. Definitely I can use a if block to define what matrix A11 should be when B is an empty matrix. But it will be too fussy because I have another 5 statement like this kind of using matrix as an index. Could you give me an example that shows me how to fix this problem? Thanks a lot!
B = np.array([0]) does not generate an empty matrix, it just converts the list [0] into a numpy array.
I suppose you meant something like B = np.zeros(0) (where the argument is a shape). Numpy's default is dtype =float64 but in order to use an array for indexing integer or boolean type is required. For a non-empty array with values that are in fact integers numpy figures out that it can just change the dtype.
To fix your problem you can simply specify the dtype (to int or boolean) when you initialize it, i.e. B = np.zeros(0, dtype=np.int) works fine. A will then be an 'empty matrix' in the sense that one of its shape dimensions is 0 - the others however do not change.
I would like to use np.ravel to create a similar return structure as seen in the MATLAB code below:
[xi yi imv1] = find(squeeze(imagee(:,:,1))+0.1);
imv1 = imv1 - 0.1;
[xi yi imv2] = find(squeeze(imagee(:,:,2))+0.1);
imv2 = imv2 - 0.1;
where imagee is a matrix corresponding to values of a picture obtained from imread().
so, the(almost) corresponding Python translation is:
imv1=np.ravel(imagee**[:,:,0]**,order='F')
Where the bolded index splicing is clearly not the same as MATLAB. How do I specify the index values in Pythonic so that my return values will be the same as that found in the MATLAB portion? I believe this MATLAB code is written as "access all rows, columns, in the specified array of the third dimension." Therefore, how to specify this third parameter in Python?
To retrieve indexes, I usually use np.where. Here's an example: You have a 2 dimensional array
a = np.asarray([[0,1,2],[3,4,5]])
and want to get the indexes where the values are above a threshold, say 2. You can use np.where with the condition a>2
idxX, idxY = np.where(a>2)
which in turn you can use to address a
print a[idxX, idxY]
>>> [3 4 5]
However, the same effect can be achieved by indexing:
print a[a>2]
>>> [3 4 5]
This works on ravel'ed arrays as well as on three dimensional. Using 3D arrays with the first method however will require you to foresee more index arrays.
I am trying to work with lists of numpy matrices and am encountering an annoying problem.
Let's say I start with a list of ten 2x2 zero matrices
para=[numpy.matrix(numpy.zeros((2,2)))]*(10)
I access individual matrices like this
para[0]
para[1]
and so on. So far so good.
Now, I want to modify the first row of the second matrix only, leaving all the others unchanged. So I do this
para[1][0]=numpy.matrix([[1,1]])
The first index points to the second matrix in the list and the second index points to the first row in that matrix, replacing it with [1,1].
But strangely enough, this command changes the first row of ALL ten matrices in the list to [1,1] instead of just the second one like I wanted. What gives?
When you multiply the initial list by 10, you end up with a list of 10 numpy arrays which are in fact references to the the same underlying structure. Modifying one will modify all of them because in fact there's only one numpy array, not 10.
If you need proof, check out this example in the REPL:
>>> a = numpy.zeros(10)
>>> a = [numpy.zeros(10)]*10
>>> a[0] is a[1]
True
>>>
The is operator checks if both objects are in fact the same(not if they are equal in value).
What you should do is use a list comprehension to generate your initial arrays instead of a multiplication, like so:
para=[numpy.matrix(numpy.zeros((2,2))) for i in range(10)]
That will call numpy.matrix() ten times instead of just once and generate 10 distinct matrixes.