I want to merge certain values that are numerically close.
In this example I'll look to merge anything that can be connected via a neighboring value with a difference less 2.
import numpy as np
a = np.arange(10)
a = np.delete(a, (3, 7))
matrix = np.abs(a.reshape(-1,1) - a)
matrix < 2
array([[ True, True, False, False, False, False, False, False],
[ True, True, True, False, False, False, False, False],
[False, True, True, False, False, False, False, False],
[False, False, False, True, True, False, False, False],
[False, False, False, True, True, True, False, False],
[False, False, False, False, True, True, False, False],
[False, False, False, False, False, False, True, True],
[False, False, False, False, False, False, True, True]])
Starting at the top left corner:
First move horizontal to find the last true value.
Then go downwards to find the last true value.
Repeat both until the square is found.
Cannot connect True values diagonally.
This would give you the first square that goes from [0,2] to [2,2].
The desired output of this example would be:
[[0, 2], [3, 5], [6, 7]]
Where the values indicate the beginning and end of the square. Is there a good way of doing this?
I'd prefer not to loop if possible.
So I've done this using np.diagonal as the basis for the logic.
bm = matrix < 2
endcoords = np.argwhere(bm[1:].diagonal() == False)
zers = np.zeros(endcoords.shape)
zers[1:] += endcoords[:-1] + 1
end = np.hstack((zers, endcoords))
end
array([[0., 2.],
[3., 5.]])
I know this excludes the last coordinate, which for my example doesn't give the 'correct' answer. This however, does give the right answer for my workflow as I'm chunking arrays together - so the last missing coordinate will be the first of my next array. It shouldn't be too hard to finagle the right answer from this anyways.
Related
I am trying to make a grid to store bool variables kinda like mine sweeper and i would like to find a better way
So far i have a very inefficient way of just declaring like 15 lists with the values set to false like this
A = [False, False, False, False, False, False, False, False, False, False]
Is there a more efficient way to do this
You can efficiently create a list of the same value with:
A = [False]*15
However, more code is required to extend this into a grid. Instead, you could use NumPy to create a grid of False (True) values by using np.zeros (np.ones). For example, a 3x4 grid of False values can be created with:
grid = np.zeros((3, 4), dtype=bool)
>> [[False False False False]
>> [False False False False]
>> [False False False False]]
You might want to use a 2D array for that:
array = [
[False, False, False, False, False, False, False, False, False, False],
[False, False, False, False, False, False, False, False, False, False],
...
]
This can also be created using a list comp:
array = [[False] * 15 for _ in range(15)]
I have the following tensor:
predictions = torch.tensor([[ True, False, False],
[False, False, True],
[False, True, True],
[ True, False, False]])
I applied conditions along the axis like below.
new_pred= []
if predictions == ([True,False,False]):
new_pred = torch.Tensor(0)
if predictions == ([False,False,True]):
new_pred = torch.Tensor(2)
if predictions == ([False,True,True]):
new_pred = torch.Tensor(2)
So I want the final output (new_pred) to be:
tensor([0, 2, 2, 0])
But I am getting a blank [] for the new_pred tensor. I think my logic must be flawed since nothing is getting stored in the new_pred. Can someone help me write this logic accurately?
The type of predictions is torch.Tensor while ([True, False, False]) is a list, first, you have to make sure both sides have the same type.
predictions == torch.tensor([True,False,False])
>>> tensor([[ True, True, True],
[False, True, False],
[False, False, False],
[True, True, True]])
Then, you are still comparing a 2d tensor to a 1d tensor, which is ambiguous in an if statement, an easy way to fix this would be to write a for loop, compare each row of the predictions to the conditions and append the result to the new_pred list. Note that you will be comparing two booleans tensors with the size of three, therefore, you have to make sure the result of the comparison is True for all of the cells.
predictions = torch.tensor([[ True, False, False],
[False, False, True],
[False, True, True],
[ True, False, False]])
conditions = torch.tensor([[True,False,False],
[False,False,True],
[False,True,True]])
new_predict = []
for index in range(predictions.size(0)):
if (predictions[index] == conditions[0]).all():
new_predict.append(0)
# ...
Alternatively, you can use slicing to achieve your expected result without any for loop.
I need to find in which index in the large list where it match with the sub list.
c = np.array(close)
EMA50 = np.array(MA50)
sublist = [False,True,True]
biglist = (c-EMA50)/c>0.01
>>>array([False, False, False, False, False, False, False, False, False,
False, False, False, False, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, False, False, True, False, True, True, False, False,
True, False, False, False, False, False, False, False, True,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, True, True, True,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, True, True, True, True], dtype=bool)
>>>sublist in biglist
>>>False
I expected True but it return False.
The desired output is
index_loc = [12,31,68,112]
One very easy solution using the sublist pattern for slicing and checking for that signature and getting the indices would be -
np.flatnonzero(~a[:-2] & a[1:-1] & a[2:]) # a as data array
Explanation
Basically, we are slicing three slices out of the data array - One that starts at 0th index and goes until leaving out last two elements, another that starts at 1st index and ends at second last element and a third slice starting at 2nd and goes on until the end. These three slices correspond to the three elements of matching required against the sublist pattern that's - [False, True, True]. We need to make sure that the first one is False, in other words, let's make sure the negation of it is True. Negation in NumPy is achieved through ~ operator. So, in essence, we get the combined mask off those three slices and get the corresponding indices with np.flatnonzero.
For the given data results in -
In [79]: np.flatnonzero(~a[:-2] & a[1:-1] & a[2:])
Out[79]: array([ 12, 31, 68, 112])
in won't check for sub-arrays. Instead, it checks for elements.
You will have to do something like this:
(Using A for big array and b for sub-list for readability.)
n = len(b)
c = [i for i in xrange(len(A)-n+1) if (b==A[i:i+n]).all()]
c is the required list of indexes.
Explanation:
This is basic List Comprehension in python.
The idea is to create sub-arrays of the bigarray and check if it matches the sublist.
Breaking down the statement for better understanding:
c = []
for i in xrange(len(A)-n+1):
if (b==A[i:i+n]).all(): # if list and arrays match
c.append(i)
Is there a way to generate an array, masking a rectangular area, without the need to initialize an empty array first?
mask = np.zeros((10,10), dtype=bool)
mask[10/2:,10/2:] = True
Im looking for an numpy expression that would replace these two lines and generate mask
If you really, really want to have a one-liner, this is a possible way. But I'd say your original code is cleaner, clearer, and overall better...
>>> np.logical_and.outer(np.arange(10) >= 5, np.arange(6) >= 3)
array([[False, False, False, False, False, False],
[False, False, False, False, False, False],
[False, False, False, False, False, False],
[False, False, False, False, False, False],
[False, False, False, False, False, False],
[False, False, False, True, True, True],
[False, False, False, True, True, True],
[False, False, False, True, True, True],
[False, False, False, True, True, True],
[False, False, False, True, True, True]], dtype=bool)
That can be done in one line:
mask = np.fromfunction(lambda i, j: (i >= 5) * (j >= 5), (10, 10), dtype=int)
but this turns out to be quite a bit slower than the original implementation. (It creates two arrays holding row and column indices).
You can use np.empty if you don't want to initialize the array and set the values manually. In this specific case, however, you might be better off using np.zeros or np.ones and flip the values for whichever is the smallest number of entries that you need to update.
I am having a slight problem in getting numpy.any() to work fine on my problem.
Consider I have a 3D matrix of N X M X M matrix, where I need to get rid of any matrix MXM that has all its elements the same [all zeros to say].
Here is an example to illustrate my issue
x = np.arange(250).reshape(10,5,5)
x[0,:,:] = 0
What I need to do is get rid of the first 5X5 matrix since it contain all zeros.
So I tried with
np.any(x,axis=0)
and expected to have a results of
[FALSE,TRUE,TRUE,TRUE,TRUE,TRUE,TRUE,TRUE,TRUE,TRUE]
but what i get is
array([[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]], dtype=bool)
Applying the follwing results with what I want but I hope that there is a better way without any loops
for i in range(x.shape[0]):
y.append(np.any(x[i,:,:]))
Did I make a mistake somewhere here?
Thanks!
In a 10x5x5 matrix with x[0,:,:] = 0 I would expect a result of:
[False, True, True, True, True, True, True, True, True, True]
because it is the first of ten 5x5 arrays which is all zero and not of five.
You get this result using
x.any(axis=1).any(axis=1)
or
x.any(axis=2).any(axis=1)
which means you first eliminate the second (axis=1) or the third (asix=2) dimension and then the remaining second (axis=1) and you get the only one dimension, which was originally the first one (axis=0).