Is there a numpy solution that would allow you to initialize an array based on the following conditions?
Number of elements in axis 1. (In the example below you have 4 places in each element of the array)
Sum of values. (All elements sum to 8)
Step size. (Using increments of 2)
Essentially this shows all the combinations of 4 values you can add to achieve the wanted sum (8) at a step size of 2.
My experiments fail when I set the axis 1 dimension to over 6 and the sum to over 100.
There has to be a better way to do this than what I've been trying.
array([[0, 0, 0, 8],
[0, 0, 2, 6],
[0, 0, 4, 4],
[0, 0, 6, 2],
[0, 0, 8, 0],
[0, 2, 0, 6],
[0, 2, 2, 4],
[0, 2, 4, 2],
[0, 2, 6, 0],
[0, 4, 0, 4],
[0, 4, 2, 2],
[0, 4, 4, 0],
[0, 6, 0, 2],
[0, 6, 2, 0],
[0, 8, 0, 0],
[2, 0, 0, 6],
[2, 0, 2, 4],
[2, 0, 4, 2],
[2, 0, 6, 0],
[2, 2, 0, 4],
[2, 2, 2, 2],
[2, 2, 4, 0],
[2, 4, 0, 2],
[2, 4, 2, 0],
[2, 6, 0, 0],
[4, 0, 0, 4],
[4, 0, 2, 2],
[4, 0, 4, 0],
[4, 2, 0, 2],
[4, 2, 2, 0],
[4, 4, 0, 0],
[6, 0, 0, 2],
[6, 0, 2, 0],
[6, 2, 0, 0],
[8, 0, 0, 0]], dtype=int64)
Here is a small code that will enable you to loop over the desired combinations. It takes 3 parameter:
itsize: Number of elements.
itsum: Sum of values.
itstep: Step size.
It may be necessary to optimize it if the computations you do in the FOR loop are light. I loop over more combinations than necessary (all the i,j,k,l that take values in 0,itstep,2*itstep,...,itsum) and keep only those verifying the condition that all sum up to itsum. The big size array is not computed and the rows are computed on-the-fly when iterating so you will not have the memory troubles:
class Combinations:
def __init__(self, itsize, itsum, itstep):
assert(itsum % itstep==0) # Sum is a multiple of step
assert(itsum >= itstep) # Sum bigger or equal than step
assert(itsize > 0) # Number of elements >0
self.itsize = itsize # Number of elements
self.itsum = itsum # Sum parameter
self.itstep = itstep # Step parameter
self.cvalue = None # Value of the iterator
def __iter__(self):
self.itvalue = None
return self
def __next__(self):
if self.itvalue is None: # Initialization of the iterator
self.itvalue = [0]*(self.itsize)
elif self.itvalue[0] == self.itsum: # We reached all combinations the iterator is restarted
self.itvalue = None
return None
while True: # Find the next iterator value
for i in range(self.itsize-1,-1,-1):
if self.itvalue[i]<self.itsum:
self.itvalue[i] += self.itstep
break
else:
self.itvalue[i] = 0
if sum(self.itvalue) == self.itsum:
break
return self.itvalue # Return iterator value
myiter = iter(Combinations(4,8,2))
for val in myiter:
if val is None:
break
print(val)
Output:
% python3 script.py
[0, 0, 0, 8]
[0, 0, 2, 6]
[0, 0, 4, 4]
[0, 0, 6, 2]
[0, 0, 8, 0]
[0, 2, 0, 6]
[0, 2, 2, 4]
[0, 2, 4, 2]
[0, 2, 6, 0]
[0, 4, 0, 4]
[0, 4, 2, 2]
[0, 4, 4, 0]
[0, 6, 0, 2]
[0, 6, 2, 0]
[0, 8, 0, 0]
[2, 0, 0, 6]
[2, 0, 2, 4]
[2, 0, 4, 2]
[2, 0, 6, 0]
[2, 2, 0, 4]
[2, 2, 2, 2]
[2, 2, 4, 0]
[2, 4, 0, 2]
[2, 4, 2, 0]
[2, 6, 0, 0]
[4, 0, 0, 4]
[4, 0, 2, 2]
[4, 0, 4, 0]
[4, 2, 0, 2]
[4, 2, 2, 0]
[4, 4, 0, 0]
[6, 0, 0, 2]
[6, 0, 2, 0]
[6, 2, 0, 0]
[8, 0, 0, 0]
I tried this out and also found that it slowed down significantly at that size. I think part of the problem is that the output array gets pretty large at that point. I'm not 100% sure my code is right, but the plot shows how the array size grows with condition 2 (sum of values in each row). I didn't do 100, but it looks like it would be about 4,000,000 rows
plot
Related
I have the following Matrix made of 0s and 1s which I want to identify its spots(elements with the value 1 and connected to eachothers).
M = np.array([[1,1,1,0,0,0,0,0,0,0,0],
[1,1,1,0,0,0,0,0,0,1,1],
[1,1,1,0,0,0,0,0,0,1,1],
[1,1,1,0,0,1,1,1,0,0,0],
[0,0,0,0,0,1,1,1,0,0,0],
[1,1,1,0,1,1,1,1,0,0,0],
[1,1,1,0,0,1,1,1,0,0,0],
[1,1,1,0,0,1,1,1,0,0,0]])
In the matrix there are four spots.
an example of my output should seem the following
spot_0 = array[(0,0),(0,1), (0,2), (1,0),(1,1), (1,2), (2,0),(2,1), (2,2), (3,0),(3,1), (3,2)]
Nbr_0 = 12
Top_Left = (0, 0)
and that is the same process for the other 3 spots
Does anyone know how can I identify each spot with the number of its elements and top_left element, using numpy functions ?
Thanks
You can use a connected component labeling to find the spots. Then, you can use np.max so to find the number of component and np.argwhere so to find the locations of each component. Here is an example:
# OpenCV provides a similar function
from skimage.measure import label
components = label(M)
# array([[1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],
# [1, 1, 1, 0, 0, 0, 0, 0, 0, 2, 2],
# [1, 1, 1, 0, 0, 0, 0, 0, 0, 2, 2],
# [1, 1, 1, 0, 0, 3, 3, 3, 0, 0, 0],
# [0, 0, 0, 0, 0, 3, 3, 3, 0, 0, 0],
# [4, 4, 4, 0, 3, 3, 3, 3, 0, 0, 0],
# [4, 4, 4, 0, 0, 3, 3, 3, 0, 0, 0],
# [4, 4, 4, 0, 0, 3, 3, 3, 0, 0, 0]])
for i in range(1, np.max(components)+1):
spot_i = np.argwhere(components == i)
Nbr_i = len(spot_i)
Top_Left_i = spot_i[0]
Note that Top_Left only make sense for a rectangular area. If they are not rectangular this point needs to be carefully defined.
Note also that this method is only efficient with few component. If there are many component, then it is better to replace the current loop by an iteration over the components array (in this case the output structure is stored in a list l and l[components[i,j]] is updated with the information found for all item location (i,j) of components). This last algorithm will be slow unless Numba/Cython are used to speed the process up.
You could use skimage.measure.label or other tools (for instance, OpenCV or igraph) to create labels for connected components:
#from #Jérôme's answer
from skimage.measure import label
components = label(M)
# array([[1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],
# [1, 1, 1, 0, 0, 0, 0, 0, 0, 2, 2],
# [1, 1, 1, 0, 0, 0, 0, 0, 0, 2, 2],
# [1, 1, 1, 0, 0, 3, 3, 3, 0, 0, 0],
# [0, 0, 0, 0, 0, 3, 3, 3, 0, 0, 0],
# [4, 4, 4, 0, 3, 3, 3, 3, 0, 0, 0],
# [4, 4, 4, 0, 0, 3, 3, 3, 0, 0, 0],
# [4, 4, 4, 0, 0, 3, 3, 3, 0, 0, 0]])
In the later part you could create a one-dimensional view of image, sort values of pixels and find dividing points of sorted label values:
components_ravel = components.ravel()
c = np.arange(1, np.max(components_ravel) + 1)
argidx = np.argsort(components_ravel)
div_points = np.searchsorted(components_ravel, c, sorter=argidx)
# Sorted label values are:
# [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
# 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
# 2, 2, 2, 2
# 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3
# 4, 4, 4, 4, 4, 4, 4, 4, 4
# So you find indices that divides these groups:
# [47, 59, 63, 79]
After that you could split array of indices that sorts your one-dimensional view of image at these points and convert them into two-dimensional ones:
spots = []
for n in np.split(argidx, div_points)[1:]: #in case there are no zeros, cancel `[1:]`
x, y = np.unravel_index(n, components.shape)
spots.append(np.transpose([x, y]))
It creates a list of spot coordinates of each group:
[array([[1, 0], [1, 2], [0, 2], [0, 1], [1, 1], [0, 0], [2, 2], [2, 1], [2, 0], [3, 2], [3, 1], [3, 0]]),
array([[2, 10], [1, 9], [2, 9], [1, 10]]),
array([[6, 5], [7, 5], [7, 6], [7, 7], [6, 7], [6, 6], [3, 5], [4, 6], [3, 6], [4, 5], [3, 7], [5, 7], [5, 6], [4, 7], [5, 5], [5, 4]]),
array([[5, 0], [5, 1], [5, 2], [6, 2], [7, 0], [6, 0], [6, 1], [7, 1], [7, 2]])]
Note that an order of pixels of each group is mixed. This is because np.argsort uses a sort which is not stable. You could fix it like so:
argidx = np.argsort(components_ravel, kind='stable')
In this case you'll get:
[array([[0, 0], [0, 1], [0, 2], [1, 0], [1, 1], [1, 2], [2, 0], [2, 1], [2, 2], [3, 0], [3, 1], [3, 2]]),
array([[1, 9], [1, 10], [2, 9], [2, 10]]),
array([[3, 5], [3, 6], [3, 7], [4, 5], [4, 6], [4, 7], [5, 4], [5, 5], [5, 6], [5, 7], [6, 5], [6, 6], [6, 7], [7, 5], [7, 6], [7, 7]]),
array([[5, 0], [5, 1], [5, 2], [6, 0], [6, 1], [6, 2], [7, 0], [7, 1], [7, 2]])]
num1 = [1,2,3,4,5]
num2 = [1,2,3,4,5]
arr1 = [[0]*(len(num2)+1)]*(len(num1)+1)
arr2 = [[0 for _ in range(len(num2)+1)] for _ in range(len(num1)+1)]
I get a different answer when I define arr1 and arr2.
Aren't arr1 and arr2 create the same 2D array?
They are not the same. arr1 is a list with (len(nums1)+1) references to the same list [0]*(len(nums2)+1). So when you modify an element in one of them, all references will see this change as well.
For example,
>>> arr1[0][0] += 1
>>> print(arr1)
[[1, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[1, 0, 0, 0, 0, 0]]
arr2 doesn't suffer from this problem because it has len(nums1)+1 distinct lists:
>>> arr2[0][0] += 1
>>> print(arr2)
[[1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]]
A better way to see the difference is to use a random number to fill the entries.
from random import randrange
num1 = [1,2,3,4,5]
num2 = [1,2,3,4,5]
arr1 = [[randrange(10)]*(len(nums2)+1)]*(len(nums1)+1)
arr2 = [[randrange(10) for _ in range(len(nums2)+1)] for _ in range(len(nums1)+1)]
print(arr1)
print(arr2)
The output is:
[[5, 5, 5, 5, 5, 5], [5, 5, 5, 5, 5, 5], [5, 5, 5, 5, 5, 5], [5, 5, 5, 5, 5, 5], [5, 5, 5, 5, 5, 5], [5, 5, 5, 5, 5, 5]]
[[7, 4, 2, 4, 0, 3], [7, 5, 1, 0, 1, 7], [4, 4, 1, 0, 2, 1], [2, 3, 6, 2, 6, 7], [6, 6, 6, 0, 3, 3], [0, 4, 5, 0, 6, 6]]
You can see that for the arr1, it populates every entry with the same number; while for arr2, the entries are all truly random. This is because arr1 is constructed by expanding a list of just one number, which is [5] here.
I have a problem. First let me show you the code:
def neon(l):
oc = 0; #naj ocena
tz = 0; #teraz ocena
#sprawdzanie poziomo
for x in range(len(l[0])):
for i in range(len(l[x])):
for y in range(i + 1,len(l[x])):
tz = l[x][i] + l[x][y] + (max(y - x, x - y) + 1) * 2;
if (tz > oc):
oc = tz;
pion = [[0] * len(l[0])] * len(l);
print(pion);
print("#######");
for i in range(len(pion)):
for y in range(len(pion[i])):
pion[i][y] = l[y][i];
print(pion);
neon([[1,2,1,2],[7,1,7,1],[1,1,1,1],[3,3,3,3]]);
The problem is that when i try to adress pion[i][y] instead of just changing that value from 0 to whatever the program changes the value in all of the inner arrays with the second index y. This is how it looks:
[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
#######
[[1, 7, 1, 3], [1, 7, 1, 3], [1, 7, 1, 3], [1, 7, 1, 3]]
[[2, 1, 1, 3], [2, 1, 1, 3], [2, 1, 1, 3], [2, 1, 1, 3]]
[[1, 7, 1, 3], [1, 7, 1, 3], [1, 7, 1, 3], [1, 7, 1, 3]]
[[2, 1, 1, 3], [2, 1, 1, 3], [2, 1, 1, 3], [2, 1, 1, 3]]
Instead, it should be:
[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
#######
[[1, 7, 1, 3], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
[[1, 7, 1, 3], [2, 1, 1, 3], [0, 0, 0, 0], [0, 0, 0, 0]]
[[1, 7, 1, 3], [2, 1, 1, 3], [1, 7, 1, 3], [0, 0, 0, 0]]
[[1, 7, 1, 3], [2, 1, 1, 3], [1, 7, 1, 3], [2, 1, 1, 3]]
Please help and thank you in advance.
This problem occurs due to the following reasons:
pionis a list of inner lists. Each element of pion just holds the reference to one inner list.
Because of the way you have created and initialized pion (using pion = [[0] * len(l[0])] * len(l)), all elements of pion hold references to the same inner list. So, in pion, instead of many distinct inner lists, you just have multiple references to a single inner list. In other words, pion[0], pion[1], pion[2], etc, are all references to the same inner list of zeros. Any modification that you make to this inner list, using a particular row index (eg, using the expression pion[3]), will be visible through all the other row indexes also, because, at all row indexes, you are just holding a reference to the same inner list.
To correct this, you need to create and initialize the list differently. Eg, if rows and cols are respectively the number of rows and columns, you could do something like this:
pion = [([0]*cols) for i in range(rows)]
is there an easy way to merge let's say n spectra (i.e. arrays of shape (y_n, 2)) with varying lengths y_n into an array (or list) of shape (y_n_max, 2*x) by filling up y_n with zeros if it is
Basically I want to have all spectra next to each other.
For example
a = [[1,2],[2,3],[4,5]]
b = [[6,7],[8,9]]
into
c = [[1,2,6,7],[2,3,8,9],[4,5,0,0]]
Either Array or List would be fine. I guess it comes down to filling up arrays with zeros?
If you're dealing with native Python lists, then you can do:
from itertools import zip_longest
c = [a + b for a, b in zip_longest(a, b, fillvalue=[0, 0])]
You also could do this with extend and zip without itertools provided a will always be longer than b. If b could be longer than a, the you could add a bit of logic as well.
a = [[1,2],[2,3],[4,5]]
b = [[6,7],[8,9]]
b.extend([[0,0]]*(len(a)-len(b)))
[[x,y] for x,y in zip(a,b)]
Trying to generalize the other solutions to multiple lists:
In [114]: a
Out[114]: [[1, 2], [2, 3], [4, 5]]
In [115]: b
Out[115]: [[6, 7], [8, 9]]
In [116]: c
Out[116]: [[3, 4]]
In [117]: d
Out[117]: [[1, 2], [2, 3], [4, 5], [6, 7], [8, 9]]
In [118]: ll=[a,d,c,b]
zip_longest pads
In [120]: [l for l in itertools.zip_longest(*ll,fillvalue=[0,0])]
Out[120]:
[([1, 2], [1, 2], [3, 4], [6, 7]),
([2, 3], [2, 3], [0, 0], [8, 9]),
([4, 5], [4, 5], [0, 0], [0, 0]),
([0, 0], [6, 7], [0, 0], [0, 0]),
([0, 0], [8, 9], [0, 0], [0, 0])]
intertools.chain flattens the inner lists (or .from_iterable(l))
In [121]: [list(itertools.chain(*l)) for l in _]
Out[121]:
[[1, 2, 1, 2, 3, 4, 6, 7],
[2, 3, 2, 3, 0, 0, 8, 9],
[4, 5, 4, 5, 0, 0, 0, 0],
[0, 0, 6, 7, 0, 0, 0, 0],
[0, 0, 8, 9, 0, 0, 0, 0]]
More ideas at Convert Python sequence to NumPy array, filling missing values
Adapting #Divakar's solution to this case:
def divakars_pad(ll):
lens = np.array([len(item) for item in ll])
mask = lens[:,None] > np.arange(lens.max())
out = np.zeros((mask.shape+(2,)), int)
out[mask,:] = np.concatenate(ll)
out = out.transpose(1,0,2).reshape(5,-1)
return out
In [142]: divakars_pad(ll)
Out[142]:
array([[1, 2, 1, 2, 3, 4, 6, 7],
[2, 3, 2, 3, 0, 0, 8, 9],
[4, 5, 4, 5, 0, 0, 0, 0],
[0, 0, 6, 7, 0, 0, 0, 0],
[0, 0, 8, 9, 0, 0, 0, 0]])
For this small size the itertools solution is faster, even with an added conversion to array.
With an array as target we don't need the chain flattener; reshape takes care of that:
In [157]: np.array(list(itertools.zip_longest(*ll,fillvalue=[0,0]))).reshape(-1, len(ll)*2)
Out[157]:
array([[1, 2, 1, 2, 3, 4, 6, 7],
[2, 3, 2, 3, 0, 0, 8, 9],
[4, 5, 4, 5, 0, 0, 0, 0],
[0, 0, 6, 7, 0, 0, 0, 0],
[0, 0, 8, 9, 0, 0, 0, 0]])
Use the zip built-in function and the chain.from_iterable function from itertools. This has the benefit of being more type agnostic than the other posted solution -- it only requires that your spectra are iterables.
a = [[1,2],[2,3],[4,5]]
b = [[6,7],[8,9]]
c = list(list(chain.from_iterable(zs)) for zs in zip(a,b))
If you want more than 2 spectra, you can change the zip call to zip(a,b,...)
I generated a lower triangular matrix, and I want to complete the matrix using the values in the lower triangular matrix to form a square matrix, symmetrical around the diagonal zeros.
lower_triangle = numpy.array([
[0,0,0,0],
[1,0,0,0],
[2,3,0,0],
[4,5,6,0]])
I want to generate the following complete matrix, maintaining the zero diagonal:
complete_matrix = numpy.array([
[0, 1, 2, 4],
[1, 0, 3, 5],
[2, 3, 0, 6],
[4, 5, 6, 0]])
Thanks.
You can simply add it to its transpose:
>>> m
array([[0, 0, 0, 0],
[1, 0, 0, 0],
[2, 3, 0, 0],
[4, 5, 6, 0]])
>>> m + m.T
array([[0, 1, 2, 4],
[1, 0, 3, 5],
[2, 3, 0, 6],
[4, 5, 6, 0]])
You can use the numpy.triu_indices or numpy.tril_indices:
>>> a=np.array([[0, 0, 0, 0],
... [1, 0, 0, 0],
... [2, 3, 0, 0],
... [4, 5, 6, 0]])
>>> irows,icols = np.triu_indices(len(a),1)
>>> a[irows,icols]=a[icols,irows]
>>> a
array([[0, 1, 2, 4],
[1, 0, 3, 5],
[2, 3, 0, 6],
[4, 5, 6, 0]])