Related
How to index the arrays into the empty_array matrix using np. ix_?
please don,t get confused by other details, dict_A and dict_B are used just to calculate the index positions.
However, because my code is too long, that's why I just mentioned the value of 'arrays' directly below here( making an np.array), without mentioning it's background and how it came.
import numpy as np
dict_A = { 1:[1,2], 2:[2,3], 3:[3,4], 4:[4,5], 5:[5,6] }
dict_B = {1:[1,2], 2:[3,4], 3:[5,6], 4:[7,8], 5:[9,10], 6:[11,12] } # these are the values used for indexing the arrays in to K matrix
empty_array = np.zeros((40,40))
arrays = np.array([[[[ 1, -7, -1, 7 ], # five matrix obtained from a loop
[ -7, 3, 7, -3],
[-1, 7, 1, -7 ],
[ 7, -3, -7, 3]]],
[[[ 1, -6, -1, 6],
[ -6, 2, 6, -2],
[-1, 6, 1, -6],
[ 6, -2,-6, 2]]],
[[[ 1, -6, -5,-2],
[ 2, 0, 5, 9 ],
[-5, 1, 6, 8 ],
[-12, 1, 4, 5 ]]],
[[[ 2, 5, 4, 2],
[ -4, 5, 1, 7],
[7, -5, -2, 3],
[ 0, 2, 5, 3]]],
[[[ 3, 0, 2, 5],
[ -2, 6, 3, 1],
[-2, 5, 3, 5],
[ 2, 6, 12, 3 ]]]])
# this is just a small process by which i am combining values of 'dict_B' to create an 'index', for positioning arrays.
a_list = []
for i in dict_A:
index= []
a_list.append(index)
for j in dict_A[i]:
index.extend(dict_B[j])
print(index)
empty_array[np.ix_(index,index)] = empty_array[np.ix_(index,index)] + arrays
#ValueError: shape mismatch: value array of shape (5,1,4,4) could not be broadcast to indexing result of shape (4,4)
print(K)
I am using the matrix for the multiple sequence alignment and this is my score matrix which I got by running the alignment algorithm.
My matrix:
[
[0, 24, -5, 3, -3, -5],
[0, -4, 8, 1, 1],
[0, 13, 1, 2],
[0, -2, 5],
[0, 4],
[0]
]
Matrix I want to build:
[
[0, 24, -5, 3, -3, -5],
[24, 0, -4, 8, 1, 1],
[-5, -4, 0, 13, 1, 2],
[3, 8, 13, 0, -2, 5],
[-3, 1, 1, 2, 0, 4],
[-5, 1, 2, 5, 4, 0]
]
I am trying to create a symmetric matrix from the output I got in python without using NumPy and additional library. I have tried to implement using NumPy but I want to implement without using NumPy.
Try the following:
upper = [[0, 24, -5, 3, -3, -5], [0, -4, 8, 1, 1], [0, 13, 1, 2], [0, -2, 5], [0, 4], [0]]
n = len(upper) # 6: num of rows and cols (assuming square)
output = []
for i in range(n): # iterate over rows
row = [(upper[i][j - i] if j >= i else output[j][i]) for j in range(n)]
output.append(row)
print(output)
# [[0, 24, -5, 3, -3, -5], [24, 0, -4, 8, 1, 1], [-5, -4, 0, 13, 1, 2], [3, 8, 13, 0, -2, 5], [-3, 1, 1, -2, 0, 4], [-5, 1, 2, 5, 4, 0]]
Let say I have a NumPy array A of shape (66,5) and B of shape (100, 66, 5).
The elements of A will index the first dimension (axis=0) of B, where the values are from 0 to 99 (i.e. the first dimension of B is 100).
A =
array([[ 1, 0, 0, 1, 0],
[ 0, 2, 0, 2, 4],
[ 1, 7, 0, 5, 5],
[ 2, 1, 0, 1, 7],
[ 0, 7, 0, 1, 4],
[ 0, 0, 3, 6, 0]
.... ]])
For example, A[4,1] will take index 7 of the first dimension of B, index 4 of the second dimension of B and index 1 of the third dimension B.
What I wanted to is to produce array C of shape (66,5) where it contains the elements in B that are selected based on the elements in A.
You can use np.take_along_axis to do that:
import numpy as np
np.random.seed(0)
a = np.random.randint(100, size=(66, 5))
b = np.random.random(size=(100, 66, 5))
c = np.take_along_axis(b, a[np.newaxis], axis=0)[0]
# Test some element
print(c[25, 3] == b[a[25, 3], 25, 3])
# True
If I understand correctly, you are looking for advances indexing of first dimension of B. You can use np.indices to create the indices required for the other two dimensions of B and use advanced indexing:
idx = np.indices(A.shape)
C = B[A,idx[0],idx[1]]
Example:
B = np.random.rand(10,20,30)
A = np.array([[ 1, 0, 0, 1, 0],
[ 0, 2, 0, 2, 4],
[ 1, 7, 0, 5, 5],
[ 2, 1, 0, 1, 7],
[ 0, 7, 0, 1, 4],
[ 0, 0, 3, 6, 0]])
print(C[4,1]==B[7,4,1])
#True
Use the following (using functions of NumPy library):
print(A)
# array([[2, 0],
# [1, 1],
# [2, 0]])
print(B)
# array([[[ 5, 7],
# [ 0, 0],
# [ 0, 0]],
# [[ 1, 8],
# [ 1, 9],
# [10, 1]],
# [[12, 22],
# [ 2, 2],
# [ 2, 2]]])
temp = A.reshape(-1) + np.cumsum(np.ones([A.reshape(-1).shape[0]])*B.shape[0], dtype = 'int') - 3
C = B.swapaxes(0, 1).swapaxes(2, 1).reshape(-1)[temp].reshape(A.shape)
print(C)
# array([[12, 7],
# [ 1, 9],
# [ 2, 0]])
is there an easy way to merge let's say n spectra (i.e. arrays of shape (y_n, 2)) with varying lengths y_n into an array (or list) of shape (y_n_max, 2*x) by filling up y_n with zeros if it is
Basically I want to have all spectra next to each other.
For example
a = [[1,2],[2,3],[4,5]]
b = [[6,7],[8,9]]
into
c = [[1,2,6,7],[2,3,8,9],[4,5,0,0]]
Either Array or List would be fine. I guess it comes down to filling up arrays with zeros?
If you're dealing with native Python lists, then you can do:
from itertools import zip_longest
c = [a + b for a, b in zip_longest(a, b, fillvalue=[0, 0])]
You also could do this with extend and zip without itertools provided a will always be longer than b. If b could be longer than a, the you could add a bit of logic as well.
a = [[1,2],[2,3],[4,5]]
b = [[6,7],[8,9]]
b.extend([[0,0]]*(len(a)-len(b)))
[[x,y] for x,y in zip(a,b)]
Trying to generalize the other solutions to multiple lists:
In [114]: a
Out[114]: [[1, 2], [2, 3], [4, 5]]
In [115]: b
Out[115]: [[6, 7], [8, 9]]
In [116]: c
Out[116]: [[3, 4]]
In [117]: d
Out[117]: [[1, 2], [2, 3], [4, 5], [6, 7], [8, 9]]
In [118]: ll=[a,d,c,b]
zip_longest pads
In [120]: [l for l in itertools.zip_longest(*ll,fillvalue=[0,0])]
Out[120]:
[([1, 2], [1, 2], [3, 4], [6, 7]),
([2, 3], [2, 3], [0, 0], [8, 9]),
([4, 5], [4, 5], [0, 0], [0, 0]),
([0, 0], [6, 7], [0, 0], [0, 0]),
([0, 0], [8, 9], [0, 0], [0, 0])]
intertools.chain flattens the inner lists (or .from_iterable(l))
In [121]: [list(itertools.chain(*l)) for l in _]
Out[121]:
[[1, 2, 1, 2, 3, 4, 6, 7],
[2, 3, 2, 3, 0, 0, 8, 9],
[4, 5, 4, 5, 0, 0, 0, 0],
[0, 0, 6, 7, 0, 0, 0, 0],
[0, 0, 8, 9, 0, 0, 0, 0]]
More ideas at Convert Python sequence to NumPy array, filling missing values
Adapting #Divakar's solution to this case:
def divakars_pad(ll):
lens = np.array([len(item) for item in ll])
mask = lens[:,None] > np.arange(lens.max())
out = np.zeros((mask.shape+(2,)), int)
out[mask,:] = np.concatenate(ll)
out = out.transpose(1,0,2).reshape(5,-1)
return out
In [142]: divakars_pad(ll)
Out[142]:
array([[1, 2, 1, 2, 3, 4, 6, 7],
[2, 3, 2, 3, 0, 0, 8, 9],
[4, 5, 4, 5, 0, 0, 0, 0],
[0, 0, 6, 7, 0, 0, 0, 0],
[0, 0, 8, 9, 0, 0, 0, 0]])
For this small size the itertools solution is faster, even with an added conversion to array.
With an array as target we don't need the chain flattener; reshape takes care of that:
In [157]: np.array(list(itertools.zip_longest(*ll,fillvalue=[0,0]))).reshape(-1, len(ll)*2)
Out[157]:
array([[1, 2, 1, 2, 3, 4, 6, 7],
[2, 3, 2, 3, 0, 0, 8, 9],
[4, 5, 4, 5, 0, 0, 0, 0],
[0, 0, 6, 7, 0, 0, 0, 0],
[0, 0, 8, 9, 0, 0, 0, 0]])
Use the zip built-in function and the chain.from_iterable function from itertools. This has the benefit of being more type agnostic than the other posted solution -- it only requires that your spectra are iterables.
a = [[1,2],[2,3],[4,5]]
b = [[6,7],[8,9]]
c = list(list(chain.from_iterable(zs)) for zs in zip(a,b))
If you want more than 2 spectra, you can change the zip call to zip(a,b,...)
I have a list A = [1, 2, 3, ..., n] and want to repeatedly shift the list to get a list of lists. The first row should be A, the second row [2, 3, 4, ...], the third row [3, 4, 5, ...], until the last row [n, 0, 0, ...]. The missing elements in the last columns should be zeros. I was trying to put them individually, but n is >= 100 so manually padding the zeros would take long. How do I do this?
edit same question for numpy arrays, which is what I really have.
>>> a = [1, 2, 3, 4]
>>> [ a[i:] + i*[0] for i in range(len(a))]
[[1, 2, 3, 4], [2, 3, 4, 0], [3, 4, 0, 0], [4, 0, 0, 0]]
How it works
To get the i'th shifted list, we can use: a[i:] + i*[0]. The list comprehension does this repeatedly for all the i's that we need.
Using numpy
+ means something different for numpy arrays than it does for normal python lists. Consequently, the above code needs from tweaks to adapt it to numpy:
>>> import numpy as np
>>> a = np.arange(1, 5)
>>> [ np.concatenate((a[i:], np.zeros(i))) for i in range(len(a))]
[array([1, 2, 3, 4]),
array([2, 3, 4, 0]),
array([3, 4, 0, 0]),
array([4, 0, 0, 0])]
If you want the final result to be a numpy array:
>>> np.array([np.concatenate((a[i:], np.zeros(i))) for i in range(len(a)) ])
array([[ 1., 2., 3., 4.],
[ 2., 3., 4., 0.],
[ 3., 4., 0., 0.],
[ 4., 0., 0., 0.]])
Extra verbose for clarity:
import pprint
A = [1,2,3,4,5]
lists = []
lists.append(A)
for _ in range(len(A)):
last_list = lists[-1] # Grab the last list element from lists
new_list = last_list[1:] + [0] # (See below)
lists.append(new_list) # Add new_list to the end of lists
pprint.pprint(lists)
Output:
[[1, 2, 3, 4, 5],
[2, 3, 4, 5, 0],
[3, 4, 5, 0, 0],
[4, 5, 0, 0, 0],
[5, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]
The new_list = last_list[1:] + [0] just means take the "1th" through last element in last_list and concatenate it with a zero to form a new list called new_list.
Note I say "1th" because python lists are 0-indexed so "1th" is the second element.
A = [1, 2, 3, 4]
matrix = []
for i in range(len(A)):
row = A[i:]
row.extend([0 for i in range(len(A)-len(row))])
matrix.append(row)
print matrix
Output:
[[1, 2, 3, 4], [2, 3, 4, 0], [3, 4, 0, 0], [4, 0, 0, 0]]
The array that you are creating is known as a Hankel matrix. If you don't mind the dependency on scipy, you can use the function scipy.linalg.hankel to create the array with a single function call:
In [21]: from scipy.linalg import hankel
In [22]: A = [1, 2, 3, 4, 5]
In [23]: hankel(A)
Out[23]:
array([[1, 2, 3, 4, 5],
[2, 3, 4, 5, 0],
[3, 4, 5, 0, 0],
[4, 5, 0, 0, 0],
[5, 0, 0, 0, 0]])
If this is the only reason to use scipy, I wouldn't bother--it is a pretty heavy dependency. But if you are already using scipy, then you might as well take advantage of the convenience of hankel.
a = [0,1,2,3,4,5,6,7,8,9]
tam = a.__len__()
cont = 0
rdo = []
for item in a:
rdo.append(a[cont:tam]+(cont)*[0])
cont+=1
print rdo
Output:
[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [1, 2, 3, 4, 5, 6, 7, 8, 9, 0], [2, 3, 4, 5, 6, 7, 8, 9, 0, 0], [3, 4, 5, 6, 7, 8, 9, 0, 0, 0], [4, 5, 6, 7, 8, 9, 0, 0, 0, 0], [5, 6, 7, 8, 9, 0, 0, 0, 0, 0], [6, 7, 8, 9, 0, 0, 0, 0, 0, 0], [7, 8, 9, 0, 0, 0, 0, 0, 0, 0], [8, 9, 0, 0, 0, 0, 0, 0, 0, 0], [9, 0, 0, 0, 0, 0, 0, 0, 0, 0]]