Related
I have an array
xx = np.arange(24).reshape(2, 12)
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])
and I would like to reshape it, to obtain
array([[[ 0, 1, 2, 3],
[12, 13, 14, 15]],
[[ 4, 5, 6, 7],
[16, 17, 18, 19]],
[[ 8, 9, 10, 11],
[20, 21, 22, 23]]])
I can achieve it via
xx.T.reshape(3, 4, 2).transpose(0, 2, 1)
But it has to be transposed twice, which seems unnecessary to me. So could somebody confirm that this is the only way of doing it or provide more readable solution otherwise?
Thanks!
It is possible to do a single transpose:
data = np.arange(24).reshape(2, 12)
data = data.reshape(2, 3, 4).transpose(1, 0, 2)
Edit:
I checked this using itertools.permutations and itertools.product:
import itertools
import numpy as np
data = np.arange(24).reshape(2, 12)
desired_data = np.array([[[ 0, 1, 2, 3],
[12, 13, 14, 15]],
[[ 4, 5, 6, 7],
[16, 17, 18, 19]],
[[ 8, 9, 10, 11],
[20, 21, 22, 23]]])
shapes = [2, 3, 4]
transpose_dims = [0, 1, 2]
shape_permutations = itertools.permutations(shapes)
transpose_permutations = itertools.permutations(transpose_dims)
for shape, transpose in itertools.product(
list(shape_permutations),
list(transpose_permutations),
):
new_data = data.reshape(*shape).transpose(*transpose)
try:
np.allclose(new_data, desired_data)
except ValueError as e:
pass
else:
break
print(f"{shape=}, {transpose=}")
shape=(2, 3, 4), transpose=(1, 0, 2)
I would do it this way: first, generate two arrays (shown separated for the sake of decomposition):
xx.reshape(2, -1, 4)
# Output:
# array([[[ 0, 1, 2, 3],
# [ 4, 5, 6, 7],
# [ 8, 9, 10, 11]],
#
# [[12, 13, 14, 15],
# [16, 17, 18, 19],
# [20, 21, 22, 23]]])
From here, I would then stack along the second dimension in order to combine them like you want:
np.stack(xx.reshape(2, -1, 4), axis=1)
# Output:
# array([[[ 0, 1, 2, 3],
# [12, 13, 14, 15]],
#
# [[ 4, 5, 6, 7],
# [16, 17, 18, 19]],
#
# [[ 8, 9, 10, 11],
# [20, 21, 22, 23]]])
You'd avoid the transposition. Hopefully it's more readable, but in the end, that's highly subjective, right? '^^
To add on top of #Paul's answer, there is some speedup from removing one of the transpose. The time gain is of ~15%:
If I have a list:
lst = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
I would like to cast the above list into an array with the following arrangements of the elements:
array([[ 1, 2, 3, 7, 8, 9]
[ 4, 5, 6, 10, 11, 12]
[13, 14, 15, 19, 20, 21]
[16, 17, 18, 22, 23, 24]])
How do I do this or what is the best way to do this? Many thanks.
I have done this in a crude way below where I will just get all the sub-matrix and then concatenate all of them at the end:
np.array(results[arr.shape[0]*arr.shape[1]*0:arr.shape[0]*arr.shape[1]*1]).reshape(arr.shape[0], arr.shape[1])
array([[1, 2, 3],
[4, 5, 6]])
np.array(results[arr.shape[0]*arr.shape[1]*1:arr.shape[0]*arr.shape[1]*2]).reshape(arr.shape[0], arr.shape[1])
array([[ 7, 8, 9],
[ 10, 11, 12]])
etc,
But I will need a more generalized way of doing this (if there is one) as I will need to do this for an array of any size.
You could use the reshape function from numpy, with a bit of indexing :
a = np.arange(24)
>>> a
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23])
Using reshape and a bit of indexing :
a = a.reshape((8,3))
idx = np.arange(2)
idx = np.concatenate((idx,idx+4))
idx = np.ravel([idx,idx+2],'F')
b = a[idx,:].reshape((4,6))
Ouptut :
>>> b
array([[ 0, 1, 2, 6, 7, 8],
[ 3, 4, 5, 9, 10, 11],
[12, 13, 14, 18, 19, 20],
[15, 16, 17, 21, 22, 23]])
Here the tuple (4,6) passed to reshape indicates that you want your array to be 2 dimensional, and have 4 arrays of 6 elements. Those values can be computed.
Then we compute the index to set the correct order of the data. Obvisouly, this a complicated bit here. As I'm not sure what you mean by "any size of data", its difficult for me to give you a agnostic way to compute that index.
Obviously, if you are using a list and not an np.array, you might have to convert the list first, for example by using np.array(your_list).
Edit :
I'm not sure if this exactly what you are after, but this should work for any array evenly divisible by 6 :
def custom_order(size):
a = np.arange(size)
a = a.reshape((size//3,3))
idx = np.arange(2)
idx = np.concatenate([idx+4*i for i in range(0,size//(6*2))])
idx = np.ravel([idx,idx+2],'F')
b = a[idx,:].reshape((size//6,6))
return b
>>> custom_order(48)
array([[ 0, 1, 2, 6, 7, 8],
[ 3, 4, 5, 9, 10, 11],
[12, 13, 14, 18, 19, 20],
[15, 16, 17, 21, 22, 23],
[24, 25, 26, 30, 31, 32],
[27, 28, 29, 33, 34, 35],
[36, 37, 38, 42, 43, 44],
[39, 40, 41, 45, 46, 47]])
I am working in numpy and have a numpy array of the form;
[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12],
[13, 14, 15],
[16, 17, 18],
[19, 20, 21],
[22, 23, 24]]
I want to use only the reshape and transpose functions and obtain the following array:
[[ 1, 2, 3, 7, 8, 9, 13, 14, 15, 19, 20, 21],
[ 4, 5, 6, 10, 11, 12, 16, 17, 18, 22, 23, 24]]
Can this be done? I have spent hours trying and am starting to think it just can't be done - am I missing something obvious?
You can reshape into columns, then transpose, then reshape with something like:
a = np.array([[ 1,2,3],
[ 4,5,6],
[ 7,8,9],
[10, 11, 12],
[13, 14, 15],
[16, 17, 18],
[19, 20, 21],
[22, 23, 24]])
a.reshape(-1, 2, 3).transpose((1, 0, 2)).reshape(2, -1)
# array([[ 1, 2, 3, 7, 8, 9, 13, 14, 15, 19, 20, 21],
# [ 4, 5, 6, 10, 11, 12, 16, 17, 18, 22, 23, 24]])
You may try to slice odd and even and pass them to np.reshape.
a_out = np.reshape([a[::2], a[1::2]], (2,-1))
Out[81]:
array([[ 1, 2, 3, 7, 8, 9, 13, 14, 15, 19, 20, 21],
[ 4, 5, 6, 10, 11, 12, 16, 17, 18, 22, 23, 24]])
First time i ask something here. I am kind of 'blocked'.
I have an array composed of n x n arrays (lets take n=3 for simplification):
[
[
[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]
],
[
[9, 10, 11],
[12, 13, 14],
[15, 16, 17]
],
[
[18, 19, 20],
[21, 22, 23],
[24, 25, 26]
]
]
(altho my array contains a lot more than 3 3*3 arrays)
i want to achieve a 2D array like this :
[0,1,2,9,10,11,18,19,20]
[3,4,5,12,13,14,21,22,23]
[6,7,8,15,16,17,24,25,26]
Is there a trick i haven't thought of because i cant think of any way to accomplish the transformation.
Thanks
Slightly cleaner than moveaxis maybe:
import numpy as np
a = np.arange(27).reshape(3,3,3)
a.swapaxes(0,1).reshape(3,-1)
array([[ 0, 1, 2, 9, 10, 11, 18, 19, 20],
[ 3, 4, 5, 12, 13, 14, 21, 22, 23],
[ 6, 7, 8, 15, 16, 17, 24, 25, 26]])
Think of this as a list of 3 arrays, that you want to join horizontally:
In [171]: arr = np.arange(27).reshape(3,3,3)
In [172]: np.hstack(arr)
Out[172]:
array([[ 0, 1, 2, 9, 10, 11, 18, 19, 20],
[ 3, 4, 5, 12, 13, 14, 21, 22, 23],
[ 6, 7, 8, 15, 16, 17, 24, 25, 26]])
In [173]: arr
Out[173]:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
Of I like to test ideas with arrays with differing dimensions. Then the various axis manipulations becomes more obvious.
In [174]: arr = np.arange(24).reshape(2,3,4)
In [175]: arr
Out[175]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
In [176]: np.hstack(arr)
Out[176]:
array([[ 0, 1, 2, 3, 12, 13, 14, 15],
[ 4, 5, 6, 7, 16, 17, 18, 19],
[ 8, 9, 10, 11, 20, 21, 22, 23]])
In [177]: np.vstack(arr)
Out[177]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]])
But there's nothing wrong with the variations on the transpose and reshape answers, if starting from a 3d array (as opposed to a list of arrays):
In [187]: arr.transpose(1,0,2).reshape(3,-1)
Out[187]:
array([[ 0, 1, 2, 9, 10, 11, 18, 19, 20],
[ 3, 4, 5, 12, 13, 14, 21, 22, 23],
[ 6, 7, 8, 15, 16, 17, 24, 25, 26]])
You could use np.block
>>> import numpy as np
>>> X = np.arange(27).reshape(3, 3, 3)
>>>
>>> np.block(list(X))
array([[ 0, 1, 2, 9, 10, 11, 18, 19, 20],
[ 3, 4, 5, 12, 13, 14, 21, 22, 23],
[ 6, 7, 8, 15, 16, 17, 24, 25, 26]])
A simple reshape doesn't suffice since you have to change the order of the axes first:
import numpy as np
a = np.array([[[0,1,2],[3,4,5],[6,7,8]],[[9,10,11],[12,13,14],[15,16,17]],[[18,19,20],[21,22,23],[24,25,26]]])
np.moveaxis(a, 0, 1).reshape(3,9)
[[ 0 1 2 9 10 11 18 19 20]
[ 3 4 5 12 13 14 21 22 23]
[ 6 7 8 15 16 17 24 25 26]]
I have a file in which I have some lines that looks like this:
[16, 1, 4, 15][0, 4, 5, 14][8, 9, 10, 3][2, 11, 12, 6][0, 1, 10, 11][1, 19, 12, 14][19, 3, 13, 15][9, 17, 14, 15][9, 2, 18, 17][8, 2, 13, 7][4, 2, 19, 12][16, 18, 3, 4][10, 3, 5, 15][16, 9, 18, 6][1, 19, 5, 7][0, 12, 6, 7][0, 17, 11, 13][16, 8, 18, 7][8, 17, 11, 13][10, 6, 5, 14]
and I want to read them, and make a list of lists with each line.
I've tried split() function but does not work.
What I've tried is:
file = open(filename, 'r')
string.split(',')
print(string[3])
But it returns ,, not [2, 11, 12, 6]
Any guesses? Thanks in advance!
You could use json and a list comprehension
import json
line = [16, 1, 4, 15][0, 4, 5, 14][8, 9, 10, 3][2, 11, 12, 6][0, 1, 10, 11][1, 19, 12, 14][19, 3, 13, 15][9, 17, 14, 15][9, 2, 18, 17][8, 2, 13, 7][4, 2, 19, 12][16, 18, 3, 4][10, 3, 5, 15][16, 9, 18, 6][1, 19, 5, 7][0, 12, 6, 7][0, 17, 11, 13][16, 8, 18, 7][8, 17, 11, 13][10, 6, 5, 14]
lst = [json.loads(sublist+']') for sublist in line.split(']') if sublist]
#[[16, 1, 4, 15],
# [0, 4, 5, 14],
# [8, 9, 10, 3],
# [2, 11, 12, 6],
# [0, 1, 10, 11],
# [1, 19, 12, 14],
# [19, 3, 13, 15],
# [9, 17, 14, 15],
# [9, 2, 18, 17],
# [8, 2, 13, 7],
# [4, 2, 19, 12],
# [16, 18, 3, 4],
# [10, 3, 5, 15],
# [16, 9, 18, 6],
# [1, 19, 5, 7],
# [0, 12, 6, 7],
# [0, 17, 11, 13],
# [16, 8, 18, 7],
# [8, 17, 11, 13],
# [10, 6, 5, 14]]
In this code, I split the line based on ']', this gives me a list of strings like '[16, 1, 4, 15', '[0, 4, 5, 14', ... Then for each of these strings, I add the ending bracket and use json to interpret it and transform it into a list.
In another website I get an answer which works (I don't know if it's a good way to do it, but it works for me).
A typical line in my file looks like this:
[16, 1, 4, 15][0, 4, 5, 14][8, 9, 10, 3][2, 11, 12, 6][0, 1, 10, 11][1, 19, 12, 14][19, 3, 13, 15][9, 17, 14, 15][9, 2, 18, 17][8, 2, 13, 7][4, 2, 19, 12][16, 18, 3, 4][10, 3, 5, 15][16, 9, 18, 6][1, 19, 5, 7][0, 12, 6, 7][0, 17, 11, 13][16, 8, 18, 7][8, 17, 11, 13][10, 6, 5, 14]
Is a string, not a list. And I want to make a list of lists from that str.
And my code now looks like this:
line=file.readline() # stores the str line from the file
line = '[' + line + ']'
line = line.replace('][', '],[')
line = ast.literal_eval(line)
Now I can access to each list within the (big) list, and each value in each list.
If you do not want to use an extra module, you can do it with list comprehensions and string split and strip methods:
[[int(s.strip()) for s in sublist.split(',')] for sublist in line[1:-1].split('][')]
#[[16, 1, 4, 15],
# [0, 4, 5, 14],
# [8, 9, 10, 3],
# [2, 11, 12, 6],
# [0, 1, 10, 11],
# [1, 19, 12, 14],
# [19, 3, 13, 15],
# [9, 17, 14, 15],
# [9, 2, 18, 17],
# [8, 2, 13, 7],
# [4, 2, 19, 12],
# [16, 18, 3, 4],
# [10, 3, 5, 15],
# [16, 9, 18, 6],
# [1, 19, 5, 7],
# [0, 12, 6, 7],
# [0, 17, 11, 13],
# [16, 8, 18, 7],
# [8, 17, 11, 13],
# [10, 6, 5, 14]]
s='[16, 1, 4, 15][0, 4, 5, 14][8, 9, 10, 3][2, 11, 12, 6][0, 1, 10, 11][1, 19, 12, 14][19, 3, 13, 15][9, 17, 14, 15][9, 2, 18, 17][8, 2, 13, 7][4, 2, 19, 12][16, 18, 3, 4][10, 3, 5, 15][16, 9, 18, 6][1, 19, 5, 7][0, 12, 6, 7][0, 17, 11, 13][16, 8, 18, 7][8, 17, 11, 13][10, 6, 5, 14]'
[l.split(',') for l in s[1:-1].split('][')]
If string is the line that you gave above, a 1-line solution using a list comprehension is:
[[int(s) for s in t.split(',')] for t in string.strip()[1:-1].split('][')]
Like thus:
>>> string = '[16, 1, 4, 15][0, 4, 5, 14][8, 9, 10, 3][2, 11, 12, 6][0, 1, 10, 11][1, 19, 12, 14][19, 3, 13, 15][9, 17, 14, 15][9, 2, 18, 17][8, 2, 13, 7][4, 2, 19, 12][16, 18, 3, 4][10, 3, 5, 15][16, 9, 18, 6][1, 19, 5, 7][0, 12, 6, 7][0, 17, 11, 13][16, 8, 18, 7][8, 17, 11, 13][10, 6, 5, 14]\n'
>>> [[int(s) for s in t.split(',')] for t in string.strip()[1:-1].split('][')]
[[16, 1, 4, 15], [0, 4, 5, 14], [8, 9, 10, 3], [2, 11, 12, 6], [0, 1, 10, 11], [1, 19, 12, 14], [19, 3, 13, 15], [9, 17, 14, 15], [9, 2, 18, 17], [8, 2, 13, 7], [4, 2, 19, 12], [16, 18, 3, 4], [10, 3, 5, 15], [16, 9, 18, 6], [1, 19, 5, 7], [0, 12, 6, 7], [0, 17, 11, 13], [16, 8, 18, 7], [8, 17, 11, 13], [10, 6, 5, 14]]
This last is clearly a list of lists of integers, and not a string, as the following output shows:
>>> [sum(nums) for nums in [[int(s) for s in t.split(',')] for t in string.strip()[1:-1].split('][')]]
[36, 23, 30, 31, 22, 46, 50, 55, 46, 30, 37, 41, 33, 49, 32, 25, 41, 49, 49, 35]
The string you are reading is almost ready:
it needs an opening "["
commas between each list
and a closing "]"
just modify your string, that you have in a variable string as I see from your question, and then parse it with json.loads, or ast.literal_eval
import json # or ast
parse = json.loads
# or
# parse = ast.literal_eval
new_string = parse("".join(["[", string.replace("][", "],["), "]"])