Related
This question already has answers here:
Numpy multiply arrays into matrix (outer product)
(2 answers)
Closed 4 years ago.
I'm trying to create a 2x2 array from a size 2 vector x by doing matrix multiplication like x * x^T:
>>> x = np.array([2, 2])
>>> x
array([2, 2])
>>> np.matmul(x,x.T)
8
As you can see, this fails. I came up with this solution:
>>> m = np.matrix(x)
>>> m
matrix([[2, 2]])
>>> m.T
matrix([[2],
[2]])
>>> np.matmul(m.T, m)
matrix([[4, 4],
[4, 4]])
Which achieves what I want. But is there a better way to do this, preferrably without resorting to using np.matrix?
EDIT: Creating a 2x1 vector is not an option because of the context outside the question.
Use np.outer:
np.outer(x, x)
# array([[4, 4],
# [4, 4]])
Alternatively, increase x's dimension by 1 before calling np.matmul:
x = x[:, None] # x = x.reshape(-1, 1)
x.shape
# (2, 1)
x # x.T # (2,1) . (1,2) => (2,2)
# array([[4, 4],
# [4, 4]])
If you reshaped x, you can use the # operator to do the multiplication:
x = np.array([2, 2])
Xprime = x.reshape(len(x), 1)
print(Xprime # Xprime.T)
#[[4 4]
# [4 4]]
np.array([2, 2]) doesn't create a 2x1 vector, it creates a 2 vector. If you want a 2x1 matrix, you need np.array([[2], [2]]). Or you can create a 1x2 matrix with np.array([[2, 2]]) and then do np.matmul(x.T,x)
You do not have a 2x1 vector here, but a 1D vector. You can see that with:
> x.shape
(2,)
To actually create a 2x1 vector, add bracets:
> x = np.array([[2, 2]])
> x.shape
(1,2)
And now you have what you want with:
> np.matmul(x.T,x)
array([[4, 4],
[4, 4]])
or x.T#x in Python3.
Let's say I have a row vector of the shape (1, 256). I want to transform it into a column vector of the shape (256, 1) instead. How would you do it in Numpy?
you can use the transpose operation to do this:
Example:
In [2]: a = np.array([[1,2], [3,4], [5,6]])
In [5]: a.shape
Out[5]: (3, 2)
In [6]: a_trans = a.T #or: np.transpose(a), a.transpose()
In [8]: a_trans.shape
Out[8]: (2, 3)
In [7]: a_trans
Out[7]:
array([[1, 3, 5],
[2, 4, 6]])
Note that the original array a will still remain unmodified. The transpose operation will just make a copy and transpose it.
If your input array is rather 1D, then you can promote the array to a column vector by introducing a new (singleton) axis as the second dimension. Below is an example:
# 1D array
In [13]: arr = np.arange(6)
# promotion to a column vector (i.e., a 2D array)
In [14]: arr = arr[..., None] #or: arr = arr[:, np.newaxis]
In [15]: arr
Out[15]:
array([[0],
[1],
[2],
[3],
[4],
[5]])
In [12]: arr.shape
Out[12]: (6, 1)
For the 1D case, yet another option would be to use numpy.atleast_2d() followed by a transpose operation, as suggested by ankostis in the comments.
In [9]: np.atleast_2d(arr).T
Out[9]:
array([[0],
[1],
[2],
[3],
[4],
[5]])
We can simply use the reshape functionality of numpy:
a=np.array([[1,2,3,4]])
a:
array([[1, 2, 3, 4]])
a.shape
(1,4)
b=a.reshape(-1,1)
b:
array([[1],
[2],
[3],
[4]])
b.shape
(4,1)
Some of the ways I have compiled to do this are:
>>> import numpy as np
>>> a = np.array([1, 2, 3], [2, 4, 5])
>>> a
array([[1, 2],
[2, 4],
[3, 5]])
Another way to do it:
>>> a.T
array([[1, 2],
[2, 4],
[3, 5]])
Another way to do this will be:
>>> a.reshape(a.shape[1], a.shape[0])
array([[1, 2],
[3, 2],
[4, 5]])
I have used a 2-dimensional array in all of these problems, the real problem arises when there is a 1-dimensional row vector which you want to columnize elegantly.
Numpy's reshape has a functionality where you pass the one of the dimension (number of rows or number of columns) you want, numpy can figure out the other dimension by itself if you pass the other dimension as -1
>>> a.reshape(-1, 1)
array([[1],
[2],
[3],
[2],
[4],
[5]])
>>> a = np.array([1, 2, 3])
>>> a.reshape(-1, 1)
array([[1],
[2],
[3]])
>>> a.reshape(2, -1)
...
ValueError: cannot reshape array of size 3 into shape (2,newaxis)
So, you can give your choice of 1-dimension without worrying about the other dimension as long as (m * n) / your_choice is an integer.
If you want to know more about this -1, head over to:
What does -1 mean in numpy reshape?
Note: All these operations return a new array and do not modify the original array.
You can use reshape() method of numpy object.
To transform any row vector to column vector, use
array.reshape(-1, 1)
To convert any column vector to row vector, use
array.reshape(1, -1)
reshape() is used to change the shape of the matrix.
So if you want to create a 2x2 matrix you can call the method like a.reshape(2, 2).
So why this -1 in the answer?
If you dont want to explicitly specify one dimension(or unknown dimension) and wants numpy to find the value for you, you can pass -1 to that dimension. So numpy will automatically calculate the the value for you from the ramaining dimensions. Keep in mind that you can not pass -1 to more than one dimension.
Thus in the first case(array.reshape(-1, 1)) the second dimension(column) is one(1) and the first(row) is unknown(-1). So numpy will figure out how to represent a 1-by-4 to x-by-1 and finds the x for you.
An alternative solutions with reshape method will be a.reshape(a.shape[1], a.shape[0]). Here you are explicitly specifying the diemsions.
Using np.newaxis can be a bit counterintuitive. But it is possible.
>>> a = np.array([1,2,3])
>>> a.shape
(3,)
>>> a[:,np.newaxis].shape
(3, 1)
>>> a[:,None]
array([[1],
[2],
[3]])
np.newaxis is equal to None internally. So you can use None.
But it is not recommended because it impairs readability
To convert a row vector into a column vector in Python can be important e.g. to use broadcasting:
import numpy as np
def colvec(rowvec):
v = np.asarray(rowvec)
return v.reshape(v.size,1)
colvec([1,2,3]) * [[1,2,3], [4,5,6], [7,8,9]]
Multiplies the first row by 1, the second row by 2 and the third row by 3:
array([[ 1, 2, 3],
[ 8, 10, 12],
[ 21, 24, 27]])
In contrast, trying to use a column vector typed as matrix:
np.asmatrix([1, 2, 3]).transpose() * [[1,2,3], [4,5,6], [7,8,9]]
fails with error ValueError: shapes (3,1) and (3,3) not aligned: 1 (dim 1) != 3 (dim 0).
I want to reduce the dimensions of an array after converting it to a list
a = np.array([[1,2],[3,4]])
print a.shape
b = np.array([[1],[3,4]])
print b.shape
Output:
(2, 2)
(2,)
I want a to have the same shape as b i.e. (2,)
>>> a = np.array([[1,2],[3,4], None])[:2]
>>> a
array([[1, 2], [3, 4]], dtype=object)
>>> a.shape
(2,)
Works, though is probably the wrong way to do it (I'm a numpy newb).
Do you understand what b is?
b = np.array([[1],[3,4]])
print(repr(b))
array([[1], [3, 4]], dtype=object)
b is a 1d array with 2 elements, each a list. np.array does this way because the 2 sublists have different length, so it can't create a 2d array.
a = np.array([[1,2],[3,4]])
print(repr(a))
array([[1, 2],
[3, 4]])
Here the 2 sublists have the same length, so it can create a 2d array. Each element is an integer. np.array tries to create the highest dimensional array that the input allows.
Probably the best way to create another array like b is to make a copy, and insert the desired lists.
a1 = b.copy()
a1[0] = [1,2]
# a1[1] = [3,4]
print(repr(a1))
array([[1, 2], [3, 4]], dtype=object)
You have to use this convoluted method because you trying to do something 'unnatural'.
You comment about using vstack. Both work:
In [570]: np.vstack((a,b)) # (3,2) array
Out[570]:
array([[1, 2],
[3, 4],
[[1], [3, 4]]], dtype=object)
In [571]: np.vstack((a1,b)) # (2,2) array
Out[571]:
array([[[1, 2], [3, 4]],
[[1], [3, 4]]], dtype=object)
Your array b is little more than the original list in an array wrapper. Is that really what you need? The 2d a is a normal numpy array. b is an oddball construction.
I have some trouble understanding what numpy's dstack function is actually doing. The documentation is rather sparse and just says:
Stack arrays in sequence depth wise (along third axis).
Takes a sequence of arrays and stack them along the third axis
to make a single array. Rebuilds arrays divided by dsplit.
This is a simple way to stack 2D arrays (images) into a single
3D array for processing.
So either I am really stupid and the meaning of this is obvious or I seem to have some misconception about the terms 'stacking', 'in sequence', 'depth wise' or 'along an axis'. However, I was of the impression that I understood these terms in the context of vstack and hstack just fine.
Let's take this example:
In [193]: a
Out[193]:
array([[0, 3],
[1, 4],
[2, 5]])
In [194]: b
Out[194]:
array([[ 6, 9],
[ 7, 10],
[ 8, 11]])
In [195]: dstack([a,b])
Out[195]:
array([[[ 0, 6],
[ 3, 9]],
[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]]])
First of all, a and b don't have a third axis so how would I stack them along 'the third axis' to begin with? Second of all, assuming a and b are representations of 2D-images, why do I end up with three 2D arrays in the result as opposed to two 2D-arrays 'in sequence'?
It's easier to understand what np.vstack, np.hstack and np.dstack* do by looking at the .shape attribute of the output array.
Using your two example arrays:
print(a.shape, b.shape)
# (3, 2) (3, 2)
np.vstack concatenates along the first dimension...
print(np.vstack((a, b)).shape)
# (6, 2)
np.hstack concatenates along the second dimension...
print(np.hstack((a, b)).shape)
# (3, 4)
and np.dstack concatenates along the third dimension.
print(np.dstack((a, b)).shape)
# (3, 2, 2)
Since a and b are both two dimensional, np.dstack expands them by inserting a third dimension of size 1. This is equivalent to indexing them in the third dimension with np.newaxis (or alternatively, None) like this:
print(a[:, :, np.newaxis].shape)
# (3, 2, 1)
If c = np.dstack((a, b)), then c[:, :, 0] == a and c[:, :, 1] == b.
You could do the same operation more explicitly using np.concatenate like this:
print(np.concatenate((a[..., None], b[..., None]), axis=2).shape)
# (3, 2, 2)
* Importing the entire contents of a module into your global namespace using import * is considered bad practice for several reasons. The idiomatic way is to import numpy as np.
Let x == dstack([a, b]). Then x[:, :, 0] is identical to a, and x[:, :, 1] is identical to b. In general, when dstacking 2D arrays, dstack produces an output such that output[:, :, n] is identical to the nth input array.
If we stack 3D arrays rather than 2D:
x = numpy.zeros([2, 2, 3])
y = numpy.ones([2, 2, 4])
z = numpy.dstack([x, y])
then z[:, :, :3] would be identical to x, and z[:, :, 3:7] would be identical to y.
As you can see, we have to take slices along the third axis to recover the inputs to dstack. That's why dstack behaves the way it does.
I'd like to take a stab at visually explaining this (even though the accepted answer makes enough sense, it took me a few seconds to rationalise this to my mind).
If we imagine the 2d-arrays as a list of lists, where the 1st axis gives one of the inner lists and the 2nd axis gives the value in that list, then the visual representation of the OP's arrays will be this:
a = [
[0, 3],
[1, 4],
[2, 5]
]
b = [
[6, 9],
[7, 10],
[8, 11]
]
# Shape of each array is [3,2]
Now, according to the current documentation, the dstack function adds a 3rd axis, which means each of the arrays end up looking like this:
a = [
[[0], [3]],
[[1], [4]],
[[2], [5]]
]
b = [
[[6], [9]],
[[7], [10]],
[[8], [11]]
]
# Shape of each array is [3,2,1]
Now, stacking both these arrays in the 3rd dimension simply means that the result should look, as expected, like this:
dstack([a,b]) = [
[[0, 6], [3, 9]],
[[1, 7], [4, 10]],
[[2, 8], [5, 11]]
]
# Shape of the combined array is [3,2,2]
Hope this helps.
Because you mention "images", I think this example would be useful. If you're using Keras to train a 2D convolution network with the input X, then it is best to keep X with the dimension (#images, dim1ofImage, dim2ofImage).
image1 = np.array([[4,2],[5,5]])
image2 = np.array([[3,1],[6,7]])
image1 = image1.reshape(1,2,2)
image2 = image2.reshape(1,2,2)
X = np.stack((image1,image2),axis=1)
X
array([[[[4, 2],
[5, 5]],
[[3, 1],
[6, 7]]]])
np.shape(X)
X = X.reshape((2,2,2))
X
array([[[4, 2],
[5, 5]],
[[3, 1],
[6, 7]]])
X[0] # image 1
array([[4, 2],
[5, 5]])
X[1] # image 2
array([[3, 1],
[6, 7]])
How do I convert a simple list of lists into a numpy array? The rows are individual sublists and each row contains the elements in the sublist.
If your list of lists contains lists with varying number of elements then the answer of Ignacio Vazquez-Abrams will not work. Instead there are at least 3 options:
1) Make an array of arrays:
x=[[1,2],[1,2,3],[1]]
y=numpy.array([numpy.array(xi) for xi in x])
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'numpy.ndarray'>
2) Make an array of lists:
x=[[1,2],[1,2,3],[1]]
y=numpy.array(x)
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'list'>
3) First make the lists equal in length:
x=[[1,2],[1,2,3],[1]]
length = max(map(len, x))
y=numpy.array([xi+[None]*(length-len(xi)) for xi in x])
y
>>>array([[1, 2, None],
>>> [1, 2, 3],
>>> [1, None, None]], dtype=object)
>>> numpy.array([[1, 2], [3, 4]])
array([[1, 2], [3, 4]])
As this is the top search on Google for converting a list of lists into a Numpy array, I'll offer the following despite the question being 4 years old:
>>> x = [[1, 2], [1, 2, 3], [1]]
>>> y = numpy.hstack(x)
>>> print(y)
[1 2 1 2 3 1]
When I first thought of doing it this way, I was quite pleased with myself because it's soooo simple. However, after timing it with a larger list of lists, it is actually faster to do this:
>>> y = numpy.concatenate([numpy.array(i) for i in x])
>>> print(y)
[1 2 1 2 3 1]
Note that #Bastiaan's answer #1 doesn't make a single continuous list, hence I added the concatenate.
Anyway...I prefer the hstack approach for it's elegant use of Numpy.
It's as simple as:
>>> lists = [[1, 2], [3, 4]]
>>> np.array(lists)
array([[1, 2],
[3, 4]])
Again, after searching for the problem of converting nested lists with N levels into an N-dimensional array I found nothing, so here's my way around it:
import numpy as np
new_array=np.array([[[coord for coord in xk] for xk in xj] for xj in xi], ndmin=3) #this case for N=3
The OP specified that "the rows are individual sublists and each row contains the elements in the sublist".
Assuming that the use of numpy is not prohibited (given that the flair numpy has been added in the OP), use vstack:
import numpy as np
list_of_lists= [[1, 2, 3], [4, 5, 6], [7 ,8, 9]]
array = np.vstack(list_of_lists)
# array([[1, 2, 3],
# [4, 5, 6],
# [7, 8, 9]])
or simpler (as mentioned in another answer),
array = np.array(list_of_lists)
As mentioned in the other answers, np.vstack() will let you convert your list-of-lists(nested list) into a 1-dimensional array of sublists. But if you are looking to convert the list of lists into a 2-dimensional numpy.ndarray. Then you can use the numpy.asarray() function.
For example, if you have a list of lists named y_true that looks like:
[[0, 1, 0], [1, 0, 0], [0, 0, 1], [1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 0, 0]]
<class 'list'>
This line y_true = np.asarray(y_true) will convert the list of lists into a 2-dimensional numpy ndarray that looks like:
[[0 1 0]
[1 0 0]
[0 0 1]
[1 0 0]
[0 1 0]
[0 0 1]
[1 0 0]]
<class 'numpy.ndarray'>
Additionally, you can also specify the dtype parameter like np.asarray(y_true, dtype = float) to have your array values in your desired data type.
I had a list of lists of equal length. Even then Ignacio Vazquez-Abrams's answer didn't work out for me. I got a 1-D numpy array whose elements are lists. If you faced the same problem, you can use the below method
Use numpy.vstack
import numpy as np
np_array = np.empty((0,4), dtype='float')
for i in range(10)
row_data = ... # get row_data as list
np_array = np.vstack((np_array, np.array(row_data)))
Just use pandas
list(pd.DataFrame(listofstuff).melt().values)
this only works for a list of lists
if you have a list of list of lists you might want to try something along the lines of
lists(pd.DataFrame(listofstuff).melt().apply(pd.Series).melt().values)