I have a long 1D array. I'd like to create an array that is the result of np.arange() applied to each value in the array plus some constant. E.g if the constant = 3 and my array looks like
[1,2,3,4,5]
I'd like to get
[[1,2,3]
[2,3,4]
[3,4,5]
[4,5,6]
[5,6,7]]
np.arange() only accepts scalars as arguments. I played around with np.vectorize() a bit to no success. Clearly I could do this with a loop, or with lists and then convert to an array, but I was wondering if there's a good numpy-only solution.
You could use addition and broadcasting:
>>> x = np.array([1,2,3,4,5])
>>> constant = 3
>>> x[:,None] + np.arange(constant)
array([[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7]])
This could also be written as np.add.outer(x, np.arange(constant)).
Related
I have three 2D np.array that mathematically are [8:1550] matrices, and I want to express them into 1D np.array of 12400 numbers (8 x 1550 = 12400...) so that I could create a DataFrame later with this code:
Exported_Data = pd.DataFrame({"UD": UD_Data, "NS": NS_Data, "EW": EW_Data})
Exported_Data.to_csv("EXCEL.csv")
To put a simpler example, if I have this:
A = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
And I want to obtain this from that:
B = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9])
What is the best way to do it?
I would suggest use reshape. It most likely creates a view and is more efficient whereas np.flatten creates a copy:
B = A.reshape(-1)
-1 implicitly takes care of required dimension size.
You can use A.flatten() to convert a 2D array to a 1D array.
Consider 2D Numpy array A and in-place function x like
A = np.arange(9).reshape(3,3)
def x(M):
M[:,2] = 0
Now, I have a list (or 1D numpy array) L pointing the rows, I want to select and apply the function f on them like
L = [0, 1]
x(A[L, :])
where the output will be written to A. Since I used index access to A, the matrix A is not affected at all:
A = array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
What I actually need is to slice the matrix such as
x(A[:2, :])
giving me the desired output
A = array([[0, 1, 0],
[3, 4, 0],
[6, 7, 8]])
The question is now, how to provide Numpy array slicing by the list L (either any automatic conversion of list to slice or if there is any build in function for that), because I am not able to convert the list L easily to slice like :2 in this case.
Note that I have both large matrix A and list L in my problem - that is the reason, why I would need the in-place operations to control the available memory.
Can you modify the function so as you can pass slice L inside it:
def func(M,L):
M[L,2] = 0
func(A,L)
print(A)
Out:
array([[0, 1, 0],
[3, 4, 0],
[6, 7, 8]])
I know it is possible to use meshgrid to get all combinations between two arrays using numpy.
But in my case I have an array of two columns and n rows and another array that I would like to get the unique combinations.
For example:
a = [[1,1],
[2,2],
[3,3]]
b = [5,6]
# The expected result would be:
final_array = [[1,1,5],
[1,1,6],
[2,2,5],
[2,2,6],
[3,3,5],
[3,3,6]]
Which method is the fastest way to get this result using only numpy?
Proposed solution
Ok got the result, but I would like to know if this is a reliable and fast solution for this task, if someone could give me any advice I will appreciate.
a_t = np.tile(a, len(b)).reshape(-1,2)
b_t = np.tile(b, len(a)).reshape(1,-1)
final_array = np.hstack((a_t,b_t.T))
array([[1, 1, 5],
[1, 1, 6],
[2, 2, 5],
[2, 2, 6],
[3, 3, 5],
[3, 3, 6]])
Kind of ugly, but here's one way:
xx = np.repeat(a, len(b)).reshape(-1, a.shape[1])
yy = np.tile(b, a.shape[0])[:, None]
np.concatenate((xx, yy), axis=1)
I have a function that returns a list. I think I use np.append to add this list as a new line in an array, my intention is as follow:
list = 4 5 6
b = 1 2 3
b = np.append(b, list)
output;
1 2 3
4 5 6
This isn't the code I use (there's a lot of messing around in between). But the output I get is this:
2016-06-01 PRINT [ 99.86 99.928 99.9 99.875 99.8 89.7933
97.60018333 98.903 99.928 0.2801201 98.95 98.93
98.87 98.94 99.05 89.097 97.6712 98.87
99.59 0.23538903 99.711 99.732 99.725 99.724
99.769 89.777 98.12053333 99.68 99.88
0.30333219 99.805 99.79 99.743 99.71 99.69
89.7728 98.06653333 99.617 99.82 0.28981292
99.882 99.879 99.865 99.84 99.9 89.9206
98.29823333 99.82 100.08 0.31420778]
Is this a 10 column by 5 row array/matrix or is this a 50 column/row array? I feel like I'm missing something here - or is it just that the output doesn't really show the shape of the array?
True list append:
In [701]: alist = [4,5,6]
In [702]: b=[1,2,3]
In [703]: b.append(alist)
In [704]: b
Out[704]: [1, 2, 3, [4, 5, 6]]
bad array operation:
In [705]: anArray=np.array([4,5,6])
In [706]: b=np.array([1,2,3])
In [707]: b=np.append(b,anArray)
In [708]: b
Out[708]: array([1, 2, 3, 4, 5, 6])
In [709]: b.shape
Out[709]: (6,)
Here I just concatenated anArray onto b, making a longer array.
I've said this before - np.append is not a good function. It looks too much like the list append, and people end up misusing it. Either they miss the fact that it returns a new array, as opposed to modifying in-place. Or they use it repeatedly.
Here's the preferred way of collecting lists or arrays and joining them into one
In [710]: alist = []
In [711]: b=np.array([1,2,3]) # could be b=[1,2,3]
In [712]: alist.append(b)
In [713]: b=np.array([4,5,6]) # b=[4,5,6]
In [714]: alist.append(b)
In [715]: alist
Out[715]: [array([1, 2, 3]), array([4, 5, 6])]
In [716]: np.array(alist)
Out[716]:
array([[1, 2, 3],
[4, 5, 6]])
In [717]: _.shape
Out[717]: (2, 3)
The result is a 2d array. List append is much faster than array append (which is real array concatenate). Build the list and then make the array.
The most common way of defining a 2d array is with a list of lists:
In [718]: np.array([[1,2,3],[4,5,6]])
Out[718]:
array([[1, 2, 3],
[4, 5, 6]])
np.concatenate is another option for joining arrays and lists. If gives more control over how they are joined, but you have to pay attention to the dimensions of the inputs (you should pay attention to those anyways).
There are several 'stack' functions which streamline the dimension handling a bit, stack, hstack, vstack and yes, append. It's worth looking at their code.
you should use hstack or vstack
import numpy as np
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])
np.vstack((a,b))
gives
array([[1, 2, 3],
[4, 5, 6]])
or
np.hstack((a,b))
gives
array([1, 2, 3, 4, 5, 6])
I tried the following:
>>> a = np.array([1,2,3])
>>> b = np.array([4,5,6])
>>> np.concatenate((a,b), axis=0)
array([1, 2, 3, 4, 5, 6])
>>> np.concatenate((a,b), axis=1)
array([1, 2, 3, 4, 5, 6])
However, I'd expect at least that one result looks like this
array([[1, 2, 3],
[4, 5, 6]])
Why is it not concatenated vertically?
Because both a and b have only one axis, as their shape is (3), and the axis parameter specifically refers to the axis of the elements to concatenate.
this example should clarify what concatenate is doing with axis. Take two vectors with two axis, with shape (2,3):
a = np.array([[1,5,9], [2,6,10]])
b = np.array([[3,7,11], [4,8,12]])
concatenates along the 1st axis (rows of the 1st, then rows of the 2nd):
np.concatenate((a,b), axis=0)
array([[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11],
[ 4, 8, 12]])
concatenates along the 2nd axis (columns of the 1st, then columns of the 2nd):
np.concatenate((a, b), axis=1)
array([[ 1, 5, 9, 3, 7, 11],
[ 2, 6, 10, 4, 8, 12]])
to obtain the output you presented, you can use vstack
a = np.array([1,2,3])
b = np.array([4,5,6])
np.vstack((a, b))
array([[1, 2, 3],
[4, 5, 6]])
You can still do it with concatenate, but you need to reshape them first:
np.concatenate((a.reshape(1,3), b.reshape(1,3)))
array([[1, 2, 3],
[4, 5, 6]])
Finally, as proposed in the comments, one way to reshape them is to use newaxis:
np.concatenate((a[np.newaxis,:], b[np.newaxis,:]))
If the actual problem at hand is to concatenate two 1-D arrays vertically, and we are not fixated on using concatenate to perform this operation, I would suggest the use of np.column_stack:
In []: a = np.array([1,2,3])
In []: b = np.array([4,5,6])
In []: np.column_stack((a, b))
array([[1, 4],
[2, 5],
[3, 6]])
A not well known feature of numpy is to use r_. This is a simple way to build up arrays quickly:
import numpy as np
a = np.array([1,2,3])
b = np.array([4,5,6])
c = np.r_[a[None,:],b[None,:]]
print(c)
#[[1 2 3]
# [4 5 6]]
The purpose of a[None,:] is to add an axis to array a.
a = np.array([1,2,3])
b = np.array([4,5,6])
np.array((a,b))
works just as well as
np.array([[1,2,3], [4,5,6]])
Regardless of whether it is a list of lists or a list of 1d arrays, np.array tries to create a 2d array.
But it's also a good idea to understand how np.concatenate and its family of stack functions work. In this context concatenate needs a list of 2d arrays (or any anything that np.array will turn into a 2d array) as inputs.
np.vstack first loops though the inputs making sure they are at least 2d, then does concatenate. Functionally it's the same as expanding the dimensions of the arrays yourself.
np.stack is a new function that joins the arrays on a new dimension. Default behaves just like np.array.
Look at the code for these functions. If written in Python you can learn quite a bit. For vstack:
return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
Suppose you have 3 NumPy arrays (A, B, C). You can contact these arrays vertically like this:
import numpy as np
np.concatenate((A, B, C), axis=1)
np.shape