I have a 1d array of ids, for example:
a = [1, 3, 4, 7, 9]
Then another 2d array:
b = [[1, 4, 7, 9], [3, 7, 9, 1]]
I would like to have a third array with the same shape of b where each item is the index of the corresponding item from a, that is:
c = [[0, 2, 3, 4], [1, 3, 4, 0]]
What's a vectorized way to do that using numpy?
this may not make sense but ... you can use np.interp to do that ...
a = [1, 3, 4, 7, 9]
sorting = np.argsort(a)
positions = np.arange(0,len(a))
xp = np.array(a)[sorting]
fp = positions[sorting]
b = [[1, 4, 7, 9], [3, 7, 9, 1]]
c = np.rint(np.interp(b,xp,fp)) # rint is better than astype(int) because floats are tricky.
# but astype(int) should work faster for small len(a) but not recommended.
this should work as long as the len(a) is smaller than the largest representable int by float (16,777,217) .... and this algorithm is of O(n*log(n)) speed, (or rather len(b)*log(len(a)) to be precise)
Effectively, this solution is a one-liner. The only catch is that you need to reshape the array before you do the one-liner, and then reshape it back again:
import numpy as np
a = np.array([1, 3, 4, 7, 9])
b = np.array([[1, 4, 7, 9], [3, 7, 9, 1]])
original_shape = b.shape
c = np.where(b.reshape(b.size, 1) == a)[1]
c = c.reshape(original_shape)
This results with:
[[0 2 3 4]
[1 3 4 0]]
Broadcasting to the rescue!
>>> ((np.arange(1, len(a) + 1)[:, None, None]) * (a[:, None, None] == b)).sum(axis=0) - 1
array([[0, 2, 3, 4],
[1, 3, 4, 0]])
A and B matrices will be different when i run the program
A = np.array([[1, 1, 1], [2, 2, 2]])
B = np.array([[1, 1, 1], [2, 2, 2], [3, 3, 3]])
The output matrix (C) should be the same dimension as matrix A.
As title says, I'm trying to multiply each row from one matrix (A) to every rows to another matrix (B) and would like to sum them.
For example,
Dimension of C = (2,3)
C = [[A(0)*B(0) + A(1)*B(0)], [A(0)*B(1) + A(1)*B(1)],[A(0)*B(1) + A(1)*B(1)]]
I would like to know if there is a numpy function does that.
Use numpy broadcasting:
C = (A * B[:, None]).sum(axis=1)
Output:
>>> C
array([[3, 3, 3],
[6, 6, 6],
[9, 9, 9]])
>>> ndarr = np.array([0, 1, 2])
>>> (lambda x: x + 1) (ndarr)
array([1, 2, 3])
I see that it replaces every element with the function applied to it.
but when I do this to a two dimensional array:
>>> ndarr = np.array([[0, 1, 2], [3, 4, 5]])
>>> (lambda x: x[0]) (ndarr)
array([0, 1, 2])
I thought this would take the two elements of the array which are [0, 1, 2] and [3, 4, 5], apply the lambda to them resulting in 0 and 3, and the result would be [0, 3]. but this applies the function to the whole array instead. why? and wat do I do to get [0, 3]?
Applying + 1 on a numpy array will cause all elements to be incremented by one, as you have noticed.
However, this does not imply that every operation you perform using a numpy array will be a "mapping". In the second example, there is simply a nested list, and you select the first element of the outer list. The exact same would happen if you just used regular Python:
>>> nlist = [[0, 1, 2], [3, 4, 5]]
>>> (lambda x: x[0]) (nlist)
[0, 1, 2]
You are looking for a mapping if you want to apply an operation on each of the nested arrays. However, this problem can be solved with a slice most easily:
>>> ndarr = np.array([[0, 1, 2], [3, 4, 5]])
>>> ndarr[:,0]
array([0, 3])
I have two arrays of same size. In general dtype of these arrays is object (dtype = 'O'). What is the best way to access elements with same indicies from both arrays.
Possibility 1:
remove_indices = [i for i in range(len(array1)) if value in array1]
array1 = np.delete(array1, remove_indices, 0)
array2 = np.delete(array2, remove_indices, 0)
Possibility 2:
array3 = np.array([[array1[i], array2[i]] for i in range(len(array1))
if value not in array1[i]])
array1 = array3[:,0]
array2 = array3[:,1]
Note that Possibility 2 is faster. Is there any other solution with similar execution time (or faster)? How could I make Possiblity 2 more readable?
Not sure to understand well your examples, but sticking to What is the best way to access elements with same indicies from both arrays. make me think about zip. But using numpy why not using transpose ?
Like:
>>> array1 = numpy.array([0, 1, 2, 3, 4])
>>> array2 = numpy.array([5, 6, 7, 8, 9])
>>> numpy.array([array1, array2])
array([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]])
>>> numpy.array([array1, array2]).T
array([[0, 5],
[1, 6],
[2, 7],
[3, 8],
[4, 9]])
How does one add rows to a numpy array?
I have an array A:
A = array([[0, 1, 2], [0, 2, 0]])
I wish to add rows to this array from another array X if the first element of each row in X meets a specific condition.
Numpy arrays do not have a method 'append' like that of lists, or so it seems.
If A and X were lists I would merely do:
for i in X:
if i[0] < 3:
A.append(i)
Is there a numpythonic way to do the equivalent?
Thanks,
S ;-)
You can do this:
newrow = [1, 2, 3]
A = numpy.vstack([A, newrow])
What is X? If it is a 2D-array, how can you then compare its row to a number: i < 3?
EDIT after OP's comment:
A = array([[0, 1, 2], [0, 2, 0]])
X = array([[0, 1, 2], [1, 2, 0], [2, 1, 2], [3, 2, 0]])
add to A all rows from X where the first element < 3:
import numpy as np
A = np.vstack((A, X[X[:,0] < 3]))
# returns:
array([[0, 1, 2],
[0, 2, 0],
[0, 1, 2],
[1, 2, 0],
[2, 1, 2]])
As this question is been 7 years before, in the latest version which I am using is numpy version 1.13, and python3, I am doing the same thing with adding a row to a matrix, remember to put a double bracket to the second argument, otherwise, it will raise dimension error.
In here I am adding on matrix A
1 2 3
4 5 6
with a row
7 8 9
same usage in np.r_
A = [[1, 2, 3], [4, 5, 6]]
np.append(A, [[7, 8, 9]], axis=0)
>> array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
#or
np.r_[A,[[7,8,9]]]
Just to someone's intersted, if you would like to add a column,
array = np.c_[A,np.zeros(#A's row size)]
following what we did before on matrix A, adding a column to it
np.c_[A, [2,8]]
>> array([[1, 2, 3, 2],
[4, 5, 6, 8]])
If you want to prepend, you can just flip the order of the arguments, i.e.:
np.r_([[7, 8, 9]], A)
>> array([[7, 8, 9],
[1, 2, 3],
[4, 5, 6]])
If no calculations are necessary after every row, it's much quicker to add rows in python, then convert to numpy. Here are timing tests using python 3.6 vs. numpy 1.14, adding 100 rows, one at a time:
import numpy as np
from time import perf_counter, sleep
def time_it():
# Compare performance of two methods for adding rows to numpy array
py_array = [[0, 1, 2], [0, 2, 0]]
py_row = [4, 5, 6]
numpy_array = np.array(py_array)
numpy_row = np.array([4,5,6])
n_loops = 100
start_clock = perf_counter()
for count in range(0, n_loops):
numpy_array = np.vstack([numpy_array, numpy_row]) # 5.8 micros
duration = perf_counter() - start_clock
print('numpy 1.14 takes {:.3f} micros per row'.format(duration * 1e6 / n_loops))
start_clock = perf_counter()
for count in range(0, n_loops):
py_array.append(py_row) # .15 micros
numpy_array = np.array(py_array) # 43.9 micros
duration = perf_counter() - start_clock
print('python 3.6 takes {:.3f} micros per row'.format(duration * 1e6 / n_loops))
sleep(15)
#time_it() prints:
numpy 1.14 takes 5.971 micros per row
python 3.6 takes 0.694 micros per row
So, the simple solution to the original question, from seven years ago, is to use vstack() to add a new row after converting the row to a numpy array. But a more realistic solution should consider vstack's poor performance under those circumstances. If you don't need to run data analysis on the array after every addition, it is better to buffer the new rows to a python list of rows (a list of lists, really), and add them as a group to the numpy array using vstack() before doing any data analysis.
You can also do this:
newrow = [1,2,3]
A = numpy.concatenate((A,newrow))
import numpy as np
array_ = np.array([[1,2,3]])
add_row = np.array([[4,5,6]])
array_ = np.concatenate((array_, add_row), axis=0)
I use 'np.vstack' which is faster, EX:
import numpy as np
input_array=np.array([1,2,3])
new_row= np.array([4,5,6])
new_array=np.vstack([input_array, new_row])
I use numpy.insert(arr, i, the_object_to_be_added, axis) in order to insert object_to_be_added at the i'th row(axis=0) or column(axis=1)
import numpy as np
a = np.array([[1, 2, 3], [5, 4, 6]])
# array([[1, 2, 3],
# [5, 4, 6]])
np.insert(a, 1, [55, 66], axis=1)
# array([[ 1, 55, 2, 3],
# [ 5, 66, 4, 6]])
np.insert(a, 2, [50, 60, 70], axis=0)
# array([[ 1, 2, 3],
# [ 5, 4, 6],
# [50, 60, 70]])
Too old discussion, but I hope it helps someone.
If you can do the construction in a single operation, then something like the vstack-with-fancy-indexing answer is a fine approach. But if your condition is more complicated or your rows come in on the fly, you may want to grow the array. In fact the numpythonic way to do something like this - dynamically grow an array - is to dynamically grow a list:
A = np.array([[1,2,3],[4,5,6]])
Alist = [r for r in A]
for i in range(100):
newrow = np.arange(3)+i
if i%5:
Alist.append(newrow)
A = np.array(Alist)
del Alist
Lists are highly optimized for this kind of access pattern; you don't have convenient numpy multidimensional indexing while in list form, but for as long as you're appending it's hard to do better than a list of row arrays.
You can use numpy.append() to append a row to numpty array and reshape to a matrix later on.
import numpy as np
a = np.array([1,2])
a = np.append(a, [3,4])
print a
# [1,2,3,4]
# in your example
A = [1,2]
for row in X:
A = np.append(A, row)