How do I use the numpy accumulator and add functions to add arrays column wise to make a basic accumulator?
import numpy as np
a = np.array([1,1,1])
b = np.array([2,2,2])
c = np.array([3,3,3])
two_dim = np.array([a,b,c])
y = np.array([0,0,0])
for x in two_dim:
y = np.add.accumulate(x,axis=0,out=y)
return y
actual output: [1,2,3]
desired output: [6,6,6]
numpy glossary says the sum along axis argument axis=1 sums over rows: "we can sum each row of an array, in which case we operate along columns, or axis 1".
"A 2-dimensional array has two corresponding axes: the first running vertically downwards across rows (axis 0), and the second running horizontally across columns (axis 1)"
With axis=1 I would expect output [3,6,9], but this also returns [1,2,3].
Of Course! neither x nor y are two-dimensional.
What am I doing wrong?
I can manually use np.add()
aa = np.array([1,1,1])
bb = np.array([2,2,2])
cc = np.array([3,3,3])
yy = np.array([0,0,0])
l = np.add(aa,yy)
m = np.add(bb,l)
n = np.add(cc,m)
print n
and now I get the correct output, [6,6,6]
I think
two_dim.sum(axis=0)
# [6 6 6]
will give you what you want.
I don't think accumulate is what you're looking for as it provides a running operation, so, using add it would look like:
np.add.accumulate(two_dim)
[[1 1 1]
[3 3 3] # = 1+2
[6 6 6]] # = 1+2+3
reduce is more like what you've descibed:
np.add.reduce(two_dim)
[6 6 6]
Related
How can I select specific rows, where these rows equal to another row in another parallel array?
in other words; can I vectorize code? here p, y are ndarray withe the same shape
for inx,val in enumerate(p):
if val ==y[inx]:
pprob.append(1)
else:
pprob.append(0)
I just ran this in a python shell in Python 3.9.4
import numpy as np
x = np.array([1,2,3,4,5])
y = np.array([1,1,3,3,5])
matching_idx = np.where(x == y) # (array([0, 2, 4]),)
x[matching_idx] # array([1, 3, 5])
Seems like x[matching_idx] is what you want
The key to this is np.where(), explained here
import numpy as np
a = np.random.normal(size=(10, 5))
b = np.random.normal(size=(10, 5))
a[1] = b[1]
a[7] = b[7]
rows = np.all(a == b, axis=1).astype(np.int32)
rows = rows.tolist() # if you really want the result to be a list
print(rows)
Result as expected
[0 1 0 0 0 0 0 1 0 0]
If you could be dealing with more than two dimensions, change the following (works for 2d as well):
# this
np.all(a == b, axis=1)
# to this
np.all(a == b, axis=tuple(range(len(a.shape)))[1:])
I am attempting to iteratively create a block matrix within a for loop in python. Is there a way to use simple indexing in which the index corresponds to the matrix index instead of a scalar index. For example, imagine the following as two 2x2 matrices in a block matrix:
4 5 6 7
1 2 3 4
Is there a way to index the sub-matrices such that:
block_matrix[0,0] =
4 5
1 2
block_matrix[0,1] =
6 7
3 4
My end goal is to have a for loop to stack these. For example:
for i in range(3):
for j in range(3):
mat = single_matrix
block_matrix[i,j] = mat
block_matrix =
matrix_1_1 matrix_1_2 matrix_1_3
matrix_2_1 matrix_2_2 matrix_2_3
matrix_3_1 matrix_3_2 matrix_3_3
Using numpy with slicing would be one good way to go.
import numpy as np
block_matrix = np.zeros((9,9)) # shape (9,9)
mat = np.reshape(np.arange(9), (3,3)) # shape (3,3)
for i in range(3):
for j in range(3):
block_matrix[i*3:i*3+3,j*3:j*3+3] = mat
# block_matrix =
# mat mat mat
# mat mat mat
# mat mat mat
There, of course I just created a simple matrix of shape (3,3) and used it for all sub-parts of block_matrix but I hope you get the gist.
I believe the functions you want are numpy.reshape and numpy.swapaxes
https://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html
https://docs.scipy.org/doc/numpy/reference/generated/numpy.swapaxes.html
import numpy as np
a = np.array([[4,5,6,7],[1,2,3,4]])
b = np.reshape(a, (2,2,2), order="C")
c = np.swapaxes(b, 0, 1)
print(c)
Output:
[[[4 5]
[1 2]]
[[6 7]
[3 4]]]
Edit
Here is a version that should work for your case, including what the loop does:
import numpy as np
a = np.random.random((6,6))
b = np.reshape(a, (3,2,3,2), order="C")
c = np.swapaxes(b, 2, 1)
print(a)
print(c[0,1])
Output:
[[0.14413028 0.32553884 0.84321485 0.52101265 0.39548678 0.04210311]
[0.06844168 0.37270808 0.0523836 0.66408026 0.29857363 0.9086674 ]
[0.30052066 0.85342026 0.42354871 0.20516629 0.47962509 0.31865669]
[0.92307636 0.36024872 0.00109126 0.66277798 0.70634145 0.02647658]
[0.18408546 0.79832633 0.92462421 0.8060224 0.51464245 0.88324207]
[0.24439081 0.61620587 0.66114919 0.50045374 0.93085541 0.85732735]]
[[0.84321485 0.52101265]
[0.0523836 0.66408026]]
I need to write a list comprehension to create a vector twice the square of the middle column of a matrix. (My matrix x = [[1,2,3],[4,5,6],[7,8,9]].) Problem is, I know how to extract the middle column BUT I don't know how to square it or double the square. Any help would be greatly appreciated (...still learning but trying my best)!
x = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(x)
z = [b[1] for b in x]
print(z)
To create a vector twice the square of the column:
import numpy as np
x = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(x)
with a list comprehension: (not recommended)
z = [2*b[1]**2 for b in x]
print(z)
The output is a python list:
[8, 50, 128]
using numpy indexing: (recommended)
more info here
z = 2 * x[:,1] ** 2
print(z)
The output is a numpy array:
[ 8 50 128]
I have two matrices in NumPy. One is larger than the other. I want to insert the smaller 2D-array (randomly) into the bigger 2D-array where there are only zeros (so no actual information in the bigger one is lost). Example:
Big array:
[0 0 0 9]
[0 0 0 7]
[0 0 0 2]
[2 3 1 5]
Small array:
[3 3]
[3 3]
(Possible) result:
[3 3 0 9]
[3 3 0 7]
[0 0 0 2]
[2 3 1 5]
I think you can use 2D convolution to find the places where the small array b can go in the large array a. If you use scipy.signal.convolve2d with mode='valid' you only get locations where the small array 'fits'. I think using the abs of the arays gets around positive and negative values (in either array) canceling, but I haven't tested any of this very rigorously.
Here's what I did, using #CypherX's fill_a_with_b function for the fill step:
import numpy as np
import scipy.signal
# Your input data.
a = np.array([[0, 0, 0, 9],
[0, 0, 0, 7],
[0, 0, 0, 2],
[2, 3, 1, 5]])
b = np.ones((2, 2)) * 3
# Find places where b can go.
allowed = scipy.signal.convolve2d(np.abs(a), np.abs(b), mode='valid')
# Get these locations as (row, col) pairs.
coords = np.stack(np.where(allowed==0)).T
# Choose one of the locations at random.
choice = coords[np.random.randint(coords.shape[0])]
# Use #CypherX's 'fill' function.
def fill_a_with_b(a, b, pos=[0, 0]):
aa = a.copy()
aa[slice(pos[0], pos[0] + b.shape[0]),
slice(pos[1], pos[1] + b.shape[1])] = b.copy()
return aa
# Do the fill thing.
fill_a_with_b(a, b, choice)
This results in (for example)...
array([[0, 0, 0, 9],
[0, 3, 3, 7],
[0, 3, 3, 2],
[2, 3, 1, 5]])
I will try giving you an example. But it will be based on some assumptions:
You know that the 0's are spanned across a contiguous rectangular block.
There are no other zeros in a.
If you would like to fill in a non-contiguous block of zeroes or, there are zeros on the columns/rows where you have some other non-zero values, you would have to think of a more sophisticated solution.
Solution: Random Insertion of array b into array a where a==0
Assumption: we know that the places where a is zero are a contiguous set of positions with a rectangular shape.
Imports
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
%config InlineBackend.figure_format = 'svg' # 'svg', 'retina'
plt.style.use('seaborn-white')
Make Data
# Make a
shape = (5,5)
a = np.zeros(shape)
a[:,-1] = np.arange(shape[0]) + 10
a[-1,:] = np.arange(shape[1]) + 10
# Make b
b = np.ones((2,2))*2
Preprocess
Here we determine possible slot-positions of the top-left element of b on a.
# Get range of positions (rows and cols) where we have zeros
target_indices = np.argwhere(a==0)
minmax = np.array([target_indices.min(axis=0), target_indices.max(axis=0)])
# Define max position (index) of topleft element of b on a
maxpos = np.dot(np.array([-1,1]), minmax) + minmax[0,:] - (np.array(b.shape) -1)
# Define min position (index) of topleft element of b on a
minpos = minmax[0,:]
Make a list of Top Left Corner Positions of b on a
Th function get_rand_topleftpos() takes in minpos and maxpos for rows and columns on a that define possible slot-positions, and returns a randomly selected valid slot-position for size=1. I have used a size=20 to create quite a few valid random slot-positions and then select only the unique positions so we could then see them as images. If you need just one slot-position at a time, select size=1.
def get_rand_topleftpos(minpos, maxpos, size=1):
rowpos = np.random.randint(minpos[0], high=maxpos[0] + 1, size=size)
colpos = np.random.randint(minpos[1], high=maxpos[1] + 1, size=size)
pos = np.vstack([rowpos, colpos]).T
return (rowpos, colpos, pos)
# Make a few valid positions where the array b could be placed
rowpos, colpos, pos = get_rand_topleftpos(minpos, maxpos, size=20)
# Select the Unique combinations so we could visualize them only
pos = np.unique(pos, axis=0)
Place b on a and Make Figures
We make a custom defined function fill_a_with_b() to fill a with b at a certain postion on a. This position will accept the top-left cell of b.
def fill_a_with_b(a, b, pos = [0,0]):
aa = a.copy()
aa[slice(pos[0], pos[0] + b.shape[0]),
slice(pos[1], pos[1] + b.shape[1])] = b.copy()
return aa
# Make a few figures with randomly picked position
# for topleft position of b on a
if pos.shape[0]>6:
nrows, ncols = int(np.ceil(pos.shape[0]/6)), 6
else:
nrows, ncols = 1, pos.shape[0]
fig, axs = plt.subplots(nrows = nrows,
ncols = ncols,
figsize=(2.5*ncols,2.5*nrows))
for i, ax in enumerate(axs.flatten()):
if i<pos.shape[0]:
aa = fill_a_with_b(a, b, pos[i,:])
sns.heatmap(aa,
vmin=np.min(aa),
vmax=np.max(aa),
annot=True,
cbar=False,
square=True,
cmap = 'YlGnBu_r',
ax = ax
);
ax.set_title('TopLeftPos: {}'.format(tuple(pos[i,:])),
fontsize=9);
else:
ax.axis('off')
plt.tight_layout()
plt.show()
Results
With array a defined as:
shape = (5,5)
a = np.zeros(shape)
a[:,-1] = np.arange(shape[0]) + 10
a[-1,:] = np.arange(shape[1]) + 10
With array a defined as:
shape = (6,5)
a = np.zeros(shape)
a[:,0] = np.arange(shape[0]) + 10
a[:,-1] = np.arange(shape[0]) + 10
a[-1,:] = np.arange(shape[1]) + 10
How to get values of each row in a matrix according to the max and secondary value indexes which I got from another matrix in tensorflow? E.g. I have a matrix tensor A as
[[1,2,3],
[6,5,4],
[7,9,8]],
and a Matrix tensor B
[[10,11,12],
[13,14,15],
[16,17,18]].
Then I get the maximum value and secondary maximum value index vector form matrix A like
[[2,1],
[0,2],
[1,2]]
by using tf.nn_topk. Then I want to get the exatly value of matrix B from these index, which is
[[12,11],
[13,15],
[17,18]].
How should I do? it seems maybe tf.gather_nd can do this work, But I don't know how to give the generate the 2-D index for it.
So for this specific case this code returns the values.
It isn't vectorized
It isn't generic
It just creates a template for gather_nd like this.
[[0 1]
[0 2]
[1 2]
[1 0]
[2 2]
[2 1]]
Others may have more compact ideas.
import tensorflow as tf
A = tf.Variable([[10,11,12],
[13,14,15],
[16,17,18]], )
B = tf.Variable([[2,1],
[0,2],
[1,2]] )
sess = tf.Session()
sess.run(tf.global_variables_initializer())
indices = sess.run(B)
incre = tf.Variable(0)
template = tf.Variable(tf.zeros([6,2],tf.int32))
sess.run(tf.global_variables_initializer())
#There are 3 rows in the indices array
row = tf.gather( indices , [0,1,2])
for i in range(0, row.get_shape()[0] ) :
newrow = tf.gather(row, i)
exprow1 = tf.concat([tf.constant([i]), newrow[1:]], axis=0)
exprow2 = tf.concat([tf.constant([i]), newrow[:1]], axis=0)
template = tf.scatter_update(template, incre, exprow1)
template = tf.scatter_update(template, incre + 1, exprow2)
#Dataflow execution dependency is enforced.
with tf.control_dependencies([template]):
incre = tf.assign(incre,incre + 2)
print(sess.run(tf.gather_nd(A,template)))
Output is this.
[11 12 15 13 18 17]