Develop Reference

Numpy dot product for group of rows

I am trying to calculate a dot product between two matrices, for each couple of rows.
I have matrix D with (u x 2) dimensions and matrix R with (u*2 x c) dimensions.
Below an example:
D = np.array([[0.02747092, 0.11233295],
[0.02747092, 0.07295284],
[0.01245856, 0.19935923],
[0.01245856, 0.13520913],
[0.11233295, 0.07295284]])
R = np.array([[-3. , 0. , 1. , -1. ],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[-2.33333333, -0.33333333, 1.66666667, -1.33333333],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[ 0. , -2. , 2. , -4. ],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[ 0.66666667, -3.33333333, 2.66666667, -4.33333333],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[-2.33333333, -0.33333333, 1.66666667, -1.33333333],
[-3. , 0. , 1. , -1. ]])
The result should be matrix M with dimensions (u x c) as follows (example of first row):
M = np.array([[-0.2185, 0.0825, 0.2195, -0.1645],
[...]])
Which is result of dot product between the first row of D and first two rows of matrix R as such:
D_ = np.array([[0.027, 0.11]])
R_ = np.array([[-3., 0., 1., -1.],
[-1.25, 0.75, 1.75, -1.25]])
D_.dot(R_)
I tried various ways of np.tensordot after reshaping the D matrix into tensor, but without any luck. I am looking for vectorized solution and to avoid loops (which is my current solution, quite slow).

Reshape R to 3D and use np.einsum -
np.einsum('ijk,ij->ik',R.reshape(len(D),2,-1),D)

In-line column assignments in Python/Numpy

I have a bunch of points and need to select a subset of them, add a value to the x coordinates and store the information in the original points.
I need to do it without loops or intermediate assignments.
import numpy as np
points=np.array([[100. , 100. , 100. ],
[ 0. , -2.75, 0. ],
[ 0. , -2.75, 5. ],
[ 0. , -1.9 , 3.15],
[ 0. , -1.9 , 3.35]])
then trying:
points[[3,4,0]][:,[0]]+=2
or
points[[3,4,0]][:,[0]]=points[[3,4,0]][:,[0]]+2
the original points variable does not change.
Any ideas? I suspect I am missing some stupid stuff...

If you are looking to edit first column of those rows use:
points[[3,4,0], 0] += 2
points
#[[ 102. 100. 100. ]
# [ 0. -2.75 0. ]
# [ 0. -2.75 5. ]
# [ 2. -1.9 3.15]
# [ 2. -1.9 3.35]]

Fill values in a numpy array given a condition

Currently I have an array as follows:
myArray = np.array(
[[ 976.77 , 152.95 , 105.62 , 53.44 , 0 ],
[ 987.61 , 156.63 , 105.53 , 51.1 , 0 ],
[1003.74 , 151.31 , 104.435, 52.86 , 0 ],
[ 968. , 153.41 , 106.24 , 58.98 , 0 ],
[ 978.66 , 152.19 , 103.28 , 57.97 , 0 ],
[1001.9 , 152.88 , 105.08 , 58.01 , 0 ],
[1024.93 , 146.59 , 107.06 , 59.94 , 0 ],
[1020.01 , 148.05 , 109.96 , 58.67 , 0 ],
[1034.01 , 152.69 , 107.64 , 59.74 , 0 ],
[ 0. , 154.88 , 102. , 58.96 , 0 ],
[ 0. , 147.46 , 100.69 , 54.95 , 0 ],
[ 0. , 149.7 , 102.439, 53.91 , 0 ]]
)
I would like the fill in the zeros in the first column with the previous last value (1034.01) however if the 0's start from index 0, for it to remain as 0.
Example of end result:
myArrayEnd = np.array(
[[ 976.77 , 152.95 , 105.62 , 53.44 , 0 ],
[ 987.61 , 156.63 , 105.53 , 51.1 , 0 ],
[1003.74 , 151.31 , 104.435, 52.86 , 0 ],
[ 968. , 153.41 , 106.24 , 58.98 , 0 ],
[ 978.66 , 152.19 , 103.28 , 57.97 , 0 ],
[1001.9 , 152.88 , 105.08 , 58.01 , 0 ],
[1024.93 , 146.59 , 107.06 , 59.94 , 0 ],
[1020.01 , 148.05 , 109.96 , 58.67 , 0 ],
[1034.01 , 152.69 , 107.64 , 59.74 , 0 ],
[1034.01 , 154.88 , 102. , 58.96 , 0 ],
[1034.01 , 147.46 , 100.69 , 54.95 , 0 ],
[1034.01 , 149.7 , 102.439, 53.91 , 0 ]]
)
I would like the code to be applicable to any array not just this one, where the situation may be different. (Column 3 might be all 0's and Column 4 might have 0's in the middle which should be filled with the last previous value).

Here's a vectorised way with pandas. This is also possible with numpy. In any case, you should not need explicit loops for this task.
import pandas as pd
import numpy as np
df = pd.DataFrame(myArray)\
.replace(0, np.nan)\
.ffill().fillna(0)
res = df.values
print(res)
[[ 976.77 152.95 105.62 53.44 0. ]
[ 987.61 156.63 105.53 51.1 0. ]
[ 1003.74 151.31 104.435 52.86 0. ]
[ 968. 153.41 106.24 58.98 0. ]
[ 978.66 152.19 103.28 57.97 0. ]
[ 1001.9 152.88 105.08 58.01 0. ]
[ 1024.93 146.59 107.06 59.94 0. ]
[ 1020.01 148.05 109.96 58.67 0. ]
[ 1034.01 152.69 107.64 59.74 0. ]
[ 1034.01 154.88 102. 58.96 0. ]
[ 1034.01 147.46 100.69 54.95 0. ]
[ 1034.01 149.7 102.439 53.91 0. ]]

Staying within numpy:
for k, c in enumerate(myArray.T):
idx = np.flatnonzero(c == 0)
if idx.size > 0 and idx[0] > 0:
myArray[idx, k] = myArray[idx[0] - 1, k]

Assuming I've understood you correctly, this should do the trick:
def fill_zeroes(array):
temp_array = array
for i in xrange(1, len(temp_array)):
if temp_array[i][0] == 0:
temp_array[i][0] = temp_array[i-1][0]
return temp_array

How about something like this (in psuedo code)?
for each col in array
for each row in col
if array[col,row] == 0 && row>0
array[col,row] = array[col,row-1]
edit: Combined with #ukemi, who has a quicker solution, but does not loop over the various columns. Also, you need to make sure to not try to index array[0][-1].

The code below requires testing:
values = myArray.to_list() # don't remember if nd_array.to_list is a method or property
result = []
last = None
for i,item in enumerate(values):
if i == 0 and item[0] == 0:
last = item
elif item[0] == 0 and last is not None:
item[0] = last
else:
last = item[0]
result.append(item)

How can I extract two separate matrices from a file?

So I have a file that looks something like this:
# 3 # Number of network ROIs
# 2 # Number of netcc matrices
# WITH_ROI_LABELS
001 002 003
1 2 3
# CC
1.0000 0.9800 0.9895
0.9800 1.0000 0.9817
0.9895 0.9817 1.0000
# FZ
4.0000 2.2965 2.6240
2.2965 4.0000 2.3426
2.6240 2.3426 4.0000
I want to extract the 3x3 matrix labelled "CC"
I want to extract the 3x3 matrix labelled "FZ"
So I did the following:
file=/users/3dfile1
A= numpy.genfromtxt(file)
m= A[:,:]
m
So the output I get looks like this:
array([[ 1. , 2. , 3. ],
[ 1. , 2. , 3. ],
[ 1. , 0.98 , 0.9895],
[ 0.98 , 1. , 0.9817],
[ 0.9895, 0.9817, 1. ],
[ 4. , 2.2965, 2.624 ],
[ 2.2965, 4. , 2.3426],
[ 2.624 , 2.3426, 4. ]])
However, my question is... if I have multiple files. Where the matrix size is NOT CONSISTENT. This means that in some files the matrix will be 3x3, some files 8x8, 1x1, etc. In this case, how can I code something that will:
differentiate the matrix CC from FZ
extract the matrix (can detect the size of matrix somehow and give me the exact matrix I'm looking for)

Try
import numpy as np
x = np.array([[ 1. , 2. , 3. ],
[ 1. , 2. , 3. ],
[ 1. , 0.98 , 0.9895],
[ 0.98 , 1. , 0.9817],
[ 0.9895, 0.9817, 1. ],
[ 4. , 2.2965, 2.624 ],
[ 2.2965, 4. , 2.3426],
[ 2.624 , 2.3426, 4. ]])
x1 = x[2:,:]
x2 = x1.reshape(2,3,3)
CC ,FZ = x2
Result:
In [23]: CC
Out[23]:
array([[ 1. , 0.98 , 0.9895],
[ 0.98 , 1. , 0.9817],
[ 0.9895, 0.9817, 1. ]])
In [24]: FZ
Out[24]:
array([[ 4. , 2.2965, 2.624 ],
[ 2.2965, 4. , 2.3426],
[ 2.624 , 2.3426, 4. ]])

matplotlib colormap giving duplicated values

Recently I was using this method to basically select 6 equally values along this colormap.
import matplotlib.pyplot as plt
import numpy as np
ints = np.linspace(0,255,6)
ints = [int(x) for x in ints]
newcm = plt.cm.Accent(ints)
Normally this would return the colormap values no problem. Now when I run this, the output I get for newcm is:
Out[25]:
array([[ 0.49803922, 0.78823529, 0.49803922, 1. ],
[ 0.4 , 0.4 , 0.4 , 1. ],
[ 0.4 , 0.4 , 0.4 , 1. ],
[ 0.4 , 0.4 , 0.4 , 1. ],
[ 0.4 , 0.4 , 0.4 , 1. ],
[ 0.4 , 0.4 , 0.4 , 1. ]])
So now things are not plotting right. I have also tried bytes=True but the behaviour is the same. Do others get the same result or is it some funny setting on my matplotlib that has gone awry?
Moreover - it seems this is happening in particular on the Accent colormap, but not necessarily others.

In general, a colormap ranges between 0 and 1. In np.linspace(0,255,6) all values except the first are larger than 1, hence you get the output corresponding to the maximum value 1 for all but the first item of that list.
If instead you use numbers = np.linspace(0,1,6), you will get 6 different values from that colormap.
import matplotlib.pyplot as plt
import numpy as np
numbers = np.linspace(0,1,6)
newcm = plt.cm.Accent(numbers)
print(newcm)
produces
[[ 0.49803922 0.78823529 0.49803922 1. ]
[ 0.74509804 0.68235294 0.83137255 1. ]
[ 1. 1. 0.6 1. ]
[ 0.21960784 0.42352941 0.69019608 1. ]
[ 0.74901961 0.35686275 0.09019608 1. ]
[ 0.4 0.4 0.4 1. ]]