I have 2 matrices and I want to add headings to them. How can I add them and call each cell with the headings of rows and columns of them? And also I have to calculate the sum of each element of rows with the elements of columns. I did it but is there a better way (a loop) to do it instead of what I did? and it returns the headings of rows and columns with the sum (for example: xy:0.022)
Hint: the headings of rows must be different with the headings of columns.
And I prefer not to use pandas.
here is my code:
import numpy as np
T = np.array([0,0.012,0.054,0,1,0.03,0.08,0.14,0.02]).reshape(3,3)
#print (T)
#print ('----------------------------')
W = np.array([1,0,0.03,0.01,0.099,0.020,2,0,0.05]).reshape(3,3)
#print (W)
x = T[0][0] + W[0][0]
y = T[0][1] + W[1][0]
z = T[0][2] + W[2][0]
v = T[1][0] + W[0][1]
n = T[1][1] + W[1][1]
m = T[1][2] + W[2][1]
s = T[2][0] + W[0][2]
g = T[2][1] + W[1][2]
k = T[2][2] + W[2][2]
print (x, y, z, v, n, m, s, g, k)
This is one way to do it:
>>> import itertools
>>> T = np.array([0,0.012,0.054,0,1,0.03,0.08,0.14,0.02]).reshape(3,3)
>>> T
array([[ 0. , 0.012, 0.054],
[ 0. , 1. , 0.03 ],
[ 0.08 , 0.14 , 0.02 ]])
>>> W = np.array([1,0,0.03,0.01,0.099,0.020,2,0,0.05]).reshape(3,3)
>>> W
array([[ 1. , 0. , 0.03 ],
[ 0.01 , 0.099, 0.02 ],
[ 2. , 0. , 0.05 ]])
>>> Y = T + W.T # The additions you do can be expressed like this.
>>> Y
array([[ 1. , 0.022, 2.054],
[ 0. , 1.099, 0.03 ],
[ 0.11 , 0.16 , 0.07 ]])
>>> labels = map(lambda x:"".join(x), itertools.product(["x","y","z"], ["a","b","c"]))
>>> labels
['xa', 'xb', 'xc', 'ya', 'yb', 'yc', 'za', 'zb', 'zc']
>>> labels = np.array(labels).reshape((3,3))
>>> labels
array([['xa', 'xb', 'xc'],
['ya', 'yb', 'yc'],
['za', 'zb', 'zc']],
dtype='|S2')
>>> for (x,y), value in np.ndenumerate(Y):
... print labels[x,y], Y[x,y]
...
xa 1.0
xb 0.022
xc 2.054
ya 0.0
yb 1.099
yc 0.03
za 0.11
zb 0.16
zc 0.07
I am not sure on which format you want the headings, but I think this method makes sense.
Related
I am trying to calculate a dot product between two matrices, for each couple of rows.
I have matrix D with (u x 2) dimensions and matrix R with (u*2 x c) dimensions.
Below an example:
D = np.array([[0.02747092, 0.11233295],
[0.02747092, 0.07295284],
[0.01245856, 0.19935923],
[0.01245856, 0.13520913],
[0.11233295, 0.07295284]])
R = np.array([[-3. , 0. , 1. , -1. ],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[-2.33333333, -0.33333333, 1.66666667, -1.33333333],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[ 0. , -2. , 2. , -4. ],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[ 0.66666667, -3.33333333, 2.66666667, -4.33333333],
[-1.25 , 0.75 , 1.75 , -1.25 ],
[-2.33333333, -0.33333333, 1.66666667, -1.33333333],
[-3. , 0. , 1. , -1. ]])
The result should be matrix M with dimensions (u x c) as follows (example of first row):
M = np.array([[-0.2185, 0.0825, 0.2195, -0.1645],
[...]])
Which is result of dot product between the first row of D and first two rows of matrix R as such:
D_ = np.array([[0.027, 0.11]])
R_ = np.array([[-3., 0., 1., -1.],
[-1.25, 0.75, 1.75, -1.25]])
D_.dot(R_)
I tried various ways of np.tensordot after reshaping the D matrix into tensor, but without any luck. I am looking for vectorized solution and to avoid loops (which is my current solution, quite slow).
Reshape R to 3D and use np.einsum -
np.einsum('ijk,ij->ik',R.reshape(len(D),2,-1),D)
I have a matrix :
matrix = np.array([[[0,0.5,0.6],[0.9,1.2,0]],[[0,0.5,0.6],[0.9,1.2,0]]])
I want to replace all the values 0.55 < x < 0.95 by 0.55.
PS : My question is similar to this question. But the answer does not work in my case.
You can use np.where:
matrix = np.array([[[0,0.5,0.6],[0.9,1.2,0]],[[0,0.5,0.6],[0.9,1.2,0]]])
matrix[np.where((matrix > 0.55) & (matrix < 0.95))] = 0.55
# Or
# matrix[(matrix > 0.55) & (matrix < 0.95)] = 0.55
Output:
>>> matrix
array([[[0. , 0.5 , 0.55],
[0.55, 1.2 , 0. ]],
[[0. , 0.5 , 0.55],
[0.55, 1.2 , 0. ]]])
I'm trying to minimize a dot product of 2 vectors but it doesn't work and I have no idea why. Can someone please help me?
I have a matrix c of this form:
c = [[c11, c12, c13, c14, c15],
[c21, c22, c23, c24, c25]]
I want to get a matrix p of this form:
p = [[p11, p12, p13, p14, p15],
[p21, p22, p23, p24, p25]]
I want to maximize this value :
c11*p11 + c12*p12 +c13*p13 + c14*p14 + c15*p15 + c21*p21 + c22*p22 +c23*p23 + c24*p24 + c25*p25
To get that I convert the c and p to 1-D vector and do the dot product so that my function to maximize is:
f(p) = c.dot(p)
The constraints are:
c11 + c12 + c13 + c14 + c15 = 1
c21 + c22 + c23 + c24 + c25 = 1
every element in p must be between 0.01 and 0.99.
I have tried scipy.optimize.linprog and it works:
from scipy.optimize import linprog
c = np.array([0. , 0. , 0. , 0. , 0. , 0. , 20094.21019108, 4624.08079143, 6625.51724138, 3834.81081081])
A_eq = np.array([[1,1,1,1,1,0,0,0,0,0],
[0,0,0,0,0,1,1,1,1,1]])
b_eq = np.array([1, 1])
res = linprog(-c, A_eq=A_eq, b_eq=b_eq, bounds=(0.01, 0.99))
res
Out[561]:
fun: -19441.285871873002
message: 'Optimization terminated successfully.'
nit: 13
slack: array([0.03, 0.98, 0.98, 0.98, 0.98, 0.98, 0.03, 0.98, 0.98, 0.98, 0. ,
0. , 0.95, 0. , 0. , 0. , 0. , 0. , 0. , 0. ])
status: 0
success: True
x: array([0.96, 0.01, 0.01, 0.01, 0.01, 0.01, 0.96, 0.01, 0.01, 0.0
But I'm trying to use scipy.optimize.minimize with SLSQP instead and that's where I get this 'Singular matrix C in LSQ subproblem' . Here is what I've done:
from scipy.optimize import minimize
def build_objective(ck, sign = -1.00):
"""
Builds the objective fuction for matrix ck
"""
# Here I turn my c matrix to a 1-D matrix
ck = np.concatenate(ck)
def objective(P):
return sign*(ck.dot(P))
return objective
def build_constraint_rows(ck):
"""
Builds the constraint functions that specify that the sum of the proportions for
each bin equals 1
"""
ncol = ck.shape[1]
nrow = ck.shape[0]
constrain_dict = []
for i in range(nrow):
vector = np.zeros((nrow,ncol))
vector[i, :] = 1
vector = np.concatenate(vector)
def row_constrain(P):
return 1 - vector.dot(P)
constrain_dict.append({'type': 'eq', 'fun': row_constrain})
return constrain_dict
# Matrix: Notice that this is not in vector form yet
c = np.array([[0. , 0. , 0. , 0., 0.],
[0. , 20094.21019108, 4624.08079143, 6625.51724138, 3834.81081081]])
# I need some initial p matrix for the function 'minimize'. I look for the value of the row that is the highest and assign it a proportion p of 0.96 and the rest 0.01 so the sum in 1 per row
P_initial = np.ones(c.shape)*0.01
nrow = test.shape[0]
for i in range(nrow):
index= np.where(c[i,] == np.max(c[i,]))[0]
if index.shape[0] > 1:
index = int(np.random.choice(index, size = 1))
else:
index = int(index)
P_initial[i,index] = 0.96
# I turn the P_initial to vector form
P_initial = np.concatenate(P_initial)
# These are the bounds of each p value
b = (0.01,0.99)
bnds = (b,)*c.size
# I then use my previous functions
objective_fun = build_objective(c)
cons = build_constraint_rows(c)
res = minimize(objective_fun,P_initial,method='SLSQP',\
bounds=bnds,constraints=cons)
This is my final result:
res
Out[546]:
fun: -19434.501741138763
jac: array([0. , 0.,0. , 0. ,0. , 0., -20094.21020508, -4624.08056641, -6625.51708984, -3834.81079102])
message: 'Singular matrix C in LSQ subproblem'
nfev: 24
nit: 2
njev: 2
status: 6
success: False
x: array([0.96 , 0.01 , 0.01 , 0.01 , 0.01 ,
0.01020202, 0.95962502, 0.01006926, 0.01001178, 0.01009192])
Please help me understand what I'm doing wrong.
Thank you in advanced,
Karol
I have two series namely train_1 and train_2,
import numpy as np
mean = 0
std = 1
num_samples = 4
train_1 = numpy.random.normal(mean, std, size=num_samples)
train_2 = numpy.random.normal(mean, std, size=num_samples)
I am entering this command:
X = np.array(train_1,train_2, dtype=float)
and taking this output:
array([[ 0.82561222, 0.95885746, 0.40454621, 1.37793967],
[ 0.93473674, -1.51716492, -0.56732792, 1.03333013]])
But, I would like these different series to match in ordered manner such this:
Y = np.array(([3,5], [5,1], [10,2], [6,1.5]), dtype=float)
Y
array([[ 3. , 5. ],
[ 5. , 1. ],
[ 10. , 2. ],
[ 6. , 1.5]])
I might be misunderstanding your question, but is this not simply the transpose?
X = np.array(train_1,train_2, dtype=float).T
Note the .T at the end. In this case X will have two columns, the first will be train_1, the second will be train_2.
and thanks in advance for the help.
Using Python (mostly numpy), I am trying to compute an upper-triangular matrix where each row "j" is the first j-terms of a geometric series, all rows using the same parameter.
For example, if my parameter is B (where abs(B)=<1, i.e. B in [-1,1]), then row 1 would be [1 B B^2 B^3 ... B^(N-1)], row 2 would be [0 1 B B^2...B^(N-2)] ... row N would be [0 0 0 ... 1].
This computation is key to a Bayesian Metropolis-Gibbs sampler, and so needs to be done thousands of times for new values of "B".
I have currently tried this two ways:
Method 1 - Mostly Vectorized:
B_Matrix = np.triu(np.dot(np.reshape(B**(-1*np.array(range(N))),(N,1)),np.reshape(B**(np.array(range(N))),(1,N))))
Essentially, this is the upper triangle part of a product of an Nx1 and 1xN set of matrices:
upper triangle ([1 B^(-1) B^(-2) ... B^(-(N-1))]' * [1 B B^2 B^3 ... B^(N-1)])
This works great for small N (algebraically it is correct), but for large N it errs out. And it produces errors out for B=0 (which should be allowed). I believe this is stemming from taking B^(-N) ~ inf for small B and large N.
Method 2:
B_Matrix = np.zeros((N,N))
B_Row_1 = B**(np.array(range(N)))
for n in range(N):
B_Matrix[n,n:] = B_Row_1[0:N-n]
So that just fills in the matrix row by row, but uses a loop which slows things down.
I was wondering if anyone had run into this before, or had any better ideas on how to compute this matrix in a faster way.
I've never posted on stackoverflow before, but didn't see this question anywhere, and thought I'd ask.
Let me know if there's a better place to ask this, and if I should provide anymore detail.
You could use scipy.linalg.toeplitz:
In [12]: n = 5
In [13]: b = 0.5
In [14]: toeplitz(b**np.arange(n), np.zeros(n)).T
Out[14]:
array([[ 1. , 0.5 , 0.25 , 0.125 , 0.0625],
[ 0. , 1. , 0.5 , 0.25 , 0.125 ],
[ 0. , 0. , 1. , 0.5 , 0.25 ],
[ 0. , 0. , 0. , 1. , 0.5 ],
[ 0. , 0. , 0. , 0. , 1. ]])
If your use of the array is strictly "read only", you can play tricks with numpy strides to quickly create an array that uses only 2*n-1 elements (instead of n^2):
In [55]: from numpy.lib.stride_tricks import as_strided
In [56]: def make_array(b, n):
....: vals = np.zeros(2*n - 1)
....: vals[n-1:] = b**np.arange(n)
....: a = as_strided(vals[n-1:], shape=(n, n), strides=(-vals.strides[0], vals.strides[0]))
....: return a
....:
In [57]: make_array(0.5, 4)
Out[57]:
array([[ 1. , 0.5 , 0.25 , 0.125],
[ 0. , 1. , 0.5 , 0.25 ],
[ 0. , 0. , 1. , 0.5 ],
[ 0. , 0. , 0. , 1. ]])
If you will modify the array in-place, make a copy of the result returned by make_array(b, n). That is, arr = make_array(b, n).copy().
The function make_array2 incorporates the suggestion #Jaime made in the comments:
In [30]: def make_array2(b, n):
....: vals = np.zeros(2*n-1)
....: vals[n-1] = 1
....: vals[n:] = b
....: np.cumproduct(vals[n:], out=vals[n:])
....: a = as_strided(vals[n-1:], shape=(n, n), strides=(-vals.strides[0], vals.strides[0]))
....: return a
....:
In [31]: make_array2(0.5, 4)
Out[31]:
array([[ 1. , 0.5 , 0.25 , 0.125],
[ 0. , 1. , 0.5 , 0.25 ],
[ 0. , 0. , 1. , 0.5 ],
[ 0. , 0. , 0. , 1. ]])
make_array2 is more than twice as fast as make_array:
In [35]: %timeit make_array(0.99, 600)
10000 loops, best of 3: 23.4 µs per loop
In [36]: %timeit make_array2(0.99, 600)
100000 loops, best of 3: 10.7 µs per loop