In theano, once the sharedvarialbe is initialized in one function, it will never be initialized again even if the function is accessed repeatedly, am I right?
def sgd_updates_adadelta(params,cost,rho=0.95,epsilon=1e-6,norm_lim=9,word_vec_name='Words'):
updates = OrderedDict({})
exp_sqr_grads = OrderedDict({})
exp_sqr_ups = OrderedDict({})
gparams = []
for param in params:
empty = np.zeros_like(param.get_value())
exp_sqr_grads[param] = theano.shared(value=as_floatX(empty),name="exp_grad_%s" % param.name)
gp = T.grad(cost, param)
exp_sqr_ups[param] = theano.shared(value=as_floatX(empty), name="exp_grad_%s" % param.name)
gparams.append(gp)
In the code above, the exp_sqr_grads variable and the exp_sqr_ups variable will not be initialized with zeros again when the sgd_updates_adadelta function is called the second time?
Shared variables are not static, if that is what you mean. My understanding of your code:
import theano
import theano.tensor as T
global_list = []
def f():
a = np.zeros((4, 5), dtype=theano.config.floatX)
b = theano.shared(a)
global_list.append(b)
Copy and paste this into an IPython and then try:
f()
f()
print global_list
The list will contain two items. They are not the same object:
In [9]: global_list[0] is global_list[1]
Out[9]: False
And they don't refer to the same memory: Do
global_list[0].set_value(np.arange(20).reshape(4, 5).astype(theano.config.floatX))
Then
In [20]: global_list[0].get_value()
Out[20]:
array([[ 0., 1., 2., 3., 4.],
[ 5., 6., 7., 8., 9.],
[ 10., 11., 12., 13., 14.],
[ 15., 16., 17., 18., 19.]])
In [21]: global_list[1].get_value()
Out[21]:
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
Having established that initializing shared variables several times leads to different variables, here is how to update a shared variable using a function. We re-use the established shared variables:
s = global_list[1]
x = T.scalar(dtype=theano.config.floatX)
g = theano.function([x], [s], updates=[(s, T.inc_subtensor(s[0, 0], x))])
g now increments the top left value of s by x at every call:
In [7]: s.get_value()
Out[7]:
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
In [8]: g(1)
Out[8]:
[array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])]
In [9]: s.get_value()
Out[9]:
array([[ 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
In [10]: g(10)
Out[10]:
[array([[ 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])]
In [11]: s.get_value()
Out[11]:
array([[ 11., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
Related
I am new to CVXPY. I have been trying to solve an easy feasibility problem. Here is the code:
from cvxpy import *
import numpy as np
dim = np.shape(Bs[0])[1]
X = Variable(dim)
objective = Minimize(0)
constraintsA = [X.T * M * X + B * X + C == 0 for M, B, C in zip(Ms, Bs, Cs)]
constraintsB = [A * X - b == 0 for A, b in zip(As, bs)]
constraints = constraintsA + constraintsB
prob = Problem(objective, constraints)
result = prob.solve()
R = X.value
print R
Below, you can find the input matrices:
Ms = [array([[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.]]),
array([[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.]]),
array([[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0.]])]
Bs = [array([[-18., 0., 0., 18., 0., 0.]]),
array([[ 0., 0., 0., 0., -36., 0.]]),
array([[ 0., 0., -18., 0., 0., -18.]])]
Cs = [270.0, 540.0, 810.0]
As = [array([[ 0., -1., 1., 0., 0., 0.],
[ 0., 0., 0., 0., -1., 1.]]),
array([[-1., 1., 0., 0., 0., 0.],
[ 0., 0., 0., -1., 1., 0.]]),
array([[-1., 0., 1., 0., 0., 0.],
[ 0., 0., 0., -1., 0., 1.]])]
bs = [array([[ 0],
[ 1.00000000e+01]]),
array([[ 20.],
[ 10.]]),
array([[ -2.00000000e+01],
[ 0]])]
Running it only for constraintsA, the code is executed and returns back a result vector as expected. However, when considering the constraintsB set too, my result is None. Any ideas where I might have messed it up? Thanks a lot!
Say I have two matrices A and B. For example,
A = numpy.zeros((5,5))
B = np.eye(5)
Is there a way to append A and B?
It sounds to me like you're looking for np.hstack:
>>> import numpy as np
>>> a = np.zeros((5, 5))
>>> b = np.eye(5)
>>> np.hstack((a, b))
array([[ 0., 0., 0., 0., 0., 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 1.]])
np.vstack will work if you want to stack them downward:
>>> np.vstack((a, b))
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 1.]])
I have an empty 'numpy.ndarray' to update.
import numpy as np
my_grid = np.zeros((5, 5))
# stat
parse = "max","min","avg"
# create a dictionary for each element of parse
grid_stat = {}
for arg in parse:
grid_stat[arg] = my_grid
grid_stat
{'avg': array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]),
'max': array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]),
'min': array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])}
I wish to update with new value each grid in the dictionary (it will be part of a loop)
ex: on dy = 0, dx = 0, max= 100, min= 50, avg = 75
grid_stat
{'avg': array([[ 75., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]),
'max': array([[ 100., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]),
'min': array([[ 50., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])}
i tried an easy solution
grid_stat['avg'][0,0] = 100 but also for max and min the value updated is 100
grid_stat
{'avg': array([[ 100., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]),
'max': array([[ 100., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]]),
'min': array([[ 100., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])}
As I suggested in the comments to the last question, you probably want to use
for arg in parse:
grid_stat[arg] = my_grid.copy()
instead of
for arg in parse:
grid_stat[arg] = my_grid
sets each value of grid_stat to the very same array, the one called my_grid. It doesn't make three separate arrays of the same shape and contents. You can confirm this by using id or is:
>>> id(my_grid)
4325774752
>>> id(grid_stat['max'])
4325774752
>>> id(grid_stat['avg'])
4325774752
>>> id(grid_stat['min'])
4325774752
>>> my_grid is grid_stat['max']
True
>>> grid_stat['max'] is grid_stat['avg']
True
etc.
I am trying to use reduce() function to create a function hstack() which horizontally stacks multiple arrays. As a simple example, lets say
>>>>M=eye((4))
>>>>M
array([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 0.],
[ 0., 0., 0., 1.]])
>>>>hstack([M,M])
array([[ 1., 0., 0., 0., 1., 0., 0., 0.],
[ 0., 1., 0., 0., 0., 1., 0., 0.],
[ 0., 0., 1., 0., 0., 0., 1., 0.],
[ 0., 0., 0., 1., 0., 0., 0., 1.]])
This works as I want. Now I define
>>>> hstackm = lambda *args: reduce(hstack, args)
And try to do the hstack() from the previous case
>>>>hstackm([M,M])
[array([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 0.],
[ 0., 0., 0., 1.]]),
array([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 0.],
[ 0., 0., 0., 1.]])]
Which is incorrect. How do I define hstackm() to obtain a proper output?
My final objective will be to create a hstackm() function to stack SPARSE matrices if it is possible. Something like,
hstackm = lambda *args: reduce(sparse.hstack, args).
The _*args_ would be csr or _lil_matrix_
thank you
In [16]: hstackm = lambda args: reduce(lambda x,y:hstack((x,y)), args)
In [17]: hstackm([M,M])
Out[17]:
array([[ 1., 0., 0., 0., 1., 0., 0., 0.],
[ 0., 1., 0., 0., 0., 1., 0., 0.],
[ 0., 0., 1., 0., 0., 0., 1., 0.],
[ 0., 0., 0., 1., 0., 0., 0., 1.]])
Your function hstack takes one parameter, a list of matrices. reduce() calls it with two parameters instead, each a matrix.
Change your hstack method to accept an arbitrary number of arguments instead:
def hstack(*matrices):
....
instead of hstack(matrices), then call it as hstack(M, M).
I'm trying to do the following with numpy (python newbie here)
Create a zeroed matrix of the rigth dimensions
num_rows = 80
num_cols = 23
A = numpy.zeros(shape=(num_rows, num_cols))
Operate on the matrix
k = 5
numpy.transpose(A)
U,s,V = linalg.svd(A)
Extract sub-matrix
sk = s[0:(k-1), 0:(k-1)]
Results on error
Traceback (most recent call last):
File "tdm2svd.py", line 40, in <module>
sk = s[0:(k-1), 0:(k-1)]
IndexError: too many indices
What am I doing wrong?
to answer your question s is only a 1d array ... (even if you did actually transpose it ... which you did not)
>>> u,s,v = linalg.svd(A)
>>> s
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
>>>
for selecting a submatrix
I think this does what you want ... there may be a better way
>>> rows = range(10,15)
>>> cols = range(5,8)
>>> A[rows][:,cols]
array([[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.]])
or probably better
>>> A[15:32, 2:7]
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])