I'm creating a code to run a perceptron algorithm and I can't create a random matrix the way I need it:
from random import choice
from numpy import array, dot, random
unit_step = lambda x: -1 if x < 0 else 1
import numpy as np
m=3 #this will be the number of rows
allys=[]
for j in range(m):
aa=np.random.rand(1,3)
tt=np.random.rand(3)
yy=dot(aa,tt)
ally = [aa, yy]
allys.append(ally)
print "allys", allys
w = random.rand(3)
errors = []
eta = 0.2
n = 10
x=[1,3]
for i in xrange(n):
print i
x, expected = choice(allys)
print "x", x
And I get the problem here:
result = dot(x,w)
error = expected - unit_step(result)
errors.append(error)
w += eta * error * x
print x, expected, w, error, result, errors
The log says
w += eta * error * x
ValueError: non-broadcastable output operand with shape (3,) doesn't
match the broadcast shape (1,3)
The idea is to get result looping randomly over the "table" allys.
How can I solve this? What is shape(3,)?
Thanks!
The error message actually tells you what is wrong. The result of your multiplication is a (1,3) array (2D array, one row, three columns), whereas you try to add it into a 3-element vector.
Both arrays have three elements in a row, so if you do this:
w = w + eta * error * x
there will be no error on that line, but the resulting vector will actually be a (1,3) array. That is unwanted, because then your dot does not work.
There are several ways to fix the problem, but possibly the easiest to read is to reshape x for the calculation to be a 3-element vector (1D array):
w += eta * error * x.reshape(3,)
Possibly a cleaner solution would be to define w as a (1,3) 2D array, as well, and then transpose w for the dot. This is really a matter of taste.
For numpy arrays, shape attribute returns array's dimensionality. In your case, w.shape is (3,). This means that w is a one-dimensional array with 3 elements. In turn, x.shape is (1,3), which means that x is a two-dimensional array with one row and 3 columns. You are getting an error, because the interpreter is confused on how to match the shapes. I am not sure what you are trying to do, so it's hard to suggest the solution. But you might want to try reshaping one of the arrays. For example, x = x.reshape((3,)) for adapting the shape of x to w.
Related
I am attemping to define y equal to (x × 0.15) + N where N is a 100 element 1D array of random values chosen from a Gaussian distribution with mean 0.0 and standard deviation 0.5 but I keep getting the error code : ValueError: operands could not be broadcast together with shapes (1,99) (100,) . Any tips on how I can revise my code to make it work would be much appreciated, thanks.
N = np.random.normal (0, 0.5,size=(100))
y = (np.dot([x],0.15)) + N
It would be helpful if you included the x array in your example above. Python is complaining that (np.dot([x],0.15)) has dimensions np.shape(1,99), which means that x has the same shape. You could x.flatten() to make it np.shape(99), but then you still have the problem that x has 99 elements, and N has 100, and python does not know how to add two arrays of dissimilar length.
Assuming this is somehow wrong, and that x should have had 100 elements, you could just do
x = np.arange(100) # Example x-array, use your own in stead
N = np.random.normal(0, 0.5,size=x.shape)
y = x*0.15 + N
since all (np.dot([x],0.15)) does in this case is to multiply each element of x with 0.15, which much simpler written as the above.
Earlier today I asked this about integer array indexing and I am having trouble taking the answer and applying it to the problem that inspired the question.
In a nutshell, p_stack1 and c_stack1 contain arrays that are derived from an image processing algorithm I am working on. p_stack1 contains probability data and c_stack1 contain integer classifications. I need to find the classification that has the highest probability for every pixel in the image with dimensions 768 x 1024. From the docs for integer array indexing, it provides a way to subset data from higher dimensional arrays using their integer indexes.
The solution of my original question works for a simplified example of n x n x n shaped arrays, but does not seem to work on l x m x n shaped arrays.
#dummy data
p_stack1 = np.reshape(np.random.uniform(0,1,2359296),(3,768,1024))
c_stack1 = np.reshape(np.random.randint(0,4,2359296),(3,768,1024))
#find where max value occurs on axis 0
ind_new=p_stack1.argmax(axis=0)
#Create assending indicies
nx, ny = 768,1024
xx = np.arange(ny)
aa= np.tile(xx,(ny,1))
bb = np.column_stack(tuple(aa))[:nx,:]
aa= np.tile(xx,(ny,1))[:nx,:]
#perform the integer array indexing
print(c_stack1[ind_new, aa,bb])
The last print statement returns the error:
IndexError: index 768 is out of bounds for axis 1 with size 768
I checked the shapes of aa and bb and both are (768, 1024)
What I am I missing?
Looks like you got your dimensions mixed up:
c_stack1.shape # (3, 768, 1024)
aa.max() # 1023
bb.max() # 767
So, when you run
c_stack1[ind_new, aa, bb]
you will be trying to index axis=1 with higher values than available, hence the error
Either turn around aa and bb, or else c_stack1[ind_new, bb, aa] will also do the trick
when I am trying to run this simple snippet of code
a= 2
G = np.random.rand(25,1)
H = np.zeros((25,a))
for i in range(a):
H[:,i] = .5 * G
I receive the
ValueError: could not broadcast input array from shape (25,1) into shape (25).
I wonder if anyone can point at a solution to this problem?
I know it happens quite a bit in image processing, but this one, I don't knwo how to circumvent.
Cheers.
To fix this you use the first column of G:
for i in range(a):
H[:,i] = .5 * G[:, 0]
Numpy broadcasting basically attempts to match dimensions of arrays (when broadcasting) by starting with the last dimension and moving to the first. In this case the second dimension of G (1) gets broadcast to 25 (the first and only dimension of H[:, i]. The first dimension of G does not match with anything. You can read more about numpy broadcasting rules here.
Note: you really don't need that for loop. H is just G column repeated twice. You can accomplish that in various ways (e.g. np.tile, np.hstack, etc.)
H = np.tile(G / 2, 2)
I'm quite new to Python and Numpy, so I apologize if I'm missing something obvious here.
I have a function that solves a system of 2 differential equations :
import numpy as np
import numpy.linalg as la
def solve_ode(x0, a0, beta, t):
At = np.array([[0.23*t, (-10**5)*t], [0, -beta*t]], dtype=np.float32)
# get eigenvalues and eigenvectors
evals, V = la.eig(At)
Vi = la.inv(V)
# get e^At coeff
eAt = V # np.exp(evals) # Vi
xt = eAt*x0
return xt
However, running it with this code :
import matplotlib.pyplot as plt
# initial values
x0 = 10**6
a0 = 2.5
beta = 0.05
t = np.linspace(0, 3600, 360)
plt.semilogy(t, solve_ode(x0, a0, beta, t))
... throws this error :
ValueError: setting an array element with a sequence.
At this line :
At = np.array([[0.23*t, (-10**5)*t], [0, -beta*t]], dtype=np.float32)
Note that t and beta are supposed to be floats. I think Python might not be able to infer this but I don't know how I could do this...
Thx in advance for your help.
You are supplying t as a numpy array of shape 360 from linspace and not simply a float. The resulting At numpy array you are trying to create is then ill formed as all columns must be the same length. In python there is an important difference between lists and numpy arrays. For example, you could do what you have here as a list of lists, e.g.
At = [[0.23*t, (-10**5)*t], [0, -beta*t]]
with dimensions [[360 x 360] x [1 x 360]].
Alternatively, if all elements of At are the length of t the array would work,
At = np.array([[0.23*t, (-10**5)*t], [t, -beta*t]], dtype=np.float32)
with shape [2, 2, 360].
When you give a list or a list of lists, or in this case, a list of list of listss, all of them should have the same length, so that numpy can automatically infer the dimensions (shape) of the resulting matrix.
In your example, it's all correctly put, except the part you put 0 as a column I guess. Not sure what to call it though, cause your expected output is a cube I suppose.
You can fix it by giving the correct number of zeros as bellow:
At = np.array([[0.23*t, (-10**5)*t], [np.zeros(len(t)), -beta*t]], dtype=np.float32)
But check the .shape of the resulting array, and make sure it's what you want.
As others note the problem is the 0 in the inner list. It doesn't match the 360 length arrays generated by the other expressions. np.array can make an object dtype array from that (2x2), but can't make a float one.
At = np.array([[0.23*t, (-10**5)*t], [0*t, -beta*t]])
produces a (2,2,360) array. But I suspect the rest of that function is built around the assumption that At is (2,2) - a 2d square array with eig, inv etc.
What is the return xt supposed to be?
Does this work?
S = np.array([solve_ode(x0, a0, beta, i) for i in t])
giving a 1d array with the same number of values as in t?
I'm not suggesting this is the fastest way of solving the problem, but it's the simplest, especially if you are only generating 360 values.
I'm trying to understand how numpy works when you try to call the dot product of two row vectors.
I have this code:
X = np.array([[1,2,3]])
THETA = np.array([[1,2,3]])
print X.dot(THETA)
This gives me the error:
ValueError: shapes (1,3) and (1,3) not aligned: 3 (dim 1) != 1 (dim 0)
I thought that you could take the dot product of two row vectors however to get:
x1*theta1 + x2*theta2 + x3*theta3
And this would also transfer to the dot product of two column vectors.
The weird part is, I have to take the transpose of the second matrix in order to actually use the dot product:
print X.dot(THETA.T)
array([[14]])
However, I didn't think this would actually work, and why it would work instead of just doing a row dot row operation. Can anyone help me understand what's going on? Is it some rule in linear algebra that I forgot from long ago?
dot for 2D input is matrix multiplication, not a dot product. What you're seeing is just the result of the normal rules of matrix multiplication. If you want a vector dot product, the easiest way is to use 1D vectors, with no superfluous second dimension:
X = np.array([1, 2, 3])
THETA = np.array([1, 2, 3])
print X.dot(THETA)
dot-ting two 1D arrays takes a dot product and produces a scalar result.
If you want to use row and column vectors, then by the standard rules of matrix multiplication, you need to multiply a 1-by-N array (a row vector) by an N-by-1 array (a column vector) to get a 1-by-1 result, and NumPy will give you a 1-by-1 array rather than a scalar.
The alignment error you're seeing is because you're trying to represent a 1D vector as a 2D array.
In [1]: import numpy as np
In [2]: X = np.array([1,2,3])
In [3]: THETA = np.array([1,2,3])
In [4]: print X.dot(THETA)
14
In [5]: print X.dot(THETA.T)
14
And:
x1*theta1 + x2*theta2 + x3*theta3 =
1*1 + 2*2 + 3*3 =
14