I am working with the multinerf Framework from Google Research.
It's great to work with and things are actually pretty clear down to the paragraph existing data loaders under Intrinsic camera poses:
Intrinsic camera poses
pixtocams= [N, 3, 4] numpy array of inverse intrinsic matrices, OR [3,
4] numpy array of a single shared inverse intrinsic matrix. These
should be in OpenCV format, e.g.
camtopix = np.array([
[focal, 0, width/2],
[ 0, focal, height/2],
[ 0, 0, 1],
])
pixtocam = np.linalg.inv(camtopix)
I don't quite understand why they are talking about an inverse intrinsic matrix (or matrices) of the dimensions [3, 4]. The inverse intrinsic matrix should have the dimensions [3, 3], shouldn't it?
I think that it's a typo, since you can't really invert a 3x4 matrix (it must be singular). You're right that the dimensions should be 3x3. You can see that immediately below the passage you quoted, they give this example of an inverse intrinsic matrix:
camtopix = np.array([
[focal, 0, width/2],
[ 0, focal, height/2],
[ 0, 0, 1],
])
pixtocam = np.linalg.inv(camtopix)
which is 3x3.
Related
I hope to generate a gradient line between two points in a numpy array. For example, if I have a numpy array like
[[1,0,0,0]
[0,0,0,0]
[0,0,0,0]
[0,0,0,4]]
What I hope to get is find a line between the [0,0] and [3,3], and have a liner gradient. So I hope to make an array like
[[1,0,0,0]
[0,2,0,0]
[0,0,3,0]
[0,0,0,4]]
The tricky part is the matrix may not be a perfect nxn. I don't care if some lines have two non-zero elements (because we cannot get a perfect diagonal for mxn matrix). The element of the same line can be the same in my case.
I am wondering is there an efficient way to make this happen?
You can use np.fill_diagonal:
np.fill_diagonal(arr, np.arange(arr[0,0], arr[-1,-1]+1))
Output:
array([[1, 0, 0, 0],
[0, 2, 0, 0],
[0, 0, 3, 0],
[0, 0, 0, 4]])
In scientific computation, we often have to construct matrices that compute differential operators. It is often easier to write the code that applies the operator than to explicitly construct the matrix. Is there a library that takes the code (assuming it only uses linear operations) and outputs the matrix, ideally in sparse form?
Example:
# computes finite differences
def myop(a):
return a[1:]-a[:-1]
a = np.array(5)
myop(a)
computes the finite differences of the "a" vector. I now want to so something like
a = some_library.array(5)
op = myop(a)
print(op.as_matrix())
which should give me the matrix representation:
[[-1, 1, 0, 0, 0],
[0, -1, 1, 0, 0],
[0, 0, -1, 1, 0],
[0, 0, 0, -1, 1]]
Having the matrix is very useful, for example for computing the transposed operator or analyzing the sparsity patterns. Technically it should be possible to use automatic differentiation tools and extract the Jacobian of op(), but I didn't find any AD libraries that efficiently deal with the whole Jacobian, in particular if it is sparse. They all seem to either do one pass per row or one per column of the Jacobian, which is horribly slow even for only a few hundreds of variables.
One easy way:
In [568]: arr = np.zeros((4,5),int)
In [569]: arr[np.arange(4),np.arange(4)]=-1
In [570]: arr[np.arange(4),np.arange(1,5)]=1
In [571]: arr
Out[571]:
array([[-1, 1, 0, 0, 0],
[ 0, -1, 1, 0, 0],
[ 0, 0, -1, 1, 0],
[ 0, 0, 0, -1, 1]])
There are various np.diag... functions, but since your don't want a square matrix I though this would be easier.
The scipy.sparse library is often used for finite-difference problems, and has various dia constructors. But that requires more reading.
I'm looking for something similar to ARRAYFUN in MATLAB, but for Python. What I need to do is to compute a matrix whose components are exp(j*dot([kx,ky], [x,y])), where [kx,ky] is a fixed known vector, and [x,y] is an element from a meshgrid.
What I was trying to do is to define
RX, RY = np.meshgrid(np.arange(N), np.arange(M))
R = np.dstack((RX,RY))
and then iterate over the R indices, filling a matrix with the same shape as R, in which each component would be exp(j*dot([kx,ky], [x,y])), with [x,y] being in R. This doesn't look efficient nor elegant.
Thanks for your help.
You could do what we used to do in MATLAB before they added ARRAYFUN - change the calculation so it works with arrays. That could be tricky in the days when everything in MATLAB was 2d; allowing more dimensions made it easier. numpy allows more than 2 dimensions.
Anyways, here a quick attempt:
In [497]: rx,ry=np.meshgrid(np.arange(3),np.arange(4))
In [498]: R=np.dstack((rx,ry))
In [499]: R.shape
Out[499]: (4, 3, 2)
In [500]: kx,ky=1,2
In [501]: np.einsum('i,jki->jk',[kx,ky],R)
Out[501]:
array([[0, 1, 2],
[2, 3, 4],
[4, 5, 6],
[6, 7, 8]])
There are other versions of dot, matmul and tensordot, but einsum is the one I like to use. I've worked with it enough to quickly set up a multidimensional dot.
Now just apply the 1j and exp to each element:
In [502]: np.exp(np.einsum('i,jki->jk',[kx,ky],R)*1j)
Out[502]:
array([[ 1.00000000+0.j , 0.54030231+0.84147098j,
-0.41614684+0.90929743j],
[-0.41614684+0.90929743j, -0.98999250+0.14112001j,
-0.65364362-0.7568025j ],
[-0.65364362-0.7568025j , 0.28366219-0.95892427j,
0.96017029-0.2794155j ],
[ 0.96017029-0.2794155j , 0.75390225+0.6569866j ,
-0.14550003+0.98935825j]])
EDIT
I realized that I did not check my mwe very well and as such asked something of the wrong question. The main problem is when the numpy array is passed in as a 2d array instead of 1d (or even when a python list is passed in as 1d instead of 2d). So if we have
x = np.array([[1], [2], [3]])
then obviously if you try to index this then you will get arrays out (if you use item you do not). this same thing also applies to standard python lists.
Sorry about the confusion.
Original
I am trying to form a new numpy array from something that may be a numpy array or may be a standard python list.
for example
import numpy as np
x = [2, 3, 1]
y = np.array([[0, -x[2], x[1]], [x[2], 0, -x[0]], [-x[1], x[0], 0]])
Now I would like to form a function such that I can make y easily.
def skew(vector):
"""
this function returns a numpy array with the skew symmetric cross product matrix for vector.
the skew symmetric cross product matrix is defined such that
np.cross(a, b) = np.dot(skew(a), b)
:param vector: An array like vector to create the skew symmetric cross product matrix for
:return: A numpy array of the skew symmetric cross product vector
"""
return np.array([[0, -vector[2], vector[1]],
[vector[2], 0, -vector[0]],
[-vector[1], vector[0], 0]])
This works great and I can now write (assuming the above function is included)
import numpy as np
x=[2, 3, 1]
y = skew(x)
However, I would also like to be able to call skew on existing 1d or 2d numpy arrays. For instance
import numpy as np
x = np.array([2, 3, 1])
y = skew(x)
Unfortunately, doing this returns a numpy array where the elements are also numpy arrays, not python floats as I would like them to be.
Is there an easy way to form a new numpy array like I have done from something that is either a python list or a numpy array and have the result be just a standard numpy array with floats in each element?
Now obviously one solution is to check to see if the input is a numpy array or not:
def skew(vector):
"""
this function returns a numpy array with the skew symmetric cross product matrix for vector.
the skew symmetric cross product matrix is defined such that
np.cross(a, b) = np.dot(skew(a), b)
:param vector: An array like vector to create the skew symmetric cross product matrix for
:return: A numpy array of the skew symmetric cross product vector
"""
if isinstance(vector, np.ndarray):
return np.array([[0, -vector.item(2), vector.item(1)],
[vector.item(2), 0, -vector.item(0)],
[-vector.item(1), vector.item(0), 0]])
else:
return np.array([[0, -vector[2], vector[1]],
[vector[2], 0, -vector[0]],
[-vector[1], vector[0], 0]])
however, it gets very tedious having to write these instance checks all over the place.
Another solution would be to cast everything to an array first and then just use the array call
def skew(vector):
"""
this function returns a numpy array with the skew symmetric cross product matrix for vector.
the skew symmetric cross product matrix is defined such that
np.cross(a, b) = np.dot(skew(a), b)
:param vector: An array like vector to create the skew symmetric cross product matrix for
:return: A numpy array of the skew symmetric cross product vector
"""
vector = np.array(vector)
return np.array([[0, -vector.item(2), vector.item(1)],
[vector.item(2), 0, -vector.item(0)],
[-vector.item(1), vector.item(0), 0]])
but I feel like this is inefficient as it requires creating a new copy of vector (in this case not a big deal since vector is small but this is just a simple example).
My question is, is there a different way to do this outside of what I've discussed or am I stuck using one of these methods?
Arrays are iterable. You can write in your skew function:
def skew(x):
return np.array([[0, -x[2], x[1]],
[x[2], 0, -x[0]],
[-x[1], x[0], 0]])
x = [1,2,3]
y = np.array([1,2,3])
>>> skew(y)
array([[ 0, -3, 2],
[ 3, 0, -1],
[-2, 1, 0]])
>>> skew(x)
array([[ 0, -3, 2],
[ 3, 0, -1],
[-2, 1, 0]])
In any case your methods ended with 1st dimension elements being numpy arrays containing floats. You'll need in any case a call on the 2nd dimension to get the floats inside.
Regarding what you told me in the comments, you may add an if condition for 2d arrays:
def skew(x):
if (isinstance(x,ndarray) and len(x.shape)>=2):
return np.array([[0, -x[2][0], x[1][0]],
[x[2][0], 0, -x[0][0]],
[-x[1][0], x[0][0], 0]])
else:
return np.array([[0, -x[2], x[1]],
[x[2], 0, -x[0]],
[-x[1], x[0], 0]])
You can implement the last idea efficiently using numpy.asarray():
vector = np.asarray(vector)
Then, if vector is already a NumPy array, no copying occurs.
You can keep the first version of your function and convert the numpy array to list:
def skew(vector):
if isinstance(vector, np.ndarray):
vector = vector.tolist()
return np.array([[0, -vector[2], vector[1]],
[vector[2], 0, -vector[0]],
[-vector[1], vector[0], 0]])
In [58]: skew([2, 3, 1])
Out[58]:
array([[ 0, -1, 3],
[ 1, 0, -2],
[-3, 2, 0]])
In [59]: skew(np.array([2, 3, 1]))
Out[59]:
array([[ 0, -1, 3],
[ 1, 0, -2],
[-3, 2, 0]])
This is not an optimal solution but is a very easy one.
You can just convert the vector into list by default.
def skew(vector):
vector = list(vector)
return np.array([[0, -vector[2], vector[1]],
[vector[2], 0, -vector[0]],
[-vector[1], vector[0], 0]])
I have a large sparse numpy/scipy matrix where each row corresponds to a point in high-dimensional space. I want make queries of the following kind:
Given a point P (a row in the matrix) and a distance epsilon, find all points with distance at most epsilon from P.
The distance metric I am using is Jaccard-similarity, so it should be possible to use Locality Sensitive Hashing tricks such as MinHash.
Is there an implementation of MinHash for sparse numpy arrays somewhere (I can't seem to find one) or is there an easy way to do this?
The reason I am not just pulling something built for non-sparse arrays off of Github is that the sparse data structures in scipy might cause explosions in time complexity.
If you have very large sparse datasets that are too large to be held in memory in a non-sparse format, I'd try out this LSH implementation that is built around the assumption of Scipy's CSR Sparse Matrices:
https://github.com/brandonrobertz/SparseLSH
It also hash support for disk-based key-value stores like LevelDB if you can't fit the tables in memory. From the docs:
from sparselsh import LSH
from scipy.sparse import csr_matrix
X = csr_matrix( [
[ 3, 0, 0, 0, 0, 0, -1],
[ 0, 1, 0, 0, 0, 0, 1],
[ 1, 1, 1, 1, 1, 1, 1] ])
# One class number for each input point
y = [ 0, 3, 10]
X_sim = csr_matrix( [ [ 1, 1, 1, 1, 1, 1, 0]])
lsh = LSH( 4,
X.shape[1],
num_hashtables=1,
storage_config={"dict":None})
for ix in xrange(X.shape[0]):
x = X.getrow(ix)
c = y[ix]
lsh.index( x, extra_data=c)
# find points similar to X_sim
lsh.query(X_sim, num_results=1)
If you definitely only want to use MinHash, you could try out https://github.com/go2starr/lshhdc, but I haven't personally tested that one out for compatibility with sparse matrices.