Suppose I have a kxn array of data with columns of vectors and a distance function defined on these vectors. How do I convert the kxn array into another array of the same shape such that the euclidean norm among the converted vectors is the norm derived by the given distance function? I know you can directly calculate the distance matrix for the data by that given distance function, and derive the coordinates in R^k thereby. But this method is really expensive espesically when the distance function has a complexity O(n^2) or more. So I wonder if there is any simpler algorithm to do that.
It sounds like you are describing multidimension scaling (MDS). One way to do it in Python is with scikit-learn's sklearn.manifold.MDS.
MDS expects the NxN distance (or "dissimilarity") matrix as input, so that doesn't get around the cost of evaluating the distance function. The distance matrix is unavoidably needed for this conversion, so if the distance function itself is expensive, it seems the best thing to do is reduce the number of samples or look for a way to compute fast approximate distances it to speed it up. Also, beware that MDS is usually only approximate. A numerical optimization looks for the best fit of Euclidean norms to the given distances.
Say I have orthogonal vectors of dimension n. I have two questions:
How to create/initialize n such orthogonal vectors in python using the existing packages (numpy, scipy, pytorch etc)? Ideally these basis vectors should be as random as possible given the constraints, that is avoiding values such as 1,0,-1 as much as possible.
How can I rotate them by an angle alpha so that they remain orthogonal in high dimensional space? Again, I would like to do this in python, preferably using existing implementation in some of the packages.
You could do a QR decomposition of a random matrix, and set the R-component to zero. This will yield a random orthogonal matrix.
Vary one of the Givens angles in the Q components and you get a random rotation.
I have an answer to your first question and some thoughts on how to approach the second.
1.
import numpy as np
#let's say we're working in 5-D space
n = 5
#set of orthogonal basis vectors
basis_vectors = []
for _ in range(n):
vector = np.random.randn(n)
for basis_vector in basis_vectors:
vector -= basis_vector.dot(vector) * vector
#uncomment following to make basis orthonormal
#vector /= np.linalg.norm(rotation_axis)
basis_vectors.append(vector)
for a_i in range(n):
for b_i (a_i + 1, n):
assert np.allclose(basis_vectors[a_i].dot(basis_vectors[b_i]), 0)
Because you want to rotate both vectors in the same manner, there must be a way to preserve information on the way each rotation is carried out (e.g. rotation matrix, rotation quaternion).
Preexisting implementations of 3D Rotation matrices include the Scipy function scipy.spatial.transform.Rotation.from_rotvec and Python's quaternion module (see henneray's answer), but these are only for 3D vectors. Unless I've overlooked something, it'd be necessary to implement ND rotation from scratch.
Here's a general outline of the steps I would take:
Find 2 linearly independent ND basis vectors of the 2D plane in which you want to rotate the two vectors. (the vectors you want to rotate, a and b, aren't necessarily on this plane)
Find the remaining (N-2)D basis vectors that are linearly independent to these first 2 vectors. Combined the N basis vectors should span the ND space.
Break up each of the two N-D orthogonal vectors you want to rotate into the sum of two vectors: 1) the vectors' projections onto the 2D plane you've constructed and 2) the "remainder" of the vector that doesn't fall on the 2D plane. Set this "remainder" aside for now.
Perform a change of basis on the projected N-D vectors so that they can be expressed as the product of a 2D vector and an Nx2 matrix, which has its columns set to each of the corresponding basis vectors calculated. Keep in mind that the 2D vector is now in a modified coordinate space, not the original.
Construct the 2D rotation matrix corresponding to the desired rotation within the 2D plane identified in the first step. Perform the rotation transformation on the 2D vectors.
Transform the rotated 2D vectors back into ND vectors in the main coordinate system by multiplying the by the Nx2 matrix.
Add the "remainder" set aside earlier back to the mapped ND vector.
The resulting two vectors have been rotated by an arbitrary angle on a particular 2D plane, but maintain orthogonality.
I hope these ideas help you. Take care.
I found a scipy function that can do 1, ortho_group, still wondering about 2.
>>> from scipy.stats import ortho_group
>>> m = ortho_group.rvs(dim=4)
>>> m
array([[-0.25952499, 0.435163 , 0.04561972, 0.86092902],
[-0.44123728, -0.38814758, -0.80217271, 0.10568846],
[ 0.16909943, -0.80707234, 0.35548632, 0.44007851],
[-0.8422362 , -0.0927839 , 0.47756387, -0.23229737]])
>>> m.dot(m.T)
array([[ 1.00000000e+00, -1.68203864e-16, 1.75471554e-16,
9.74154717e-17],
[-1.68203864e-16, 1.00000000e+00, -1.18506045e-16,
-1.81879209e-16],
[ 1.75471554e-16, -1.18506045e-16, 1.00000000e+00,
1.16692720e-16],
[ 9.74154717e-17, -1.81879209e-16, 1.16692720e-16,
1.00000000e+00]])
I am trying to get the phase distribution of a 2D aperture using FFT.
The input is a circle, where everything inside the circle has value 1, outside it has value 0.
In order to make a good transform, I use an input array that is 200x as large as the radius of the circle, and make a 5000x5000 grid out of it. This ensures that the circle is actually circular and there is enough room around in order that no Nyquist things happen.
I need to 2D Fourier transform the aperture and then calculate the phase of the Fourier transform at every point.
The function I use for creating the input (aperture):
creating the input aperture
Next do the numpy fft2 2D fourier transform:
Fourier transforming aperture
And the result of this is a 2D complex array (as expected!), BUT with the imaginary parts so much much much smaller than the real parts (17 orders of magnitude difference imaginary parts ~10E-17).
This is not expected and most probably wrong. What went wrong?
The FFT phase result of a perfectly symmetric input is zero, e.g. a strictly real result, thus atan2(Im,Re) == 0 , (imaginary components all zero, except for rounding noise).
(even symmetry with respect to (0,0) circularly, or to (n/2,n/2))
The phase will become non-zero (thus a non-zero imaginary component in the FFT result) when the input is moved off center or otherwise made non-symmetric.
I know that for fft.fft, I can use fft.fftfreq. But there seems to be no such thing as fft.fftfreq2. Can I somehow use fft.fftfreq to calculate the frequencies in 2 dimensions, possibly with meshgrid? Or is there some other way?
Yes. Apply fftfreq to each spatial vector (x and y) separately. Then create a meshgrid from those frequency vectors.
Note that you need to use fftshift if you want the typical representation (zero frequencies in center of spatial spectrum) to both the output and your new spatial frequencies (before using meshgrid).
I am trying to implement an algorithm in python, but I am not sure when I should use fftshift(fft(fftshift(x))) and when only fft(x) (from numpy). Is there a rule of thumb based on the shape of input data?
I am using fftshift instead of ifftshift due to the even number of values in the vector x.
It really just depends on what you want. The DFT (and hence the FFT) is periodic in the frequency domain with period equal to 2pi.
The fft() function will return the approximation of the DFT with omega (radians/s) from 0 to pi (i.e. 0 to fs, where fs is the sampling frequency). All fftshift() does is swap the output vector of the fft() right down the middle. So the output of fftshift(fft()) is now from -pi/2 to pi/2.
Usually, people like to plot a good approximation of the DTFT (or maybe even the CTFT) using the FFT, so they zero-pad the input with a huge amount of zeros (the function fft() does this on it's own) and then they use the fftshift() function to plot between -pi and pi.
In other words, use fftshift(fft()) for plotting, and fft() for the math!
fft(fftshift(x)) rotates the input vector so the the phase of the complex FFT result is relative to the center of the original data window. If the input waveform is not exactly integer periodic in the FFT width, phase relative to the center of the original window of data may make more sense than the phase relative to some averaging between the discontinuous beginning and end. fft(fftshift(x)) also has the property that the imaginary component of a result will always be positive for a positive zero crossing at the center of the window of any antisymmetric waveform component.
fftshift(fft(y)) rotates the FFT results so that the DC bin is in the center of the result, halfway between -Fs/2 and Fs/2, which is a common spectrum display format.