I am going to decompose a 4D tensor using tucker decomposition in python. I found a library, tensorly, to do this.
I only want to perform the decomposition on the first and second dimensions. To perform tucker decomposition on some modes (not all modes) using tensorly I have to use partial_tucker command. This is my code:
F = 256
D = 96
h = 5
w = 6
ranks = [89, 48]
modes = [0, 1]
tensor = tl.tensor((np.arange(F*D*h*w).reshape((F, D, h, w))).astype(np.float64))
core, factors = partial_tucker(tensor, modes=modes, rank=ranks)
This code works well, but when I am trying to change the rank list, for example:
ranks = [3,4]
I get an error as follows:
Traceback (most recent call last):
File "D:\PhD_Thessaloniki\Codes\LRF_Convolutional\tucker-decomposition.py", line 49, in <module>
core, factors = partial_tucker(tensor, modes=modes, rank=ranks)
File "C:\Users\Milad\Anaconda3\envs\tensorly\lib\site-packages\tensorly\decomposition\_tucker.py", line 109, in partial_tucker
eigenvecs, _, _ = svd_fun(unfold(core_approximation, mode), n_eigenvecs=rank[index], random_state=random_state)
File "C:\Users\Milad\Anaconda3\envs\tensorly\lib\site-packages\tensorly\backend\core.py", line 913, in partial_svd
S, V = scipy.sparse.linalg.eigsh(
File "C:\Users\Milad\Anaconda3\envs\tensorly\lib\site-packages\scipy\sparse\linalg\_eigen\arpack\arpack.py", line 1689, in eigsh
params.iterate()
File "C:\Users\Milad\Anaconda3\envs\tensorly\lib\site-packages\scipy\sparse\linalg\_eigen\arpack\arpack.py", line 571, in iterate
raise ArpackError(self.info, infodict=self.iterate_infodict)
scipy.sparse.linalg._eigen.arpack.arpack.ArpackError: ARPACK error 3: No shifts could be applied during a cycle of the Implicitly restarted Arnoldi iteration. One possibility is to increase the size of NCV relative to NEV.
I don't know if there is a constraint to define the rank in tucker decomposition or not, but when I am trying to perform decomposition on only one dimension, for example:
ranks = [3]
modes = [0]
or
ranks = [4]
modes = [1]
works well again.
I want to know:
Is this an algorithmic or code (tensorly) problem (constraint)?
What is this problem?
What rank lists are valid?
Thanks
Tucker relies on higher-order PCA. The error you are seeing is is in the sparse SVD, applied to an unfolding of the main tensor.
You can try a different SVD function (svd parameter in partial_tucker), you can see the available option using tensorly.tenalg.svd.SVD_FUNS.
You also might want to try a tensor with random elements, using tensorly.random.random_tensor or tensorly.randn.
Related
i've recreated a code of Haar Tranform matrix from matlab to python it's a success upon entering the value of n for 2 and 4 but when i'm trying to input 8 there's an error
"Traceback (most recent call last):
File "python", line 20, in
ValueError: shape too large to be a matrix."
here's my code
import numpy as np
import math
n=8
# check input parameter and make sure it's the power of 2
Level1 = math.log(n, 2)
Level = int(Level1)+1
#Initialization
H = [1]
NC = 1 / math.sqrt(2) #normalization constant
LP = [1, 1]
HP = [1,-1]
for i in range(1,Level):
H = np.dot(NC, [np.matrix(np.kron(H, LP)), np.matrix(np.kron(np.eye(len(H)), HP))])
print H
I'm assuming you got the definition of the haar transform from the wikipedia article or a similar source, so I'll try to stick to their notation.
The problem with your code is that on the wikipedia article a slight abuse of notation is used. In the equation defining H_2N in terms of H_N, two matrices are stacked on top of eachother with brackets around them. Technically, this would be something like an array consisting of 2 arrays, but they mean it to be a single array where the top half of the values is equal to the one matrix and the bottom half equal to the other matrix.
In your code, the array of two matrices is the following part:
[np.matrix(np.kron(H, LP)), np.matrix(np.kron(np.eye(len(H)), HP))]
You can make this into a single matrix as described above using the np.concatenate function as follows:
H = np.dot(NC, np.concatenate([np.matrix(np.kron(H, LP)), np.matrix(np.kron(np.eye(len(H)), HP))]))
I'm trying to rewrite Zhao Koch steganography method from matlab into python and I am stuck right at the start.
The first two procedures as they are in matlab:
Step 1:
A = imread(casepath); # Reading stegonography case image and aquiring it's RGB values. In my case it's a 400x400 PNG image, so it gives a 400x400x3 array.
Step 2:
D = dct2(A(:,:,3)); # Applying 2D DCT to blue values of the image
Python code analog:
from scipy import misc
from numpy import empty,arange,exp,real,imag,pi
from numpy.fft import rfft,irfft
arr = misc.imread('casepath')# 400x480x3 array (Step 1)
arr[20, 30, 2] # Getting blue pixel value
def dct(y): #Basic DCT build from numpy
N = len(y)
y2 = empty(2*N,float)
y2[:N] = y[:]
y2[N:] = y[::-1]
c = rfft(y2)
phi = exp(-1j*pi*arange(N)/(2*N))
return real(phi*c[:N])
def dct2(y): #2D DCT bulid from numpy and using prvious DCT function
M = y.shape[0]
N = y.shape[1]
a = empty([M,N],float)
b = empty([M,N],float)
for i in range(M):
a[i,:] = dct(y[i,:])
for j in range(N):
b[:,j] = dct(a[:,j])
return b
D = dct2(arr) # step 2 anlogue
However, when I try to execute the code I get the following error:
Traceback (most recent call last):
File "path to .py file", line 31, in <module>
D = dct2(arr)
File "path to .py file", line 25, in dct2
a[i,:] = dct(y[i,:])
File "path to .py file", line 10, in dct
y2[:N] = y[:]
ValueError: could not broadcast input array from shape (400,3) into shape (400)
Perhaps someone could kindly explain to me what am I doing wrong?
Additional Info:
OS: Windows 10 Pro 64 bit
Python: 2.7.12
scipy:0.18.1
numpy:1.11.2
pillow: 3.4.1
Your code works fine, but it is designed to only accept a 2D array, just like dct2() in Matlab. Since your arr is a 3D array, you want to do
D = dct2(arr[...,2])
As mentioned in my comment, instead or reinventing the wheel, use the (fast) built-in dct() from the scipy package.
The code from the link in my comment effectively provides you this:
import numpy as np
from scipy.fftpack import dct, idct
def dct2(block):
return dct(dct(block.T, norm='ortho').T, norm='ortho')
def idct2(block):
return idct(idct(block.T, norm='ortho').T, norm='ortho')
But again, I must stress that you have to call this function for each colour plane individually. Scipy's dct() will happily accept any N-dimensional array and will apply the transform on the last axis. Since that's your colour planes and not your rows and columns of your pixels, you'll get the wrong result. Yes, there is a way to address this with the axis input parameter, but I won't unnecessarily overcomplicate this answer.
Regarding the various DCT implementations involved here, your version and scipy's implementation give the same result if you omit the norm='ortho' parameter from the snippet above. But with that parameter included, scipy's transform will agree with Matlab's.
I have a similarity score between 0 and 1 from each entry to every other entry in an 100 by 100 matrix. So e.g. [0,0] would be 1, [0,1] might be .54 etc. The similarity score was generated using Shannon Jensen divergence.
I want to use ward clustering (but am open to other suggestions) to cluster these together and I tried the following code:
x = np.array(mylist)
print x.shape#(100,100)
clustering = scipy.cluster.hierarchy.ward(x)
scipy.cluster.hierarchy.dendrogram(clustering)
but I am getting the error:
Traceback (most recent call last):
File "C:/Python27/ward.py", line 38, in <module>
clustering = scipy.cluster.hierarchy.ward(y)
File "C:\Python27\lib\site-packages\scipy\cluster\hierarchy.py", line 465, in ward
return linkage(y, method='ward', metric='euclidean')
File "C:\Python27\lib\site-packages\scipy\cluster\hierarchy.py", line 620, in linkage
y = _convert_to_double(np.asarray(y, order='c'))
File "C:\Python27\lib\site-packages\scipy\cluster\hierarchy.py", line 928, in _convert_to_double
X = X.astype(np.double)
ValueError: setting an array element with a sequence.
Do I need to do some transformation on my array first or use some other method?
I am trying to fit some data to a curve in Python using scipy.optimize.curve_fit. I am running into the error ValueError: array must not contain infs or NaNs.
I don't believe either my x or y data contain infs or NaNs:
>>> x_array = np.asarray_chkfinite(x_array)
>>> y_array = np.asarray_chkfinite(y_array)
>>>
To give some idea of what my x_array and y_array look like at either end (x_array is counts and y_array is quantiles):
>>> type(x_array)
<type 'numpy.ndarray'>
>>> type(y_array)
<type 'numpy.ndarray'>
>>> x_array[:5]
array([0, 0, 0, 0, 0])
>>> x_array[-5:]
array([2919, 2965, 3154, 3218, 3461])
>>> y_array[:5]
array([ 0.9999582, 0.9999163, 0.9998745, 0.9998326, 0.9997908])
>>> y_array[-5:]
array([ 1.67399000e-04, 1.25549300e-04, 8.36995200e-05,
4.18497600e-05, -2.22044600e-16])
And my function:
>>> def func(x,alpha,beta,b):
... return ((x/1)**(-alpha) * ((x+1*b)/(1+1*b))**(alpha-beta))
...
Which I am executing with:
>>> popt, pcov = curve_fit(func, x_array, y_array)
resulting in the error stack trace:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 426, in curve_fit
res = leastsq(func, p0, args=args, full_output=1, **kw)
File "/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 338, in leastsq
cov_x = inv(dot(transpose(R),R))
File "/usr/lib/python2.7/dist-packages/scipy/linalg/basic.py", line 285, in inv
a1 = asarray_chkfinite(a)
File "/usr/lib/python2.7/dist-packages/numpy/lib/function_base.py", line 590, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs
I'm guessing the error might not be with respect to my arrays, but rather an array created by scipy in an intermediate step? I've had a bit of a dig through the relevant scipy source
files, but things get hairy pretty quickly debugging the problem that way. Is there something obvious I'm doing wrong here? I've seen casually mentioned in other questions that sometimes certain initial parameter guesses (of which I currently don't have any explicit) might result in these kind of errors, but even if this is the case, it would be good to know a) why that is and b) how to avoid it.
Why it is failing
Not your input arrays are entailing nans or infs, but evaluation of your objective function at some X points and for some values of the parameters results in nans or infs: in other words, the array with values func(x,alpha,beta,b) for some x, alpha, beta and b is giving nans or infs over the optimization routine.
Scipy.optimize curve fitting function uses Levenberg-Marquardt algorithm. It is also called damped least square optimization. It is an iterative procedure, and a new estimate for the optimal function parameters is computed at each iteration. Also, at some point during optimization, algorithm is exploring some region of the parameters space where your function is not defined.
How to fix
1/Initial guess
Initial guess for parameters is decisive for the convergence. If initial guess is far from optimal solution, you are more likely to explore some regions where objective function is undefined. So, if you can have a better clue of what your optimal parameters are, and feed your algorithm with this initial guess, error while proceeding might be avoided.
2/Model
Also, you could modify your model, so that it is not returning nans. For those values of the parameters, params where original function func is not defined, you wish that objective function takes huge values, or in other words that func(params) is far from Y values to be fitted.
Also, at points where your objective function is not defined, you may return a big float, for instance AVG(Y)*10e5 with AVG the average (so that you make sure to be much bigger than average of Y values to be fitted).
Link
You could have a look at this post: Fitting data to an equation in python vs gnuplot
Your function has a negative power (x^-alpha) this is the same as (1/x)^(alpha). If x is ever 0 your function will return inf and your curve fit operation will break, I'm surprised a warning/error isn't thrown earlier informing you of a divide by 0.
BTW why are you multiplying and dividing by 1?
I was able to reproduce this error in python2.7 like so:
from sklearn.decomposition import FastICA
X = load_data.load("stuff") #this sets X to a 2d numpy array containing
#large positive and negative numbers.
ica = FastICA(whiten=False)
print(np.isnan(X).any()) #this prints False
print(np.isinf(X).any()) #this prints False
ica.fit(X) #this produces the error:
Which always produces the Error:
/usr/lib64/python2.7/site-packages/sklearn/decomposition/fastica_.py:58: RuntimeWarning: invalid value encountered in sqrt
return np.dot(np.dot(u * (1. / np.sqrt(s)), u.T), W)
Traceback (most recent call last):
File "main.py", line 43, in <module>
ica()
File "main.py", line 18, in ica
ica.fit(X)
File "/usr/lib64/python2.7/site-packages/sklearn/decomposition/fastica_.py", line 523, in fit
self._fit(X, compute_sources=False)
File "/usr/lib64/python2.7/site-packages/sklearn/decomposition/fastica_.py", line 479, in _fit
compute_sources=compute_sources, return_n_iter=True)
File "/usr/lib64/python2.7/site-packages/sklearn/decomposition/fastica_.py", line 335, in fastica
W, n_iter = _ica_par(X1, **kwargs)
File "/usr/lib64/python2.7/site-packages/sklearn/decomposition/fastica_.py", line 108, in _ica_par
- g_wtx[:, np.newaxis] * W)
File "/usr/lib64/python2.7/site-packages/sklearn/decomposition/fastica_.py", line 55, in _sym_decorrelation
s, u = linalg.eigh(np.dot(W, W.T))
File "/usr/lib64/python2.7/site-packages/scipy/linalg/decomp.py", line 297, in eigh
a1 = asarray_chkfinite(a)
File "/usr/lib64/python2.7/site-packages/numpy/lib/function_base.py", line 613, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs
Solution:
from sklearn.decomposition import FastICA
X = load_data.load("stuff") #this sets X to a 2d numpy array containing
#large positive and negative numbers.
ica = FastICA(whiten=False)
#this is a column wise normalization function which flattens the
#two dimensional array from very large and very small numbers to
#reasonably sized numbers between roughly -1 and 1
X = (X - np.mean(X, axis=0)) / np.std(X, axis=0)
print(np.isnan(X).any()) #this prints False
print(np.isinf(X).any()) #this prints False
ica.fit(X) #this works correctly.
Why does that normalization step fix the error?
I found the eureka moment here: sklearn's PLSRegression: "ValueError: array must not contain infs or NaNs"
What I think is happening is that numpy is being fed gigantic numbers and very tiny numbers, and inside it's tiny brain it's creating NaN's and Inf's. So it's a bug in the sklearn. The work around is to flatten your input data to the algorithm so that there are no very large or very small numbers.
Bad sklearn! NO biscuit!
I process rather large matrices in Python/Scipy. I need to extract rows from large matrix (which is loaded to coo_matrix) and use them as diagonal elements. Currently I do that in the following fashion:
import numpy as np
from scipy import sparse
def computation(A):
for i in range(A.shape[0]):
diag_elems = np.array(A[i,:].todense())
ith_diag = sparse.spdiags(diag_elems,0,A.shape[1],A.shape[1], format = "csc")
#...
#create some random matrix
A = (sparse.rand(1000,100000,0.02,format="csc")*5).astype(np.ubyte)
#get timings
profile.run('computation(A)')
What I see from the profile output is that most of the time is consumed by get_csr_submatrix function while extracting diag_elems. That makes me think that I use either inefficient sparse representation of initial data or wrong way of extracting row from a sparse matrix. Can you suggest a better way to extract a row from a sparse matrix and represent it in a diagonal form?
EDIT
The following variant removes bottleneck from the row extraction (notice that simple changing 'csc' to csr is not sufficient, A[i,:] must be replaced with A.getrow(i) as well). However the main question is how to omit the materialization (.todense()) and create the diagonal matrix from the sparse representation of the row.
import numpy as np
from scipy import sparse
def computation(A):
for i in range(A.shape[0]):
diag_elems = np.array(A.getrow(i).todense())
ith_diag = sparse.spdiags(diag_elems,0,A.shape[1],A.shape[1], format = "csc")
#...
#create some random matrix
A = (sparse.rand(1000,100000,0.02,format="csr")*5).astype(np.ubyte)
#get timings
profile.run('computation(A)')
If I create DIAgonal matrix from 1-row CSR matrix directly, as follows:
diag_elems = A.getrow(i)
ith_diag = sparse.spdiags(diag_elems,0,A.shape[1],A.shape[1])
then I can neither specify format="csc" argument, nor convert ith_diags to CSC format:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.6/profile.py", line 70, in run
prof = prof.run(statement)
File "/usr/local/lib/python2.6/profile.py", line 456, in run
return self.runctx(cmd, dict, dict)
File "/usr/local/lib/python2.6/profile.py", line 462, in runctx
exec cmd in globals, locals
File "<string>", line 1, in <module>
File "<stdin>", line 4, in computation
File "/usr/local/lib/python2.6/site-packages/scipy/sparse/construct.py", line 56, in spdiags
return dia_matrix((data, diags), shape=(m,n)).asformat(format)
File "/usr/local/lib/python2.6/site-packages/scipy/sparse/base.py", line 211, in asformat
return getattr(self,'to' + format)()
File "/usr/local/lib/python2.6/site-packages/scipy/sparse/dia.py", line 173, in tocsc
return self.tocoo().tocsc()
File "/usr/local/lib/python2.6/site-packages/scipy/sparse/coo.py", line 263, in tocsc
data = np.empty(self.nnz, dtype=upcast(self.dtype))
File "/usr/local/lib/python2.6/site-packages/scipy/sparse/sputils.py", line 47, in upcast
raise TypeError,'no supported conversion for types: %s' % args
TypeError: no supported conversion for types: object`
Here's what I came up with:
def computation(A):
for i in range(A.shape[0]):
idx_begin = A.indptr[i]
idx_end = A.indptr[i+1]
row_nnz = idx_end - idx_begin
diag_elems = A.data[idx_begin:idx_end]
diag_indices = A.indices[idx_begin:idx_end]
ith_diag = sparse.csc_matrix((diag_elems, (diag_indices, diag_indices)),shape=(A.shape[1], A.shape[1]))
ith_diag.eliminate_zeros()
Python profiler said 1.464 seconds versus 5.574 seconds before. It takes advantage of the underlying dense arrays (indptr, indices, data) that define sparse matrices. Here's my crash course: A.indptr[i]:A.indptr[i+1] defines which elements in the dense arrays correspond to the non-zero values in row i. A.data is a dense 1d array of non-zero the values of A and A.indptr is the column where those values go.
I would do some more testing to make very certain this does the same thing as before. I only checked a few cases.