I have a 100x200 input, and a 1x100 target matrix that I am using to run a gridsearch and create a classifier in python. However, I get errors that my training set of target data is not an array. I've tried:
target=np.asarray(matTarget)
Where the matTarget is just my target imported from Matlab using scipy.io.loadmat.
My exact error is
len() of unsized object
When I try target.size I get a blank size as well.
If I do not do the array conversion, then I get
Expected array-like (array or non string sequence) got {'_header_': b'Matlab matfile ... Array([[1],[1]...)}
I still have the original matrix in Matlab and have also tried using np.array instead of asarray.
If I do print(matTarget.keys()) then I get ('header`,'version','globals','y_train'])
y_train is the name of the mat file itself
According to the documentation of scipy.io.loadmat it returns a dictionary where the values are the contained matrices.
Returns: mat_dict : dict
dictionary with variable names as keys, and loaded matrices as values.
So you need to select your matrix by its name before using it with numpy:
matrix = matTarget['name of matrix']
Related
I am trying to convert an array of 3D tensors (images) to a single 4D one, so I can pass them as values to the model.fit which does not seem to accept Tensor3D arrays.
The idea would be
4dTensor = tf.tensor4d(batch)
I am actually using javascript, but either a python or js solution would probably work as the Tensorflow API is similar.
The error of this procedure is:
Argument of type 'Tensor4D' is not assignable to parameter of type 'Tensor3D[]'.
Type 'Tensor<Rank.R4>' is missing the following properties from type 'Tensor3D[]': length, pop, push, join, and 26 more.ts(2345)
You may want to use tf.stack():
4dTensor = tf.stack(batch)
So the logistic regression from the sklearn library from Python has the .fit() function which takes x_train(features) and y_train(labels) as arguments to train the classifier.
It seems that x_train.shape = (number_of_samples, number_of_features)
For x_train I should use the extracted xvector.scp file, which I am reading like so:
b = kaldiio.load_scp('xvector.scp')
And I can print the content like so:
for file_id in b:
xvector = b[file_id]
print(xvector)
Right now the b variable is like a dictionary and you can get the x-vector value of the corresponding id. I want to use sklearn Logistic Regression to classify the x-vectors and in order to use the .fit() method I should pass an array as an argument.
My question is how can I make an array that contains only the xvector variables?
PS: the file_ids are like 1 million and each xvector has length of 512, which is too big for an array
It seems you are trying to store the dictionary into a numpy array. If the dictionary is small, you can directly store the values as:
import numpy as np
x = np.array(list(b.values()))
However, this will run into OOM issues if the dictionary is large. In this case, you would need to use np.memmap as explained here: https://ipython-books.github.io/48-processing-large-numpy-arrays-with-memory-mapping/
Essentially, you have to add rows to the array one at a time, and flush it when you have run out of memory. The array is stored directly on the disk, so it avoids OOM issues.
I have an array of strings in Matlab and I want to use it with python. I tried to do the same process as for numeric arrays i.e.
np_array = scipy.io.loadmat(mat_array)
which works with numeric data.
But when I try to use a string array like
mat_str_array = [“This”, “is”, “a”, “string”]
I get an error/warning stating
cannot be transformed using the same method
Any suggestions?
Thanks in advance.
I have a tensorflow placeholder defined as:
fs = tf.placeholder(tf.float32, shape=(nn, mm))
Further in the code, I want to feed it.
I will obtain a numpy array "features" with shape = (nn, mm) and I write:
feed_dict.update({fs, features})
However, I get the error:
TypeError: Unhashable type"numpy.ndarray"
Because I already could feed a list with lenght = nn to a placeholder with shape = (nn,)
So before feeding the numpy array to the placeholder, I wrote
features = features.tolist() #to make them as a list, not numpy array!
again, I got a similar error:
TypeError: Unhashable type "list"
So, I was wondering how can I feed a 2d numpy array into a 2d tensorflow placeholder?
I also have checked that they all have np.float32 and tf.float32 datatypes!
I am using python3 with tensorflow version 1.1
There is a minor typo in your code. Where you wrote:
feed_dict.update({fs, features})
you should have written:
feed_dict.update({fs: features})
note the comma is replaced by a colon.
What's going on
In your code you accidentally tried to create a set containing fs and features, where what you meant to do was create a dictionary. In order to be placed in a set, a python object must implement a method called __hash__. Not all objects implement this method (for good reasons) and that includes lists and numpy arrays. So the reason you got the error message about an "unhashable type" is because you inadvertently tried to create a set containing features.
To visualize TensorType(float64, matrix) as an image using imshow, How could I that?I cannot directly use imshow on Tensor since it gives me this error
mat is not a numpy array, neither a scalar
When I try to convert datatype to array using numpy.asarray I get
mat data type = 17 is not supported
Is there any way to convert to uint8 datatype?
Theano tensors are symbolic variables, with no value attached to them. If you want to plot something it is probably a shared variable (the weights of your network), in which case you can grab the values with my_shared.get_value(), or the output of the network when you pass an input, in which case I believe it should already be a numpy array.