I need to have each numpy file from a folder like an array with 1 dimension; This is my code:
path ='E:\\t'
traces= os.listdir(path)
print("tempTracesHW_Creation=", tempTracesHW)
for i in range(len(traces)):
HW = tempHW[i]
for trace in os.listdir(path):
file_array= np.load(os.path.join(path, trace))
print file_array
tempTracesHW[HW].append(file_array)
The result of file_array is:
file_array= [[-0.0006447 -0.00094265 -0.0012406 ..., -0.02096185 -0.0210646
-0.02114679]]
But what I want is:
file_array= [-0.0006447 -0.00094265 -0.0012406 ..., -0.02096185 -0.0210646
-0.02114679]
I would be very grateful if you could help me please?
The numpy load function loads the file and return the array.
The file_array is two dimensional because your input to numpy.load is two dimensional.
Check the trace file, you need to make it one-dimensional array.
For example:
example = numpy.save("example",numpy.array([1,2,3]))
result = numpy.load("example.npy")
print result
[1,2,3]
See if this helps.
More of the code snippet would have help understanding your problem.(About the Trace file)
You can use flatten to turn this (1, x) array into a (x, ) array. flatten can be used differently, but in this case, it will do what you're looking for.
>>> import numpy as np
>>> a = np.array([[1, 2, 3, 4, 5]])
>>> a
array([[1, 2, 3, 4, 5]])
>>> a.shape
(1, 5)
>>> a.flatten()
array([1, 2, 3, 4, 5])
>>> a.flatten().shape
(5,)
Related
I'm trying to create a 2d list with shape of [n,784] (the same shape as the MNIST image batches) using multiple [1,784] lists.
mylist.append(element) doesn't give me what I'm looking for, where mylist is the 2d [n,784] list and element is the [1,784] lists. It would return a list with shape [n,1,784].
I've also tried mylist[index].append(element), and I got a [784] 1d list instead.
Any idea how to solve my problem?
Thanks a lot
import numpy as np
myarray = np.array(mylist)
newarray = np.concatenate((myarray, element))
And if you want to turn it back into a list:
newlist = newarray.tolist()
a = [[1,1],[2,2]]
b = np.concatenate([a, a], axis=1).tolist()
The output will be:
[[1, 1, 1, 1], [2, 2, 2, 2]]
For example,I want to make two datasets, one is Input ,the other is Output
The data in Input and Output are multi-dims.
such as
But I notice in h5py,input_node and output_node is fixed.
Input = f.create_dataset('Input', (3,input_node ),dtype='float', chunks=True)
Output = f.create_dataset('Output', (3,output_node),dtype='float', chunks=True)
But hdf5 can't handle this,this code can prove it
import h5py
X = [[1,2,3,4],[1,2],[1,2,3,4,5,6]]
with h5py.File('myfile.hdf5', "w") as ofile:
ofile.create_dataset("X", data=X)
TypeError: Object dtype dtype('O') has no native HDF5 equivalent
So how to make a multi-dims dataset in h5py?
I don't quite follow what your {...} denote. In Python those are used for dictionaries and sets. [] are used for lists, () for tuples. Array shape is expressed as a tuple.
Anyways, your code produces
In [68]: X
Out[68]:
array([ list([0.6503719194043309, 0.8703218883225239, -1.4139639093161405, 2.3288987644271835, -1.7957516518177206]),
list([-0.1781710442823114, 0.9591992379396287, -0.6319292685053243]),
list([0.7104492662861611, -0.8951817329357393, -0.8925882332063567, 1.5587934871464815]),
list([-1.2384976614455354, 0.9044140291496179, 1.1277220227448401]),
list([1.1386910680393805, -0.1775792543137636, 1.0567836199711476]),
list([2.7535019220459707, 0.29518918092088386, -0.32166742909305196, 1.5269788560083497, 0.29633276686886767]),
list([1.6397535315116918, -0.8839570613086122, -0.4491121599234047, -2.4461439611764333, -0.6884616200199412, -1.1920165045444608]),
list([1.3240629024597295, 1.170019287452736, 0.5999977019629572, -0.38338543090263366, 0.6030856099472732]),
list([-0.013529997305716175, -0.7093551284624415, -1.8611980839518099, 0.9165791506693297]),
list([2.384081118320432, -0.6158201308053464, 0.8802896893269192, -0.7636283160361232])], dtype=object)
In [69]: y
Out[69]: array([1, 1, 0, 0, 0, 1, 1, 0, 1, 0])
y is a simple array. h5py should have no problem saving that.
X is an object dtype array, containing lists of varying size
In [72]: [len(l) for l in X]
Out[72]: [5, 3, 4, 3, 3, 5, 6, 5, 4, 4]
h5py cannot save that kind of array. At best you can write each element to a different dataset. It will save each as an array.
....
for i, item in enumerate(X):
ofile.create_dataset('name%s'%i, data=item)
I am trying to append a new row to an existing numpy array in a loop. I have tried the methods involving append, concatenate and also vstack none of them end up giving me the result I want.
I have tried the following:
for _ in col_change:
if (item + 2 < len(col_change)):
arr=[col_change[item], col_change[item + 1], col_change[item + 2]]
array=np.concatenate((array,arr),axis=0)
item+=1
I have also tried it in the most basic format and it still gives me an empty array.
array=np.array([])
newrow = [1, 2, 3]
newrow1 = [4, 5, 6]
np.concatenate((array,newrow), axis=0)
np.concatenate((array,newrow1), axis=0)
print(array)
I want the output to be [[1,2,3][4,5,6]...]
The correct way to build an array incrementally is to not start with an array:
alist = []
alist.append([1, 2, 3])
alist.append([4, 5, 6])
arr = np.array(alist)
This is essentially the same as
arr = np.array([ [1,2,3], [4,5,6] ])
the most common way of making a small (or large) sample array.
Even if you have good reason to use some version of concatenate (hstack, vstack, etc), it is better to collect the components in a list, and perform the concatante once.
If you want [[1,2,3],[4,5,6]] I could present you an alternative without append: np.arange and then reshape it:
>>> import numpy as np
>>> np.arange(1,7).reshape(2, 3)
array([[1, 2, 3],
[4, 5, 6]])
Or create a big array and fill it manually (or in a loop):
>>> array = np.empty((2, 3), int)
>>> array[0] = [1,2,3]
>>> array[1] = [4,5,6]
>>> array
array([[1, 2, 3],
[4, 5, 6]])
A note on your examples:
In the second one you forgot to save the result, make it array = np.concatenate((array,newrow1), axis=0) and it works (not exactly like you want it but the array is not empty anymore). The first example seems badly indented and without know the variables and/or the problem there it's hard to debug.
just wondering if there is any clever way to do the following.
I have an N dimensional array representing a 3x3 grid
grid = [[1,2,3],
[4,5,6],
[7,8,9]]
In order to get the first row I do the following:
grid[0][0:3]
>> [1,2,3]
In order to get the first column I would like to do something like this (even though it is not possible):
grid[0:3][0]
>> [1,4,7]
Does NumPy support anything similar to this by chance?
Any ideas?
Yes, there is something like that in Numpy:
import numpy as np
grid = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
grid[0,:]
# array([1, 2, 3])
grid[:,0]
# array([1, 4, 7])
You can use zip to transpose a matrix represented as a list of lists:
>>> zip(*grid)[0]
(1, 4, 7)
Anything more than just that, and I'd use Numpy.
To get the columns in Python you could use:
[row[0] for row in grid]
>>> [1,4,7]
You could rewrite your code for getting the row as
grid[0][:]
because [:] just copies the whole array, no need to add the indices.
However, depending on what you want to achieve, I'd say it's better to just write a small matrix class to hide this implementation stuff.
How do I convert a NumPy array into a Python List?
Use tolist():
>>> import numpy as np
>>> np.array([[1,2,3],[4,5,6]]).tolist()
[[1, 2, 3], [4, 5, 6]]
Note that this converts the values from whatever numpy type they may have (e.g. np.int32 or np.float32) to the "nearest compatible Python type" (in a list). If you want to preserve the numpy data types, you could call list() on your array instead, and you'll end up with a list of numpy scalars. (Thanks to Mr_and_Mrs_D for pointing that out in a comment.)
c = np.array([[1,2,3],[4,5,6]])
list(c.flatten())
The numpy .tolist method produces nested lists if the numpy array shape is 2D.
if flat lists are desired, the method below works.
import numpy as np
from itertools import chain
a = [1,2,3,4,5,6,7,8,9]
print type(a), len(a), a
npa = np.asarray(a)
print type(npa), npa.shape, "\n", npa
npa = npa.reshape((3, 3))
print type(npa), npa.shape, "\n", npa
a = list(chain.from_iterable(npa))
print type(a), len(a), a`
tolist() works fine even if encountered a nested array, say a pandas DataFrame;
my_list = [0,1,2,3,4,5,4,3,2,1,0]
my_dt = pd.DataFrame(my_list)
new_list = [i[0] for i in my_dt.values.tolist()]
print(type(my_list),type(my_dt),type(new_list))
Another option
c = np.array([[1,2,3],[4,5,6]])
c.ravel()
#>> array([1, 2, 3, 4, 5, 6])
# or
c.ravel().tolist()
#>> [1, 2, 3, 4, 5, 6]
also works.
The easiest way to convert array to a list is using the numpy package:
import numpy as np
#2d array to list
2d_array = np.array([[1,2,3],[8,9,10]])
2d_list = 2d_array.tolist()
To check the data type, you can use the following:
type(object)