I'm trying to use numpy's savetxt function to generate a bunch of files as inputs for another piece of software.
I'm trying to write an array of the form:
a=np.array([1,2,3,4,...])
a.shape=>(1,n)
to a text file with the formatting
1,2,3,4,...
when I enter the command
np.savetxt('test.csv',a,fmt='%d',delimiter=',')
I get a file that looks like:
1
2
3
4
...
savetxt works as I would expect for a 2D array, but I can't get all of the values for a 1D array onto a single line
Any suggestions?
Thanks
EDIT:
I solved the problem. Using np.atleast_2d(a) as the input to savetxt forces savetxt to write the array as a row, not a column
There are different ways to fix this. The one closest to your current approach is:
np.savetxt('test.csv', a[None], fmt='%d', delimiter=',')
i.e. add the slicing [None] to your array to make it two-dimensional with only a single line.
If you only want to save a 1D array, it's actually a lot faster to use this method:
>>> x = numpy.array([0,1,2,3,4,5])
>>> ','.join(map(str, x.tolist()))
'0,1,2,3,4,5'
Related
I'm generating a number of test files iteratively, the process derives a 0, 1 or 2 dimensional numpy array, then writes that array to CSV, at least that's the intent.
Does anyone have a good solution for this?
My code (expectedly) fails when the output is zero-dimensional (scalar):
for key in testfiles:
tname = key + ".csv"
np.savetxt(tname, testfiles[key], delimiter=",",newline=';',fmt='%0.15f')
There are a couple of ways to ensure that your input is not a scalar in numpy. For example, you could use np.array:
arr = np.array(testfiles[key], ndmin=1, copy=False)
Another option is np.atleast_1d:
arr = np.atleast_1d(testfiles[key])
Both options will attempt to make an object without copying the data. In both cases, pass arr to np.savetxt instead of testfiles[key].
I am working on a Deep Learning project (in Python), and I had a problem with my code. I get an output like this:
[[-0.00111287 -0.97692661 -0.9939433 -0.99474857]]
and I want to change that to this:
[-0.00111287 -0.97692661 -0.9939433 -0.99474857]
Can anyone help me with a simple function. I could not find any function that did the job.
Thanks in advance!
I assume that you have a numpy.ndarray? You can flatten any array into 1D by doing array.ravel(). Additionally, you can reshape an array into any desired shape (compatible with the number of elements in the array) using array.reshape. array.reshape(-1) is the same as array.ravel(). See here for more on reshaping arrays. Finally, you could also get array_1d = array_2d[0], but I don't find this as clear in code because it seems to the reader like you are just selecting the first row of a 2D array. Doing array_1d = array_2d.ravel() makes it clear that you are keeping all elements, just flattening the array.
Try this:
import numpy as np
my_matrix = [[-0.00111287, -0.97692661, -0.9939433, -0.99474857]]
result = list(np.array(my_matrix).reshape(-1))
print(result)
# [-0.00111287, -0.97692661, -0.9939433, -0.99474857]
If you're not working with a numpy array, itertools provides a function called chain() that can be used to flatten a list:
from itertools import chain
array = list(chain(*matrix))
Which is the most performant way
to convert something like that
problem = [ [np.array([1,2,3]), np.array([4,5])],
[np.array([6,7,8]), np.array([9,10])]]
into
desired = np.array([[1,2,3,4,5],
[6,7,8,9,10]])
Unfortunately, the final number of columns and rows (and length of subarrays) is not known in advance, as the subarrays are read from a binary file, record by record.
How about this:
problem = [[np.array([1,2,3]), np.array([4,5])],
[np.array([6,7,8]), np.array([9,10])]]
print np.array([np.concatenate(x) for x in problem])
I think this:
print np.array([np.hstack(i) for i in problem])
Using your example, this runs in 0.00022s, wherease concatenate takes 0.00038s
You can also use apply_along_axis although this runs in 0.00024s:
print np.apply_along_axis(np.hstack, 1, problem)
I loaded a text file containing a two column matrix (e.g. below)
[ 1 3
2 4
3 5
2 0]
My calculation is just to sum each row i.e. 1+3, 2+4, 3+5 and 2+0. I am using the below code:
data=np.loadtxt(fname="textfile.txt")## to load the above two column
xy= data
for XY in xy:
i=0
Z=XY(i,0)+XY(i,1)
i=i+1
print (Z)
But I received an error saying numpy.ndarray object is not callable. Why does this happen? How can I do this simple calculation? Thanks.
The error TypeError: 'numpy.ndarray' object is not callable means that you tried to call a numpy array as a function.
Use
Z=XY[0]+XY[1]
Instead of
Z=XY(i,0)+XY(i,1)
Sometimes, when a function name and a variable name to which the return of the function is stored are same, the error is shown. Just happened to me.
Avoid loops. What you want to do is:
import numpy as np
data=np.loadtxt(fname="data.txt")## to load the above two column
print data
print data.sum(axis=1)
Avoid the for loopfor XY in xy:
Instead read up how the numpy arrays are indexed and handled.
Numpy Indexing
Also try and avoid .txt files if you are dealing with matrices.
Try to use .csv or .npy files, and use Pandas dataframework to load them just for clarity.
Not for the example asked above but sometimes this error happens because you forget to specify brackets [] instead of parenthesis for your numpy.ndarray. Such as writing arr(x,y) in a for loop that explores x and y in arr instead of its correct form: arr[x,y].
Let's say i'm making a loop, and after each iteration, y want to extend some array.
iter 1 ------------> iter 2 --------------> iter 3-------------->....
shape=[2,4]---->shape=[2,12]----->shape=[2,36]---->....
in fortran i used to do this by appending the new numbers to a binary file with:
OPEN(2,file='array.in',form='unformatted',status='unknown',access='stream')
write(2) newarray
so this would extend the old array with new values at the end.
i wish to do the same in python. This is my attempt so far:
import numpy as np
#write 2x2 array to binfile
bintest=open('binfile.in','wb')
np.ndarray.tofile(np.array([[1.0,2.0],[3.0,4.0]]),'binfile.in')
bintest.close()
#read array from binfile
artest=np.fromfile('binfile.in',dtype=np.float64).reshape(2,2)
But i can't get it to extend the array. Lets say.. by appeding another [[5.0,5.0],[5.0,5.0]] at the end,
#append new values.
np.ndarray.tofile(np.array([[5.0,5.0],[5.0,5.0]]),'binfile.in')
to make it [[1.0,2.0,5.0,5.0],[3.0,4.0,5.0,5.0]] after the reading.
How can i do this?
The other problem i have, is that i would like to be able to make this without knowing the shape of the final array (i know it would be 2 x n ). But this is not so important.
edit: the use of 'access=stream' is only to skip having to read format headers and tails.
This does the trick:
import numpy as np
#write
bintest=open('binfile.in','ab')
a=np.array([[1.0,2.0],[3.0,2.0]])
a.tofile(bintest)
bintest.close()
#read
array=np.fromfile('binfile.in',dtype=np.float64)
this way, each time its run, it appends the new array to the end of the file.