In pytorch, I want to write a tensor to a file and visually read the file contents. For example, consider T = torch.tensor([3,4,5,6]). I want to write the tensor T to a file, say file_T.txt, and want to visually read the contents of the file_T.txt, which will be 3,4,5 and 6. How can I achieve this?
You can use numpy:
import numpy as np
np.savetxt('my_file.txt', torch.Tensor([3,4,5,6]).numpy())
You can use numpy.savetxt() but it won't work if your tensor's dimensions are higher than 2. Moreover, this method could be used only if your tensor is on a CPU.
Related
Greeting I know how to import .raw data images into the jupyter notebook but I don.t seem to understand the logic explanation.. Below is the code give, please explain these line to me in as simple way as possible
# Read input RAW file
raw_file = np.fromfile('Sandstones/'+name+'_2d25um_binary.raw', dtype=np.uint8)
im = (raw_file.reshape(1000,1000,1000))
im = im==0;
I tried alternate ways but this was the best as well as the preferred. I have run it but I don't seem to understand the last line
name is some variable. Let's say it is name of your file. np.fromfile takes in a path, and the datatype it wants to read, and convert it into an array.
so
'Sandstones/'+name+'_2d25um_binary.raw'
is the path, if the name is file1 then it will be
'Sandstones/file1_2d25um_binary.raw'
You read this file into a numpy array which is a special data structure as uint8.
In the next line of code, you simply reshape the values you read from the file to a 3D array with the shape
1000, 1000, 1000
The last line changes the im array values. If you notice, im is your numpy array, All the values, which are equal to 0 will be changed to 1 as im==0 will be true, rest will be changed to false. You can think of these as a Boolean array of 0s and 1s where 1 are those number which were 0 in the original array. You can learn more about it here
I have a big file saved from matlab with version -v7.3, when reading it by python, the shape of matrix change !! is that normal ?
For example, let's have the below matrix in MATLAB,
clear all, clcl
A = randn(10,3) + randn(10,3)*i;
save('example.mat','-v7.3'); %% The saved file is example.mat with version 7.3
above, the saved file is example.mat a matrix of size (10,3)
so, let's go to python to read that file :
import numpy as np
import h5py as h5
data_try = h5.File('example.mat', 'r')
A = np.array(data_try)
A = A.view(np.complex) #here the matrix equivalent to that one in matlab
but what i find that A in python is of size (3,10) !! and also when having matrix of three dimensions, the shape is changing !!
Is that normal that python reads the transpose of matrix coming from matlab ??!! or something wrong is happening !
However when using the other way as below:
import scipy.io as spio
Data = spio.loadmat('example.mat', squeeze_me=True)
A = Data[‘A’]
in that case, everything is really nice, but unfortunately we can not use that way for big matrices !!!
please, any solution for that issue ?
You might face a problem with different memory alignment in Matlab (column-major) and Numpy (row-major)... check e.g. this question for related discussion and a solution (reshaping in Fortran-style, which is also column-major).
SciPy's .mat interface automatically takes care of this reinterpretation, which is why you don't encounter the problem when using it.
So i'm working on a code where i need to write and read files. I'm using python and numpy, but the numpy fromfile function does not seem to work propperly. First i create an array with 500 elements, and save it with the savetxt function. I check the file and it is all right, just how i wanted.
import numpy as np
w = np.zeros(500, float)
np.savetxt("weights.txt", weight, '%.100f', )
print(weight[2])
But after i change the line where i create the array with the one where i read it from a file a problem accours. The zeros turn into really small numbers. I can't guess why. Here is the line where i read from file:
weight = np.fromfile("weights.txt", float, -1)
Should i write a custom function that turns files into arrays or is there a way to make it work?
What is the most efficient way of saving a numpy masked array? Unfortunately numpy.save doesn't work:
import numpy as np
a = np.ma.zeros((500, 500))
np.save('test', a)
This gives a:
NotImplementedError: Not implemented yet, sorry...
One way seems to be using pickle, but that unfortunately is not very efficient (huge file sizes), and not platform-independent. Also, netcdf4 seems to work, but it has a large overhead just to save a simple array.
Anyone has had this problem before? I'm tempted just to do numpy.save of array.data and another for the mask.
import numpy as np
a = np.ma.zeros((500, 500))
a.dump('test')
then read it with
a = np.load('test')
The current accepted answer is somewhat obsolete, and badly inefficient if the array being stored is sparse (it relies on uncompressed pickling of the array).
A better way to save/load a masked array would be to use an npz file:
import numpy as np
# Saving masked array 'arr':
np.savez_compressed('test.npz', data=arr.data, mask=arr.mask)
# Loading array back
with np.load('test.npz') as npz:
arr = np.ma.MaskedArray(**npz)
If you have a fixed mask that doesn't need to be saved, then you can just save the valid values:
a = np.ma.MaskedArray(values,mask)
np.save('test', a.compressed())
You can then recover it doing something like:
compressed = np.load('test')
values = np.zeros_like(mask, dtype=compressed.dtype)
np.place(values, ~mask, compressed)
a = np.ma.MaskedArray(values, mask)
A simple way to do it would be to save the data and mask of the masked array separately:
np.save('DIN_WOA09.npy',DIN_woa.data)
np.save('mask_WOA09.npy',DIN_woa.mask)
Then later, you can reconstruct the masked array from the data and mask.
Saving it inside a dictionary will allow you to keep its original format and mask without any trouble. Something like:
b={}
b['a'] = a
np.save('b', b)
should work fine.
I'm trying to extract a slice from a volume using numpy.
The volume is 512x512x132 and I want the slice number 66.
Each voxel is an unsigned 16-bit integer.
My code is:
import numpy
original = numpy.fromfile(filepath, dtype=numpy.uint16)
original = numpy.reshape(original, (512,512,132))
slice = original[:,:,66]
f = open('test.rawl', 'w')
slice.tofile(f)
f.close()
The code complete cleanly, but when I open the slice with an external program, it is not the slice data but garbage.
What I am doing wrong?
Thanks
Your first problem is that you have your axes wrong. Assuming you have 132 layers of 512x512 images you want to use:
original = numpy.fromfile(filepath, dtype=numpy.uint16).reshape((132, 512, 512))
Then for the slice take:
slc = original[66]
Also, binary data such as Numpy arrays use:
f = open('test.raw', 'wb')
The 'b' in the mode is for binary. Otherwise Python will assume you're trying to write text and do things like convert newlines to the appropriate format for the system, among other things.
By the way, the ndarray.tofile() method also takes a filename, so unless you have a particular reason to it's not necessary to create a file handle first. You can just use
arr.tofile('test.raw')
One final note: Try not to use slice as a variable. That's a builtin name in Python and you could run into trouble by shadowing it with something else.