Can I attach metadata to NumPy arrays? - python

I am writing a pipeline-style image processing program. I would like to log all intermediate steps it takes.
Currently, it loads all images from a directory and stores them all as NumPy arrays (for further processing with OpenCV). I would like to attach a string tag to each array to see the name of the original file it came from, so I don't have to drag that name everywhere. Is there a simple way to do that?

Related

how to change field variable/cell values in vti file with python script

I have a vti file which contains certain geometry with hexagonal mesh. After a loading step a field variable name "concentration" changes and must be changed back zero. There is one possibility in paraview by hard way. Can any body share a way how to open, edit a field variable and overwrite a vti file with python.
Thanks.
You can use vtk python module to do that.
read with vtkXMLImageDataReader
Get the array to modify array = reader.GetOutput().GetCellData().GetArray("concentration")
modify the array values by index: array.InsertTuple(i, 0)
write back with vtkXMLImageDataWriter
See the read/write example
That is the native VTK solution. There is some other ways, as using numpy to modify the data array, or do it in ParaView python scripting

Python - vtkDICOMImageReader array input?

I'm building a system for viewing DICOM files. DICOM files located in the specified folder are read with dcmread and put in a list. I check the metadata to separate the series by the series number in the information, then I create a dictionary with several lists, one for each series, which contain the respective scans. In the program, therefore, I can select which series to display with the 3D reconstruction. I noticed however that with vtkDICOMImageReader I can only specify a file or a directory. Can I also select a list containing DICOM files in some way?
vtkDICOMImageReader derives from vtkImageReader2, you can use the vtkImageReader2::SetFileNames(vtkStringArray *)
https://vtk.org/doc/nightly/html/classvtkImageReader2.html#ac084edcfab5d27247a7683892a95617e
By design vtkImageReader2 needs to read the files from disk (think UpdateExtent != WholeExtent).
If you are looking to import a c-array as an image into VTK, you should instead start using:
https://vtk.org/doc/nightly/html/classvtkImageImport.html

how to load a very large mat file in chunks?

okay so the code is like this
X1 is the loaded hyperspectral image with dimensions (512x512x91)
what i am trying to do is basically crop 64x64x91 sized matrices with a changing stride of 2. this gives me a total of 49952 images each of 64x64x91 size however when i run the for loop i get the memory error.
my system has 8 GB ram.
data_images_0=np.zeros((49952,256,256,91))
k=0
for i in range(0,512-64,2):
r=64
print(k)
for j in range (0,512-64,2):
#print(k)
data_images_0[k,:,:,:]=X1[i:i+r,j:j+r,:]
k=k+1
I have a hyperspectral image loaded as a Mat file and the dimensions are (512x512x91). I want to use chunks of this image as the input to my CNN for example using crops of 64x64x91. The problem is once i create the crops out of the original image i have trouble loading the data as loading all the crops at once gives me memory error.
Is there something i can do to load my cropped data in batches so that i dont receive such a memory error.
Should i convert my data into some other format or proceed the problem in some other way?
You are looking for the matfile function. It allows you to access the array on your harddisk and then only load parts of it.
Say your picture is named pic, then you can do something like
data = matfile("filename.mat");
part = data.pic(1:64,1:64,:);
%Do something
then only the (1:64,1:64,:) part of the variable pic will be loaded into part.
As always it should be noted, that working on the harddisk is not exactly fast and should be avoided. On the other hand if your variable is too large to fit in the memory, there is no other way around it (apart from buying more memory).
I think you might want to use the matfile function, which basically opens a .mat file without pulling its entire content into the RAM. You basically read a header from your .mat file that contains information about the stored elements, like size, data type and so on. Imagine your .mat file hyperspectralimg.mat containing the matrix myImage. You'd have to proceed like this:
filename = 'hyperspectralimg.mat';
img = matfile(filename);
A = doStuff2MyImg(img.myImage(1:64,1:64,:)); % Do stuff to your imageparts
img.myImage(1:64,1:64,:) = A; %Return changes to your file
This is a brief example how you can use it in case you haven't used matfile before. If you have already used it and it doesn't work, let us know and as a general recommendation upload code snippets and data samples regarding your issues, it helps.
A quick comment regarding the tags: If your concern regards matlab, then don't tag python and similar things.
You can use numpy memory map. This is equivalent to matfile of MatLAB.
https://numpy.org/doc/stable/reference/generated/numpy.memmap.html

How to build a file to save different Python objects as a project file?

I'm creating a GUI using PyQt4 and Python 2.7, I have to process different images with advanced techniques. The images are processed as NumPy arrays, however, in the program there are more than one image and other data (dictionaries, tuples, lists, NumPy arrays) that should be stored preferably in just one file. It would be great that all this data could be stored in a file as file_name.project (with a custom extension). I don't know how to do this. There is an easy way to do this stuff? What do you recommend me?

Unittest binary file output

I have an array of pixels which I wish to save to an image file. Python appears to have a few libraries which can do this for me, so I'm going to use one of them, passing in my pixel array and using functions I didn't write to write the image headers and data to disk.
How do I do unit testing for this situation?
I can:
Test that the pixel array I'm passing to the external library is what I expect it to be.
Test that the external library functions I call give me the expected return values.
Manually verify that the image looks like I'm expecting (by opening the image and eyeballing it).
I can't:
Test that the image file is correct. To do that I'd have to either generate an image to compare to (but how do I generate that 'trustworthy' image?), or write a unit-testable image-writing module (so I wouldn't need to bother with the external library at all).
Is this enough to provide coverage for my code? Is testing the interface between my code and the external library sufficient, leaving me to trust that the output of the external library (the image file) is correct through manual eyeballing?
How do you write unit tests to ensure that the external libraries you use do what you expect them to?
Bit old on Python.
But this is how I would approach it.
Grab the image doing a manual test. Compute a check sum (MD5 perhaps). Then the automated tests need to compare it by computing the MD5 (in this example) with the one done on the manual test.
Hope this helps.

Categories

Resources