I have some .bsq files with corresponding header files (.hdr). I want to open and edit them in Python, using the spectral (SPy) module.
Since I need to edit the files from within a Python Toolbox in ArcMap (that is, a single Python script which uses the Python installation that comes with ArcMap), I decided to copy the GitHub repository of the SPy module and import it as
import sys
sys.path.append("/path/to/module")
import spatial as spy
in order to be able to run the toolbox on any computer without having to install pip or other software. (I intend to just copy the Toolbox and the module folder, or, in a later step, create a single Python script comprising the Toolbox as well as the SPy-module code.)
I opened some .bsq-file and tried to subsequently edit it as memmap, following this example.
First, I opened the image as spectral.io.bsqfile.BsqFile:
path = "/path/to/image_header.hdr"
img = spy.open_image(path)
I am able to apply various methods to img (such as: view metadata, read bands as array, etc), hence, I assume there were no issues with the path or the image file. I can also read single bands with memmap = True:
mem_band = img.read_band(0, use_memmap = True)
Reading a single band results in an array of shape (lines, samples) with dtype: float64 and where lines and sample correspond to the respective values in the .hdr file.
However, trying to apply the .open_memmap() method to the BsqFile instance as follows:
mem = img.open_memmap(writable = True)
results in the following error:
Runtime error
Traceback (most recent call last):
File "<string>", line 1, in <module>
File
"/path/to/module/spectral/io/spyfile.py", line 809, in open_memmap dst_inter))
File "/path/to/lib/site-packages/numpy/core/fromnumeric.py", line 536, in transpose
return _wrapit(a, 'transpose', axes)
File "/path/to/lib/site-packages/numpy/core/fromnumeric.py", line 45, in _wrapit
result = getattr(asattr(obj), method)(*args, **kwds)
ValueError: axes don't match array
Is this due to some incompatibilities between the numpy version that comes with the ArcMap-Python-Installation which I am required to use (numpy version 1.9.2)? Or are there other issues with the code or set-up?
Python version: 2.7.10
numpy version: 1.9.2
spectral version: 0.23.1
Edit
With the given Python version, spectral.envi.create_image() cannot create images of the given size due to an OverflowError. Potentially, this older Python version does not handle large numbers "correctly"?
Using another Python installation, the .open_memmap() method worked without issues.
Related
i have a .mat file containing annotations for some images, I need to read and manipulate specified values from this file using python and am stuck I tried to use h5py it doesn't work for me. with scipy I can read the file and print the whole file but I can't get a specified value.
I captured the structure of my mat file using octave can anyone help me to get the BBox values for each ImgName and save them into a variable.
screen shot of the annotation.mat file
Problem solved using this ( it may help some beginners like me) :
import scipy.io as spio
anno=spio.loadmat('annotation_1.mat')
#BBox. X y
#print(anno['annot'][0][1][0])
listX=anno['annot'][0][1][0]
print(listX[1][1])
#ImgName
#print(anno['annot'][0][1][1])
img=anno['annot'][0][1][1]
print(img)
I generate a 4D complex numpy array
for example:
numpy_array = np.tile(np.array([3, 4, 0], dtype=np.complex64), (100,100,100,1))
i want to save the numpy array as a nifti file
I tried using vtk and SimpleITK, but both of them don't support complex numbers (only vectors which are real numbers)
It seems that only nibabel support complex vectors and i manage to save the file, but i can only load it with nibabel, when i try to load it with ITK-SNAP or with Slicer it doesn't open. I know that ITK-SNAP can open complex vector of nifti files because i already have those files saved with another script using matlab
import numpy as np
import nibabel as nib
import SimpleITK as sitk
import vtk
from vtk.util.numpy_support import numpy_to_vtk
numpy_array = np.tile(np.array([3, 4, 0], dtype=np.complex64), (100,100,100,1))
nibabel save try:
image = nib.Nifti2Image(numpy_array, affine=np.eye(4))
nib.save(image , r'file_name.nii')
SimpleITK save try:
image = sitk.GetImageFromArray(numpy_array)
sitk.writeimage(image, r'file_name.nii')
vtk save try:
array = np.concatenate((np.real(numpy_array), np.imag(numpy_array)), axis=3)
stacked_array = array.reshape(-1, array.shape[-1])
vtk_array = numpy_to_vtk(stacked_array, deep=True,
array_type=vtk.VTK_FLOAT)
vtk_image = vtk.vtkImageData()
vtk_image.SetDimensions(numpy_array.shape[0], numpy_array.shape[1],
numpy_array.shape[2])
vtk_image.GetPointData().SetScalars(vtk_array)
writer = vtk.vtkNIFTIImageWriter()
writer.SetFileName(file_name)
writer.SetInputData(vtk_image)
writer.Write()
nibabel output:
nibabel creates a nifti file with vector but with other programs like ITK-SNAP it doens't open
ITK-SNAP error:
Error: Unsupported or missing image file format. ITK-SNAP failed to create an
ImageIO object for the image 'file_name.nii' using format ''.
SimpleITK error:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\SimpleITK\SimpleITK.py", line 3366, in _get_sitk_vector_pixelid
return _np_sitk[numpy_array_type.dtype]
KeyError: dtype('complex64')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\ProgramData\Anaconda3\lib\site-packages\SimpleITK\SimpleITK.py", line 3431, in GetImageFromArray
id = _get_sitk_vector_pixelid( z )
File "C:\ProgramData\Anaconda3\lib\site-packages\SimpleITK\SimpleITK.py", line 3371, in _get_sitk_vector_pixelid
raise TypeError('dtype: {0} is not supported.'.format(numpy_array_type.dtype))
TypeError: dtype: complex64 is not supported.
vtk output:
vtk creates a vector nifti file but with 6 components instead of 3
(consider the imaginary part as components too), i saw in the
documentation of numpy_to_vtk that it doesn't support copmlex arrays,
maybe someone know about a workaround.
The SimpleITK 1.2 release supports writing 4D complex images. The missing feature is support for complex number in stk.GetImageFromArray, but it's already added to the development branch and your code works if you a compiled version of the current SimpleITK master branch or wait for the 1.3 release scheduled for October 2019.
In the mean time you can convert the complex number components to SimpleITK separately like this:
image = sitk.RealAndImaginaryToComplexImageFilter(sitk.GetImageFromArray(numpy_array.real),
sitk.GetImageFromArray(numpy_array.imag))
sitk.WriteImage(image, r'file_name.nii')
I tried tflite_convert to convert my saved_model.pb(object detection API) file to .tflite but when i execute this command on cmd on the directory of C:\Users\LENOVO-PC\tensorflow> where tensorflow git is cloned,
tflite_convert \ --output_file=/saved_model/maonani.tflite \ --saved_model_dir=/saved_model/saved_model
I get an error saying
ImportError: No module named 'tensorflow.contrib.lite.python.tflite_convert'
the complete sourcelog is
C:\Users\LENOVO-PC\tensorflow>tflite_convert \ --output_file=/saved_model/maonani.tflite \ --saved_model_dir=/saved_model/saved_model
Traceback (most recent call last):
File "c:\users\lenovo-pc\appdata\local\programs\python\python35\lib\runpy.py", line 184, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\lenovo-pc\appdata\local\programs\python\python35\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\LENOVO-PC\AppData\Local\Programs\Python\Python35\Scripts\tflite_convert.exe\__main__.py", line 5, in <module>
ImportError: No module named 'tensorflow.contrib.lite.python.tflite_convert'
is there anyway to convert my .pb file to .tflite on WINDOWS?
Hi my solution was using linux in the following way
Windows Subsystem for Linux - see
then install from store ubunto
then need to pip3 install --upgrade "tensorflow=1.7*"
then if you try to run toco it would not be recognized.
the solution is go to folder
~/.local/bin/
there you would find toco, this is a python file.
run
python3 ~/.local/bin/toco
you would get the "exe" of toco.
to convert you can run the command explained in https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/#2
just change the -graph_def_file=tf_files/retrained_graph.pb to --input_file=tf_files/retrained_graph.pb
Hope this helps someone
Note:
if you are missing pip3, you would need to install it
I follow the instruction on this site:
https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/#2
However, it seems that tflearn_convert doesn't support Windows anymore. So i decided
to use Ubuntu on Windows. After having created a virtual environment to install tensorflow, i checked "toco" by typing toco in the terminal. And there go the instruction to use toco.
usage: /home/hieu/venv/bin/toco
Flags:
--input_file="" string Input file (model of any supported format). For Protobuf formats, both text and binary are supported regardless of file extension.
--output_file="" string Output file. For Protobuf formats, the binary format will be used.
--input_format="" string Input file format. One of: TENSORFLOW_GRAPHDEF, TFLITE.
--output_format="" string Output file format. One of TENSORFLOW_GRAPHDEF, TFLITE, GRAPHVIZ_DOT.
--default_ranges_min=0.000000 float If defined, will be used as the default value for the min bound of min/max ranges used for quantization.
--default_ranges_max=0.000000 float If defined, will be used as the default value for the max bound of min/max ranges used for quantization.
--inference_type="" string Target data type of arrays in the output file (for input_arrays, this may be overridden by inference_input_type). One of FLOAT, QUANTIZED_UINT8.
--inference_input_type="" string Target data type of input arrays. If not specified, inference_type is used. One of FLOAT, QUANTIZED_UINT8.
--input_type="" string Deprecated ambiguous flag that set both --input_data_types and --inference_input_type.
--input_types="" string Deprecated ambiguous flag that set both --input_data_types and --inference_input_type. Was meant to be a comma-separated list, but this was deprecated before multiple-input-types was ever properly supported.
--drop_fake_quant=false bool Ignore and discard FakeQuant nodes. For instance, to generate plain float code without fake-quantization from a quantized graph.
--reorder_across_fake_quant=false bool Normally, FakeQuant nodes must be strict boundaries for graph transformations, in order to ensure that quantized inference has the exact same arithmetic behavior as quantized training --- which is the whole point of quantized training and of FakeQuant nodes in the first place. However, that entails subtle requirements on where exactly FakeQuant nodes must be placed in the graph. Some quantized graphs have FakeQuant nodes at unexpected locations, that prevent graph transformations that are necessary in order to generate inference code for these graphs. Such graphs should be fixed, but as a temporary work-around, setting this reorder_across_fake_quant flag allows TOCO to perform necessary graph transformaitons on them, at the cost of no longer faithfully matching inference and training arithmetic.
--allow_custom_ops=false bool If true, allow TOCO to create TF Lite Custom operators for all the unsupported TensorFlow ops.
--drop_control_dependency=false bool If true, ignore control dependency requirements in input TensorFlow GraphDef. Otherwise an error will be raised upon control dependency inputs.
--debug_disable_recurrent_cell_fusion=false bool If true, disable fusion of known identifiable cell subgraphs into cells. This includes, for example, specific forms of LSTM cell.
And many more...
After that, i used this command to convert file:
toco --input_file="tf_files/retrained_graph.pb" --output_file="tf_files/optimized_graph.lite" --input_format="TENSORFLOW_GRAPHDEF" --output_format="TFLITE" --input_shape="1,224,224,3" --input_array="input" --output_array="final_result" --inference_type="FLOAT" --input_data_type="FLOAT"
Then optimized_graph.lite should be found in tf_files
According to this thread : Tensorflow discussions
The issue really is that the module as of now is not supported on windows. You can follow the thread and see if there is an update regarding the same.
P.S.: some people claim that a git-clone and bazel build helped resloved the issue, instead of pip install, you can try that as well but have serious doubts if that will work.
I would like to read a tif file with basically the following code:
import matplotlib.pyplot as plt
filename = 'test.tif'
plt.imread(filename)
This results in the following error message (just the last lines):
File ".../miniconda2/lib/python2.7/site-packages/PIL/Image.py", line 692, in tobytes
self.load()
File ".../miniconda2/lib/python2.7/site-packages/PIL/TiffImagePlugin.py", line 1013, in load
return super(TiffImageFile, self).load()
File ".../miniconda2/lib/python2.7/site-packages/PIL/ImageFile.py", line 204, in load
decoder.setimage(self.im, extents)
ValueError: tile cannot extend outside image
When I open the tif image with imagemagick's display and save it without making changes, everything works normally.
Nevertheless, I think it could a problem with my python environment/version, as my colleague who's working with the same code and the same files did not have this problem.
I tried many files and it ends in the same for all of them. I am aware that, for this mini example, I could make a workaround or use e.g. gdal (which works fine). But as these lines are just a part of a larger code and are supposed to work for new files immediately, I would like to have a real solution.
I am working with openSUSE 11.4, conda 4.3.23, Python 2.7.13, matplotlib 2.0.2 .
This could be a duplicate of Value Error in reading tif image with pil in python?. Due to the lack of specific information there, I open a new question. Sorry if that's the wrong way - my first post here...
Remark: I see that my tif file gets uploaded as png here. Is there a way to change that?
I'm experimenting a little bit working with images in Python for a project I'm working on.
This is the first time ever for me programming in Python and I haven't found a tutorial that deals with the issues I'm facing.
I'm experimenting with different image decompositions, and I want to define some variable A as a set image from a specified folder. Basically I'm looking for Python's analog of Matlab's imread.
After googling for a bit, I found many solutions but none seem to work for me for some reason.
For example even this simple code
import numpy as np
import cv2
# Load an color image in grayscale
img = cv2.imread('messi5.jpg',0)
which is supposed to work (taken from http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_gui/py_image_display/py_image_display.html) yields the error "No module named cv2".
Why does this happen? How can I read an image?
Another thing I tried is
import numpy as np
import skimage.io as io
A=io.imread('C:\Users\Oria\Desktop\test.jpg')
io.imshow(A)
which yields the error "SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape"
All I want to do is be able to read an image from a specified folder, shouldn't be hard...Should also be noted that the database I work with is ppm files. So I want to read and show ppm images.
Edit: My enviornment is Pyzo. If it matters for anything.
Edit2: Changing the back slashes into forward slashes changes the error to
Traceback (most recent call last):
File "<tmp 1>", line 3, in <module>
A=io.imread('C:/Users/Oria/Desktop/test.jpg')
File "F:\pyzo2015a\lib\site-packages\skimage\io\_io.py", line 97, in imread
img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
File "F:\pyzo2015a\lib\site-packages\skimage\io\manage_plugins.py", line 209, in call_plugin
return func(*args, **kwargs)
File "F:\pyzo2015a\lib\site-packages\matplotlib\pyplot.py", line 2215, in imread
return _imread(*args, **kwargs)
File "F:\pyzo2015a\lib\site-packages\matplotlib\image.py", line 1258, in imread
'more images' % list(six.iterkeys(handlers.keys)))
File "F:\pyzo2015a\lib\site-packages\six.py", line 552, in iterkeys
return iter(d.keys(**kw))
AttributeError: 'builtin_function_or_method' object has no attribute 'keys'
The closest analogue to Matlab's imread is scipy.misc.imread, part of the scipy package. I would write this code as:
import scipy.misc
image_array = scipy.misc.imread('filename.jpg')
Now to your broader questions. The reason this seems hard is because you're coming from Matlab, which uses a different philosophy. Matlab is a monolithic install that comes out of the box with a huge number of functions. Python is modular. The built-in library is relatively small, and then you install packages depending on what you want to do. For instance, the packages scipy (scientific computing), cv2 (computer vision), and PIL (image processing) can all read simple images from disk, so you choose between them depending on what else from the package you might want to use.
This provides a lot more flexibility, but it does require you to become comfortable installing packages. Sadly this is much more difficult on Windows than on Linux-like systems, due to the lack of a "package manager". On Linux I can sudo apt-get install scipy and install all of scipy in one line. In Windows, you might be better off installing something like conda that smooths the package installation process.