Trying to load a chest x-ray DICOM file that has JPEG2000 compression, extract the pixel array, crop it, and then save as a new DICOM file. Tried this on a Windows10 and MacOS machine, but getting similar errors. Running Python 3.6.13, GDCM 2.8.0, OpenJpeg 2.3.1, Pillow 8.1.2 in a conda environment (installed OpenJPEG and GDCM first before installing Pillow and Pydicom).
My initial code:
file_list = [f.path for f in os.scandir(basepath)]
ds = pydicom.dcmread(file_list[0])
arr = ds.pixel_array
arr = arr[500:1500,500:1500]
ds.Rows = arr.shape[0]
ds.Columns = arr.shape[1]
ds.PixelData = arr.tobytes()
outputpath = os.path.join(basepath, "test.dcm")
ds.save_as(outputpath)
Subsequent error: ValueError: With tag (7fe0, 0010) got exception: (7FE0,0010) Pixel Data has an undefined length indicating that it's compressed, but the data isn't encapsulated as required. See pydicom.encaps.encapsulate() for more information
I then tried modifying the ds.PixelData line to ds.PixelData = pydicom.encaps.encapsulate([arr.tobytes()]) which creates the .dcm without error, but when I open the .dcm to view it doesn't show any image (all black).
My next attempt was to see if I needed to somehow compress back to JPEG2000, so I attempted:
arr = Image.fromarray(arr)
output = io.BytesIO()
arr.save(output, format='JPEG2000')
but then I get error: OSError: encoder jpeg2k not available. I also tried format='JPEG' but then it tells me OSError: cannot write mode I;16 as JPEG ...
Any help much appreciated!
Was able to get this working by using the imagecodecs library and the jpeg2k_encode function. One potential pitfall is you need to .copy() the array to meet the function's C contiguous requirement, which you can confirm by running arr_crop.flag if you needed to. Here is the updated code that worked best for me:
import os
import numpy as np
import matplotlib.pyplot as plt
import pydicom
from pydicom.encaps import encapsulate
from pydicom.uid import JPEG2000
from imagecodecs import jpeg2k_encode
file_list = [f.path for f in os.scandir(basepath)]
ds = pydicom.dcmread(file_list[0])
arr = ds.pixel_array
#Need to copy() to meet jpeg2k_encodes C contiguous requirement
arr_crop = arr[500:1500,500:1500].copy()
# jpeg2k_encode to perform JPEG2000 compression
arr_jpeg2k = jpeg2k_encode(arr_crop)
# convert from bytearray to bytes before saving to PixelData
arr_jpeg2k = bytes(arr_jpeg2k)
ds.Rows = arr_crop.shape[0]
ds.Columns = arr_crop.shape[1]
ds.PixelData = encapsulate([arr_jpeg2k])
outputpath = os.path.join(basepath, "test.dcm")
ds.save_as(outputpath)
I also ended up using the interactivecrop package to relatively quickly get the crop indices I needed (a tip in case future folks try this in jupyter). In case it's helpful, here's a snippet of code for that (which is run before the above):
from interactivecrop.interactivecrop import main as crop
file_names = [os.path.split(f)[1].split(".")[0] for f in file_list]
image_list = []
for x in file_list:
ds = pydicom.dcmread(x)
arr = ds.pixel_array
image_list.append(arr)
crop(image_list, file_names, optimize=True)
#After cropping all images, will print a dictionary
#copied and pasted this dictionary to a new cell as crop_dict
#use the below function to convert the output to actual indices
def convert_crop_to_index(fname, crop_dict):
x = [crop_dict[fname][1], crop_dict[fname][1] + crop_dict[fname][3]]
y = [crop_dict[fname][0], crop_dict[fname][0] + crop_dict[fname][2]]
return x, y
arr_crop = arr[x[0]:x[1],y[0]:y[1]].copy()
Was never able to quite figure out why ds.decompress() and saving the decompressed dicom was generating an all black image. I feel like that should have been the easiest method, but the above ended up working for me, so I'm happy was able to figure it out.
Related
I am using the following code to read geotiff images (~300). Each image file contains 15 bands.
code:
import gdal
inputPath="/content/drive/MyDrive/Notebook/test/tiled_stack"
images = []
# Load in the images
for filepath in os.listdir(inputPath):
images.append(gdal.Open(inputPath+'/{0}'.format(filepath)))
image = images.ReadAsArray()
print(type(images[1]))
error:
AttributeError: 'list' object has no attribute 'ReadAsArray'
Another code from this link:
myImages= glob.glob(os.path.join(inputPath, '*.tif'))
for data_path in myImages:
images=gdal.Open(data_path, gdal.GA_ReadOnly)
type(images)
osgeo.gdal.Dataset
How can I modify the code to resolve the error and have images in form of (width, height, number of bands)?
You may read the image using GDAL as described in the following post.
Example:
import gdal
filepath = r'c:\Tmp\ds\images\remote_sensing\otherDatasets\sentinel_2\tif\Forest\Forest_1.tif'
# https://stackoverflow.com/questions/43684072/how-to-import-multiple-bands-from-an-image-into-numpy
# Load one GeoTIFF image using GDAL
dataset = gdal.Open(filepath)
image = dataset.ReadAsArray()
print(type(image)) # <class 'numpy.ndarray'>
print(image.shape) # (13, 64, 64)
print(image.dtype) # uint16
Installing GDAL in Windows is not so straight forward.
I started working by the following instructions, but it's not up to date.
I downloaded gdal-204-1911-x64-core.msi and GDAL-2.4.4.win-amd64-py3.6.msi (matches Python 3.6) from here.
I updated the environment variables with C:\Program Files opposed to C:\Program Files (x86) (as described in the instructions).
For some reason it's working only when I Run the Python script, but gives an error when I Debug the script.
Update:
images is a Python list.
You can't use images.ReadAsArray(), because Python list has no method ReadAsArray().
You may append an image to the list after executing image = dataset.ReadAsArray() - you have to call the method for every image, and can't apply it to the list.
Example:
import gdal
inputPath="/content/drive/MyDrive/Notebook/test/tiled_stack"
images = []
# Load the images, and append them to a list.
for filepath in os.listdir(inputPath):
dataset = gdal.Open(inputPath+'/{0}'.format(filepath))
image = dataset.ReadAsArray() # Returned image is a NumPy array with shape (13, 64, 64) for example.
images.append(image) # Append the NumPy array to the list.
print(type(images[1]))
I'm working on a project that involves Python. I've NEVER used it along with OpenCV. The objective is to take a 16x16 section of an video, I'm practicing with a single Image, and get it's RBG value. I'm suppose to run this for thousands of frame on a video, which i dont know how to loop. Once I have it ([ 71 155 90]) for example, I want to save it to a notepad, excel sheet, or some simple way or referring to my results.
I've tried looking up tutorials on how to export values, but they've used so many different terms that I don't know where to start.
import numpy as np
import cv2
img = cv2.imread('dog.jpg', cv2.IMREAD_COLOR)
px = img[16,16]
print(px)
The only thing I get is the RBG output [ 71 155 90] in the terminal. I don't know where to do from there. I don't know how to export the value.
you can use openpyxl or
import numpy as np import cv2
img = cv2.imread('dog.jpg', cv2.IMREAD_COLOR)
px = img[16,16]
import pandas as px
df = pd.DataFrame(px)
df.to_excel('filename.xlsx')
You'll need to open a file and then write the results to that file, here is one possible way to do this (although perhaps not the most optimal):
fp = open('output.csv', 'w')
fp.write('{},{},{}'.format(px[0],px[1],px[2])
# write more values here
fp.close() # do this at the end of your writes
I am currently working on something similar, instead of videos I am working with images so I went around searching for tutorials on how to do bulk export of images/ frames from a folder and saving the data into a numpy array.
This is a sample of my code *not sure how much errors are inside but it is able to load and save image frames into an array. I use tqdm to show a simple progress bar so I know what is the status of the image loading when I call this function.
def img_readph(path):
readph =[i for i in listdir(path) if isfile(join(path,i))]
img = np.empty(len(readph), dtype=object)
for j in tqdm(range(0, len(readph))):
img[j] = cv2.imread(join(path,readph[j]))
return img
In order to load and work on the images that are currently saved in a numpy array stack, I use this set of code to do the extraction, perform a basic psnr calculation and save the data to a .txt (learning how to convert the result to a .csv that I can load/save/append in python for future edits as well).
for index in tqdm(range(len(img))):
(psnr, meanerror) = calculate_psnr(imgGT[index], imgSR[index])
print('Image No.{} has average mean square error of {} and the average PSNR is {}'.format(index,meanerror,psnr))
Doing it this way lets me loop every video frame I have in the previous numpy array to perform my operation to calculate psnr.
What you could do is to try write your code to get the RGB values into a function using,
txtfilename = input("enter filename: ")
with open(str(txtfilename)+".txt","w") as results:
for index in tqdm(range(0, len(img))) #the array from imread
img = cv2.imread('img', cv2.IMREAD_COLOR)
px = img[width, height]
print("The RBG values are {}".format(px), file=results)
Something along the lines of this I guess, hope it helps.
I'm trying to convert some PDFs to high res jpegs using imagemagick . I'm working on win 10, 64 with python 3.62 - 64 bit and wand 0.4.4. At the command line I have :
$ /e/ImageMagick-6.9.9-Q16-HDRI/convert.exe -density 400 myfile.pdf -scale 2000x1000 test3.jpg.
which is working well for me.
In python:
from wand.image import Image
file_path = os.path.dirname(os.path.abspath(__file__))+os.sep+"myfile.pdf"
with Image(filename=file_path, resolution=400) as image:
image.save()
image_jpeg = image.convert('jpeg')
Which is giving me low res JPEGs . How do I translate this into my wand code to do the same thing?
edit:
I realized that the problem is that the input pdf has to be read into the Image object as a binary string, so based on http://docs.wand-py.org/en/0.4.4/guide/read.html#read-blob I tried:
with open(file_path,'rb') as f:
image_binary = f.read()
f.close()
with Image(blob=image_binary,resolution=400) as img:
img.transform('2000x1000', '100%')
img.make_blob('jpeg')
img.save(filename='out.jpg')
This reads the file in ok, but the output is split into 10 files. Why? I need to get this into 1 high res jpeg.
EDIT:
I need to send the jpeg to an OCR api, so I was wondering if I could write the output to a file like object. Looking at https://www.imagemagick.org/api/magick-image.php#MagickWriteImageFile, I tried :
emptyFile = Image(width=1500, height=2000)
with Image(filename=file_path, resolution=400) as image:
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand,
True)
library.MagickWriteImagesFile(resource_pointer,emptyFile)
This gives:
File "E:/ENVS/r3/pdfminer.six/ocr_space.py", line 113, in <module>
test_file = ocr_stream(filename='test4.jpg')
File "E:/ENVS/r3/pdfminer.six/ocr_space.py", line 96, in ocr_stream
library.MagickWriteImagesFile(resource_pointer,emptyFile)
ctypes.ArgumentError: argument 2: <class 'TypeError'>: wrong type
How can I get this working?
Why? I need to get this into 1 high res jpeg.
The PDF contains pages that ImageMagick considers individual images in a "stack". The wand library provides a wand.image.Image.sequance to work with each page.
However, to append all images into a single JPEG. You can either iterate over each page & stitch them together, or call C-API's method MagickAppendImages.
from wand.image import Image
from wand.api import library
import ctypes
# Map C-API not provided by wand library.
library.MagickAppendImages.argtypes = [ctypes.c_void_p, ctypes.c_int]
library.MagickAppendImages.restype = ctypes.c_void_p
with Image(filename="path_to_document.pdf", resolution=400) as image:
# Do all your preprocessing first
# Ether word directly on the wand instance, or iterate over each page.
# ...
# To write all "pages" into a single image.
# Reset the stack iterator.
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand,
True)
# Write C resource directly to disk.
library.MagickWriteImages(resource_pointer,
"output.jpeg".encode("ASCII"),
False)
Update:
I need to send the jpeg to an OCR api ...
Assuming your using OpenCV's python API, you'll only need to iterate over each page, and pass the image-file data to the OCR via numpy buffers.
from wand.image import Image
import numpy
import cv2
def ocr_process(file_data_buffer):
""" Replace with whatever your OCR-API calls for """
mat_instance = cv2.imdecode(file_data_buffer)
# ... work ...
source_image="path_to_document.pdf"
with Image(filename=source_image, resolution=400) as img:
for page in img.sequence:
file_buffer = numpy.asarray(bytearray(page.make_blob("JPEG")),
dtype=numpy.uint8)
ocr_process(file_buffer)
so I was wondering if I could write the output to a file like object
Don't assume that python "image" objects (or underlining C structures) from different libraries are comparable with each other.
Without knowing the OCR api, I can't help you past the wand part, but I can suggest one of the following...
Use temporary intermediate files. (slower I/O, but easier to learn/develop/debug)
with Image(filename=INPUT_PATH) as img:
# work
img.save(filename=OUTPUT_PATH)
# OCR work on OUTPUT_PATH
Use file descriptors if the OCR API supports it. (Same as above)
with open(INPUT_PATH, 'rb') as fd:
with Image(file=fd) as img:
# work
# OCR work ???
Use blobs. (faster I/O but need a lot more memory)
buffer = None
with Image(filename=INPUT_PATH) as img:
# work
buffer = img.make_blob(FORMAT)
if buffer:
# OCR work ???
Even More Updates
Wrapping all the comments together, a solution might be...
from wand.image import Image
from wand.api import library
import ctypes
import requests
# Map C-API not provided by wand library.
library.MagickAppendImages.argtypes = [ctypes.c_void_p, ctypes.c_int]
library.MagickAppendImages.restype = ctypes.c_void_p
with Image(filename='path_to_document.pdf', resolution=400) as image:
# ... Do pre-processing ...
# Reset the stack iterator.
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand, True)
# Convert to JPEG.
library.MagickSetImageFormat(resource_pointer, b'JPEG')
# Create size sentinel.
length = ctypes.c_size_t()
# Write image blob to memory.
image_data_pointer = library.MagickGetImagesBlob(resource_pointer,
ctypes.byref(length))
# Ensure success
if image_data_pointer and length.value:
# Create buffer from memory address
payload = ctypes.string_at(image_data_pointer, length.value)
# Define local filename.
payload_filename = 'my_hires_image.jpg'
# Post payload as multipart encoded image file with filename.
requests.post(THE_URL, files={'file': (payload_filename, payload)})
What about something like:
ok = Image(filename=file_path, resolution=400)
with ok.transform('2000x1000', '100%') as image:
image.compression_quality = 100
image.save()
or:
with ok.resize(2000, 1000)
related:
https://github.com/dahlia/wand/blob/13c4f544bd271fe298ac8dde44fbf178b349361a/docs/guide/resizecrop.rst
Python 3 Wand How to make an unanimated gif from multiple PDF pages
I am working with web services using requests to get an image based on parameters passed. The first response I get is a XML schema with file reference URL.
<?xml version="1.0"?>
<Coverages schemaLocation="http://localhost/server/schemas/wcs/1.1.1/wcsCoverages.xsd">
<Coverage>
<Title>Filename</Title>
<Abstract/>
<Identifier>Filename</Identifier>
<Reference href="http://localhost/server/temp/filename.tif"/>
</Coverage>
</Coverages>
Next using xml.etree.ElementTree I extracted the URL. Next what I need is to dsiplay that tiff image (or any other image) on the Jupyter Notebook without downloading (as one image can be more than 50 or 100 MB sometimes)
Currently I am downloading and plotting the file after reading and converting file data into array ( as pyplot plot image array/matrix ).
import requests as req # request wcs urls
import xml.etree.ElementTree as ET # used for xml parsing
import matplotlib.pyplot as plt # display image
import gdal
# Download the File to local directory using Chunks
chunk_size=1024
local_filename = url.split('/')[-1] # Filename from url
r = req.get(url, stream=True)
with open(local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size):
if chunk:
f.write(chunk)
# Read File Raster Data as Array using Gdal
gtif = gdal.Open(local_filename)
georaster = gtif.ReadAsArray()
# Plot image using matplotlib
plt.imshow(georaster)
plt.title(local_filename)
plt.show()
So is there any way to convert the raw response from requests API for file directly into image array (in chunks or whole) and display it on notebook (without downloading and taking space on local directory)
The raw response from get request for tiff file is below
resp2 = req.get('tiffFielurl')
rawdata = resp2.content
rawdata[0:10]
Output: b'MM\x00*\x00\x00\x00\x08\x00\x10'
I tried searching for this question but not found any good answer on it so if there is any related question or duplicate provide me the link.
You can try plotting tiff images using ipyplot package
import ipyplot
ipyplot.plot_images(
['https://file-examples.com/wp-content/uploads/2017/10/file_example_TIFF_1MB.tiff',
'https://file-examples.com/wp-content/uploads/2017/10/file_example_TIFF_1MB.tiff',
'https://file-examples.com/wp-content/uploads/2017/10/file_example_TIFF_1MB.tiff'],
img_width=250)
After doing so much research and trying different solutions it seems to me the only way to do the above procedure i.e displaying Tiff file for now is downloading and reading the data using gdal and converting into array and display using matplotlib .
As the solution mentioned in following link is only accepting "PNG" files.
How to plot remote image (from http url)
whcih comes to conclusion we need PIL library that I also tried and fails.
from PIL import Image
resp2 = req.get('tiffFielurl')
resp2.raw.decode_content = True
im = Image.open(resp2.raw)
im
Gives Output:
<PIL.TiffImagePlugin.TiffImageFile image mode=I;16BS size=4800x4800 at 0x11CB7C50>
and converting the PIL object to numpy array or even getting data or pixels from PIL object gives error of unrecognized mode error.
im.getdata()
im.getpixels((0,0))
numpy.array(im)
All gives same error
257 if not self.im or\
258 self.im.mode != self.mode or self.im.size != self.size:
--> 259 self.im = Image.core.new(self.mode, self.size)
260 # create palette (optional)
261 if self.mode == "P":
ValueError: unrecognized mode
It comes out that PIL even don't support 16bit Signed integer pixel that is defined in Tiff object of PIL above.
https://pillow.readthedocs.io/en/4.0.x/handbook/concepts.html#concept-modes
I'm streaming a png image from my iPhone to my MacBook over tcp. The MacBook code is from http://docs.python.org/library/socketserver.html#requesthandler-objects. How can the image be converted for use with OpenCV? A png was selected because they are efficient, but other formats could be used.
I wrote a test program that reads the rawImage from a file, but not sure how to convert it:
# Read rawImage from a file, but in reality will have it from TCPServer
f = open('frame.png', "rb")
rawImage = f.read()
f.close()
# Not sure how to convert rawImage
npImage = np.array(rawImage)
matImage = cv2.imdecode(rawImage, 1)
#show it
cv.NamedWindow('display')
cv.MoveWindow('display', 10, 10)
cv.ShowImage('display', matImage)
cv. WaitKey(0)
#Andy Rosenblum's works, and it might be the best solution if using the outdated cv python API (vs. cv2).
However, because this question is equally interesting for users of the latest versions, I suggest the following solution. The sample code below may be better than the accepted solution because:
It is compatible with newer OpenCV python API (cv2 vs. cv). This solution is tested under opencv 3.0 and python 3.0. I believe only trivial modifications would be required for opencv 2.x and/or python 2.7x.
Fewer imports. This can all be done with numpy and opencv directly, no need for StringIO and PIL.
Here is how I create an opencv image decoded directly from a file object, or from a byte buffer read from a file object.
import cv2
import numpy as np
#read the data from the file
with open(somefile, 'rb') as infile:
buf = infile.read()
#use numpy to construct an array from the bytes
x = np.fromstring(buf, dtype='uint8')
#decode the array into an image
img = cv2.imdecode(x, cv2.IMREAD_UNCHANGED)
#show it
cv2.imshow("some window", img)
cv2.waitKey(0)
Note that in opencv 3.0, the naming convention for the various constants/flags changed, so if using opencv 2.x, you will need to change the flag cv2.IMREAD_UNCHANGED. This code sample also assumes you are loading in a standard 8-bit image, but if not, you can play with the dtype='...' flag in np.fromstring.
another way,
also in the case of a reading an actual file this will work for a unicode path (tested on windows)
with open(image_full_path, 'rb') as img_stream:
file_bytes = numpy.asarray(bytearray(img_stream.read()), dtype=numpy.uint8)
img_data_ndarray = cv2.imdecode(file_bytes, cv2.CV_LOAD_IMAGE_UNCHANGED)
img_data_cvmat = cv.fromarray(img_data_ndarray) # convert to old cvmat if needed
I figured it out:
# Read rawImage from a file, but in reality will have it from TCPServer
f = open('frame.png', "rb")
rawImage = f.read()
f.close()
# Convert rawImage to Mat
pilImage = Image.open(StringIO(rawImage));
npImage = np.array(pilImage)
matImage = cv.fromarray(npImage)
#show it
cv.NamedWindow('display')
cv.MoveWindow('display', 10, 10)
cv.ShowImage('display', matImage)
cv. WaitKey(0)
This works for me (these days):
import cv2
import numpy as np
data = open('016e263c726a.raw').read()
x = np.frombuffer(data, dtype='uint8').reshape(2048,2448)
cv2.imshow('x',x); cv2.waitKey(); cv2.destroyAllWindows()
But it reads a RAW image saved without any specific format.
(Your question seems to be tagged objective-c but you ask for Python and so is your example, so I'll use that.)
My first post on Stack Overflow!
The cv.LoadImageM method seems to be what you are looking for.
http://opencv.willowgarage.com/documentation/python/reading_and_writing_images_and_video.html
Example use:
http://opencv.willowgarage.com/wiki/PythonInterface/
LoadImage(filename, iscolor=CV_LOAD_IMAGE_COLOR) → None
Loads an image from a file as an IplImage.
Parameters:
filename (str) – Name of file to be loaded.
iscolor (int) –
Specific color type of the loaded image:
CV_LOAD_IMAGE_COLOR the loaded image is forced to be a 3-channel color image
CV_LOAD_IMAGE_GRAYSCALE the loaded image is forced to be grayscale
CV_LOAD_IMAGE_UNCHANGED the loaded image will be loaded as is.
The function cvLoadImage loads an image from the specified file and
returns the pointer to the loaded image. Currently the following file
formats are supported:
Windows bitmaps - BMP, DIB
JPEG files - JPEG, JPG, JPE
Portable Network Graphics - PNG
Portable image format - PBM, PGM, PPM
Sun rasters - SR, RAS
TIFF files - TIFF, TIF
Note that in the current implementation the alpha channel, if any, is
stripped from the output image, e.g. 4-channel RGBA image will be
loaded as RGB.
When you have to load from file, this simple solution does the job (tested with opencv-python-3.2.0.6):
import cv2
img = cv2.imread(somefile)