I am writing a python script that will be running on the server and will resize images (namely Indesign links). It all works fine, except when I am trying to open bigger tiff files (800 MB).
I am using Pillow's Image.open() in order to resize it, but I am getting MemoryError error. I tried with other libraries like ImageMagick and tifffile with the same results. I could split it in smaller chunks, resize those and then combine them, but I have no idea how to do that without opening file first.
I searched extensively, but cannot find solution that would seem appropriate to my case. I was also observing memory consumption and it doesn't seem to be irregularly high. It's safe to say I am completely lost.
The whole trace goes like this:
Traceback (most recent call last):
File "app.py", line 87, in <module>
convert()
File "X:\Development\Python\zipper\converter.py", line 112, in convert
if saveAsJPEG(file, name + ".jpg"): converted_images = updateCounter(all_images, converted_images)
File "X:\Development\Python\zipper\converter.py", line 37, in saveAsJPEG
print Image.open(a).size
File "C:\Python27\lib\site-packages\PIL\Image.py", line 2266, in open
im = factory(fp, filename)
File "C:\Python27\lib\site-packages\PIL\ImageFile.py", line 97, in __init__
self._open()
File "C:\Python27\lib\site-packages\PIL\TiffImagePlugin.py", line 637, in _open
self._seek(0)
File "C:\Python27\lib\site-packages\PIL\TiffImagePlugin.py", line 672, in _seek
self.tag.load(self.fp)
File "C:\Python27\lib\site-packages\PIL\TiffImagePlugin.py", line 458, in load
data = ImageFile._safe_read(fp, size)
File "C:\Python27\lib\site-packages\PIL\ImageFile.py", line 521, in _safe_read
block = fp.read(min(size, SAFEBLOCK))
MemoryError
Thank you all for any help!
--- EDIT
This is the code that throws error
def saveAsJPEG(file, newfile):
print "\nopening:", file
try:
with Image.open(file) as im:
if not im.size[0] < size[0] or im.size[1] < size[1]:
new_im = im.resize(size, Image.ANTIALIAS)
new_im.save(newfile, 'JPEG', quality=100, dpi=(72, 72))
# Force a memory dump. Otherwise memory will get cluttered up -> I am not sure if this is necessary as it doesn't seem to do anything.
del im
del new_im
collect()
return True
except:
print "image is too big -> try something else. But what?"
# this line below throws error (MemoryError). It was the same without try-except part before.
Image.open(file)
The function is used in for-loop where "images" is array (list) of full file paths (X:\path\to\my\image.tiff)
for file in images:
#gets extension
name = basename(splitext(file)[0])
ext = splitext(file)[1]
# check for extension type and run appropriate function
if ext == '.jpg' or ext == '.jpeg' or ext == '.tif' or ext == '.tiff':
if saveAsJPEG(file, name + ".jpg"): converted_images = updateCounter(all_images, converted_images)
del file, name, ext
Related
The below error is thrown when trying to read the image URL referenced below. (Note, I can't even upload the image to SO because it throws an error when I try to upload it.)
https://s3.amazonaws.com/comicgeeks/comics/covers/large-7441962.jpg
image = imread('https://s3.amazonaws.com/comicgeeks/comics/covers/large-7441962.jpg', as_gray=True)
This is the stack trace.
Traceback (most recent call last):
....
return imread(image_or_path, as_gray=True)
File "skimage/io/_io.py", line 48, in imread
img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
File "skimage/io/manage_plugins.py", line 209, in call_plugin
return func(*args, **kwargs)
File "skimage/io/_plugins/imageio_plugin.py", line 10, in imread
return np.asarray(imageio_imread(*args, **kwargs))
File "imageio/__init__.py", line 86, in imread
return imread_v2(uri, format=format, **kwargs)
File "imageio/v2.py", line 159, in imread
with imopen(uri, "ri", plugin=format) as file:
File "imageio/core/imopen.py", line 333, in imopen
raise err_type(err_msg)
ValueError: Could not find a backend to open `/var/folders/82/rky4yjcx75n1zskhy5570v0m0000gn/T/tmpy9xg7dvb.jpg`` with iomode `ri`.
After looking into this further I believe the image was originally a TIFF file that was just renamed to a .jpg file manually, but I'm not sure. If I download the file and try to open it with Photoshop I get the following message.
Could not open “large-7441962.jpeg” because an unknown or invalid JPEG marker type is found.
If I simply change the extension to a .tiff file it will not open as it states it is an invalid tiff file.
The only way I can open it with photoshop is if I open it with the preview.app and then save a copy of the image as a .tiff file. Then I can open it in photoshop.
This is an issue with a potentially large number of images so re-saving them one-by-one is not an option.
Are there any possible ways to re-save this file when this error is thrown? Or somehow figure out how to handle it even though imread() is failing?
I was able to work around this by using the following.
from PIL import Image
from skimage.io import imread
try:
image = imread(url, as_gray=True)
return image
except:
image = Image.open(requests.get(url, stream=True).raw)
return image
However it is worth noting that when having to make this request using PIL it is significantly slower.
I am trying to convert a Computed Radiography dicom file to jpeg using the code below but its raising an OSError: cannot write mode I;16 as JPEG. Here is the code I am using to convert:
ds = read_file('Dicom_files/' + dicomname)
im = fromarray(ds.pixel_array)
im.save('./Jpeg/' + dicomname + '.jpg')
It's raising the following error:
Traceback (most recent call last):
File "/home/wisdom/Desktop/DCM_OUPUT_WRITING/info2.py", line 118, in <module>
img = take_dicom(i)
File "/home/wisdom/Desktop/DCM_OUPUT_WRITING/info2.py", line 28, in take_dicom
final_img = im.save('./Jpeg/' + dicomname + '.jpg')
File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 2240, in save
save_handler(self, fp, filename)
File "/usr/local/lib/python3.9/dist-packages/PIL/JpegImagePlugin.py", line 631, in _save
raise OSError(f"cannot write mode {im.mode} as JPEG") from e
OSError: cannot write mode I;16 as JPEG
How can I solve this?
Regular JPEG uses 8 bits per sample. If your data is 16-bit, you'll need to either:
scale it down (thereby losing radiometric resolution) and use JPEG, or
use a format that supports 16-bit such as PNG or TIFF.
Goal:
Convert finite number of files in .jpg format and merge them into one PDF file.
Expected result:
Files from folder are successfully converted and merged into one pdf file at specified location.
Problem:
When size of files exceed certain number, in my tests it was around 400 mb the program crashes with following message:
Traceback (most recent call last):
File "C:\Users\kaczk\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\ImageFile.py", line 498, in _save
fh = fp.fileno()
io.UnsupportedOperation: fileno
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "MakePDF.py", line 10, in <module>
im1.save(pdf1_filename, "PDF" ,resolution=1000.0, save_all=True, append_images=imageList)
File "C:\Users\kaczk\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\Image.py", line 2084, in save
save_handler(self, fp, filename)
File "C:\Users\kaczk\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\PdfImagePlugin.py", line 46, in _save_all
_save(im, fp, filename, save_all=True)
File "C:\Users\kaczk\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\PdfImagePlugin.py", line 175, in _save
Image.SAVE["JPEG"](im, op, filename)
File "C:\Users\kaczk\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\JpegImagePlugin.py", line 770, in _save
ImageFile._save(im, fp, [("jpeg", (0, 0) + im.size, 0, rawmode)], bufsize)
File "C:\Users\kaczk\AppData\Local\Programs\Python\Python38-32\lib\site-packages\PIL\ImageFile.py", line 513, in _save
fp.write(d)
MemoryError
After running the program with task manager i noticed that indeed the computer runs out of ram memory when executing this program. Below is the code used.
import os
from PIL import Image
fileList = os.listdir(r'C:\location\of\photos\folder')
imageList = []
im1 = Image.open(os.path.join(r'C:\location\of\photos\folder',fileList[0]))
for file in fileList[1:]:
imageList.append(Image.open(os.path.join(r'C:\location\of\photos\folder',file)))
pdf1_filename = r'C:\location\of\pdf\destination.pdf'
im1.save(pdf1_filename, "PDF" ,resolution=500.0, save_all=True, append_images=imageList)
Is there an easy mistake I am making here regarding memory usage? Is there different module that would make the task easier while working with more and larger files? I will be very grateful for all help.
This question is quite old but since I got there struggling with the same issue, here is an answer.
You simply have to close your images after using them:
im1.close()
for i in imageList:
i.close()
This solved it for me.
PS: take a look at glob, it eases working with paths a lot.
I'm looking for a way to open and crop several tiff images and then save the new croped images created in the same folder (related to my script folder).
My current code looks like this:
from PIL import Image
import os,platform
filespath = os.path.join(os.environ['USERPROFILE'],"Desktop\Python\originalImagesfolder")
for file in os.listdir(filespath):
if file.endswith(".tif"):
im = Image.open(file)
im.crop((3000, 6600, 3700, 6750)).save(file+"_crop.tif")
This script is returning me the error:
Traceback (most recent call last):
File "C:\Users...\Desktop\Python\script.py", line 22, in
im = Image.open(file)
File "C:\Python34\lib\site-packages\PIL\Image.py", line 2219, in open
fp = builtins.open(fp, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'Image1Name.tif'
'Image1Name.tif' is the first tif image I'm trying to process in the folder. I don't get how the script can give the file's name without being able to find it. Any Help?
PS: I have 2 days experience in python and codes generaly speaking. Sorry if the answer is obvious
[EDIT/Update]
After modifying my initial code thanks to vttran and ChrisGuest answers, turning then into this:
from PIL import Image
import os,platform
filespath = os.path.join(os.environ['USERPROFILE'],"Desktop\Python\originalImagesfolder")
for file in os.listdir(filespath):
if file.endswith(".tif"):
filepath = os.path.join(filespath, file)
im = Image.open(filepath)
im.crop((3000, 6600, 3700, 6750)).save("crop"+file)
the script is returning me a new error message:
Traceback (most recent call last):
File "C:/Users/.../Desktop/Python/script.py", line 11, in
im.crop((3000, 6600, 3700, 6750)).save("crop"+file)
File "C:\Python34\lib\site-packages\PIL\Image.py", line 986, in crop
self.load()
File "C:\Python34\lib\site-packages\PIL\ImageFile.py", line 166, in load
self.load_prepare()
File "C:\Python34\lib\site-packages\PIL\ImageFile.py", line 250, in
load_prepare
self.im = Image.core.new(self.mode, self.size) ValueError: unrecognized mode
A maybe-useful information, it's a Landsat8 image in GeoTiff format. The TIFF file therefore include geoposition, projection... informations. The script works perfectly fine if I first open and re-save them with a software like Photoshop (16int tiff format).
When you are search for the file names you use filespath to specify the directory.
But then when you open the file, you are only using the base filename.
So you could replace
im = Image.open(file)
with
filepath = os.path.join(filespath, file)
im = Image.open(filepath)
Also consider using the glob module, as you can do glob.glob(r'path\*.tif) .
It is also good practice to avoid using builtin functions like file as variable names.
TLDR; I'm trying to take a TIFF, resize it, then save it. However it returns an error. This works fine if I change the saved filetype to png or jpg.
System: Windows 7
Tried using both Python 3.4 and 2.7.
Code:
from PIL import Image
try: #test file exists
im = Image.open(r"c:\temp\file.tif")
except:
print("Error opening image")
multiply = 5 #how much bigger
processing = tuple([multiply*x for x in im.size]) #maths
saved = (r"c:\temp\biggerfile.tif") #save location
imB = im.resize((processing)) #resizing
imB.save(saved) #saving
I need to resize a TIFF because I'm using tesseract-ocr, and resizing the image to get a better output. The program seems to work best with a TIFF.
The error I receive is:
_TIFFVSetField: c:\temp\biggerfile.tif: Bad value 2 for "ExtraSamples" tag.
Traceback (most recent call last):
File "step1.py", line 15, in <module>
imB.save(saved)
File "C:\Python34\lib\site-packages\PIL\Image.py", line 1684, in save
save_handler(self, fp, filename)
File "C:\Python34\lib\site-packages\PIL\TiffImagePlugin.py", line 1185, in _save
e = Image._getencoder(im.mode, 'libtiff', a, im.encoderconfig)
File "C:\Python34\lib\site-packages\PIL\Image.py", line 430, in _getencoder
return encoder(mode, *args + extra)
RuntimeError: Error setting from dictionary
Thanks!
Try to install libtiff
http://gnuwin32.sourceforge.net/packages/tiff.htm
File "C:\Python34\lib\site-packages\PIL\TiffImagePlugin.py", line 1185, in _save
e = Image._getencoder(im.mode, 'libtiff', a, im.encoderconfig)
Looks like that's the error that is holding you up. It's trying to access libtiff and you don't have it installed so it's failing.
Had the same issue, when using PIL to combine multiple images to one and adding a label.
I could fix this easily by converting the .tif file to a .png file in MS Paint (pls don't hate me for using MS :D). Quality of the final merged image was not reduced.