I want to make a generator that generates batches of images from urls to train a keras model. I have another generator that feeds me images urls.
What I currently do is download the image to disk and then load the image from the disk.
def loadImage(URL):
with urllib.request.urlopen(URL) as url:
with open('temp.jpg', 'wb') as f:
f.write(url.read())
img_path = 'temp.jpg'
img = image.load_img(img_path, target_size=(125, 125))
os.remove(img_path)
x = image.img_to_array(img)
return x
def imageGenerator(batch_size):
i = 0
batch = []
for URL in imageUrlGenerator():
if i>batch_size:
yield batch
batch = []
i=0
batch.append(loadImage(URL))
i+=1
This works but I wonder if there isn't a faster way to load images from the web without having to write and read in/from disk.
Assuming you are actually using keras and that this image.load_img is the method you are calling, it would call a function which should ultimately be PIL.Image.open. In the documentation for PIL.image.open, the first argument fp could be a string filename (which is what you are currently passing), or a stream-like object that implements read, seek, and tell. While the object returned by urllib.request.urlopen does provide all three methods, it does not implement seek at all, so it cannot be used directly. However, the entire buffer can be read into an BytesIO object which does implement seek, so it should be usable. Putting this together, your loadImage function may be reduced to something like the following:
from io import BytesIO
def loadImage(URL):
with urllib.request.urlopen(URL) as url:
img = image.load_img(BytesIO(url.read()), target_size=(125, 125))
return image.img_to_array(img)
This keeps the images downloaded fully in memory.
This is the simplest solution I've seen.
from PIL import Image
from urllib import request
from io import BytesIO
url = "https://github.com/ironmanciti/MachineLearningBasic/blob/master/datasets/TransferLearningData/watch.jpg?raw=true"
res = request.urlopen(url).read()
Sample_Image = Image.open(BytesIO(res)).resize((150,150))
plt.imshow(Sample_Image)
Got this from Github issues
from io import BytesIO
from PIL import Image
import requests
def loadImage(url):
response = requests.get(url)
img_bytes = BytesIO(response.content)
img = Image.open(img_bytes)
img = img.convert('RGB')
img = img.resize((250,250), Image.NEAREST)
img = img_to_array(img)
return img
2 Quick fixes:
Consider moving the os.remove(img_path) line. I assume this is removing the file from your drive, but I'm thinking you can save this for the end. Your model just needs all the information possible. Once it has the information, you can either asynchronously begin deleting the files, or just wait until the model is trained and cleanup. Doing it one by one as you have, may be slowing you down.
Use fast storage devices and configurations, SSD, USB 3.x, USB C, etc.
Other fixes:
Is it possible to hold things in a cache?
Could you hold things in an array? I don't think so, but it may be possible.
Do you need the whole image? Could you decrease the quality of the image?
How nested is the image? Parsing for the image likely isn't an issue, but it can't hurt to check.
Related
I have found How to return a numpy array as an image using FastAPI?, however, I am still struggling to show the image, which appears just as a white square.
I read an array into io.BytesIO like so:
def iterarray(array):
output = io.BytesIO()
np.savez(output, array)
yield output.get_value()
In my endpoint, my return is StreamingResponse(iterarray(), media_type='application/octet-stream')
When I leave the media_type blank to be inferred a zipfile is downloaded.
How do I get the array to be displayed as an image?
Option 1 - Return image as bytes
The below examples show how to convert an image loaded from disk, or an in-memory image (in the form of numpy array), into bytes (using either PIL or OpenCV libraries) and return them using a custom Response. For the purposes of this demo, the below code is used to create the in-memory sample image (numpy array), which is based on this answer.
# Function to create a sample RGB image
def create_img():
w, h = 512, 512
arr = np.zeros((h, w, 3), dtype=np.uint8)
arr[0:256, 0:256] = [255, 0, 0] # red patch in upper left
return arr
Using PIL
Server side:
You can load an image from disk using Image.open, or use Image.fromarray to load an in-memory image (Note: For demo purposes, when the case is loading the image from disk, the below demonstrates that operation inside the route. However, if the same image is going to be served multiple times, one could load the image only once at startup and store it on the app instance, as described in this answer). Next, write the image to a buffered stream, i.e., BytesIO, and use the getvalue() method to get the entire contents of the buffer. Even though the buffered stream is garbage collected when goes out of scope, it is generally better to call close() or use the with statement, as shown here and below.
from fastapi import Response
from PIL import Image
import numpy as np
import io
#app.get('/image', response_class=Response)
def get_image():
# loading image from disk
# im = Image.open('test.png')
# using an in-memory image
arr = create_img()
im = Image.fromarray(arr)
# save image to an in-memory bytes buffer
with io.BytesIO() as buf:
im.save(buf, format='PNG')
im_bytes = buf.getvalue()
headers = {'Content-Disposition': 'inline; filename="test.png"'}
return Response(im_bytes, headers=headers, media_type='image/png')
Client side:
The below demonstrates how to send a request to the above endpoint using Python requests module, and write the received bytes to a file, or convert the bytes back into PIL Image, as described here.
import requests
from PIL import Image
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
# write raw bytes to file
with open('test.png', 'wb') as f:
f.write(r.content)
# or, convert back to PIL Image
# im = Image.open(io.BytesIO(r.content))
# im.save('test.png')
Using OpenCV
Server side:
You can load an image from disk using cv2.imread() function, or use an in-memory image, which—if it is in RGB order, as in the example below—needs to be converted, as OpenCV uses BGR as its default colour order for images. Next, use cv2.imencode() function, which compresses the image data (based on the file extension you pass that defines the output format, i.e., .png, .jpg, etc.) and stores it in an in-memory buffer that is used to transfer the data over the network.
import cv2
#app.get('/image', response_class=Response)
def get_image():
# loading image from disk
# arr = cv2.imread('test.png', cv2.IMREAD_UNCHANGED)
# using an in-memory image
arr = create_img()
arr = cv2.cvtColor(arr, cv2.COLOR_RGB2BGR)
# arr = cv2.cvtColor(arr, cv2.COLOR_RGBA2BGRA) # if dealing with 4-channel RGBA (transparent) image
success, im = cv2.imencode('.png', arr)
headers = {'Content-Disposition': 'inline; filename="test.png"'}
return Response(im.tobytes() , headers=headers, media_type='image/png')
Client side:
On client side, you can write the raw bytes to a file, or use the numpy.frombuffer() function and cv2.imdecode() function to decompress the buffer into an image format (similar to this)—cv2.imdecode() does not require a file extension, as the correct codec will be deduced from the first bytes of the compressed image in the buffer.
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
# write raw bytes to file
with open('test.png', 'wb') as f:
f.write(r.content)
# or, convert back to image format
# arr = np.frombuffer(r.content, np.uint8)
# img_np = cv2.imdecode(arr, cv2.IMREAD_UNCHANGED)
# cv2.imwrite('test.png', img_np)
Useful Information
Since you noted that you would like the image displayed similar to a FileResponse, using a custom Response to return the bytes should be the way to do this, instead of using StreamingResponse (as shown in your question). To indicate that the image should be viewed in the browser, the HTTP response should include the following header, as described here and as shown in the above examples (the quotes around the filename are required, if the filename contains special characters):
headers = {'Content-Disposition': 'inline; filename="test.png"'}
Whereas, to have the image downloaded rather than viewed (use attachment instead):
headers = {'Content-Disposition': 'attachment; filename="test.png"'}
If you would like to display (or download) the image using a JavaScript interface, such as Fetch API or Axios, have a look at the answers here and here.
As for the StreamingResponse, if the numpy array is fully loaded into memory from the beginning, StreamingResponse is not necessary at all. StreamingResponse streams by iterating over the chunks provided by your iter() function (if Content-Length is not set in the headers—unlike StreamingResponse, other Response classes set that header for you, so that the browser will know where the data ends). As described in this answer:
Chunked transfer encoding makes sense when you don't know the size of
your output ahead of time, and you don't want to wait to collect it
all to find out before you start sending it to the client. That can
apply to stuff like serving the results of slow database queries, but
it doesn't generally apply to serving images.
Even if you would like to stream an image file that is saved on disk (which you should rather not, unless it is a rather large file that can't fit into memory. Instead, you should use use FileResponse), file-like objects, such as those created by open(), are normal iterators; thus, you can return them directly in a StreamingResponse, as described in the documentation and as shown below (if you find yield from f being rather slow when using StreamingResponse, please have a look at this answer for solutions):
#app.get('/image')
def get_image():
def iterfile():
with open('test.png', mode='rb') as f:
yield from f
return StreamingResponse(iterfile(), media_type='image/png')
or, if the image was loaded into memory instead, and was then saved into a BytesIO buffered stream in order to return the bytes, BytesIO is a file-like object (like all the concrete classes of io module), which means you could return it directly in a StreamingResponse:
from fastapi import BackgroundTasks
#app.get('/image')
def get_image(background_tasks: BackgroundTasks):
arr = create_img()
im = Image.fromarray(arr)
buf = BytesIO()
im.save(buf, format='PNG')
buf.seek(0)
background_tasks.add_task(buf.close)
return StreamingResponse(buf, media_type='image/png')
Thus, for your case scenario, it is best to return a Response with your custom content and media_type, as well as setting the Content-Disposition header, as described above, so that the image is viewed in the browser.
Option 2 - Return image as JSON-encoded numpy array
The below should not be used for displaying the image in the browser, but it is rather added here for the sake of completeness, showing how to convert an image into a numpy array (preferably, using asarray() function), then return the data in JSON format, and finally, convert the data back to image on client side, as described in this and this answer. For faster alternatives to the standard Python json library, see this answer.
Using PIL
Server side:
from PIL import Image
import numpy as np
import json
#app.get('/image')
def get_image():
im = Image.open('test.png')
# im = Image.open('test.png').convert('RGBA') # if dealing with 4-channel RGBA (transparent) image
arr = np.asarray(im)
return json.dumps(arr.tolist())
Client side:
import requests
from PIL import Image
import numpy as np
import json
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
arr = np.asarray(json.loads(r.json())).astype(np.uint8)
im = Image.fromarray(arr)
im.save('test_received.png')
Using OpenCV
Server side:
import cv2
import json
#app.get('/image')
def get_image():
arr = cv2.imread('test.png', cv2.IMREAD_UNCHANGED)
return json.dumps(arr.tolist())
Client side:
import requests
import numpy as np
import cv2
import json
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
arr = np.asarray(json.loads(r.json())).astype(np.uint8)
cv2.imwrite('test_received.png', arr)
I'm trying to collect img data for my ML/DL project.
I need facial data so, I have to detect the face and crop around it.
I have a bunch of img URLs that I searched online.
Normally I would save them in a file using requests library, but would it be possible to do it in-memory?
response = requests.get(IMG_URL)
img_byte = response.content
# Do image processing without saving to a file
I looked at some image libraries in python such as PIL or OpenCV,
but they all seems to get images loaded from a file in first place.
I think I could speed up the process if I don't save temporary files. (I/O is a big bottleneck)
I tried playing around with BytesIO functions but I couldn't figure it out.
BytesIO is the way to go!. You can store the binary data returned by the request to the In-memory byte buffer(BytesIO) and pass it to Image.open
>>> from PIL import Image
>>> from io import BytesIO
>>> img_obj = Image.open(BytesIO(r.content))
The only requirement of the Image.open API is that the fp argument must be a filename (string), pathlib.Path object or a file object. The file object must implement file.read, file.seek, and file.tell methods, and be opened in binary mode.
BytesIO implements all these methods.
>>> from io import BytesIO
>>>
>>> buffer = BytesIO()
>>> hasattr(buffer, 'read') and hasattr(buffer, 'tell') and hasattr(buffer, 'seek')
True
I have a function that takes a directory of images, reads them, and stores them in a list.
When pytesting a basic example of reading 3 images, I can't pass the test because the images have an allocation in memory data that makes the assertion to fail.
import os
from PIL import Image
def getImages(imageDir):
files = os.listdir(imageDir)
images = []
for file in files:
# Getting the full image name
filePath = os.path.abspath(os.path.join(imageDir, file))
try:
# explicit load to prevent resources crunch
fp = open(filePath, "rb")
im = Image.open(fp)
images.append(im)
# force loading the image data from file
im.load()
# close the file
fp.close()
except Exception:
# skip
print("Invalid image: %s" % (filePath,))
return images
def test_for_clean_data():
assert getImages("test_images") == [Image.open("test_images/01.jpg"),
Image.open("test_images/02.jpg"),
Image.open("test_images/03.jpg")]
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=2448x2765 at 0x1E48F5872C8>
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=2448x2765 at 0x1E48F5878C8>
As shown in the example error provided by the console, same image will have different properties when tested.
Function to test is PIL.Image based.
Perhaps, as someone suggested the test is flawed in its origin. If anyone knows a better way to pytest that the function is properly working, I would be more than happy to try a new idea. There's so much to learn.
Suggestions for correct test naming are also welcome.
Eyeballing the code, it looks like one potential reason your objects differ is that you call im.load() in getImages(), but not when you open your test images. Does this work? This is just a quick guess, I haven't tested it.
assert getImages("test_images") == [Image.open("test_images/01.jpg").load(),
Image.open("test_images/02.jpg").load(),
Image.open("test_images/03.jpg").load()]
I'm trying to convert some PDFs to high res jpegs using imagemagick . I'm working on win 10, 64 with python 3.62 - 64 bit and wand 0.4.4. At the command line I have :
$ /e/ImageMagick-6.9.9-Q16-HDRI/convert.exe -density 400 myfile.pdf -scale 2000x1000 test3.jpg.
which is working well for me.
In python:
from wand.image import Image
file_path = os.path.dirname(os.path.abspath(__file__))+os.sep+"myfile.pdf"
with Image(filename=file_path, resolution=400) as image:
image.save()
image_jpeg = image.convert('jpeg')
Which is giving me low res JPEGs . How do I translate this into my wand code to do the same thing?
edit:
I realized that the problem is that the input pdf has to be read into the Image object as a binary string, so based on http://docs.wand-py.org/en/0.4.4/guide/read.html#read-blob I tried:
with open(file_path,'rb') as f:
image_binary = f.read()
f.close()
with Image(blob=image_binary,resolution=400) as img:
img.transform('2000x1000', '100%')
img.make_blob('jpeg')
img.save(filename='out.jpg')
This reads the file in ok, but the output is split into 10 files. Why? I need to get this into 1 high res jpeg.
EDIT:
I need to send the jpeg to an OCR api, so I was wondering if I could write the output to a file like object. Looking at https://www.imagemagick.org/api/magick-image.php#MagickWriteImageFile, I tried :
emptyFile = Image(width=1500, height=2000)
with Image(filename=file_path, resolution=400) as image:
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand,
True)
library.MagickWriteImagesFile(resource_pointer,emptyFile)
This gives:
File "E:/ENVS/r3/pdfminer.six/ocr_space.py", line 113, in <module>
test_file = ocr_stream(filename='test4.jpg')
File "E:/ENVS/r3/pdfminer.six/ocr_space.py", line 96, in ocr_stream
library.MagickWriteImagesFile(resource_pointer,emptyFile)
ctypes.ArgumentError: argument 2: <class 'TypeError'>: wrong type
How can I get this working?
Why? I need to get this into 1 high res jpeg.
The PDF contains pages that ImageMagick considers individual images in a "stack". The wand library provides a wand.image.Image.sequance to work with each page.
However, to append all images into a single JPEG. You can either iterate over each page & stitch them together, or call C-API's method MagickAppendImages.
from wand.image import Image
from wand.api import library
import ctypes
# Map C-API not provided by wand library.
library.MagickAppendImages.argtypes = [ctypes.c_void_p, ctypes.c_int]
library.MagickAppendImages.restype = ctypes.c_void_p
with Image(filename="path_to_document.pdf", resolution=400) as image:
# Do all your preprocessing first
# Ether word directly on the wand instance, or iterate over each page.
# ...
# To write all "pages" into a single image.
# Reset the stack iterator.
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand,
True)
# Write C resource directly to disk.
library.MagickWriteImages(resource_pointer,
"output.jpeg".encode("ASCII"),
False)
Update:
I need to send the jpeg to an OCR api ...
Assuming your using OpenCV's python API, you'll only need to iterate over each page, and pass the image-file data to the OCR via numpy buffers.
from wand.image import Image
import numpy
import cv2
def ocr_process(file_data_buffer):
""" Replace with whatever your OCR-API calls for """
mat_instance = cv2.imdecode(file_data_buffer)
# ... work ...
source_image="path_to_document.pdf"
with Image(filename=source_image, resolution=400) as img:
for page in img.sequence:
file_buffer = numpy.asarray(bytearray(page.make_blob("JPEG")),
dtype=numpy.uint8)
ocr_process(file_buffer)
so I was wondering if I could write the output to a file like object
Don't assume that python "image" objects (or underlining C structures) from different libraries are comparable with each other.
Without knowing the OCR api, I can't help you past the wand part, but I can suggest one of the following...
Use temporary intermediate files. (slower I/O, but easier to learn/develop/debug)
with Image(filename=INPUT_PATH) as img:
# work
img.save(filename=OUTPUT_PATH)
# OCR work on OUTPUT_PATH
Use file descriptors if the OCR API supports it. (Same as above)
with open(INPUT_PATH, 'rb') as fd:
with Image(file=fd) as img:
# work
# OCR work ???
Use blobs. (faster I/O but need a lot more memory)
buffer = None
with Image(filename=INPUT_PATH) as img:
# work
buffer = img.make_blob(FORMAT)
if buffer:
# OCR work ???
Even More Updates
Wrapping all the comments together, a solution might be...
from wand.image import Image
from wand.api import library
import ctypes
import requests
# Map C-API not provided by wand library.
library.MagickAppendImages.argtypes = [ctypes.c_void_p, ctypes.c_int]
library.MagickAppendImages.restype = ctypes.c_void_p
with Image(filename='path_to_document.pdf', resolution=400) as image:
# ... Do pre-processing ...
# Reset the stack iterator.
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand, True)
# Convert to JPEG.
library.MagickSetImageFormat(resource_pointer, b'JPEG')
# Create size sentinel.
length = ctypes.c_size_t()
# Write image blob to memory.
image_data_pointer = library.MagickGetImagesBlob(resource_pointer,
ctypes.byref(length))
# Ensure success
if image_data_pointer and length.value:
# Create buffer from memory address
payload = ctypes.string_at(image_data_pointer, length.value)
# Define local filename.
payload_filename = 'my_hires_image.jpg'
# Post payload as multipart encoded image file with filename.
requests.post(THE_URL, files={'file': (payload_filename, payload)})
What about something like:
ok = Image(filename=file_path, resolution=400)
with ok.transform('2000x1000', '100%') as image:
image.compression_quality = 100
image.save()
or:
with ok.resize(2000, 1000)
related:
https://github.com/dahlia/wand/blob/13c4f544bd271fe298ac8dde44fbf178b349361a/docs/guide/resizecrop.rst
Python 3 Wand How to make an unanimated gif from multiple PDF pages
I want to serve matplotlib generated images with django.
If the image is a static png file, the following code works great:
from django.http import HttpResponse
def static_image_view(request):
response = HttpResponse(mimetype='image/png')
with open('test.png', 'rb') as f:
response.write(f.read())
return response
However, if the image is dynamically generated:
import numpy as np
import matplotlib
matplotlib.use('Agg')
from matplotlib import pyplot as plt
def dynamic_image_view(request):
response = HttpResponse(mimetype='image/png')
fig = plt.figure()
plt.plot(np.random.rand(100))
plt.savefig(response, format='png')
plt.close(fig)
return response
When accessing the url in Chrome (v36.0), the image will show up for a few seconds, then disappear and turn to the alt text. It seems that the browser doesn't know the image has already finished loading and waits until timeout. Checking with Chrome > Tools > Developer tools > Network supports this hypothesis: although the image appears after only about 1 sec, the status of the corresponding http request becomes "failed" after about 5 sec.
Note again, this strange phenomenon occurs only with the dynamically generated image, so it shouldn't be Chrome's problem (though it doesn't happen with IE or FireFox, presumably due to different rules in dealing with timeout requests).
To make it more tricky (i.e., hard to reproduce), it seems to be network speed dependent. It happens if I access the url from an IP in China, but not if via a proxy in the US (which seems to be faster visiting the host on which django is running)...
According to #HSquirrel, I tested writing the png into temporary disk file. Strangely, saving file with matplotlib didn't work,
plt.savefig('MPL.png', format='png')
with open('MPL.png', 'rb') as f:
response.write(f.read())
while saving file with PIL worked:
import io
from PIL import Image
f = io.BytesIO()
plt.savefig(f, format='png')
f.seek(0)
im = Image.open(f)
im.save('PIL.png', 'PNG')
Attempt of getting rid of temp file failed:
im.save(response, 'PNG')
However, if I generate the image data stream with PIL rather than matplotlib, temporary disk file would be unnecessary. The following code works:
from PIL import Image, ImageDraw
im = Image.new('RGBA', (256,256), (0,255,0,255))
draw = ImageDraw.Draw(im)
draw.line((100,100, 150,200), fill=128, width=3)
im.save(response, 'PNG')
Finally, savefig(response, 'jepg') has no problem at all.
Have you tried saving the image to disk and then returning that? (you can periodically clear your disk of such generated images based on their time of creation)
If that gives the same problem, it might be a problem with the way the png is generated. Than you could use some kind of image library (like PIL) to make sure all your png's are (re)generated in a way that works with all browsers.
EDIT:
I've checked the png you've linked and I've played around with it a bit, opening and saving it with different programs and with PIL. I get different binary data every time. It seems each program decides which chunks to keep and which to remove. They all encode the png image data differently as well (as far as I can see, I am by no means a specialist in this, I just looked at the binary data based on the specs).
There are a few different paths you can take:
1.The quick and dirty one:
import io
from PIL import Image
f = io.BytesIO()
plt.savefig(f, format='png')
f.seek(0)
im = Image.open(f)
tempfilename = generatetempfilename()
im.save(tempfilename, 'PNG')
with open(tempfilename, 'rb') as f:
response.write(f.read())
2.Adapt how matplotlib makes PNG files (possibly by just using PIL for
it as well). See
http://matplotlib.org/users/customizing.html#customizing-matplotlib
3.If it's an option for you, use jpeg.
4.Figure out what's wrong with the PNG generated by matplotlib and fix
it binary (I don't recommend this). You can use xxd (linux command: xxd test.png) to figure out how the files look in binary and then see how things go using the png spec: overview chunk spec