I have found How to return a numpy array as an image using FastAPI?, however, I am still struggling to show the image, which appears just as a white square.
I read an array into io.BytesIO like so:
def iterarray(array):
output = io.BytesIO()
np.savez(output, array)
yield output.get_value()
In my endpoint, my return is StreamingResponse(iterarray(), media_type='application/octet-stream')
When I leave the media_type blank to be inferred a zipfile is downloaded.
How do I get the array to be displayed as an image?
Option 1 - Return image as bytes
The below examples show how to convert an image loaded from disk, or an in-memory image (in the form of numpy array), into bytes (using either PIL or OpenCV libraries) and return them using a custom Response. For the purposes of this demo, the below code is used to create the in-memory sample image (numpy array), which is based on this answer.
# Function to create a sample RGB image
def create_img():
w, h = 512, 512
arr = np.zeros((h, w, 3), dtype=np.uint8)
arr[0:256, 0:256] = [255, 0, 0] # red patch in upper left
return arr
Using PIL
Server side:
You can load an image from disk using Image.open, or use Image.fromarray to load an in-memory image (Note: For demo purposes, when the case is loading the image from disk, the below demonstrates that operation inside the route. However, if the same image is going to be served multiple times, one could load the image only once at startup and store it on the app instance, as described in this answer). Next, write the image to a buffered stream, i.e., BytesIO, and use the getvalue() method to get the entire contents of the buffer. Even though the buffered stream is garbage collected when goes out of scope, it is generally better to call close() or use the with statement, as shown here and below.
from fastapi import Response
from PIL import Image
import numpy as np
import io
#app.get('/image', response_class=Response)
def get_image():
# loading image from disk
# im = Image.open('test.png')
# using an in-memory image
arr = create_img()
im = Image.fromarray(arr)
# save image to an in-memory bytes buffer
with io.BytesIO() as buf:
im.save(buf, format='PNG')
im_bytes = buf.getvalue()
headers = {'Content-Disposition': 'inline; filename="test.png"'}
return Response(im_bytes, headers=headers, media_type='image/png')
Client side:
The below demonstrates how to send a request to the above endpoint using Python requests module, and write the received bytes to a file, or convert the bytes back into PIL Image, as described here.
import requests
from PIL import Image
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
# write raw bytes to file
with open('test.png', 'wb') as f:
f.write(r.content)
# or, convert back to PIL Image
# im = Image.open(io.BytesIO(r.content))
# im.save('test.png')
Using OpenCV
Server side:
You can load an image from disk using cv2.imread() function, or use an in-memory image, which—if it is in RGB order, as in the example below—needs to be converted, as OpenCV uses BGR as its default colour order for images. Next, use cv2.imencode() function, which compresses the image data (based on the file extension you pass that defines the output format, i.e., .png, .jpg, etc.) and stores it in an in-memory buffer that is used to transfer the data over the network.
import cv2
#app.get('/image', response_class=Response)
def get_image():
# loading image from disk
# arr = cv2.imread('test.png', cv2.IMREAD_UNCHANGED)
# using an in-memory image
arr = create_img()
arr = cv2.cvtColor(arr, cv2.COLOR_RGB2BGR)
# arr = cv2.cvtColor(arr, cv2.COLOR_RGBA2BGRA) # if dealing with 4-channel RGBA (transparent) image
success, im = cv2.imencode('.png', arr)
headers = {'Content-Disposition': 'inline; filename="test.png"'}
return Response(im.tobytes() , headers=headers, media_type='image/png')
Client side:
On client side, you can write the raw bytes to a file, or use the numpy.frombuffer() function and cv2.imdecode() function to decompress the buffer into an image format (similar to this)—cv2.imdecode() does not require a file extension, as the correct codec will be deduced from the first bytes of the compressed image in the buffer.
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
# write raw bytes to file
with open('test.png', 'wb') as f:
f.write(r.content)
# or, convert back to image format
# arr = np.frombuffer(r.content, np.uint8)
# img_np = cv2.imdecode(arr, cv2.IMREAD_UNCHANGED)
# cv2.imwrite('test.png', img_np)
Useful Information
Since you noted that you would like the image displayed similar to a FileResponse, using a custom Response to return the bytes should be the way to do this, instead of using StreamingResponse (as shown in your question). To indicate that the image should be viewed in the browser, the HTTP response should include the following header, as described here and as shown in the above examples (the quotes around the filename are required, if the filename contains special characters):
headers = {'Content-Disposition': 'inline; filename="test.png"'}
Whereas, to have the image downloaded rather than viewed (use attachment instead):
headers = {'Content-Disposition': 'attachment; filename="test.png"'}
If you would like to display (or download) the image using a JavaScript interface, such as Fetch API or Axios, have a look at the answers here and here.
As for the StreamingResponse, if the numpy array is fully loaded into memory from the beginning, StreamingResponse is not necessary at all. StreamingResponse streams by iterating over the chunks provided by your iter() function (if Content-Length is not set in the headers—unlike StreamingResponse, other Response classes set that header for you, so that the browser will know where the data ends). As described in this answer:
Chunked transfer encoding makes sense when you don't know the size of
your output ahead of time, and you don't want to wait to collect it
all to find out before you start sending it to the client. That can
apply to stuff like serving the results of slow database queries, but
it doesn't generally apply to serving images.
Even if you would like to stream an image file that is saved on disk (which you should rather not, unless it is a rather large file that can't fit into memory. Instead, you should use use FileResponse), file-like objects, such as those created by open(), are normal iterators; thus, you can return them directly in a StreamingResponse, as described in the documentation and as shown below (if you find yield from f being rather slow when using StreamingResponse, please have a look at this answer for solutions):
#app.get('/image')
def get_image():
def iterfile():
with open('test.png', mode='rb') as f:
yield from f
return StreamingResponse(iterfile(), media_type='image/png')
or, if the image was loaded into memory instead, and was then saved into a BytesIO buffered stream in order to return the bytes, BytesIO is a file-like object (like all the concrete classes of io module), which means you could return it directly in a StreamingResponse:
from fastapi import BackgroundTasks
#app.get('/image')
def get_image(background_tasks: BackgroundTasks):
arr = create_img()
im = Image.fromarray(arr)
buf = BytesIO()
im.save(buf, format='PNG')
buf.seek(0)
background_tasks.add_task(buf.close)
return StreamingResponse(buf, media_type='image/png')
Thus, for your case scenario, it is best to return a Response with your custom content and media_type, as well as setting the Content-Disposition header, as described above, so that the image is viewed in the browser.
Option 2 - Return image as JSON-encoded numpy array
The below should not be used for displaying the image in the browser, but it is rather added here for the sake of completeness, showing how to convert an image into a numpy array (preferably, using asarray() function), then return the data in JSON format, and finally, convert the data back to image on client side, as described in this and this answer. For faster alternatives to the standard Python json library, see this answer.
Using PIL
Server side:
from PIL import Image
import numpy as np
import json
#app.get('/image')
def get_image():
im = Image.open('test.png')
# im = Image.open('test.png').convert('RGBA') # if dealing with 4-channel RGBA (transparent) image
arr = np.asarray(im)
return json.dumps(arr.tolist())
Client side:
import requests
from PIL import Image
import numpy as np
import json
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
arr = np.asarray(json.loads(r.json())).astype(np.uint8)
im = Image.fromarray(arr)
im.save('test_received.png')
Using OpenCV
Server side:
import cv2
import json
#app.get('/image')
def get_image():
arr = cv2.imread('test.png', cv2.IMREAD_UNCHANGED)
return json.dumps(arr.tolist())
Client side:
import requests
import numpy as np
import cv2
import json
url = 'http://127.0.0.1:8000/image'
r = requests.get(url=url)
arr = np.asarray(json.loads(r.json())).astype(np.uint8)
cv2.imwrite('test_received.png', arr)
Related
I'm trying to collect img data for my ML/DL project.
I need facial data so, I have to detect the face and crop around it.
I have a bunch of img URLs that I searched online.
Normally I would save them in a file using requests library, but would it be possible to do it in-memory?
response = requests.get(IMG_URL)
img_byte = response.content
# Do image processing without saving to a file
I looked at some image libraries in python such as PIL or OpenCV,
but they all seems to get images loaded from a file in first place.
I think I could speed up the process if I don't save temporary files. (I/O is a big bottleneck)
I tried playing around with BytesIO functions but I couldn't figure it out.
BytesIO is the way to go!. You can store the binary data returned by the request to the In-memory byte buffer(BytesIO) and pass it to Image.open
>>> from PIL import Image
>>> from io import BytesIO
>>> img_obj = Image.open(BytesIO(r.content))
The only requirement of the Image.open API is that the fp argument must be a filename (string), pathlib.Path object or a file object. The file object must implement file.read, file.seek, and file.tell methods, and be opened in binary mode.
BytesIO implements all these methods.
>>> from io import BytesIO
>>>
>>> buffer = BytesIO()
>>> hasattr(buffer, 'read') and hasattr(buffer, 'tell') and hasattr(buffer, 'seek')
True
UPDATE BELOW
Is there a way to pass a PIL Image to google cloud vision?
I tried to use io.Bytes, io.String and Image.tobytes() but I always get:
Traceback (most recent call last):
"C:\Users\...\vision_api.py", line 20, in get_text
image = vision.Image(content)
File "C:\...\venv\lib\site-packages\proto\message.py", line 494, in __init__
raise TypeError(
TypeError: Invalid constructor input for Image:b'Ma\x81Ma\x81La\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81La\x81Ma\x81Ma\x81Ma\x81Ma\x80Ma\x81La\x81Ma\x81Ma\x81Ma\x80Ma\x81Ma\x81Ma\x81Ma\x8 ...
or this if I pass the PIL-Image directly:
TypeError: Invalid constructor input for Image: <PIL.Image.Image image mode=RGB size=480x300 at 0x1D707131DC0>
This is my code:
image = Image.open(path).convert('RGB') # Opening the saved image
cropped_image = image.crop((30, 900, 510, 1200)) # Cropping the image
vision_image = vision.Image(# I passed the different options) # Here I need to pass the image, but I don't know how
client = vision.ImageAnnotatorClient()
response = client.text_detection(image=vision_image) # Text detection using google-vision-api
FOR CLARITY:
I want google text detection to only analyse a certain part of an image saved on my disk. So my idea was to crop the image using PIL and then pass the cropped image to google-vision. But it is not possible to pass an PIL-Image to vision.Image, as I get the error above.
The documentation from Google.
This can be found in the vision.Image class:
Attributes:
content (bytes):
Image content, represented as a stream of bytes. Note: As
with all ``bytes`` fields, protobuffers use a pure binary
representation, whereas JSON representations use base64.
Currently, this field only works for BatchAnnotateImages
requests. It does not work for AsyncBatchAnnotateImages
requests.
A working option is to save the PIL-Image as a PNG/JPG on my disk and load it using:
with io.open(file_name, 'rb') as image_file:
content = image_file.read()
vision_image = vision.Image(content=content)
But this is slow and seems unnecessary. And the whole point for me behind using google-vision-api is the speed comaped to open-cv.
UPDATE as of 25/9/2021
from PIL import Image
from io import BytesIO
from google.cloud import vision
with open('images/screenshots/screenshot.png', 'rb') as image_file:
data = image_file.read()
try:
image = vision.Image(content=data)
print('worked')
except TypeError:
print('failed')
im = Image.open('images/screenshots/screenshot.png')
buffer = BytesIO()
im.save(buffer, format='PNG')
try:
image = vision.Image(buffer.getvalue())
print('worked')
except TypeError:
print('failed')
The first version works as expected, but I can't get the second one to work as #Mark Setchell recommended. The first few characters (~50) are the same, the rest is completely different.
UPDATE as of 26/9/2021
Both inputs are of type <class 'bytes'>. The complete error stack can be seen at the top of the question.
Using this code:
print(input_data[:200])
print(type(input_data))
i get the following output:
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x048\x00\x00\x07\x80\x08\x06\x00\x00\x00+a\xe7\n\x00\x00\x00\x04sBIT\x08\x08\x08\x08|\x08d\x88\x00\x00 \x00IDATx\x9c\xec\xbdy\xd8-\xc7Y\x1f\xf8\xab\xea>\xe7\xdb\xef\xaa\xbbk\xb3%\xcb\x8b\x16[\x12\xc6\xc8\xbb,\x1b\x03\x06\xc6\x8111\x93#2y\xc2381\x8b1\x90\x10\x9e\xf18\x93\x10\x0811\x84\x192\x0c3\x9e\x1020\x03\x03\xc3\xb0\x04\xf0C0\xc6\x96m\xc9\x96m\xed\xb2dI\x96\xaetu\xf7\xed\xdb\xcf\xe9\xae\x9a?j\xe9\xea\xbd\xba\xbb\xbaO\x9f\xef\x9e\xd7\xd6\xfd\xfat\xbf\xf5Vu-o\xbd\xf5\xeb\xb7\xde"\xef\xff\xc7\'8\x1c\x13\x07\x00\xd2\x82\xcc6\xe5\xc6\xa8B&'
<class 'bytes'>
for the working input.
And:
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x048\x00\x00\x07\x80\x08\x06\x00\x00\x00+a\xe7\n\x00\x01\x00\x00IDATx\x9c\xec\xbdw\x80$\xc7u\x1f\xfc\xab\xea\xeeI\x9bw/\'\x1cr\xce\x04#\x10\x04A\x82`\x84\x95%J"\x95,\xcb\x1f%\x91T\xb0$*}\x1fM\xd9\x96\x95EY\x94(\xc9\xb6\x92i+\x90\x12\x83(3)0\x82\x08$rN\x07\\\xce\xb7\xb7yBw\xd5\xf7G\x85\xaeN3\xdd=\xdd\xb3\xb3{\xfb\xc8\xc3\xceLW\xbd\xca\xaf\xde\xfb\xf5\xabW\xe4{\xdeu\x84\xa3`\xe2\x00#J\xe0Y&\xdf\x00e($\x94\x94\'p\xcc\xc3\xda\xe7Y\x0c\xf1Te\x13\xbf\xcc>\xfa:]Y=x\x84\x7f\xe8\xc23u\x1f\x91l\xfd\x99'
<class 'bytes'>
for the failing input.
As far as I can tell, you start off with a PIL Image and you want to obtain a PNG image in memory without going to disk. So you need this:
#!/usr/bin/env python3
from PIL import Image
from io import BytesIO
# Create PIL Image like you have - filled with red
im = Image.new('RGB', (320,240), (255,0,0))
# Create in-memory PNG - like you want for Google Cloud Vision
buffer = BytesIO()
im.save(buffer, format="PNG")
# Look at first few bytes
PNG = buffer.getvalue()
print(PNG[:20])
It prints this, which is exactly what you would get if you wrote the image to disk as a PNG and then read it back as binary - except this does it in memory without going to disk:
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01#'
It would be good to have whole error stack and more accurate code snippet. But form presented information this seems to be confusion of two different "Images". Probably the some copy/paste error, as the tutorials have exactly the same line:
response = client.text_detection(image=image)
But mentioned tutorials image is created by vision.Image() so I think in presented code this should be:
response = client.text_detection(image=vision_image)
As, at least if I understand correctly the code snippet, image is PIL Image, while vision_image is Vision Image that should be passed to text_detection method. So whatever is done in vision.Image() does not have effect on the error massage.
I want to make a generator that generates batches of images from urls to train a keras model. I have another generator that feeds me images urls.
What I currently do is download the image to disk and then load the image from the disk.
def loadImage(URL):
with urllib.request.urlopen(URL) as url:
with open('temp.jpg', 'wb') as f:
f.write(url.read())
img_path = 'temp.jpg'
img = image.load_img(img_path, target_size=(125, 125))
os.remove(img_path)
x = image.img_to_array(img)
return x
def imageGenerator(batch_size):
i = 0
batch = []
for URL in imageUrlGenerator():
if i>batch_size:
yield batch
batch = []
i=0
batch.append(loadImage(URL))
i+=1
This works but I wonder if there isn't a faster way to load images from the web without having to write and read in/from disk.
Assuming you are actually using keras and that this image.load_img is the method you are calling, it would call a function which should ultimately be PIL.Image.open. In the documentation for PIL.image.open, the first argument fp could be a string filename (which is what you are currently passing), or a stream-like object that implements read, seek, and tell. While the object returned by urllib.request.urlopen does provide all three methods, it does not implement seek at all, so it cannot be used directly. However, the entire buffer can be read into an BytesIO object which does implement seek, so it should be usable. Putting this together, your loadImage function may be reduced to something like the following:
from io import BytesIO
def loadImage(URL):
with urllib.request.urlopen(URL) as url:
img = image.load_img(BytesIO(url.read()), target_size=(125, 125))
return image.img_to_array(img)
This keeps the images downloaded fully in memory.
This is the simplest solution I've seen.
from PIL import Image
from urllib import request
from io import BytesIO
url = "https://github.com/ironmanciti/MachineLearningBasic/blob/master/datasets/TransferLearningData/watch.jpg?raw=true"
res = request.urlopen(url).read()
Sample_Image = Image.open(BytesIO(res)).resize((150,150))
plt.imshow(Sample_Image)
Got this from Github issues
from io import BytesIO
from PIL import Image
import requests
def loadImage(url):
response = requests.get(url)
img_bytes = BytesIO(response.content)
img = Image.open(img_bytes)
img = img.convert('RGB')
img = img.resize((250,250), Image.NEAREST)
img = img_to_array(img)
return img
2 Quick fixes:
Consider moving the os.remove(img_path) line. I assume this is removing the file from your drive, but I'm thinking you can save this for the end. Your model just needs all the information possible. Once it has the information, you can either asynchronously begin deleting the files, or just wait until the model is trained and cleanup. Doing it one by one as you have, may be slowing you down.
Use fast storage devices and configurations, SSD, USB 3.x, USB C, etc.
Other fixes:
Is it possible to hold things in a cache?
Could you hold things in an array? I don't think so, but it may be possible.
Do you need the whole image? Could you decrease the quality of the image?
How nested is the image? Parsing for the image likely isn't an issue, but it can't hurt to check.
I'm trying to convert some PDFs to high res jpegs using imagemagick . I'm working on win 10, 64 with python 3.62 - 64 bit and wand 0.4.4. At the command line I have :
$ /e/ImageMagick-6.9.9-Q16-HDRI/convert.exe -density 400 myfile.pdf -scale 2000x1000 test3.jpg.
which is working well for me.
In python:
from wand.image import Image
file_path = os.path.dirname(os.path.abspath(__file__))+os.sep+"myfile.pdf"
with Image(filename=file_path, resolution=400) as image:
image.save()
image_jpeg = image.convert('jpeg')
Which is giving me low res JPEGs . How do I translate this into my wand code to do the same thing?
edit:
I realized that the problem is that the input pdf has to be read into the Image object as a binary string, so based on http://docs.wand-py.org/en/0.4.4/guide/read.html#read-blob I tried:
with open(file_path,'rb') as f:
image_binary = f.read()
f.close()
with Image(blob=image_binary,resolution=400) as img:
img.transform('2000x1000', '100%')
img.make_blob('jpeg')
img.save(filename='out.jpg')
This reads the file in ok, but the output is split into 10 files. Why? I need to get this into 1 high res jpeg.
EDIT:
I need to send the jpeg to an OCR api, so I was wondering if I could write the output to a file like object. Looking at https://www.imagemagick.org/api/magick-image.php#MagickWriteImageFile, I tried :
emptyFile = Image(width=1500, height=2000)
with Image(filename=file_path, resolution=400) as image:
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand,
True)
library.MagickWriteImagesFile(resource_pointer,emptyFile)
This gives:
File "E:/ENVS/r3/pdfminer.six/ocr_space.py", line 113, in <module>
test_file = ocr_stream(filename='test4.jpg')
File "E:/ENVS/r3/pdfminer.six/ocr_space.py", line 96, in ocr_stream
library.MagickWriteImagesFile(resource_pointer,emptyFile)
ctypes.ArgumentError: argument 2: <class 'TypeError'>: wrong type
How can I get this working?
Why? I need to get this into 1 high res jpeg.
The PDF contains pages that ImageMagick considers individual images in a "stack". The wand library provides a wand.image.Image.sequance to work with each page.
However, to append all images into a single JPEG. You can either iterate over each page & stitch them together, or call C-API's method MagickAppendImages.
from wand.image import Image
from wand.api import library
import ctypes
# Map C-API not provided by wand library.
library.MagickAppendImages.argtypes = [ctypes.c_void_p, ctypes.c_int]
library.MagickAppendImages.restype = ctypes.c_void_p
with Image(filename="path_to_document.pdf", resolution=400) as image:
# Do all your preprocessing first
# Ether word directly on the wand instance, or iterate over each page.
# ...
# To write all "pages" into a single image.
# Reset the stack iterator.
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand,
True)
# Write C resource directly to disk.
library.MagickWriteImages(resource_pointer,
"output.jpeg".encode("ASCII"),
False)
Update:
I need to send the jpeg to an OCR api ...
Assuming your using OpenCV's python API, you'll only need to iterate over each page, and pass the image-file data to the OCR via numpy buffers.
from wand.image import Image
import numpy
import cv2
def ocr_process(file_data_buffer):
""" Replace with whatever your OCR-API calls for """
mat_instance = cv2.imdecode(file_data_buffer)
# ... work ...
source_image="path_to_document.pdf"
with Image(filename=source_image, resolution=400) as img:
for page in img.sequence:
file_buffer = numpy.asarray(bytearray(page.make_blob("JPEG")),
dtype=numpy.uint8)
ocr_process(file_buffer)
so I was wondering if I could write the output to a file like object
Don't assume that python "image" objects (or underlining C structures) from different libraries are comparable with each other.
Without knowing the OCR api, I can't help you past the wand part, but I can suggest one of the following...
Use temporary intermediate files. (slower I/O, but easier to learn/develop/debug)
with Image(filename=INPUT_PATH) as img:
# work
img.save(filename=OUTPUT_PATH)
# OCR work on OUTPUT_PATH
Use file descriptors if the OCR API supports it. (Same as above)
with open(INPUT_PATH, 'rb') as fd:
with Image(file=fd) as img:
# work
# OCR work ???
Use blobs. (faster I/O but need a lot more memory)
buffer = None
with Image(filename=INPUT_PATH) as img:
# work
buffer = img.make_blob(FORMAT)
if buffer:
# OCR work ???
Even More Updates
Wrapping all the comments together, a solution might be...
from wand.image import Image
from wand.api import library
import ctypes
import requests
# Map C-API not provided by wand library.
library.MagickAppendImages.argtypes = [ctypes.c_void_p, ctypes.c_int]
library.MagickAppendImages.restype = ctypes.c_void_p
with Image(filename='path_to_document.pdf', resolution=400) as image:
# ... Do pre-processing ...
# Reset the stack iterator.
library.MagickResetIterator(image.wand)
# Call C-API Append method.
resource_pointer = library.MagickAppendImages(image.wand, True)
# Convert to JPEG.
library.MagickSetImageFormat(resource_pointer, b'JPEG')
# Create size sentinel.
length = ctypes.c_size_t()
# Write image blob to memory.
image_data_pointer = library.MagickGetImagesBlob(resource_pointer,
ctypes.byref(length))
# Ensure success
if image_data_pointer and length.value:
# Create buffer from memory address
payload = ctypes.string_at(image_data_pointer, length.value)
# Define local filename.
payload_filename = 'my_hires_image.jpg'
# Post payload as multipart encoded image file with filename.
requests.post(THE_URL, files={'file': (payload_filename, payload)})
What about something like:
ok = Image(filename=file_path, resolution=400)
with ok.transform('2000x1000', '100%') as image:
image.compression_quality = 100
image.save()
or:
with ok.resize(2000, 1000)
related:
https://github.com/dahlia/wand/blob/13c4f544bd271fe298ac8dde44fbf178b349361a/docs/guide/resizecrop.rst
Python 3 Wand How to make an unanimated gif from multiple PDF pages
For a program I am writing, I am transferring an image from one computer - using base64.b64encode(f.read(image)) - and trying to read it in the receiving script without saving it to hard drive (in an effort to minimize process time). I'm having a hard time figuring out how to read the image into OpenCV without saving it locally.
Here is what my code for sending the image looks like:
f = open(image.jpg)
sendthis = f.read()
f.close()
databeingsent = base64.b64encode(sendthis)
client.publish('/image',databeingsent,0)
# this is an MQTT publish, details for SO shouldn't be relevant
Meanwhile, here is the code receiving it. (This is in an on_message function, since I'm using MQTT for the transfer.)
def on_message(client, userdata, msg): # msg.payload is incoming data
img = base64.b64decode(msg.payload)
source = cv2.imread(img)
cv2.imshow("image", source)
After the message decodes, I have the error:
"TypeError: Your input type is not a numpy array".
I've done some searching, and I can't seem to find a relevant solution - some exist regarding converting from text files to numpy using b64, but none really relate to using an image and immediately reading that decoded data into OpenCV without the intermediary step of saving it to the harddrive (using the inverse process used to read the file in the "send" script).
I'm still pretty new to Python and OpenCV, so if there's a better encoding method to send the image - whatever solves the problem. How the image is sent is irrelevant, so long as I can read it in on the receiving end without saving it as a .jpg to disk.
Thanks!
You can get a numpy array from you decoded data using:
import numpy as np
...
img = base64.b64decode(msg.payload)
npimg = np.fromstring(img, dtype=np.uint8)
Then you need imdecode to read the image from a buffer in memory. imread is meant to load an image from a file.
So:
import numpy as np
...
def on_message(client, userdata, msg): # msg.payload is incoming data
img = base64.b64decode(msg.payload);
npimg = np.fromstring(img, dtype=np.uint8);
source = cv2.imdecode(npimg, 1)
From the OpenCV documentation we can see that:
imread : Loads an image from a file.
imdecode : Reads an image from a buffer in memory.
Seem a better way to do what you want.