Develop Reference

Pass PIL Image to google cloud vision without saving and reading

UPDATE BELOW
Is there a way to pass a PIL Image to google cloud vision?
I tried to use io.Bytes, io.String and Image.tobytes() but I always get:
Traceback (most recent call last):
"C:\Users\...\vision_api.py", line 20, in get_text
image = vision.Image(content)
File "C:\...\venv\lib\site-packages\proto\message.py", line 494, in __init__
raise TypeError(
TypeError: Invalid constructor input for Image:b'Ma\x81Ma\x81La\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81Ma\x81La\x81Ma\x81Ma\x81Ma\x81Ma\x80Ma\x81La\x81Ma\x81Ma\x81Ma\x80Ma\x81Ma\x81Ma\x81Ma\x8 ...
or this if I pass the PIL-Image directly:
TypeError: Invalid constructor input for Image: <PIL.Image.Image image mode=RGB size=480x300 at 0x1D707131DC0>
This is my code:
image = Image.open(path).convert('RGB') # Opening the saved image
cropped_image = image.crop((30, 900, 510, 1200)) # Cropping the image
vision_image = vision.Image(# I passed the different options) # Here I need to pass the image, but I don't know how
client = vision.ImageAnnotatorClient()
response = client.text_detection(image=vision_image) # Text detection using google-vision-api
FOR CLARITY:
I want google text detection to only analyse a certain part of an image saved on my disk. So my idea was to crop the image using PIL and then pass the cropped image to google-vision. But it is not possible to pass an PIL-Image to vision.Image, as I get the error above.
The documentation from Google.
This can be found in the vision.Image class:
Attributes:
content (bytes):
Image content, represented as a stream of bytes. Note: As
with all ``bytes`` fields, protobuffers use a pure binary
representation, whereas JSON representations use base64.
Currently, this field only works for BatchAnnotateImages
requests. It does not work for AsyncBatchAnnotateImages
requests.
A working option is to save the PIL-Image as a PNG/JPG on my disk and load it using:
with io.open(file_name, 'rb') as image_file:
content = image_file.read()
vision_image = vision.Image(content=content)
But this is slow and seems unnecessary. And the whole point for me behind using google-vision-api is the speed comaped to open-cv.
UPDATE as of 25/9/2021
from PIL import Image
from io import BytesIO
from google.cloud import vision
with open('images/screenshots/screenshot.png', 'rb') as image_file:
data = image_file.read()
try:
image = vision.Image(content=data)
print('worked')
except TypeError:
print('failed')
im = Image.open('images/screenshots/screenshot.png')
buffer = BytesIO()
im.save(buffer, format='PNG')
try:
image = vision.Image(buffer.getvalue())
print('worked')
except TypeError:
print('failed')
The first version works as expected, but I can't get the second one to work as #Mark Setchell recommended. The first few characters (~50) are the same, the rest is completely different.
UPDATE as of 26/9/2021
Both inputs are of type <class 'bytes'>. The complete error stack can be seen at the top of the question.
Using this code:
print(input_data[:200])
print(type(input_data))
i get the following output:
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x048\x00\x00\x07\x80\x08\x06\x00\x00\x00+a\xe7\n\x00\x00\x00\x04sBIT\x08\x08\x08\x08|\x08d\x88\x00\x00 \x00IDATx\x9c\xec\xbdy\xd8-\xc7Y\x1f\xf8\xab\xea>\xe7\xdb\xef\xaa\xbbk\xb3%\xcb\x8b\x16[\x12\xc6\xc8\xbb,\x1b\x03\x06\xc6\x8111\x93#2y\xc2381\x8b1\x90\x10\x9e\xf18\x93\x10\x0811\x84\x192\x0c3\x9e\x1020\x03\x03\xc3\xb0\x04\xf0C0\xc6\x96m\xc9\x96m\xed\xb2dI\x96\xaetu\xf7\xed\xdb\xcf\xe9\xae\x9a?j\xe9\xea\xbd\xba\xbb\xbaO\x9f\xef\x9e\xd7\xd6\xfd\xfat\xbf\xf5Vu-o\xbd\xf5\xeb\xb7\xde"\xef\xff\xc7\'8\x1c\x13\x07\x00\xd2\x82\xcc6\xe5\xc6\xa8B&'
<class 'bytes'>
for the working input.
And:
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x048\x00\x00\x07\x80\x08\x06\x00\x00\x00+a\xe7\n\x00\x01\x00\x00IDATx\x9c\xec\xbdw\x80$\xc7u\x1f\xfc\xab\xea\xeeI\x9bw/\'\x1cr\xce\x04#\x10\x04A\x82`\x84\x95%J"\x95,\xcb\x1f%\x91T\xb0$*}\x1fM\xd9\x96\x95EY\x94(\xc9\xb6\x92i+\x90\x12\x83(3)0\x82\x08$rN\x07\\\xce\xb7\xb7yBw\xd5\xf7G\x85\xaeN3\xdd=\xdd\xb3\xb3{\xfb\xc8\xc3\xceLW\xbd\xca\xaf\xde\xfb\xf5\xabW\xe4{\xdeu\x84\xa3`\xe2\x00#J\xe0Y&\xdf\x00e($\x94\x94\'p\xcc\xc3\xda\xe7Y\x0c\xf1Te\x13\xbf\xcc>\xfa:]Y=x\x84\x7f\xe8\xc23u\x1f\x91l\xfd\x99'
<class 'bytes'>
for the failing input.

As far as I can tell, you start off with a PIL Image and you want to obtain a PNG image in memory without going to disk. So you need this:
#!/usr/bin/env python3
from PIL import Image
from io import BytesIO
# Create PIL Image like you have - filled with red
im = Image.new('RGB', (320,240), (255,0,0))
# Create in-memory PNG - like you want for Google Cloud Vision
buffer = BytesIO()
im.save(buffer, format="PNG")
# Look at first few bytes
PNG = buffer.getvalue()
print(PNG[:20])
It prints this, which is exactly what you would get if you wrote the image to disk as a PNG and then read it back as binary - except this does it in memory without going to disk:
b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01#'

It would be good to have whole error stack and more accurate code snippet. But form presented information this seems to be confusion of two different "Images". Probably the some copy/paste error, as the tutorials have exactly the same line:
response = client.text_detection(image=image)
But mentioned tutorials image is created by vision.Image() so I think in presented code this should be:
response = client.text_detection(image=vision_image)
As, at least if I understand correctly the code snippet, image is PIL Image, while vision_image is Vision Image that should be passed to text_detection method. So whatever is done in vision.Image() does not have effect on the error massage.

Python: how to convert an image in memory?

I have an image in memory (downloaded from an online source) and I want to convert it to a different format before sending it on to a different online location.
The conversion is .webp to .jpg but that's not really relevant.
With Pillow I can easily convert local images and save them back to disc, but I can't get it to work with an image in memory.
I don't necessarily need to use Pillow. Any way to convert the image without having to save anything to disc is fine.

I am new to BytesIO with PIL, so just check my code attempt, with my test image it works, let me know
from PIL import Image
from io import BytesIO
img = Image.open('test.webp')
print('image : ', img.format)
img.show()
# Write PIL Image to in-memory PNG
membuf = BytesIO()
img.save(membuf, format="png")
img = Image.open(membuf)
print('image : ', img.format)
img.show()

conveting bytes to image using tinytag and PIL

I am using tinytags module in python to get the cover art of a mp3 file and want to display or store it. The return type of the variable is showing to be bytes. I have tried fumbling around with PIL using frombytes but to no avail. Is there any method to convert the bytes to image?
from tinytag import TinyTag
tag = TinyTag.get("03. Me, Myself & I.mp3", image=True)
img = tag.get_image()

I actually got a PNG image when I called tag.get_image() but I guess you might get a JPEG. Either way, you can wrap it in a BytesIO and open it with PIL/Pillow or display it. Carrying on from your code:
from PIL import Image
import io
...
im = tag.get_image()
# Make a PIL Image
pi = Image.open(io.BytesIO(im))
# Save as PNG, or JPEG
pi.save('cover.png')
# Display
pi.show()
Note that you don't have to use PIL/Pillow. You could look at the first few bytes and if they are a PNG signature (\x89PNG) save data as binary with PNG extension. If the signature is JPEG (\xff \xd8) save data as binary with JPEG extension.

Display TIFF Image On Juypyter Notebook using Python Without Saving / Downloading

I am working with web services using requests to get an image based on parameters passed. The first response I get is a XML schema with file reference URL.
<?xml version="1.0"?>
<Coverages schemaLocation="http://localhost/server/schemas/wcs/1.1.1/wcsCoverages.xsd">
<Coverage>
<Title>Filename</Title>
<Abstract/>
<Identifier>Filename</Identifier>
<Reference href="http://localhost/server/temp/filename.tif"/>
</Coverage>
</Coverages>
Next using xml.etree.ElementTree I extracted the URL. Next what I need is to dsiplay that tiff image (or any other image) on the Jupyter Notebook without downloading (as one image can be more than 50 or 100 MB sometimes)
Currently I am downloading and plotting the file after reading and converting file data into array ( as pyplot plot image array/matrix ).
import requests as req # request wcs urls
import xml.etree.ElementTree as ET # used for xml parsing
import matplotlib.pyplot as plt # display image
import gdal
# Download the File to local directory using Chunks
chunk_size=1024
local_filename = url.split('/')[-1] # Filename from url
r = req.get(url, stream=True)
with open(local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size):
if chunk:
f.write(chunk)
# Read File Raster Data as Array using Gdal
gtif = gdal.Open(local_filename)
georaster = gtif.ReadAsArray()
# Plot image using matplotlib
plt.imshow(georaster)
plt.title(local_filename)
plt.show()
So is there any way to convert the raw response from requests API for file directly into image array (in chunks or whole) and display it on notebook (without downloading and taking space on local directory)
The raw response from get request for tiff file is below
resp2 = req.get('tiffFielurl')
rawdata = resp2.content
rawdata[0:10]
Output: b'MM\x00*\x00\x00\x00\x08\x00\x10'
I tried searching for this question but not found any good answer on it so if there is any related question or duplicate provide me the link.

You can try plotting tiff images using ipyplot package
import ipyplot
ipyplot.plot_images(
['https://file-examples.com/wp-content/uploads/2017/10/file_example_TIFF_1MB.tiff',
'https://file-examples.com/wp-content/uploads/2017/10/file_example_TIFF_1MB.tiff',
'https://file-examples.com/wp-content/uploads/2017/10/file_example_TIFF_1MB.tiff'],
img_width=250)

After doing so much research and trying different solutions it seems to me the only way to do the above procedure i.e displaying Tiff file for now is downloading and reading the data using gdal and converting into array and display using matplotlib .
As the solution mentioned in following link is only accepting "PNG" files.
How to plot remote image (from http url)
whcih comes to conclusion we need PIL library that I also tried and fails.
from PIL import Image
resp2 = req.get('tiffFielurl')
resp2.raw.decode_content = True
im = Image.open(resp2.raw)
im
Gives Output:
<PIL.TiffImagePlugin.TiffImageFile image mode=I;16BS size=4800x4800 at 0x11CB7C50>
and converting the PIL object to numpy array or even getting data or pixels from PIL object gives error of unrecognized mode error.
im.getdata()
im.getpixels((0,0))
numpy.array(im)
All gives same error
257 if not self.im or\
258 self.im.mode != self.mode or self.im.size != self.size:
--> 259 self.im = Image.core.new(self.mode, self.size)
260 # create palette (optional)
261 if self.mode == "P":
ValueError: unrecognized mode
It comes out that PIL even don't support 16bit Signed integer pixel that is defined in Tiff object of PIL above.
https://pillow.readthedocs.io/en/4.0.x/handbook/concepts.html#concept-modes

Python PIL reading PNG from STDIN

I am having a problem reading png images from STDIN using PIL. When the image is written by PIL it is all scrambled, but if I write the file using simple file open, write and close the file is saved perfectly.
I have a program that dumps png files to stdout in a sequence, with no compression, and I read that stream using a python script which is suposed to read the data and do some routines on almost every png. The program that dumps the data writes a certain string to delimiter the PNGs files, the string is "{fim:FILE_NAME.png}"
The script is something like:
import sys
import re
from PIL import Image
png = None
for linha in sys.stdin:
if re.search('{fim:', linha):
fname = linha.replace('{fim:','')[:-2]
# writes data directly to file, works fine
#f = open("/tmp/%s" % fname , 'w')
#f.write(png)
#f.close()
# create a PIL Image from data and writes to disk, fails fine
im = Image.frombuffer("RGB",(640,480),png, "raw", "RGB", 0, 1)
#im = Image.fromstring("RGB",(640,480),png)
im.save("/tmp/%s" % fname)
png = None
else:
if png is None:
png = linha
else:
png+= linha
imagemagick identify from a wrong image:
/tmp/1349194042-24.png PNG 640x480 640x480+0+0 8-bit DirectClass 361KiB 0.010u 0:00.019
imagemagick identify from a working image:
/tmp/1349194586-01.png PNG 640x480 640x480+0+0 8-bit DirectClass 903KiB 0.010u 0:00.010
Does any one have an idea of what is happening? Is it something about little/big endians? I have tried Image.frombuffer, Image.fromstring, different modes, but nothing. It seems that there is more information on the buffer that the PIL expects.
Thanks,

If the png variable contains the binary data from a PNG file, you can't read it using frombuffer; that's used for reading raw pixel data. Instead, use io.StringIO and Image.open, i.e.:
import io
from PIL import Image
img = Image.open(io.StringIO(png))

png variable is uninitialized on the first call to Image.frombuffer(). You need to initialize it to something from stdin.
I'm not sure about your use of for linha in sys.stdin:. That gives you line-buffered input. You probably want to use block buffered input of size N, like sys.stdin.read(N). This will read a specific number of bytes and then you can parse the data, like cutting your filename delimiter out and filling the input buffer for Image.frombuffer().