Retrieve and show image from MongoDB using PyMongo - python

I am using PyMongo to retrieve collection data as dictionary and then trying to convert the image key's respective value from the dict to proper base64 encoded string to show the image, but it doesn't just work. When I use mongoengine, it works fine as the string returned from the latter is b64 encoded already, unlike PyMongo.
Following is my sample working Mongoengine code to retrieve a respective image and show as Pillow image:
from mongoengine import connect, Document, fields
import mongoengine
import io
import PIL.Image as Image
print(mongoengine.__version__)
# To save image
connect(db='video_analytics', host='127.0.0.1', port=27017)
class User(Document):
meta = {"collection": "users"}
username = fields.StringField(required=True)
profile_image = fields.ImageField(thumbnail_size=(150, 150, False))
user = User.objects(username='Doggy').first()
# print(user)
img_data = user.profile_image.read()
# Test
print(img_data)
# Ends
bytes = bytearray(img_data)
image = Image.open(io.BytesIO(bytes))
print(image)
# image.show()
And following is my PyMongo code which gives no error but doesn't show the image:
client = MongoClient("mongodb://127.0.0.1:27017/")
# select database
db = client['frame_read']
# select the collection within the database
test = db["random"]
# print("collections count -->" + str(test.count()))
cursor = test.find()
for rec in cursor:
# print(rec['profile_image'])
f = rec['profile_image']
img = Image.open(f)
print(img.size)
img.show()
I tried many ways, but it is just not working. Here I read by default MongoDB stores in BSON bytes format. Is there a way how to retrieve data properly?
I followed this too, but to no avail.
Note: I'm intentionally not using GridFS because my image size is much lesser than 16 MB. If you say GridFS is still better performant for retrieval, then I shall plan using the same.

Related

Sending images as if they were embedded, with FastAPI [duplicate]

Using the python module fastAPI, I can't figure out how to return an image. In flask I would do something like this:
#app.route("/vector_image", methods=["POST"])
def image_endpoint():
# img = ... # Create the image here
return Response(img, mimetype="image/png")
what's the corresponding call in this module?
If you already have the bytes of the image in memory
Return a fastapi.responses.Response with your custom content and media_type.
You'll also need to muck with the endpoint decorator to get FastAPI to put the correct media type in the OpenAPI specification.
#app.get(
"/image",
# Set what the media type will be in the autogenerated OpenAPI specification.
# fastapi.tiangolo.com/advanced/additional-responses/#additional-media-types-for-the-main-response
responses = {
200: {
"content": {"image/png": {}}
}
}
# Prevent FastAPI from adding "application/json" as an additional
# response media type in the autogenerated OpenAPI specification.
# https://github.com/tiangolo/fastapi/issues/3258
response_class=Response,
)
def get_image()
image_bytes: bytes = generate_cat_picture()
# media_type here sets the media type of the actual response sent to the client.
return Response(content=image_bytes, media_type="image/png")
See the Response documentation.
If your image exists only on the filesystem
Return a fastapi.responses.FileResponse.
See the FileResponse documentation.
Be careful with StreamingResponse
Other answers suggest StreamingResponse. StreamingResponse is harder to use correctly, so I don't recommend it unless you're sure you can't use Response or FileResponse.
In particular, code like this is pointless. It will not "stream" the image in any useful way.
#app.get("/image")
def get_image()
image_bytes: bytes = generate_cat_picture()
# ❌ Don't do this.
image_stream = io.BytesIO(image_bytes)
return StreamingResponse(content=image_stream, media_type="image/png")
First of all, StreamingResponse(content=my_iterable) streams by iterating over the chunks provided by my_iterable. But when that iterable is a BytesIO, the chunks will be \n-terminated lines, which won't make sense for a binary image.
And even if the chunk divisions made sense, chunking is pointless here because we had the whole image_bytes bytes object available from the start. We may as well have just passed the whole thing into a Response from the beginning. We don't gain anything by holding data back from FastAPI.
Second, StreamingResponse corresponds to HTTP chunked transfer encoding. (This might depend on your ASGI server, but it's the case for Uvicorn, at least.) And this isn't a good use case for chunked transfer encoding.
Chunked transfer encoding makes sense when you don't know the size of your output ahead of time, and you don't want to wait to collect it all to find out before you start sending it to the client. That can apply to stuff like serving the results of slow database queries, but it doesn't generally apply to serving images.
Unnecessary chunked transfer encoding can be harmful. For example, it means clients can't show progress bars when they're downloading the file. See:
Content-Length header versus chunked encoding
Is it a good idea to use Transfer-Encoding: chunked on static files?
I had a similar issue but with a cv2 image. This may be useful for others. Uses the StreamingResponse.
import io
from starlette.responses import StreamingResponse
app = FastAPI()
#app.post("/vector_image")
def image_endpoint(*, vector):
# Returns a cv2 image array from the document vector
cv2img = my_function(vector)
res, im_png = cv2.imencode(".png", cv2img)
return StreamingResponse(io.BytesIO(im_png.tobytes()), media_type="image/png")
All the other answer(s) is on point, but now it's so easy to return an image
from fastapi.responses import FileResponse
#app.get("/")
async def main():
return FileResponse("your_image.jpeg")
It's not properly documented yet, but you can use anything from Starlette.
So, you can use a FileResponse if it's a file in disk with a path: https://www.starlette.io/responses/#fileresponse
If it's a file-like object created in your path operation, in the next stable release of Starlette (used internally by FastAPI) you will also be able to return it in a StreamingResponse.
Thanks to #biophetik's answer, with an important reminder that caused me confusion: If you're using BytesIO especially with PIL/skimage, make sure to also do img.seek(0) before returning!
#app.get("/generate")
def generate(data: str):
img = generate_image(data)
print('img=%s' % (img.shape,))
buf = BytesIO()
imsave(buf, img, format='JPEG', quality=100)
buf.seek(0) # important here!
return StreamingResponse(buf, media_type="image/jpeg",
headers={'Content-Disposition': 'inline; filename="%s.jpg"' %(data,)})
The answer from #SebastiánRamírez pointed me in the right direction, but for those looking to solve the problem, I needed a few lines of code to make it work. I needed to import FileResponse from starlette (not fastAPI?), add CORS support, and return from a temporary file. Perhaps there is a better way, but I couldn't get streaming to work:
from starlette.responses import FileResponse
from starlette.middleware.cors import CORSMiddleware
import tempfile
app = FastAPI()
app.add_middleware(
CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]
)
#app.post("/vector_image")
def image_endpoint(*, vector):
# Returns a raw PNG from the document vector (define here)
img = my_function(vector)
with tempfile.NamedTemporaryFile(mode="w+b", suffix=".png", delete=False) as FOUT:
FOUT.write(img)
return FileResponse(FOUT.name, media_type="image/png")
My needs weren't quite met from the above because my image was built with PIL. My fastapi endpoint takes an image file name, reads it as a PIL image, and generates a thumbnail jpeg in memory that can be used in HTML like:
<img src="http://localhost:8000/images/thumbnail/bigimage.jpg">
import io
from PIL import Image
from fastapi.responses import StreamingResponse
#app.get('/images/thumbnail/{filename}',
response_description="Returns a thumbnail image from a larger image",
response_class="StreamingResponse",
responses= {200: {"description": "an image", "content": {"image/jpeg": {}}}})
def thumbnail_image (filename: str):
# read the high-res image file
image = Image.open(filename)
# create a thumbnail image
image.thumbnail((100, 100))
imgio = io.BytesIO()
image.save(imgio, 'JPEG')
imgio.seek(0)
return StreamingResponse(content=imgio, media_type="image/jpeg")
You can use a FileResponse if it's a file in disk with a path:
import os
from fastapi import FastAPI
from fastapi.responses import FileResponse
app = FastAPI()
path = "/path/to/files"
#app.get("/")
def index():
return {"Hello": "World"}
#app.get("/vector_image", responses={200: {"description": "A picture of a vector image.", "content" : {"image/jpeg" : {"example" : "No example available. Just imagine a picture of a vector image."}}}})
def image_endpoint():
file_path = os.path.join(path, "files/vector_image.jpg")
if os.path.exists(file_path):
return FileResponse(file_path, media_type="image/jpeg", filename="vector_image_for_you.jpg")
return {"error" : "File not found!"}
If when following the top answer and you are attempting to return a BytesIO object like this in your Response
buffer = BytesIO(my_data)
# Return file
return Response(content=buffer, media_type="image/jpg")
You may receive an error that looks like this (as described in this comment)
AttributeError: '_io.BytesIO' object has no attribute 'encode'
This is caused by the render function in Response which explicitly checks for a bytes type here. Since BytesIO != bytes it attempts to encode the value and fails.
The solution is to get the bytes value from the BytesIO object with getvalue()
buffer = BytesIO(my_data)
# Return file
return Response(content=buffer.getvalue(), media_type="image/jpg")
You can do something very similar in FastAPI
from fastapi import FastAPI, Response
app = FastAPI()
#app.post("/vector_image/")
async def image_endpoint():
# img = ... # Create the image here
return Response(content=img, media_type="image/png")

How to retrieve data stored from MongoDB using BSON (Python)?

so far in my code bellow I managed to store my data into mongoDB.
Now I want to be able to retrieve the data I have stored.
As you can see I have been trying but keep on getting an error.
With BSON do I have to first decode the data to retrieve it from mongoDB?
Any help would be greatly appreciated!
(Apologies for the messy code, I am just practicing through trial and error)
import json
from json import JSONEncoder
import pymongo
from pymongo import MongoClient
from bson.binary import Binary
import pickle
#Do this for each
client = MongoClient("localhost", 27017)
db = client['datacampdb']
coll = db.personpractice4_collection #creating a collection in the database
#my collection on the database is called personpractice4_collection
class Person:
def __init__(self, norwegian, dame, brit, german, sweed):
self.__norwegian = norwegian
self.__dame = dame
self.__brit = brit
self.__german = german #private variable
self.__sweed = sweed
# create getters and setters later to make OOP
personone = Person("norwegian", "dame", "brit", "german","sweed")
class PersonpracticeEncoder(JSONEncoder):
def default(self, o):
return o.__dict__
#Encode Person Object into JSON"
personpracticeJson = json.dumps(personone, indent=4, cls=PersonpracticeEncoder)
practicedata = pickle.dumps(personpracticeJson)
coll.insert_one({'bin-data': Binary(practicedata)})
#print(personpracticeJson)
#print(db.list_collection_names()) #get then names of my collections in DB
#retriving data from mongodb
#Retrieving a Single Document with find_one()
print(({'bin-data': Binary(practicedata)}).find_one()) #not working
the find_one method should be called on a collection
{'bin-data': Binary(practicedata)} is a query to find a document
coll.find_one({'bin-data': Binary(practicedata)})
Witch means : Find a document in the collection coll where bin-data is equal to Binary(practicedata)

unotools insert image into document (libreoffice)

I'm trying to insert an image into a libreoffice document that is handled/controlled by unotools.
Therefore I start LibreOffice with this command:
soffice --accept='socket,host=localhost,port=8100;urp;StarOffice.Service'
Inside my python code I can connect to LibreOffice:
from unotools import Socket, connect
from unotools.component.writer import Writer
context = connect(Socket('localhost', 8100))
writer = Writer(context)
(This code is taken from this documentation: https://pypi.org/project/unotools/)
By using writer.set_string_to_end() I can add some text to the document. But I also want to insert an image into the document. So far I couldn't find any resource where this was done. The image is inside of my clipboard, so ideally I want to insert the image directly from there. Alternatively I can save the image temporarily and insert the saved file.
Is there any known way how to insert images by using unotools? Any alternative solution would also be great.
I've found a way to insert images by using uno instead of unotools:
import uno
from com.sun.star.awt import Size
from pythonscript import ScriptContext
def connect_to_office():
if not 'XSCRIPTCONTEXT' in globals():
localContext = uno.getComponentContext()
resolver = localContext.ServiceManager.createInstanceWithContext(
'com.sun.star.bridge.UnoUrlResolver', localContext )
client = resolver.resolve("uno:socket,host=localhost,port=8100;urp;StarOffice.ComponentContext" )
global XSCRIPTCONTEXT
XSCRIPTCONTEXT = ScriptContext(client, None, None)
def insert_image(doc):
size = Size()
path = uno.systemPathToFileUrl('/somepath/image.png')
draw_page = self.doc.DrawPage
image = doc.createInstance( 'com.sun.star.drawing.GraphicObjectShape')
image.GraphicURL = path
draw_page.add(image)
size.Width = 7500
size.Height = 5000
image.setSize(size)
image.setPropertyValue('AnchorType', 'AT_FRAME')
connect_to_office()
doc = XSCRIPTCONTEXT.getDocument()
insert_image(doc)
sources:
https://ask.libreoffice.org/en/question/38844/how-do-i-run-python-macro-from-the-command-line/
https://forum.openoffice.org/en/forum/viewtopic.php?f=45&t=80302
I still don't know how to insert an image from my clipboard, I worked around that problem by saving the image first. If someone knows a way to insert the image directly from the clipboard that would still be helpful.

cURL method in Python for JSON feed [duplicate]

This question already has answers here:
How to download a file over HTTP?
(30 answers)
Closed 7 years ago.
While building a flask website, I'm using an external JSON feed to feed the local mongoDB with content. This feed is parsed and fed while repurposing keys from the JSON to keys in Mongo.
One of the available keys from the feed is called "img_url" and contains, guess what, an url to an image.
Is there a way, in Python, to mimic a php style cURL? I'd like to grab that key, download the image, and store it somewhere locally while keeping other associated keys, and have that as an entry to my db.
Here is my script up to now:
import json
import sys
import urllib2
from datetime import datetime
import pymongo
import pytz
from utils import slugify
# from utils import logger
client = pymongo.MongoClient()
db = client.artlogic
def fetch_artworks():
# logger.debug("downloading artwork data from Artlogic")
AL_artworks = []
AL_artists = []
url = "http://feeds.artlogic.net/artworks/artlogiconline/json/"
while True:
f = urllib2.urlopen(url)
data = json.load(f)
AL_artworks += data['rows']
# logger.debug("retrieved page %s of %s of artwork data" % (data['feed_data']['page'], data['feed_data']['no_of_pages']))
# Stop we are at the last page
if data['feed_data']['page'] == data['feed_data']['no_of_pages']:
break
url = data['feed_data']['next_page_link']
# Now we have a list called ‘artworks’ in which all the descriptions are stored
# We are going to put them into the mongoDB database,
# Making sure that if the artwork is already encoded (an object with the same id
# already is in the database) we update the existing description instead of
# inserting a new one (‘upsert’).
# logger.debug("updating local mongodb database with %s entries" % len(artworks))
for artwork in AL_artworks:
# Mongo does not like keys that have a dot in their name,
# this property does not seem to be used anyway so let us
# delete it:
if 'artworks.description2' in artwork:
del artwork['artworks.description2']
# upsert int the database:
db.AL_artworks.update({"id": artwork['id']}, artwork, upsert=True)
# artwork['artist_id'] is not functioning properly
db.AL_artists.update({"artist": artwork['artist']},
{"artist_sort": artwork['artist_sort'],
"artist": artwork['artist'],
"slug": slugify(artwork['artist'])},
upsert=True)
# db.meta.update({"subject": "artworks"}, {"updated": datetime.now(pytz.utc), "subject": "artworks"}, upsert=True)
return AL_artworks
if __name__ == "__main__":
fetch_artworks()
First, you might like the requests library.
Otherwise, if you want to stick to the stdlib, it will be something in the lines of:
def fetchfile(url, dst):
fi = urllib2.urlopen(url)
fo = open(dst, 'wb')
while True:
chunk = fi.read(4096)
if not chunk: break
fo.write(chunk)
fetchfile(
data['feed_data']['next_page_link'],
os.path.join('/var/www/static', uuid.uuid1().get_hex()
)
With the correct exceptions catching (i can develop if you want, but i'm sure the documentation will be clear enough).
You could put the fetchfile() into a pool of async jobs to fetch many files at once.
https://docs.python.org/2/library/json.html
https://docs.python.org/2/library/urllib2.html
https://docs.python.org/2/library/tempfile.html
https://docs.python.org/2/library/multiprocessing.html

insert image in text widget without using imageTK

Unfortunately I'm developing under a osx machine where I've been unable to install imageTK...
I need to insert an image taken from a dictionary of images, saved thus:
i = open(filename, 'rb')
i.seek(0)
w = i.read()
i.close()
allicons[d]=w
so at insert time I don't have a filename, just the dict.
with imageTk, this works:
c = cStringIO.StringIO()
c.write(allicons[key])
c.seek(0)
im = Image.open(c)
iconimage=ImageTk.PhotoImage(im)
config.text.image_create(INSERT, image=iconimage)
but I don't know how to do it without imageTk.
Just with "import Image", I tried:
im = Image.open(c)
iconimage = PhotoImage(im)
and iconimage is recognized (TkInter.PhotoImage object ...),
but inserting it I get a "TypeError: str returned non-string (type instance) ... in image_create *self._options(cnf, kw))"
... any help really appreciated, thanks!
alessandro
PhotoImage has a 'data' option. You can supply it with gif data that's been encoded using base64. something like:
dat = base64.encodestring(allicons[key])
pic = PhotoImage(data=dat)
...

Categories

Resources