For few days now I am trying to find how the BytesIO is working with what I want to do.
So basically I want to download a pdf that its base64 encoded. After downloading it I want to save the file but not on the disk but in memory. And then I need to upload the path of that file to an API. Does anyone know how I could do that?
Until now this is what I have so far.
def string_io_save(document_string: str, attach_id: str, file_name: str, file_type: str):
img_data = b64decode(document_string)
path = attach_id
out_stream = BytesIO()
with open('imageToSave.pdf', 'wb') as f:
f.write(img_data)
bytesio = BytesIO(f.read())
return bytesio
You can create a file-like BytesIO like this: BytesIO(img_data)
Your function this way is much simpler:
def string_io_save(document_string: str):
img_data = b64decode(document_string)
return BytesIO(img_data)
now you can work with the return value of string_io_save as if it were a regular file (which is very useful for testing purposes):
fp = string_io_save("base64-encoded-sting....")
foo = fp.read()
fp.close()
Related
with open("image.jpg", 'rb') as original_file:
original = original_file.read()
# write part
with open("dubs.jpg", 'wb') as duplicate_file:
duplicate_file.write(original)
#directly if i pass original to write and create an image it'll work. But i want to save that byte information and able to recreate the same image using same byte data, even if im on different pc.
How about this? Does this work for your case?
import base64
file_name = '<image_file_path>'
# open the file as bytes
with open(file_name, 'rb') as f:
in_jpg_encoding = f.read()
# Encode the bytes object to base64
base64_encoding = base64.b64encode(in_jpg_encoding)
# Store the base64 encoding in text file
with open("base64.txt", "wb") as fw:
fw.write(base64_encoding)
# -------------------------------------------------
# Open the base64 encoding text file
with open("base64.txt", 'rb') as f:
result = f.read()
# Apply the base64 decoding
out_b64_decoding = base64.b64decode(result)
# Store the file as .jpg
with open("recreate.jpg", "wb") as fw:
fw.write(out_b64_decoding)
You can use the following to convert any type of image files to bytes :
import base64
with open("example.png", "rb") as imageFile:
bytes = base64.b64encode(imageFile.read())
print(bytes)
then you can convert the "bytes" variable resulted from the above code by using the following :
from PIL import Image
from io import BytesIO
image = Image.open(BytesIO(base64.b64decode(bytes)))
image.save('output\\path\\example.png', 'PNG')
I am using image compression to reduce the image size. When submitting the post request, I am not getting any error, but can't figure out why the images do not get saved. Here is my code:
#app.post("/post_ads")
async def create_upload_files(title: str = Form(),body: str = Form(),
db: Session = Depends(get_db), files: list[UploadFile] = File(description="Multiple files as UploadFile")):
for file in files:
im = Image.open(file.file)
im = im.convert("RGB")
im_io = BytesIO()
im = im.save(im_io, 'JPEG', quality=50)
PIL.Image.open() takes as fp argumnet the following:
fp – A filename (string), pathlib.Path object or a file object. The
file object must implement file.read(), file.seek(), and
file.tell() methods, and be opened in binary mode.
Using a BytesIO stream, you would need to have something like the below (as shown in client side of this answer):
Image.open(io.BytesIO(file.file.read()))
However, you don't really have to use an in-memory bytes buffer, as you can get the the actual file object using the .file attribute of UploadFile. As per the documentation:
file: A SpooledTemporaryFile (a file-like object).
This is the actual Python file that you can pass directly to other
functions or libraries that expect a "file-like" object.
Example - Saving image to disk:
# ...
from fastapi import HTTPException
from PIL import Image
#app.post("/upload")
def upload(file: UploadFile = File()):
try:
im = Image.open(file.file)
if im.mode in ("RGBA", "P"):
im = im.convert("RGB")
im.save('out.jpg', 'JPEG', quality=50)
except Exception:
raise HTTPException(status_code=500, detail='Something went wrong')
finally:
file.file.close()
im.close()
Example - Saving image to an in-memory bytes buffer (see this answer):
# ...
from fastapi import HTTPException
from PIL import Image
#app.post("/upload")
def upload(file: UploadFile = File()):
try:
im = Image.open(file.file)
if im.mode in ("RGBA", "P"):
im = im.convert("RGB")
buf = io.BytesIO()
im.save(buf, 'JPEG', quality=50)
# to get the entire bytes of the buffer use:
contents = buf.getvalue()
# or, to read from `buf` (which is a file-like object), call this first:
buf.seek(0) # to rewind the cursor to the start of the buffer
except Exception:
raise HTTPException(status_code=500, detail='Something went wrong')
finally:
file.file.close()
buf.close()
im.close()
For more details and code examples on how to upload files/images using FastAPI, please have a look at this answer and this answer. Also, please have a look at this answer for more information on defining your endpoint with def or async def.
I assume you are writing to a BytesIO to get an "in memory" JPEG without slowing yourself down by writing to disk and cluttering your filesystem.
If so, you want:
from PIL import Image
from io import BytesIO
im = Image.open(file.file)
im = im.convert("RGB")
im_io = BytesIO()
# create in-memory JPEG in RAM (not disk)
im.save(im_io, 'JPEG', quality=50)
# get the JPEG image in a variable called JPEG
JPEG = im_io.get_value()
I have a python code that creates a temporary pdf files from tableau views and sends it to the slack channel separately.
I want to combine them together into one file but I can't figure out how to do it.
I am fairly new to python and would really appreciate some help in how to use PdfFileMerger in the code below.
i've tried to use
merger.append(f)
after f variable but it doesn't work giving me ar error ** AttributeError: 'dict' object has no attribute 'seek'
** what should I put in brackets?
for view_item in all_views :
with tempfile.NamedTemporaryFile(suffix='.pdf', delete=True) as temp_file:
#server.views.populate_image(view_item, req_options=image_req_option)
server.views.populate_pdf(view_item, req_options = pdf_req_option)
print('got the image')
temp_file.write(view_item.pdf)
temp_file.file.seek(0)
print('in the beginnign again')
f = {'file': (temp_file.name,temp_file, 'pdf')}
merger.append(f)
response = requests.post(url='https://slack.com/api/files.upload', data=
{'token': bot_token, 'channels': slack_channels[0], 'media': f,'title': '{} {}'.format(view_item.name, yesterday), 'initial_comment' :''},
headers={'Accept': 'application/json'}, files=f)
print('the image is in the channel')
You'll need to feed PdfFileMerger the file objects, like so, not a dict.
Since PdfFileMerger will do things in-memory anyway, there's no need to write to tempfiles on the disk, a BytesIO in memory will do fine.
import io
merger = PdfFileMerger()
for view_item in all_views:
server.views.populate_pdf(view_item, req_options=pdf_req_option)
# Write retrieved data into memory file
tf = io.BytesIO()
tf.write(view_item.pdf)
tf.seek(0)
# Add it to the merger
merger.append(tf)
# Write merged data into memory file
temp_file = io.BytesIO()
merger.write(temp_file)
temp_file.seek(0)
f = {'file': ('merged.pdf', temp_file, 'pdf')}
# Slack stuff here...
I am resizing images client side before sending them to my flask app.
The resized image, which is drawn into a canvas to be resized, is sent via a POST request.
In my app the image is decoded via base64:
def resize_image(item):
content = item.split(';')[1]
image_encoded = content.split(',')[1]
body = base64.decodestring(image_encoded.encode('utf-8'))
return body
The imagedata is stored as type String in the body variable. I can save the data to my local machine and it works:
filename = 'some_image.jpg'
with open(filename, 'wb') as f:
print "written"
f.write(body)
What I need is to upload the resized image to AWS3. On one point I need to read() the image contents, but until the image is saved somewhere as a file it is still a String, so it fails:
file_data = request.values['a']
imagedata = resize_image(file_data)
s3 = boto.connect_s3(app.config['MY_AWS_ID'], app.config['MY_AWS_SECRET'], host='s3.eu-central-1.amazonaws.com')
bucket_name = 'my_bucket'
bucket = s3.get_bucket(bucket_name)
k = Key(bucket)
# fails here
file_contents = imagedata.read()
k.key = "my_images/" + "test.png"
k.set_contents_from_string(file_contents)
Unless there is an other solution, I thought I save the image temporarily to my server (Heroku) and upload it and then delete it, how would this work? Deleting afterwards is important here!
set_contents_from_string takes a string as a parameter, you could probably just pass your image string data directly to it for upload to S3
Solution:
Delete this part:
file_contents = imagedata.read()
Use imagedata directly here:
k.set_contents_from_string(imagedata)
If you need to call .read() on your data, but don't need save file on disk use StringIO:
import StringIO
output = StringIO.StringIO()
output.write('decoded image')
output.seek(0)
output.read()
Out[1]: 'decoded image'
I am using a script to strip exif data from uploaded JPGs in Python, before writing them to disk. I'm using Flask, and the file is brought in through requests
file = request.files['file']
strip the exif data, and then save it
f = open(file)
image = f.read()
f.close()
outputimage = stripExif(image)
f = ('output.jpg', 'w')
f.write(outputimage)
f.close()
f.save(os.path.join(app.config['IMAGE_FOLDER'], filename))
Open isn't working because it only takes a string as an argument, and if I try to just set f=file, it throws an error about tuple objects not having a write attribute. How can I pass the current file into this function before it is read?
file is a FileStorage, described in http://werkzeug.pocoo.org/docs/datastructures/#werkzeug.datastructures.FileStorage
As the doc says, stream represents the stream of data for this file, usually under the form of a pointer to a temporary file, and most function are proxied.
You probably can do something like:
file = request.files['file']
image = file.read()
outputimage = stripExif(image)
f = open(os.path.join(app.config['IMAGE_FOLDER'], 'output.jpg'), 'w')
f.write(outputimage)
f.close()
Try the io package, which has a BufferedReader(), ala:
import io
f = io.BufferedReader(request.files['file'])
...
file = request.files['file']
image = stripExif(file.read())
file.close()
filename = 'whatever' # maybe you want to use request.files['file'].filename
dest_path = os.path.join(app.config['IMAGE_FOLDER'], filename)
with open(dest_path, 'wb') as f:
f.write(image)