This is what I'm currently using to send images to the API:
import requests
response = requests.post("http://someAPI",
auth=(username, password),
files={'imgFile': open(filepath, 'rb')}
)
It works for local files, but I would like to be able to send images from a URL as well. I know how to do this by saving the file, then opening it as a local file, sending it to the API and then removing it.
Is there any way of doing this without having to save and remove the file?
You can use StringIO. StringIO is file compatible object which has open,seek,read,write. So you can load data to it and serve from that, without writing file to disk. I normally use this approch for creating CAPTCH.
import StringIO
import requests
# Imagine you read the image file in image as base64 encoded...
buff = StringIO.StringIO()
buff.write(image.decode('base64'))
buff.seek(0)
response = requests.post("http://someAPI",
auth=(username, password),
files={'imgFile': buff}
)
Related
;TLDR
I want to send a file with requests.send() using multipart/form-data request without storing the file on a hard drive. Basically, I'm looking for an alternative for open() function for bytes object
Hello, I'm currently trying to send multipart/form-data request and pass in-memory files in it, but I can't figure out how to do that.
My app receives images from one source and sends them to another. Currently it sends get request directly to file, (e.g. requests.get('https://service.com/test.jpeg')), reads image's bytes and writes them into new file on the hard drive. The sending code that works looks like this:
def send_file(path_to_image: str)
url = get_upload_link()
data = {'photo': open(path_to_image, 'rb')}
r = requests.post(url, files=data)
send_file("test.jpeg")
The main issue I have with this approach is that I have to keep files on my hard drive. Sure, I can use my drive as some sort of a "temporary buffer" and just delete them after I no longer need these files, but I believe there's much more simple way to do that.
I want my function to receive bytes object and then send it. I actually tried doing that, but the backend doesn't accept them. Here's what I tried to do
Attempt 1
def send_file(image: bytes)
url = get_upload_link()
data = {'photo': open(image, 'rb')}
r = requests.post(url, files=data)
I get "ValueError: embedded null byte"
Attempt 2
def upload_photo(image: bytes):
url = get_upload_link()
file = BytesIO(image)
data = {'photo': file}
r = requests.post(url, files=data)
Backend server doesn't process my files correctly. It's like passing files=None, same response
I also tried:
sending the returning value of the methods: file.getbuffer() and file.read()
file.write(image) and then sending file
StringsIO object
etc.
Final notes
I noticed, that open() returns _io.BufferedReader object. I also looked for a way to construct its instance, but couldn't fund a way. Can someone help me, please?
UPD:
If anyone is interested, the receiving api is this
From the official documentation:
POST a Multipart-Encoded File
...
If you want, you can send strings to
be received as files:
url = 'https://httpbin.org/post'
files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}
r = requests.post(url, files=files)
I want to upload short videos through an API connection (which one is not relevant for the question). The videos that will be uploaded are already on a server (publicly accessible, so that is not the issue) with a direct link (eg: 'https://nameofcompany.com/uploads/videoname.mp4").
I am using the Requests library, so the post request looks like this:
requests.post(url, files={'file': OBJECT_GOES_HERE}, headers=headers)
The object should be a 'bytes-like object', so with a local file we can do:
requests.post(url, files={'file': open('localfile.mp4', 'rb')}, headers=headers)
I tested this with a local file and this works. However, as mentioned I need to upload it from the link, so how do I do that? Is there some method (or some library with a method) that would return the same type of response like the open() method does for local files? If not, how could I create one myself?
import requests
from io import BytesIO
url = 'https://nameofcompany.com/uploads/videoname.mp4'
r = requests.get(url)
video = r.content
# This is probably enough:
requests.post(url2, files={'file': video}, headers=headers)
# But if not, here's an example of using BytesIO to treat bytes as a file:
requests.post(url2, files={'file': open(BytesIO(video), 'rb')}, headers=headers)
I'm wondering if there is a library that would enable me to upload a file to a remote server via ftp. I know there is ftplib, but from what i can tell, it only allows uploading from your own files. So if I had a URL like, https://vignette.wikia.nocookie.net/disney/images/d/db/Donald_Duck_Iconic.png , could I make a program to directly download/upload this to my ftp server. Instead of first having to download it to my own computer, then upload it to the server.
Sorry for formatting I'm on mobile.
You can use requests to download the file, and put the content into a BytesIO for upload:
from io import BytesIO
import requests
url = 'https://vignette.wikia.nocookie.net/disney/images/d/db/Donald_Duck_Iconic.png'
response = requests.get(url)
f = BytesIO(response.content)
Then, f is a file-like object that is suitable for FTP.storbinary.
I am working on Django, python and app engine, Can anyone please tell me hoe to send a pdf file to a url using urllib2,(file is InMemoryUploadedFile). I know there is a question in SOF for sending data using urllib2 with the data being in JSON format..But here I want to send a InMemoryUploadedFile which is a pdf uploaded file from html page. Thanks in advance...
You might want to look at Python: HTTP Post a large file with streaming.
You will need to use mmap for streaming the file in memory, then set it to the request and set the headers to appropriate mime type i.e application/pdf before opening the url.
import urllib2
import mmap
# Open the file as a memory mapped string. Looks like a string, but
# actually accesses the file behind the scenes.
f = open('somelargefile.pdf','rb')
mmapped_file_as_string = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
# Do the request
request = urllib2.Request(url, mmapped_file_as_string)
request.add_header("Content-Type", "application/pdf")
response = urllib2.urlopen(request)
#close everything
mmapped_file_as_string.close()
f.close()
Since Google app engine doesn't have mmap, you might want to write the file in request.FILES to the disk temporarily
#f is the file from request.FILES
def handle_uploaded_file(f):
with open('some/file/name.txt', 'wb+') as destination:
for chunk in f.chunks():
destination.write(chunk)
And then read the file from there directly using standard file operations.
Another option is to use StringIO to write your file to in-memory as a string and then pass it to urlib2.request. This can be inefficient in a multi-user environment compared to using a stream.
I want to download file to my server and automatically send it to online storage(minus or dropbox) via minus or dropbox API, without saving the downloaded file in my server. So, its like streaming or pipe the HTTP connection. Right now im using minus.com API, but its require file object or local file as parameter. I can't figure out how to convert http response to file object.
It is possible to do this? if possible, how?
concept :
FILE_ON_ANOTHER_SERVER ----(http)---> MY_SERVER ----(http)----> ONLINE_STORAGE
thanks
You can get the data from a response via the read() method
response = urllib2.urlopen(request)
data = response.read()
The variable data has the binary data from the response.
Now you can create a StringIO Object which handles the data as a file like object.
import StringIO
datastream = StringIO.StringIO()
datastream.write(data)
datastream.seek(0)
#create dropbox client
client.put_file('/test', datastream)
urllib2.urlopen(url) will return a file-like object. Can you pass that directly to your minus api? See the urllib2 docs at
http://docs.python.org/library/urllib2