This question already has answers here:
How to upload file with python requests?
(9 answers)
Closed 4 years ago.
I was trying to create a minimal flask server/client python movie file uploader but my client-code doesn't seem to be working properly and I'm wondering if I need more then what I have?
Server.py
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['GET', 'POST', 'PUT'])
def hello_world():
file = request.files
return(str(file))
running as: flask run
Uploader.py
import requests
files = {'file': open("BigBuckBunny_320x180.mp4")}
r = requests.post("http://127.0.0.1:5000/", files)
print(r.text)
running as: python Uploader.py
However the hello_world method returns ImmutableMultiDict([])
For Debugging purposes I've used this following curl snippet which seems to work:
curl -i -X PUT -F filedata=#BigBuckBunny_320x180.mp4 "http://localhost:5000/"
and returns
ImmutableMultiDict([('file', <FileStorage: u'BigBuckBunny_320x180.mp4' ('application/octet-stream')>)])
Any Idea why the Uploader.py fails?
I tried example you gave us and I think I managed to find a solution.
After some digging, I found that you can stream requests with requests module:
Requests supports streaming uploads, which allow you to send large streams or files without reading them into memory.
You just need to provide a file to be streamed and open it in read binary mode, rb.
app.py
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['GET', 'POST', 'PUT'])
def hello_world():
# 'bw' is write binary mode
with open("BigBuckBunny_320x180_flask_upload.mp4", "bw") as f:
chunk_size = 4096
while True:
chunk = request.stream.read(chunk_size)
if len(chunk) == 0:
return 'Done'
f.write(chunk)
if __name__ == '__main__':
app.run()
Uploader.py
import requests
# Give proper path to your file, I used mine from flask app directory
with open('BigBuckBunny_320x180.mp4', 'rb') as f:
requests.post('http://127.0.0.1:5000/', data=f)
Check this article.
Related
I have a Flask application that shall provide an endpoint to download a large file. However, instead of providing it from the file system or generating the file on-the-fly, this file has to be downloaded first from another server via HTTP.
Of course, I could perform a GET request to the external server first, download the file completely and store it in the file system or in memory and then as a second step provide it as a result for the original request. This would look for example like this (also including a basic authentication to indicate why a simple proxy on a lower layer is not sufficient):
#!flask/bin/python
from flask import Flask, jsonify
import os
import requests
from requests.auth import HTTPBasicAuth
app = Flask(__name__)
#app.route('/download')
def download():
auth = HTTPBasicAuth("some_user", "some_password")
session = requests.Session()
session.auth = auth
response = session.get("http://example.com")
return response.content
if __name__ == '__main__':
app.run(host='0.0.0.0', port=1234, debug=True)
However, this increases both the latency and the storage requirements of the application. And also, even if the receiver only requires to perform a partial download (i.e. it performs a HTTP range request) of the file, it has to be fetched from the external server completely, first.
Is there a more elegant option to solve this, i.e. to provide support for HTTP range requests that are directly forwarded to the external server?
According to Proxying to another web service with Flask, Download large file in python with requests and Flask large file download I managed to make a Flask HTTP proxy in stream mode.
from flask import Flask, request, Response
import requests
PROXY_URL = 'http://ipv4.download.thinkbroadband.com/'
def download_file(streamable):
with streamable as stream:
stream.raise_for_status()
for chunk in stream.iter_content(chunk_size=8192):
yield chunk
def _proxy(*args, **kwargs):
resp = requests.request(
method=request.method,
url=request.url.replace(request.host_url, PROXY_URL),
headers={key: value for (key, value) in request.headers if key != 'Host'},
data=request.get_data(),
cookies=request.cookies,
allow_redirects=False,
stream=True)
excluded_headers = ['content-encoding', 'content-length', 'transfer-encoding', 'connection']
headers = [(name, value) for (name, value) in resp.raw.headers.items()
if name.lower() not in excluded_headers]
return Response(download_file(resp), resp.status_code, headers)
app = Flask(__name__)
#app.route('/', defaults={'path': ''})
#app.route('/<path:path>')
def download(path):
return _proxy()
if __name__ == '__main__':
app.run(host='0.0.0.0', port=1234, debug=True)
download_file() will open the request in stream mode and yield every chunk as soon as they got streamed.
_proxy() create the request then just create and return a Flask Response using the iterator download_file() as content.
I tested it with https://www.thinkbroadband.com/download where several archive files are free to download for test purpose. (be careful, archives are corrupted, so you better use checksum to make sure you got the expected file).
Some examples:
curl 'http://0.0.0.0:1234/100MB.zip' --output /tmp/100MB.zip
curl 'http://0.0.0.0:1234/20MB.zip' --output /tmp/20MB.zip
I also performed some other tests on random websites to get large images. So far I got no issues.
I have created a flask api and it is currently running in local machine
From one machine I ran the python script to post the text file:
import json
import requests
datas = {'var1' : 'var1','var2' : 'var2',}
#my file to be sent
local_file_to_send = 'C:\\Users\\tmp.txt'
with open(local_file_to_send, 'w') as f:
f.write('I am a file\n')
url = "http://127.0.0.1:5000/customerupdate"
files = [
('document', (local_file_to_send, open(local_file_to_send, 'rb'), 'application/octet')),
('datas', ('datas', json.dumps(datas), 'application/json')),
]
r = requests.post(url, files=files)
print("sd")
print(str(r.content, 'utf-8'))
Python flask API:
import json
from flask import Flask, request
from flask import send_file
app = Flask(__name__)
#app.route('/',methods=['GET'])
def hello_world():
return 'Hello World!'
#app.route('/customerupdate',methods=['GET','POST'])
def customerupdate():
posted_file = str(request.files['document'].read(), 'utf-8')
posted_data = json.load(request.files['datas'])
print(posted_file)
print(posted_data)
return '{}\n{}\n'.format(posted_file, posted_data)
if __name__=='__main__':
app.run(debug='true',threaded=True)
And the python get method to receive the file:
import requests
r = requests.get('http://127.0.0.1:5000')
print(r.text)
But here in get method I have received the contents of the file.But I want it for csv file.I tried by making that .csv instead of .txt.But can't able to send the file to api .So, Is it possible to send the csv file from one machine and receive the csv file in another machine via flask and please post the code if solutions available.
Already tried with this link:Python Flask: Send file and variable
I struggling lot to resolve it.
the api should include one function called "write text to file" and inputs a string parameter
as for the function to write to the disk I have no problem and I implemented the code my problem is how to set the rest API using python.
EDIT:
this is my code:
from flask import (
Flask,
render_template
)
import SocketServer
import SimpleHTTPServer
import re
app = Flask(__name__, template_folder="templates")
#app.route('/index', methods=['GET'])
def index():
return 'Welcome'
#app.route('/write_text_to_file', methods=['POST'])
def write_text_to_file():
f = open("str.txt", "w+")
f.write("hello world")
f.close()
if __name__ == '__main__':
app.run(debug=True)
anyhow when I try to test my rest api:
http://127.0.0.1:5000/write_text_to_file
I am getting the following error:
Now I'm trying to test my rest-api , however how can I make my code to start the server and to the test the post request api, this is my test_class:
import requests
import unittest
API_ENDPOINT="http://127.0.0.1:5000/write_text_to_file"
class test_my_rest_api(unittest.TestCase):
def test_post_request(self):
"""start the server"""
r = requests.post(API_ENDPOINT)
res = r.text
print(res)
also when runnning my request using postman I am getting internal_server_error:
You're doing a GET request for this url, but you've specified that this endpoint can only accept POST:
#app.route('/write_text_to_file', methods=['POST'])
Also, the SocketServer and SimpleHTTPServer imports are not needed with Flask.
The method is not allowed because Chrome (or any browser) makes GET requests.
Whereas, you defined it as POST
#app.route('/write_text_to_file', methods=['POST'])
Either change it to a GET method, or use a tool such as POSTMan to perform other HTTP call types
This question already has an answer here:
Testing code that requires a Flask app or request context
(1 answer)
Closed 5 years ago.
First of all, I am very new at programming.
I am trying to save a variable from bash shell
>curl http://169.254.169.254/latest/meta-data/
this line would return data such as local-ipv4. And I am trying to use phython and flask to save those variables. I wrote
from flask import Flask, request
app = Flask(__name__)
#app.route('/')
def testRequest():
url1 = "http://169.254.169.254/latest/meta-data/"
name1 = request.get(url1)
nameText = name1.text
return nameText
testOutput = testRequest()
print testOutput
But this gives me runtime error : working outside of the request context.
can someone guide me to where to look for possible solution?
To things to comment here:
First, it seems that you'll be just fine by using requests, a highly recommended library for HTTP requests. With it you could do:
import requests
url = "http://169.254.169.254/latest/meta-data/"
resp = requests.get(url)
print resp.text
Regards to the error you're getting runtime error : working outside of the request context, is because by testOutput = testRequest() you're calling a method that's part of the Flask app app. Another thing related to the error is that you never ran the Flask app. To do this, include this at the end of your code.
if __name__ == '__main__':
app.run()
But again, Flask is rather a web framework that it's useful to create web sites, APIs, web apps, etc. It's very useful, but I don't think you may need it for what you're trying to achieve.
Further info about requests and Flask:
http://docs.python-requests.org/
http://flask.pocoo.org/
Since you only need to make an HTTP GET request and print the response, you don't need Flask. You can use the urllib standard library to send the GET request (https://docs.python.org/3/library/urllib.request.html):
import urllib.request
def testRequest():
url1 = "http://169.254.169.254/latest/meta-data/"
response = urllib.request.urlopen(url1)
nameText = response.read().decode('utf-8')
return nameText
testOutput = testRequest()
print testOutput
I am trying to upload a file from my desktop to Flask running on my local machine without using a web form at all. I have looked at the examples for BOTH Python Requests and Poster.
Here is the code for my Flask API
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
#app.route('/upload', methods= ['GET', 'POST'])
def upload_file():
if request.method == 'POST':
f = request.files['files']
f.save('/Users/djangod/newTest.txt')
return '200'
else:
return 'Upload Page'
if __name__ == '__main__':
app.debug = True
app.run()
Here is the code from my Python script using Requests
import requests
url = "http://127.0.0.1:5000/upload"
files = {'file': open('/Users/djangod/text.txt', 'rb')}
r = requests.post(url, files=files)
I get an error 400 and cannot figure out why but for some strange reason running this curl command works
curl -i -X POST -F files=#test.txt http://127.0.0.1:5000/upload
I initially thought Flask was configured incorrectly, until curl worked. I am running on OS X 10.8.5 and using virtualenv. Any help would be greatly appreciated. Thanks in advance.
Your requests code is wrong (probably a typo). Below is what's different between what should work and what you have:
files = {'files': open('/Users/djangod/text.txt', 'rb')}
# ^ missing an s
You're telling Flask to look for a file called 'files' so you need to tell requests to send that as the filename.