Im using Python bottle framework to upload a file..The controller receives the file and transmits to another system using Java rest service..
Everything works fine when the file is small but if the file is huge (it takes 5 mints) the application returns a blank page instead of a html page that its supposed to return.
#app.route('/DataLoads',method='GET')
def pptriv():
return template('dataload',
footer_html=FOOTER_HTML,feedback_html=FEEDBACK_HTML)
#app.route('/DataLoads',method='POST')
def pptriv():
username = request.forms.get('username')
password = request.forms.get('password')
uploadfile = request.files.get('File Upload')
....
..use Python requests module to transmit the files.....
...
print 'r.status_code = ', r.status_code
return template('dataload',
footer_html=FOOTER_HTML,feedback_html=FEEDBACK_HTML)
I see the print stmt r.status_code but the dataload html page, if its small file everything looks good..
Have you tried increasing MEMFILE_MAX?
import bottle
bottle.BaseRequest.MEMFILE_MAX = 1024 * 1024 * 200
Python bottle module causes "Error: 413 Request Entity Too Large"
Also: are you using a browser or a command-line client to upload the file? If browser, then I'd suggest using curl to see exactly what (if anything) is returned from your server.
Related
I am working to create a pipeline with the spotify API that logs my streaming history. I am planning to automate it by uploading it as a lambda function and scheduling it to run every few hours. I have everything mostly in order, except for that on the first run the API requires web authentication. Here is my code:
import spotipy
import spotipy.util as util
import urllib3
un = USERNAME
scope = 'user-read-recently-played'
cid = CLIENT_ID
csid = CLIENT_SECRET_ID
redr = r'http://localhost:8888/callback/'
token = util.prompt_for_user_token(un,scope,cid,csid,redr)
When this is run for the first time, this message pops up:
User authentication requires interaction with your
web browser. Once you enter your credentials and
give authorization, you will be redirected to
a url. Paste that url you were directed to to
complete the authorization.
Opened <LINK HERE> in your browser
Enter the URL you were redirected to:
And then I have to copy the link from my browser into that space. I can get the URL that I need to paste using urllib3:
req_adr = ADDRESS_IT_OPENS_IN_BROWSER
http = urllib3.PoolManager()
resp = http.request('GET',req_adr)
redrurl = resp.geturl()
But I don't know how to pass it into the input prompt from the util.prompt_for_user_token response
Any suggestions would be very welcome.
So it turns out there is a workaround. You can run it one time on a local machine and that generates a a file called .cache-USERNAME. If you include that file in your deployment package you don't have to copy/paste the URL and it is able to be automated with a lambda function in AWS.
this is a two-part question: I have seen individual pieces discussed, but can't seem to get the recommended suggestions to work together. I want to create a web service to store images and their metadata passed from a caller and run a test call from Postman to make sure it is working. So to pass an image (Drew16.jpg) to the web service via Postman, it appears I need something like this:
For the web service, I have some python/flask code to read the request (one of many variations I have tried):
from flask import Flask, jsonify, request, render_template
from flask_restful import Resource, Api, reqparse
...
def post(self, name):
request_data = request.get_json()
userId = request_data['UserId']
type = request_data['ImageType']
image = request.files['Image']
Had no problem with the data portion and straight JSON but adding the image has been a bugger. Where am I going wrong on my Postman config? What is the actual set of Python commands for reading the metadata and the file from the post? TIA
Pardon the almost blog post. I am posting this because while you can find partial answers in various places, I haven't run across a complete post anywhere, which would have saved me a ton of time. The problem is you need both sides to the story in order to verify either.
So I want to send a request using Postman to a Python/Flask web service. It has to have an image along with some metadata.
Here are the settings for Postman (URL, Headers):
And Body:
Now on to the web service. Here is a bare bones service which will take the request, print the metadata and save the file:
from flask import Flask, request
app = Flask(__name__)
# POST - just get the image and metadata
#app.route('/RequestImageWithMetadata', methods=['POST'])
def post():
request_data = request.form['some_text']
print(request_data)
imagefile = request.files.get('imagefile', '')
imagefile.save('D:/temp/test_image.jpg')
return "OK", 200
app.run(port=5000)
Enjoy!
Make sure `request.files['Image'] contains the image you are sending and follow http://flask.pocoo.org/docs/1.0/patterns/fileuploads/ to save the file to your file system. Something like
file = request.files['Image']
file.save('./test_image.jpg')
might do what you want, while you will have to work out the details of how the file should be named and where it should be placed.
I'm new to Python, using Python 3.5.1 (windows)
I'm trying to write a code to automate a login module for an API server
The first inputs are an API key and secret which I provide and then I get a URL from the API server and using that we open up the browser pointing to that page which requires us to manually enter the username and password to get authenticated. == so far so good.
Once authenticated the server opens up a webpage and displays a token ref in the url parameters, something like
http://www.xxxx.xx.x.../xxxxxxtoken_ref=a1234xyx
I need to extract the token_ref value and use that for further authentication.
I am trying to use a local server at the redirect IP so I can capture the token_ref using Bottle but I think I am doing something wrong.
My code is below:
from kiteconnect import KiteConnect
from kiteconnect import websocket
from bottle import run , request , route
import webbrowser
login to kite
myapi_key="abcdxyz1234"
myapi_secret = "xxxxyyyyzzzz1234"
kite = KiteConnect(api_key=myapi_key)
url = kite.login_url()
webbrowser.open(url)
get token_request (this is where I'm stuck)
#route('/')
def root():
tk_request = request.query.token_request
return 'the token request is' + tk_request
run()
get access token
user = kite.request_access_token(request_token=tk_request,secret=api_secret)
I was trying to export a Google Spreadsheet in csv format using the Google client library for Python:
# OAuth and setups...
req = g['service'].files().export_media(fileId=fileid, mimeType=MIMEType)
fh = io.BytesIO()
downloader = http.MediaIoBaseDownload(fh, req)
# Other file IO handling...
This works for MIMEType: application/pdf, MS Excel, etc.
According to Google's documentation, text/csv is supported. But when I try to make a request, the server gives a 500 Internal Error.
Even using google's Drive API playground, it gives the same error.
Tried:
Like in v2, I added a field:
gid = 0
to the request to specify the worksheet, but then it's a bad request.
This is a known bug in Google's code. https://code.google.com/a/google.com/p/apps-api-issues/issues/detail?id=4289
However, if you manually build your own request, you can download the whole file in bytes (the media management stuff won't work).
With file as the file ID, http as the http object that you've authorized against you can download a file with:
from apiclient.http import HttpRequest
def postproc(*args):
return args[1]
data = HttpRequest(http=http,
postproc=postproc,
uri='https://docs.google.com/feeds/download/spreadsheets/Export?key=%s&exportFormat=csv' % file,
headers={ }).execute()
data here is a bytes object that contains your CSV. You can open it something like:
import io
lines = io.TextIOWrapper(io.BytesIO(data), encoding='utf-8', errors='replace')
for line in lines:
#Do whatever
You just need to implement an Exponential Backoff.
Looking at this documentation of ExponentialBackOffPolicy.
The idea is that the servers are only temporarily unavailable, and they should not be overwhelmed when they are trying to get back up.
The default implementation requires back off for 500 and 503 status codes. Subclasses may override if different status codes are required.
Here is an snippet of an implementation of Exponential Backoff from the first link:
ExponentialBackOff backoff = ExponentialBackOff.builder()
.setInitialIntervalMillis(500)
.setMaxElapsedTimeMillis(900000)
.setMaxIntervalMillis(6000)
.setMultiplier(1.5)
.setRandomizationFactor(0.5)
.build();
request.setUnsuccessfulResponseHandler(new HttpBackOffUnsuccessfulResponseHandler(backoff));
You may want to look at this documentation for the summary of the ExponentialBackoff implementation.
I've a Python/Flask app that is working ok locally. I have deployed it to the cloud (pythonanywhere) and it is all working on there as well except for a file that is being downloaded to the user that is coming as html, so the empty lines of the file are being excluded. That file is txt. When the user click on that, it opens on notepad. If opening that file in notepad++ the empty lines are there in the way it should be.
Following the Flask code to send that file:
response = make_response(result)
response.headers["Content-Disposition"] = "attachment; filename=file_to_user.txt"
If I use "inline instead of attachment", the empty lines are showed OK directly on the browser.
I've tried to add "Content type text/plain" before "Content-Disposition", but I believe that it is the default, so, no effect.
Anyone knows how could the user see that as txt file, instead of html when opening directly using notepad for example?
If you're just trying to send an existing file on the server, use send_from_directory.
If you're trying to make a response (for example, if you're generating data in memory, make_response defaults to text/html (it's just a shortcut which isn't applicable in your case). Create a response even more directly in order to override that using app.response_class.
This is a small example demonstrating both techniques.
from flask import Flask, send_from_directory
app = Flask(__name__)
#app.route('/file')
def download_file():
# change app.root_path to whatever the directory actually is
# this just serves this python file (named example.py) as plain text
return send_from_directory(
app.root_path, 'example.py',
as_attachment=True, mimetype='text/plain'
)
#app.route('/mem')
def download_mem():
# instantiate the response class directly
# pass the mimetype
r = app.response_class('test data\n\ntest data', mimetype='text/plain')
# add the attachment header
r.headers.set('Content-Disposition', 'attachment', filename='test_data.txt')
return r
app.run('localhost', debug=True)