I generate Image with Celery task in this code. this is the part of an ajax view function.
task_result = generate_result_image.apply((answer, combi.id, lang))
if task_result.state == "SUCCESS":
response['ok'] = 'yes'
return HttpResponse(json.dumps(response), content_type='application/json')
this is working locally perfectly, but in prod server, i am getting 502 bad gateway. this is the browser console msg:
what am I doing wrong? this is really mysterious..
this is the uwsgi log, the line where the cursor stands is the call I am doing. I seems, the worker is dying right after my call... no idea why..
problem solved - the bit was not my code, nor my settings but the remote server I was sending files to, to generate image. Many thanks to Alex for showing the right direction..
Related
I am using Flask and blueprints on IIS. My issue is that when the 500 error is triggered it loads the default IIS 500 template and not what I want.
I've tried several things and this is where I am. I have the following code
from flask import render_template,Blueprint
errorbp = Blueprint("errorbp",__name__)
#errorbp.app_errorhandler(404)
def page_not_found(e):
return "404", 404
#errorbp.app_errorhandler(500)
def internal_server_error(e):
return "500", 500
If I visit a page that does not exist, I get "404" back as intended. If I create an error on purpose, I get the following
Any suggestions as to what to do? I presume I may need to do something with IIS at this point but what? I've edited/remove the 5xx status codes/page and still nothing
What you need to do is, open the error pages module, double-click the 500 status code, set the path of your template in the file path, and IIS will send the content of the file as a custom error response.
In addition, IIS has two other ways to respond to an error: by executing an URL or by redirecting the request.
I am programming a home Web server for home automation. I've seen several times 'bots' scanning the ports of my server. To avoid give any kind of activity signs to undesired scans, I'm trying to avoid generate any kind of answer for specific URLs, like '/', ie. configure a silent mode for the typical scanned URL's.
I've tried with void .route decorators, error addressing and void pages, but all of them generated some kind of response.
It's that possible in Flask with Python?
Any workaround?
Thanks
What I would suggest is to return a custom error code for urls you are getting scanned, like HTTP_410_GONE.
From: http://www.flaskapi.org/api-guide/status-codes/
#app.route('/')
def empty_view(self):
content = {'please move along': 'nothing to see here'}
return content, status.HTTP_410_GONE
Put nginx in front of your flask app and use a fail2ban config to watch for this error code and start banning ips that are constantly hitting these urls.
From: https://github.com/mikechau/fail2ban-configs/blob/master/filter.d/nginx-404.conf
# Fail2Ban configuration file
[Definition]
failregex = <HOST> - - \[.*\] "(GET|POST).*HTTP.* 410
ignoreregex =
I had exactly the same need as you and decided to just return from 404 and 500 handlers:
#staticmethod
#bottle.error(404)
def error404(error):
log.warning("404 Not Found from {0} # {1}".format(bottle.request.remote_addr, bottle.request.url))
#staticmethod
#bottle.error(500)
def error500(error):
log.warning("500 Internal Error from {0} # {1}".format(bottle.request.remote_addr, bottle.request.url))
The example is for bottle but you can adapt it with flask.
Beside catching the obvious 404 I decided to do the same with 500 in case an unexpected call from the scanners would crash my script so that no traceback information is provided.
I am writing a small websites using Flask, the database is mysql and using flask-sqlalchemy to handle it. but I am stuck by a strange 502 error, I do have search a lot via Google and debug for quite a long time but still fail. Wish you guys can help.
In Debug mode, everything works fine, but when setting app.debug = False, it will return 502 error.
I have debug this for quite a long time and one of the useful finding is that the 502 error occurs when running the db.session.commit().
I am not sure whether this problems is related to the jsonify.
Part of the code:
#app.route('/sell/update', methods=('GET', 'POST'))
#login_required
def sell_update():
"""docstring for sell_update"""
res = {}
# It should be call with post method
id = int(request.form.get('id', 0))
status = int(request.form.get('status', 0))
sell = lib.get_sell_by_id(id)
if sell.user.id != g.user.id:
res['error'] = MSG_SELL_PERMISSION_INVALID
if not id or not sell:
res['error'] = MSG_SELL_ID_INVALID
res['status'] = res.get('error') and 'ERROR' or 'OK'
sell.status = status
# return 'TEST' # this line works fine in both debug and production
db.session.commit() # Error occurs right here
return jsonify(**res)
Any ideas and suggestions are welcome. Thanks in advance.
Finally, I solved the problem myself by coincidence.
The problem is that I also use flask-whooshalchemy for search feature and the whoosh need to write on the index file.
When running debug mode, the index file is created with ownership of current user, but when deploy the production mode, the user is www-data (with nginx in Ubuntu) so it have no permission to write.
The key point of finding this problem is to use try except and return the message of the exception as the return of views. Without this, we can not see any trackback or something like that in production deployment.
I have a file upload handler for multiple file uploads, and have set the MAX_CONTENT_SIZE. The docs mention that Flask throws a 413 exception when the total file size exceeds the limit, so I've also written a 413 error handler with a custom 413 page. However, when testing the file upload, I can see that the 413 error is definitely thrown, but the connection seems to break everytime instead of rendering my error page. FYI, I'm using the Flask dev server currently.
Code:
app.config['MAX_CONTENT_LENGTH'] = 50 * 1024 * 1024 # 50 Mb limit
#app.route('/upload', methods=['POST'])
def upload_files():
if request.method == 'POST':
uploaded_files = request.files.getlist('uploaded_files[]')
# do some stuff with these files
#app.errorhandler(413)
def error413(e):
return render_template('413.html'), 413
UPDATE:
Ok strange, this problem seems to only occur when using the Flask dev server. I'm testing it on Apache, and my 413 error page renders fine.
Use a production WSGI server will solve this problem (e.g. Gunicorn, Waitress). Below is a simple timeline of this issue.
2015
In this snippet (gone) that posted by Armin Ronacher, he said:
You might notice that if you start not accessing .form or .files on incoming POST requests, some browsers will honor this with a connection reset message. This can happen if you start rejecting uploads that are larger than a given size.
Some WSGI servers solve that problem for you, others do not. For instance the builtin Flask webserver is pretty dumb and will not attempt to fix this problem.
2018
I added a tip in the Flask's file uploading docs (flask #2662):
Connection Reset Issue
When using the local development server, you may get a connection reset error instead of a 413 response. You will get the correct status response when running the app with a production WSGI server.
2021
I think/hope it will be fixed in Werkzeug in the near future (werkzeug #1513).
Github offers to send Post-receive hooks to an URL of your choice when there's activity on your repo.
I want to write a small Python command-line/background (i.e. no GUI or webapp) application running on my computer (later on a NAS), which continually listens for those incoming POST requests, and once a POST is received from Github, it processes the JSON information contained within. Processing the json as soon as I have it is no problem.
The POST can come from a small number of IPs given by github; I plan/hope to specify a port on my computer where it should get sent.
The problem is, I don't know enough about web technologies to deal with the vast number of options you find when searching.. do I use Django, Requests, sockets,Flask, microframeworks...? I don't know what most of the terms involved mean, and most sound like they offer too much/are too big to solve my problem - I'm simply overwhelmed and don't know where to start.
Most tutorials about POST/GET I could find seem to be concerned with either sending or directly requesting data from a website, but not with continually listening for it.
I feel the problem is not really a difficult one, and will boil down to a couple of lines, once I know where to go/how to do it. Can anybody offer pointers/tutorials/examples/sample code?
First thing is, web is request-response based. So something will request your link, and you will respond accordingly. Your server application will be continuously listening on a port; that you don't have to worry about.
Here is the similar version in Flask (my micro framework of choice):
from flask import Flask, request
import json
app = Flask(__name__)
#app.route('/',methods=['POST'])
def foo():
data = json.loads(request.data)
print "New commit by: {}".format(data['commits'][0]['author']['name'])
return "OK"
if __name__ == '__main__':
app.run()
Here is a sample run, using the example from github:
Running the server (the above code is saved in sample.py):
burhan#lenux:~$ python sample.py
* Running on http://127.0.0.1:5000/
Here is a request to the server, basically what github will do:
burhan#lenux:~$ http POST http://127.0.0.1:5000 < sample.json
HTTP/1.0 200 OK
Content-Length: 2
Content-Type: text/html; charset=utf-8
Date: Sun, 27 Jan 2013 19:07:56 GMT
Server: Werkzeug/0.8.3 Python/2.7.3
OK # <-- this is the response the client gets
Here is the output at the server:
New commit by: Chris Wanstrath
127.0.0.1 - - [27/Jan/2013 22:07:56] "POST / HTTP/1.1" 200 -
Here's a basic web.py example for receiving data via POST and doing something with it (in this case, just printing it to stdout):
import web
urls = ('/.*', 'hooks')
app = web.application(urls, globals())
class hooks:
def POST(self):
data = web.data()
print
print 'DATA RECEIVED:'
print data
print
return 'OK'
if __name__ == '__main__':
app.run()
I POSTed some data to it using hurl.it (after forwarding 8080 on my router), and saw the following output:
$ python hooks.py
http://0.0.0.0:8080/
DATA RECEIVED:
test=thisisatest&test2=25
50.19.170.198:33407 - - [27/Jan/2013 10:18:37] "HTTP/1.1 POST /hooks" - 200 OK
You should be able to swap out the print statements for your JSON processing.
To specify the port number, call the script with an extra argument:
$ python hooks.py 1234
I would use:
https://github.com/carlos-jenkins/python-github-webhooks
You can configure a web server to use it, or if you just need a process running there without a web server you can launch the integrated server:
python webhooks.py
This will allow you to do everything you said you need. It, nevertheless, requires a bit of setup in your repository and in your hooks.
Late to the party and shameless autopromotion, sorry.
If you are using Flask, here's a very minimal code to listen for webhooks:
from flask import Flask, request, Response
app = Flask(__name__)
#app.route('/webhook', methods=['POST'])
def respond():
print(request.json) # Handle webhook request here
return Response(status=200)
And the same example using Django:
from django.http import HttpResponse
from django.views.decorators.http import require_POST
#require_POST
def example(request):
print(request.json) # Handle webhook request here
return HttpResponse('Hello, world. This is the webhook response.')
If you need more information, here's a great tutorial on how to listen for webhooks with Python.
If you're looking to watch for changes in any repo...
1. If you own the repo that you want to watch
In your repo page, Go to settings
click webhooks, new webhook (top right)
give it your ip/endpoint and setup everything to your liking
use any server to get notified
2. Not your Repo
take the url you want i.e https://github.com/fire17/gd-xo/
add /commits/master.atom to the end such as:
https://github.com/fire17/gd-xo/commits/master.atom
Use any library you want to get that page's content, like:
filter out the keys you want, for example the element
response = requests.get("https://github.com/fire17/gd-xo/commits/master.atom").text
response.split("<updated>")[1].split("</updated>")[0]
'2021-08-06T19:01:53Z'
make a loop that checks this every so often and if this string has changed, then you can initiate a clone/pull request or do whatever you like