I'm trying to create a simple flask server that redirects any http requests to https. I've created a certificate and key file and registered a before_request hook to see if the request is secure and redirect appropriately, following advise this SO answer.
The flask server responds to https requests as expected. However, when I send an http request, the before_request hook never gets called and ther server hangs forever. If I send the http request from the browser, I see an "ERR_EMPTY_RESPONSE". The server doesn't even respond to https requests afterwards. No logs are printed either.
Running the app with gunicorn didn't help either. The only difference was that gunicorn is able to detect that the worker is frozen and eventually kills and replaces it. I've also tried using flask-talisman, with the same results.
Below is the code I'm running
### server.py
from flask import Flask, request, redirect
def verify_https():
if not request.is_secure:
url = request.url.replace("http://", "https://", 1)
return redirect(url, 301)
def create_flask_app():
app = Flask(__name__)
app.before_request(verify_https)
app.add_url_rule('/', 'root', lambda: "Hello World")
return app
if __name__ == '__main__':
app = create_flask_app()
app.run(
host="0.0.0.0",
port=5000,
ssl_context=('server.crt', 'server.key')
)
Running it with either python3.8 server.py or gunicorn --keyfile 'server.key' --certfile 'server.crt' --bind '0.0.0.0:5000' 'server:create_flask_app()' and opening a browser window to localhost:5000 causes the server to hang.
Talking about freezes, its not. Flask and gunicorn can serve only one variant of connection. So it's not freezing because your browser canceled the request and is idling.
I think it is better to use a faster web server, for example, Nginx, if you want to change HTTP to HTTPS. I would recommend it to you.
But it's possible to trigger your verify_https function if you run multiple instances of gunicorn at the same time.
I took your example, generated a certificate, and then run this script in my console (it contains a background job and can be runned in twoo separate ter)
gunicorn --bind '0.0.0.0:80' 'server:create_flask_app()' & gunicorn --certfile server.crt --keyfile server.key --bind '0.0.0.0:443' 'server:create_flask_app()'
now chrome goes to the secure page as expected.
Typically servers don't listen for both http and https on the same port. I have a similar requirement for my personal portfolio, but I use nginx to forward http requests (port 80) to https (port 443) and then the https server passes it off to my uwsgi backend, which listens on port 3031. That's probably more complex than you need, but a possible solution. If you go that route I would recommend letsencrypt for your certificate needs. It will set up the certificates AND the nginx.conf for you.
If you don't want to go the full nginx/apache route I think your easiest solution is the one suggested here on that same thread that you linked.
Related
I have a web process and an api process (a "worker" process similar to what is described in the docs and these other docs). However, I don't understand how the networking between different processes works.
Both processes are listening on 0.0.0.0, and the worker process is bound to port 5000. However, when I try to make a request from the web process, I get a ConnectionRefusedError:
ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=5000): Max retries exceeded with url: /tokenize?sentence=hello (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd307019dc0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Am I supposed to figure out the IP of the other process? Am I doing something wrong here?
Procfile:
api: python app.py
web: voila UI.ipynb --port $PORT --Voila.ip=0.0.0.0
app.py:
from flask import Flask, request
from ie_utils import tokenize
app = Flask(__name__)
#app.route("/")
def home():
return {
"message": "Hello world!",
"version": "0.1",
}
if __name__ == "__main__":
import os
port = 5000
app.run(host="0.0.0.0", port=port)
Relevant code in UI.ipynb:
import requests
requests.get("http://0.0.0.0:5000/")
Full source code: https://github.com/astrojuanlu/ie-titanic-utils-a/tree/test-procfile
I don't understand how the networking between different processes works
Unless you are using Private Spaces (part of Heroku's enterprise offering), it doesn't:
The Common Runtime provides strong isolation by firewalling all dynos off from one another. The only traffic that can reach a dyno is web requests forwarded from the router to web processes listening on the port number specified in the $PORT environment variable. Worker and one-off dynos cannot receive inbound requests.
I'm not totally clear what you're trying to do here, but you won't be able to do it by making an HTTP request to localhost. You might have better luck running this as two separate apps—in that case your API could be its own web process and your Voilà app could make requests to the API app's hostname.
Side note: even on a system where this is permitted your request should not go to 0.0.0.0. That isn't a real IP address.
Telling Flask to bind to 0.0.0.0 really means that it should listen on all available IP addresses. When you want to make a request you should use one of the real hostnames or IP addresses that your system is using, e.g. localhost or 127.0.0.1 or one of its public IP addresses or hostnames.
I'm trying to make JavaScript client to a Python websocket server through an Apache2 proxy.
The client is dead simple:
const socket = io({
transports: ['websocket']
});
I have a NodeJS websocket server and a working Apache2 reverse proxy setup.
Now I want to replace the NodeJS server with a Python server - but none of the example implementations from socket.io works. With each of the my client reports an "error 400" when setting up the websocket connection.
The Python server examples come from here:
https://github.com/miguelgrinberg/python-socketio/tree/master/examples/server
Error 400 stands for "Bad Request" - but I know that my requests are fine because my NodeJS server understands them.
When not running behind a proxy then all Python examples work fine.
What could be the problem?
I found the solution - all the Python socket.io server examples that I refered to are not configured to run behind a reverse proxy. The reason is, that the socket.io server is managing a list of allowed request origins and the automatic list creation is failing in the reverse proxy situation.
This function creates the automatic list of allowed origins (engineio/asyncio_server.py):
def _cors_allowed_origins(self, environ):
default_origins = []
if 'wsgi.url_scheme' in environ and 'HTTP_HOST' in environ:
default_origins.append('{scheme}://{host}'.format(
scheme=environ['wsgi.url_scheme'], host=environ['HTTP_HOST']))
if 'HTTP_X_FORWARDED_HOST' in environ:
scheme = environ.get(
'HTTP_X_FORWARDED_PROTO',
environ['wsgi.url_scheme']).split(',')[0].strip()
default_origins.append('{scheme}://{host}'.format(
scheme=scheme, host=environ['HTTP_X_FORWARDED_HOST'].split(
',')[0].strip()))
As you can see, it only adds URLs with {scheme} as a protocol. When behind a reverse proxy, {scheme} will always be "http". So if the initial request was HTTPS based, it will not be in the list of allowed origins.
The solution to this problem is very simple: when creating the socket.io server, you have to either tell him to allow all origins or specify your origin:
import socketio
sio = socketio.AsyncServer(cors_allowed_origins="*") # allow all
# or
sio = socketio.AsyncServer(cors_allowed_origins="https://example.com") # allow specific
The API for Python's wsgiref module precludes hop-by-hop headers (as defined in RFC 2616).
I'm unclear on how to get the server to terminate a connection after a response (since there doesn't seem to be a way to add Connection: close).
This problem comes up in testing small WSGI apps and Bottle micro-services. Calls from curl get blocked by open connections from a browser. I have to click a browser refresh to terminate the connection so that the pending curl request can be answers.
Obviously, this should be a server side decision (terminate connection after a response) rather than client-side. I'm unclear how to implement this.
This is really predicated on your WSGI server you are hosting your framework via. The best solution with bottle is to run it through gevent.
botapp = bottle.app()
for Route in (mainappRoute,): #handle multiple files containing routes
botapp.merge(Route)
botapp = SessionMiddleware(botapp, beakerconfig) #in case you are using beaker sessions
botapp = WhiteNoise(botapp) #in case you want whitenoise to handle static files
botapp.add_files(staticfolder, prefix='static/') #add static route to whitenoise
server = WSGIServer(("0.0.0.0", int(80)), botapp) #gevent async web server
def shutdown():
print('Shutting down ...')
server.stop(timeout=60)
exit(signal.SIGTERM)
gevent.signal(signal.SIGTERM, shutdown)
gevent.signal(signal.SIGINT, shutdown) #CTRL C
server.serve_forever() #spawn the server
You can purge the whitenoise and bottle configs if they aren't necessary, I kept them there as an example, and a suggestion that you use them if this is outward facing.
This is purely asynchronous on every connection.
I have managed to to install flask and run the hello-world script:
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
I was impressed how easy it was. Then I wanted to make my web-server visible externally. As recommended, I put host='0.0.0.0' as the argument of run function. Then I found my IP address (in Google) and put it in the address line of my web browser (while the hello-world-script was running).
As a result I got: "A username and password are being requested" and a dialogue box where I need to put a user name and password. I am not sure but I think it comes from my wireless network. Is there a way to change this behaviour?
How are you trying to run your application? If you run flask as app.run() - flask creates its own WSGI server on your host (by default 127.0.0.1) and port (by default 5000) (need permissions if port < 1000). If you run flask using nginx + WSGI or etc. your server resolves host and port.
Now it looks like you want get application by port which resolved your server like nginx or Apache. Try to get flask application by http://your-server-host-or-ip:5000 with the default port or try to change the port (set explicit) like app.run('0.0.0.0', 8080) and get it by http://your-server-host-or-ip:8080.
By the way, you can always get IP address using command-line tools e.g. ifconfig for Unix-like systems, or ipconfig /all for Windows.
To elaborate a little bit onto what #tbicr said, that password prompt indicates that you're trying to connect to your IP on port 80, which is most likely hosting an administration page for your router/modem. You want to connect to your IP on port 5000, the default port for Flask apps run with app.run().
I want a simple python web server for the following use case:
I want to write a simple server that will accept HTTP requests from my application running on Google App Engine.
The server will accept HTTP requests, and then send iphone notifications. (Basically, I need this extra server to account for the lack of socket support in google app engine).
I guess I need the server to be able to maintain this persistent connection with Apple's Push Notification Service. So I'll need to have some sort of thread always open for this. So I need some sort of web server that can accept the request pass it off to the other thread with the persistent connection to APNS.
Maybe multiple processes and one of pythons queuing tools to communicate between them? Accept the HTTP request, then enqueue a message to the other process?
I was wondering what someone with a bit of experience would suggest. I'm starting to think that maybe even writing my own simple server is a good option (http://fragments.turtlemeat.com/pythonwebserver.php).
One option would be the (appropriately named) SimpleHTTPServer, which is part of the Python standard library. Another, more flexible but more complicated option would be to write your server in Twisted.
I've been writing simple http servers using gevent and bottle -- an example:
#!/usr/bin/env python
import gevent.monkey
gevent.monkey.patch_all()
import bottle
bottle.debug(True)
import gevent.wsgi
from bottle import route, run, request, response, static_file, abort
#route('/echo')
def echo():
s = request.GET.get('s', 'o hai')
return '<html><head><title>echo server</title></head><body>%s</body></html>\r\n' % (s)
#route('/static/:filename')
def send_static(filename):
root = os.getcwd() + '/static'
return static_file(filename, root=root)
if __name__ == '__main__':
app = bottle.app()
wsgi_server = gevent.wsgi.WSGIServer(('0.0.0.0', 8000), app)
print 'Starting wsgi search on port 8000'
wsgi_server.serve_forever()
So you could write a simple server that sticks a job into a Queue (see gevent.queue) and have another worker greenlet that handles reading requests from the queue and processing them...