Develop Reference

(Python) Add certificate to Bottle server

I am stuck with a problem for some time and can't find a right solution for it.
I have a python server based on Bottle (Python 3) written with PyCharm. I'm converting my files with "pyinstaller" to one "exe" to start the server on a fixed PC (win7). The server works fine for the things it is needed for but now I want to add more secuity to it.
I have a signed certificate (not self-signed) and a key, which I want to add. I tried to start the server with them but I'm not sure, if I have to do something else with them, because the certificate is not shown on the homepage in the information and the website is still shown as not save.
My normal server is running with:
from bottle import run, ...
...
if __name__ == "__main__":
...
run(host=IP, port=PORT)
I have tried some frameworks for bottle and I end up with cherrypy as the one, that starts my server in a proper way.
The server is running with:
run(host=IP, port=PORT, server='cherrypy', certfile='./static/MyCert.pem', keyfile='./static/key.pem')
It is not working with the current version of cherrypy, so I downgraded it (after some search) to ">=3.0.8, <9.0.0".
The server is running, but the website is still not save. And I don't know if it just does not load the certificate or I miss something else. I tried things like leaving the "keyfile" in the code or adding the key to my certificate, but it does not change anything.
Another framework I tried was gevent:
from gevent import monkey; monkey.patch_all()
...
if __name__ == "__main__":
run(host=IP, port=PORT, reloader=False, server='gevent', certfile='./static/MyCert.pem', keyfile='./static/key.pem')
But here I can't get to the website. When I try, the terminal asks me for the PEM phrase, but I can't add it (or just don't know how) and getting an error I have never seen before:
terminal_error
Like in my cherrypy-example I tried to use some combinations of leaving parts of the code away or changing the certificate but it always ends up here.
It would be nice if someone has a solution for my problem or can give me a hint of what I'm missing or just have not thought of yet. I would like to stay with cherrypy or another framework for Bottle, so I don't have to change much of my current code.
Thanks
P.

It sounds to me like you added a passphrase to your certificate. Regenerate your cert without a passphrase and try again.
Additionally, a word of advice. I highly recommend running your bottle/cherrypy server behind nginx in reverse proxy mode. This simplifies your configuration by letting nginx handle the termination of your SSL session, and then your python web server never needs to know anything about a certificate.
Here's a redacted copy of the nginx config we're using to terminate our (self-signed) SSL cert and reverse proxy our cherrypy site running on localhost on port 9000:
server {
listen example.com:80;
server_name test.example.com;
access_log /var/log/nginx/test.example.com.access.log main;
return 301 https://test.example.com$request_uri;
}
server {
listen example.com:443;
access_log /var/log/nginx/test.example.com.access.log main;
server_name test.example.com;
root /usr/local/www/test.example.com/html;
ssl on;
ssl_certificate /etc/ssl/test.example.com.crt;
ssl_certificate_key /etc/ssl/test.example.com.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # don't use SSLv3 ref: POODLE
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
client_max_body_size 16M;
# Block access to "hidden" files and directories whose names begin with a
# period. This includes directories used by version control systems such
# as Subversion or Git to store control files.
location ~ (^|/)\. {
return 403;
}
location / {
proxy_pass http://127.0.0.1:9000;
proxy_set_header X-REAL-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}

Authentication on NGINX aganist Tornado

well, I just setup NGINX and now its working.
As my BackEnd WebServer under NGINX I have Python Tornado running:
I only use NGINX for allow big uploads (large-sized), so one of my URL (for upload) is served on NGINX and the rest of URLs are served by Tornado.
I use Sessions provided by Tornado (running at http://localhost:8080/), and NGINX is running at http://localhost:8888/;
Well this is my nginx config file:
location /images/upload {
upload_pass /after_upload;
.....
.....
.....
}
location /after_upload {
proxy_pass http://localhost:8080/v1/upload/;
}
As you see, there aren't anything about authentication on NGINX.
URL for proxy_pass requiere a session valid (provided by Tornado)
This is scheme of the system is the following:
When users log in the system, system create a Tornado (tornado sessions) session in server and in user's Browser, so I need pass authentication to NGINX and continue this authentication process again in Tornado Service.
How I change NginX for authenticate against Tornado?
Thanks in advance

Well, Nginx works as a Proxy, therefore is not necessary make changes in Tornado or in your application. For my application I just add rewrites from NGINX urls towards Tornado urls. So this includes all traffic (auth, etc.) and all HTTP structures like if you were working in Tornado.
server {
listen 8888; ## listen for ipv4
server_name localhost;
access_log /var/log/nginx/localhost.access.log;
client_max_body_size 100000M;
location / {
#Real Location URL for Tornado.
proxy_pass http://localhost:8080/;
}
}
Key is proxy_pass , where every requests for 8888 port are passed to 8080 port in localhost.
Everything is passed to Tornado BackEnd from Nginx.

Flask app gives ubiquitous 404 when proxied through nginx

I've got a flask app daemonized via supervisor. I want to proxy_pass a subfolder on the localhost to the flask app. The flask app runs correctly when run directly, however it gives 404 errors when called through the proxy. Here is the config file for nginx:
upstream apiserver {
server 127.0.0.1:5000;
}
location /api {
rewrite /api/(.*) /$1 break;
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass http://apiserver;
proxy_next_upstream error timeout http_502;
proxy_buffering off;
}
For instance, when I go to http://127.0.0.1:5000/me, I get a valid response from the app. However when I go to http://127.0.0.1/api/me I get a 404 from the flask app (not nginx). Also, the flask SERVER_NAME variable is set to 127.0.0.1:5000, if that's important.
I'd really appreciate any suggestions; I'm pretty stumped! If there's anything else I need to add, let me know!

I suggest not setting SERVER_NAME.
If SERVER_NAME is set, it will 404 any requests that don't match the value.

Since Flask is handling the request, you could just add a little bit of information to the 404 error to help you understand what's passing through to the application and give you some real feedback about what effect your nginx configuration changes cause.
from flask import request
#app.errorhandler(404)
def page_not_found(error):
return 'This route does not exist {}'.format(request.url), 404
So when you get a 404 page, it will helpfully tell you exactly what Flask was handling, which should help you to very quickly narrow down your problem.

I ran into the same issue. Flask should really provide more verbose errors here since the naked 404 isn't very helpful.
In my case, SERVER_NAME was set to my domain name (e.g. example.com).
nginx was forwarding requests without the server name, and as #Zoidberg notes, this caused Flask to trigger a 404.
The solution was to configure nginx to use the same server name as Flask.
In your nginx configuration file (e.g. sites_available or nginx.conf, depending on where you're defining your server):
server {
listen 80;
server_name example.com; # this should match Flask SERVER_NAME
...etc...

Django+Nginx+uWSGI = 504 Gateway Time-out

I am running Ubuntu 10.04, Django 1.3, Nginx 0.8.54 and uWSGI 0.9.7.
Both Nginx and uWSGI load without error. However, when you access my site, it sits for a LONG time and then eventually loads a "504 Gateway Time-out" error.
Here is my Nginx Virtual Host conf file:
server {
listen 80;
server_name www.mysite.com mysite.com;
error_log /home/mysite/log/error.log;
access_log /home/mysite/log/access.log;
location / {
auth_basic "Restricted";
auth_basic_user_file /home/mysite/public/passwd;
include uwsgi_params;
uwsgi_pass unix:///home/mysite/public/myapp.sock;
}
location /media {
alias /home/mysite/public/myapp/media;
}
error_page 401 /coming_soon.html;
location /coming_soon.html {
root /home/mysite/public/error_pages/401;
}
location /401/images {
alias /home/mysite/public/error_pages/401/images;
}
location /401/style {
alias /home/mysite/public/error_pages/401/style;
}
}
My site log shows this:
SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request / !!!
My error log show this:
upstream timed out (110: Connection timed out) while reading response header from upstream
I have two other sites on this server with the same configuration and they load PERFECTLY.
Has anyone else encountered this problem? There are several threads on here that are similar to my issue and I've tried several of those solutions but nothing seems to work.
Thank you in advance for your help!

That error is produced when requests exceed the NGINX uwsgi_read_timeout setting. After NGINX exceeds this limit it closes the socket and then uWSGI tries to write to the closed socket, producing the error that you see from uWSIG.
Make sure your NGINX timeouts are at least as high as uWSGI timeouts (HARAKIRI_TIMEOUT).

unix:///home/mysite/public/myapp.sock;
syntax not correct, use like this:
unix:/home/mysite/public/myapp.sock;

AWS - Running webservice - Cherrypy + Python

I have a linux box (Ubuntu 10.10 server edition) in ec2. I have written a web service using cherrypy framework. Let's say this is the code that I have written.
import sys
sys.path.insert(0,'cherrypy.zip')
import cherrypy
from cherrypy import expose
class Service:
#expose
def index(self):
return 'Hello World'
cherrypy.quickstart(Service())
I have copied this file, the cherrypy.zip file to /var/www in my ec2 instance. [I should inform that I created the www directory manually, as it wasn't there]. Then I ran
python webservice.py
and got the message
[01/Apr/2011:13:50:04] ENGINE Bus STARTED
However, when I try to run
(I have masked my public ip)
ec2-1**-2**-1**-**.ap-southeast-1.compute.amazonaws.com/
in my browser, I get connection failed. Can anyone tell me where I have gone wrong? or what I should do?
EDIT:
Okay, here is something interesting that I found. When I do
python webservice.py
I see
ENGINE Serving on 127.0.0.1:8080
Which means, the webservice will run only for the local machine. How do I make set the service 0.0.0.0 (that is, to serve any IP address?)
Hope this detail is sufficient for understanding the problem I'm facing. Help, please :)
EDIT 2:
Oh well, found the solution :-) Have to add this before cherrypy.quickstart() call
cherrypy.config.update({'server.socket_host': '0.0.0.0',
'server.socket_port': 80,
})

The cherrypy.quickstart function takes a config argument, which can be a dict, an open configuration file, or a path to a configuration file. I favor using a path to a configuration file because that minimizes the hardcoding of settings that you might prefer to control from a startup script.
In addition, since you control the server, you could configure a reverse proxy to route requests to the CherryPy application. This gives you quite a bit of flexibility. For example, if you wanted to, you could run multiple instances of the CherryPy application in parallel, each configured to listen on a different port.
Here's a sample configuration file for nginx, instructing it to forward requests to a single instance of your CherryPy application:
server
{
server_name your.hostname.com;
  location / {
    proxy_pass http://127.0.0.1:8080/;
  }
}
And here's a sample configuration file instructing nginx to load-balance across two instances of your application, which are listening on the loopback address at ports 33334 and 33335:
upstream myapps {
server 127.0.0.1:33334;
server 127.0.0.1:33335;
}
server {
server_name your.hostname.com;
location / {
proxy_pass http://myapps;
}
}