So I have been working on a Badgr server for another department. I have built it using Python 2.7 and django. From what I have heard Django is only used for dev websites.
I want to take this project and convert it to run on something meant for a production environment. But I really have no idea how to proceed. Sorry if this is a really noob question, I am a system administrator not a dev.
(env)[root#badgr code]# ./manage.py runserver &
Performing system checks...
System check identified no issues (0 silenced).
August 08, 2016 - 16:31:48
Django version 1.7.1, using settings 'mainsite.settings'
Starting development server at #####//127.0.0.1:8000/
Quit the server with CONTROL-C.
But I can't seem to connect to it when I go to #####//myserver:8000,
I know the traffic from my PC is hitting the server because I see it in tcpdump on TCP 8000. I have been told runserver blocks traffic from external sources because of it being meant for dev only.
After talking with some people they recommend that I switch to Apache or Gunicorn?
Here are some instructions I was sent from the Django documentation: https://docs.djangoproject.com/en/1.7/howto/deployment/wsgi/ Although I can't really make heads or tails of what I should do. Any input would be appreciated. Thanks
I recommend you to use gunicorn and Nginx to run a Django project on your production server. Both are easy to google for official docs and recipes, and their combination is one of the fastest, as long as your code is not to slow. (Nginx+uWSGI is another good option, but a bit harder for beginners).
Gunicorn can be installed with pip install unicorn or the same way you installed Django and then launched with simple gunicorn yourproject.wsgi (refer to docs for more configuration options).
Nginx (use your distribution's package manager to install it) should be configured for reverse proxy mode and also to serve static/media files from your respective static/media root (manage.py collectstatic must be used to keep static files up-to-date). Read documentation to understand basic principles and use this except as an example for your /etc/nginx/sites-enabled/yoursite.conf:
server {
listen 80 default;
server_name example.com;
root /path/to/project/root/static;
location /media {
alias /path/to/project/root/media;
}
location /static {
alias /path/to/project/root/static;
}
location /favicon.ico {
alias /path/to/project/root/static/favicon.ico;
}
location / {
proxy_pass http://localhost:8000;
include proxy_params;
}
}
There's more to it if you need ssl or www/non-www redirect (both are highly recommended to set up) but this example should be enough for you to get started.
To run gunicorn automatically you can either use supervisor or system unit system (be it systemd or something else).
Note: All of this assumes you're using linux. You probably should not use anything else on a production server anyway.
Consider getting some professional help if you feel you can't understand how to deal with all this, there are many freelance sysadmins who would be happy to help you for a reasonable fee.
First of all, you should really be using a "long term support" version of Django, not 1.7.1. The current LTS release is 1.8.14; see https://www.djangoproject.com/download/ for details.
The Django documentation link you were given is just one part of what you need to understand. A better place to start is actually the first link on that page, which is https://docs.djangoproject.com/en/1.8/howto/deployment/wsgi/modwsgi/.
Related
I am setting up a test server at home. The web app is built with flask and python 3.6. For setting up the web server with nginx and gunicorn I followed the guide Real Python Tutorial. This use supervisor for the final management engine.
The problem is I don't know where to start looking to find the cause of my issue. The app has a file upload form which when submitted gets 502 bad gateway if the server is start with supervisor. But if you start gunicorn directly the app works with no issues and uploads the files as expected.
Any help on how to debug this would be great. I don't know what information to share so just ask and I will get what ever you if I can. The server is Ubuntu 16.04.
I spent some days with a similar issue, but with a Django app. Basically, the Bad Gateway problems with Nginx have to do with permission issues for the connection between Nginx and Gunicorn. I suggest you look at your nginx.conf file, and check the user used. The user must be the one that administrates the system (have the necessary permisions for executing/reading). Also, check that you include the configuration files contained in sites-enabled/ (in my case I don't have this directory, but directly overwrote the nginx.conf file). Then, check that you have the #proxy_to_app, location / , and upstream blocks like explained here https://rukbottoland.com/blog/django-gunicorn-nginx-supervisor/
Hope this helps!
Tip: check the nginx logs at /var/log/nginx/ for any error message. That helps debugging.
In the years past, I've always used Angular 1.x for my front end which was as simple as including a single JS file at the time. Now with the rise of TypeScript in Angular, React, and others, there's now a need for a dedicated NPM build serves which compiles the source into vanilla JavaScript before serving the front-end as an independent project. Please correct me if my understanding of this is in anyway flawed.
At the moment I use Django for all of my backend applications, particularly a REST framework that will enable the frontend to function as a SPA. I've taken a liking to Vue.js as my frontend weapon of choice. My question is (and please forgive me if it seems a bit too broad) how do I setup my project and configure it properly so that I'm not running two servers for both the frontend and backend?
Again, back in the days of Angular 1.x and vanilla JavaScript all I had to do was include a minified angular.js in my static files folder in the backend and the same goes for the standalone version of Vue. Now I'm just so confused what is what and how to properly setting up a full stack project. If you guys, could weigh in, I'd very much appreciate it. Thank you in advance!
npm run dev # running the frontend for Vue.js
manage.py runserver # and a Django DRF backend
If you want to do a full-featured VueJS project with webpack template, here is what you can do: Run server and client apps on separate ports, and proxy all server requests (like APIs) using proxyTable in webpack config.
Here are the detailed steps:
Step 1: Create your usual Python server (or any other server that serves APIs, static files, etc.) in a separate folder / terminal window. Let's say it runs on port 8080. So, your development server is available at http://localhost:8080 which also hopefully serves some api requests.
Step 2: Create your VueJS project in a different folder (preferably using webpack template). When you run npm run dev, your client app will be available at http://localhost:4200 (default webpack setup). Do not start your client app yet, till you finish Step 3 below.
Step 3: Now open the config file: VueJS_Webpack_Project_Folder/config/index.js and setup proxyTable as follows:
...
...
module.exports = {
build: {...},
dev: {
...,
port: 4200,
...,
proxyTable: {
'/api': {
target: 'http://localhost:8080',
changeOrigin: true
},
'/static': {
target: 'http://localhost:8080',
changeOrigin: true
}
},
...
}
}
...
As you can see above, all requests going to /api and /static are redirected to server. This is just an example, you can proxy more requests that are relevant for your setup.
You can also change the port if your server is running on 4200 or if you have multiple VueJS apps in your dev setup.
Step 4: Now you can finally start your Python server (your development server) and webpack server (VueJS client app) and do your regular development.
EDIT: Final steps to deploy app
After you are done with development, when you finally run npm run build, it will create a dist folder within your VueJS project setup, containing the minified build files like vendor.hash.js, app.hash.js, etc.
You will need to copy these build files to the right folders within your Django server setup. Either you may do this step manually, or simply write a python script that removes old build files and copies new build files into the right folders.
EDIT 2: Explaining the reasons for the above setup
The reason we have to do all the complicated steps above is because of the following:
Today we may use Django. Tomorrow we may change the server side technology to something else, or may even go serverless. To ensure that our client app is not affected during this migration process, it is in our best interest to keep it independent.
Server and client have different roles: Server handles the databases, APIs, user sessions, authentication, etc. Client App focuses only on having the right user experience. Therefore it helps to separate these environments.
VueJS with webpack template is a very powerful way to do VueJS development. You get to change code and view results immediately in browser without even having to refresh. You get detailed error logs or warnings that allow you to fix bugs immediately without having to do a full build. The development cycle gets shortened significantly when you use VueJS + webpack. The only way to get it working properly is by having it as a stand-alone project.
And some non-technical reasons:
Keeping separate client helps in splitting work in future. In a small project, the single developer may do both client and server development. Later when the project becomes a big success, a separate team will handle the server, and a different team will handle the client app. Having it coupled will lead to a lot of synchronization issues.
And finally, when you have a big revenue-generating project eventually, you don't get developers with Django + VueJS skillset. Developers (unlike tech founders) focus on one particular technology at a time. When you have two different projects that fully conform to the regular development processes, your team can ramp up fast and get things done quickly.
[Disclaimer: This is just my understanding after a little research. Hope it helps.]
It seems today that there is now a build step when creating a frontend with various frameworks. This step only needs to happen when you make changes to your front end, e.g. you update your TypeScript or templates, then build/bundle them into some js package. I believe you can just do this on your development machine. You can then serve the output js statically just as you did before.
I believe the frontend servers exist to streamline the process. So, when you make changes to your frontend, instead of running the build at home and manually adding the output to your static files, the server will take your input files and refresh the output js automatically. But this isn't required.
You should still be able to use vue.js by simply linking to it:
Direct Include
Simply download and include with a script tag. Vue will be registered as a global variable.
https://v2.vuejs.org/v2/guide/installation.html
I'd suggest studying that page for other ways of using vue.js along with the documentation for npm, WebPack, and Browserify and then posting a more specific question if you're still lost.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I'm trying to set up my first web server using the combination of Flask, uWSGI, and nginx. I've had some success getting the Flask & uWSGI components running. I've also gotten many tips from various blogs on how to set this up. However, there is no consistency and the articles suggest many different ways of setting this up especially where folder structures, nginx configurations and users/permissions are concerned (I've tried some of these suggestions and many do work, but I am not sure which is best). So is there one basic "best practice" way of setting up this stack?
nginx + uwsgi + flask make for a potent stack! I add supervisor to the mix and configure it as follows.
Run both uwsgi and nginx out of supervisor for better process control. You can then start supervisor at boot and it will run uwsgi and nginx in the right order. It will also intelligently try to keep them alive if they die. See a sample supervisor configuration below.
If you are running nginx and uwsgi on the same host, use unix sockets rather than HTTP.
The nginx master process must run as root if your web server is listening on port 80. I usually run my web server on some other port (like 8080) and use a load balancer in front to listen on port 80 and proxy to nginx.
Make sure your uwsgi server has access to read/write to the socket file you choose as well as proper permissions to any app code and data directories.
Don't worry too much about your folder structure, especially if you're using a Linux distro like Ubuntu that has sensible defaults. The main supervisor config file can include files from a subdirectory like /etc/supervisor/conf.d/ to separate your app-specific configuration from the supervisor core config. Same goes for nginx, only /etc/nginx/sites-enabled.
Sample supervisor config for uwsgi and nginx:
$ cat /etc/supervisor/conf.d/app.conf
[program:app]
command=/usr/local/bin/uwsgi
--enable-threads
--single-interpreter
--vacuum
--chdir /path/to/app
--uid www-data
--log-syslog
--processes 4
--socket /tmp/app.sock
-w mypython:app
--master
directory=/path/to/app
autostart=true
autorestart=true
priority=999
stopsignal=INT
environment=SOMEVAR=somevalue
[program:nginx]
command=/usr/sbin/nginx
autostart=true
autorestart=true
priority=999
Sample nginx.conf:
$ cat /etc/nginx/sites-enabled/myapp.conf
server {
listen 8080;
client_max_body_size 4G;
server_name localhost;
keepalive_timeout 5;
location / {
include uwsgi_params;
uwsgi_pass unix:///tmp/app.sock;
}
}
There are two parts to this, one is setting up the system itself (by this I mean, the operating system and its various path/filesystems) and the second part is installing and configuring the components.
I will concentrate on the second part, which I believe is the crux of your question:
nginx should be installed by your operating system's native package management utilities. This will make sure that all permissions are set correctly, and the configuration files are where you (or anyone else) would expect them. For example this means on debian-like systems (such as ubuntu and its various cousins), the configurations are in /etc/nginx/, sites are configured by adding files in /etc/nginx/sites-available/ and so on. It also means that if and when updates are pushed by your OS vendor, they will automatically be installed by your packaging software.
uWSGI you should install by source; because it has a very fast development cycle and improvements in uwsgi are going to have a positive affect on your application. The installation process is simple, and there are no special permissions required; other than the normal root/superuser permissions you would need to install applications system-wide.
Your application's source files. For this, I would strongly advise creating separate user roles for each application and isolate all permissions and all related files (for example, log files generated by uwsgi) so that they are all owned by the same user. This makes sure that other applications/users cannot read error messages/log files, and that one user has all the permissions to read/debug everything related to that application without using tools such as sudo.
Other than the three points mentioned above, actually getting all these components to work together is a standard process:
Create your wsgi process/handler in your application. For flask, the default flask application already provides this interface.
Run this file using your wsgi engine. This is uwsgi or gunicorn or similar. Make sure you are using the binary protocol.
Map your static files to a location that is serviced by your web proxy (this is nginx); and create a upstream server that points to the location that the wsgi process is expecting connections. This could be a port or a pipe (depending on how you have setup the components).
Optional use a process manager like supervisor to control the wsgi processes so that they are restarted upon system reboot and are easier to manage.
Everything else is subject to personal preferences (especially when it comes to file system layouts). For large applications, the creators of flask provide blueprints but again note that they do not extol any file system/directory layout.
As a general rule for any Python package, I would refer you to this link.
What you are asking for is not "best practices", but "conventions". No, there are no conventions in the project about paths, privileges and so on. Each sysadmin (or developer) has its needs and tastes, so if you are satisfied with your current setup... well "mission accomplished". There are no uWSGI gods to make happy :) Obviously distro-supplied packages must have their conventions, but again they are different from distro to distro.
I'd like to serve django application on windows XP/Vista.
The application is an at hoc web interface to a windows program so it won't be put under heavy load (around 100 requests per second).
Do you know any small servers that can be easily deployed on windows to serve a django app? (IIS is not an option as the app should work on all versions of windows)
cherrypy includes a good server. Here's how you set it up to work with django and some benchmarks.
twisted.web has wsgi support and that could be used to run your django application. Here's how you do it.
In fact any wsgi server will do. Here's one more example, this time using spawning:
$ spawn --factory=spawning.django_factory.config_factory mysite.settings
And for using paste, the info is gathered here.
Of course, you could use apache with mod_wsgi. It would be just another wsgi server. Here are the setup instructions.
If you want to give Apache a go, check out XAMPP to see if it'll work for you. You can do a lightweight (read: no installation) "installation." Of course, you'll also want to install mod_python to run Django. This post may help you set everything up. (Note: I have not used python/Django with XAMPP myself.)
Edit: Before someone points this out, XAMPP is not generally a production-ready tool. It's simply a useful way to see whether Apache will work for you. Also, I saw that you're using SQLite after the fact.
Why not Apache ?
Nokia have developed a scaled down version of apache to run on their mobile phones. It supports python.
http://research.nokia.com/research/projects/mobile-web-server/
Also do you need anything else such as database support etc?
I have tried following guides like this one but it just didnt work for me.
So my question is this: What is a good guide for deploying Django, and how do you deploy your Django.
I keep hearing that capastrano is pretty nifty to use, but i have no idea as to how to work it or what it does (apart from automation of deploying code), or even if i want/need to use it or not.
mod_wsgi in combination with a virtualenv for all the dependencies, a mercurial checkout into the virtualenv and a fabric recipe to check out the changes on the server.
I wrote an article about my usual workflow: Deploying Python Web Applications. Hope that helps.
I have had success with mod_wsgi
In my previous work we had real genius guy on deployment duties, he deployed application (Python, SQL, Perl and Java code) as set of deb files built for Ubuntu. Unfortunately now, I have no such support. We are deploying apps manually to virtualenv-ed environments with separate nginx configs for FastCGI. We use paver to deploy to remote servers. It's painful, but it works.
This looks like a good place to start: http://www.unessa.net/en/hoyci/2007/06/using-capistrano-deploy-django-apps/
I use mod_python, and have every site in a git repository with the following subdirs:
mysite
template
media
I have mysite/settings.py in .gitignore, and work like this:
do development on my local machine
create remote repository on webserver
push my changes to webserver repo
set up apache vhost config file, tweak live server settings.py
run git checkout && git reset --hard && sudo /etc/init.d/apache2 restart on webserver repo to get up-to-date version to its working copy and restart apache
repeat steps 1, 3 and 5 whenever change request comes
The easiest way would be to use one of the sites on http://djangofriendly.com/hosts/ that will provide the hosting and set up for you, but even if you're wanting to roll your own it will allow you to see what set up other sites are using.