I currently have a Django project running on a server behind Cloudflare. However, a number of the apps contain functionality that requires synchronizing data with certain web services. This is a security risk, because these web services may reveal the IP address of my server. Therefore I need a solution to prevent this.
So far I came up two alternatives: using a proxy or splitting the project to two servers. One server responsible for responding to requests through Cloudflare and one server responsible for synchronizing data with other web services. The IP address of the latter server will be exposed to the public, however attacks on this server will not cause the website to be offline. I prefer the second solution, because this will also split the load between two servers.
The problem is that I do not know how I should do this with Django without duplicating code. I know I can re-use apps, but for most of them counts that I, for instance, only need the models and the serializers and not the views etc. How should I solve this? What is the best approach to take? In addition, what is an appropriate naming for the two servers? Thanks
This sounds like a single project that is being split as part of the deployment strategy. So it makes sense to use just a single codebase for it, rather than splitting it into two projects. If that's the case, re-use is a non-issue since both servers are using the same code.
To support multiple deployments, then, you just create two settings files and load the appropriate one on the appropriate server. The degree to which they are different is up to you. If you want to support different views you can use different ROOT_URLCONF settings. The INSTALLED_APPS can be different, and so forth.
Related
Currently in the company I have developed about six applications with Django, these applications are managed in different databases since they are focused on different departments. They are asking me that access to these applications be done through a single credential, that is, if a user must handle two applications, they should not have two different access credentials, but rather they can access both applications with a single credential.
I have consulted various sources, one person recommended me to develop an application that is only responsible for access, but I am a bit disoriented with this problem.
I appreciate the help!
I am trying to design a SaaS app using pyramid. Clients will be able to design their facet of the app and their facet should be able to be served via the app.
To make things simpler to understand, let's say for example that we want to create an e-commerce SaaS app. The users should be able to create their own e-commerce site within the app and then access it with their own fqdn from the app if they wish.
I understand that we will need to address at least two issues and probably a third one:
Which fqdn the specific request is about.
How to distinguish static assets for each different fqdn.
How to connect to a different database for each fqdn.
Issue (1) can be easily addressed using request.domain (or request.host, request.host_url, etc.) as far as views and routes are concerned (I think). I am not sure, though, how I can address (2) and (3). The reason I am mentioning (3) is because I think that it's better to scale horizontally (having separate databases for each different fqdn) than vertically (having one database with huge tables/collections/whatevers containing data for all fqdns).
Moreover, let's keep in mind that our app should be able to live on a multiple-application-servers setup, which means that if configuration functions are needed to be called on one application server to e.g. update views dynamically, then the rest of the application servers hosting the same app should be able to be somehow informed about this configuration change as well.
Any ideas on how to resolve these issues (with an example, please) or any suggestions on how the architecture of this app could change to be more scalable and easy to develop are more than appreciated!
Thanks all in advance!
EDIT:
OK, I thought of a "hack" that generates different views for each different fqdn without using any static views in my configuration. I won't post the hack as a self reply yet, because it doesn't seem very elegant to me; but if I come with no better solutions and nobody proposes a better solution I'll post it :).
The general idea is to create a view (not a static view) that will serve our static paths with a *stararg (eg config.add_route('my_static_view', '/static/*subpaths') that will handle the static assets along with its route (eg. config.add_route('my_static_view', '/static')).
We will also need a static_view object within the same scope (let's call it static_view_obj) that will be used to serve our static content (we don't need to reinvent the wheel, we'll use pyramid's satic_view's functionality to serve our static assets). Within our my_static_view function we can alter static_view_obj's content to serve a different static path for each different request.domain.
I'm working on a Flask application, in which we will have multiple clients (10-20) each with their own configuration (for the DB, client specific settings etc.) Each client will have a subdomain, like www.client1.myapp.com, www.cleint2.myapp.com. I'm using uWSGI as the middleware and nginx as the proxy server.
There are a couple of ways to deploy this I suppose, one would be to use application dispatching, and a single instance of uwsgi. Another way would be to just run a separate uwsgi instance for each client, and forward the traffic with nginx based on subdomain to the right app. Does anyone know of the pros and cons of each scenario? Just curious, how do applications like Jira handle this?
I would recommend having multiple instances, forwarded to by nginx. I'm doing something similar with a PHP application, and it works very well.
The reason, and benefit of doing it this way is that you can keep everything completely separate, and if one client's setup goes screwy, you can re-instance it and there's no problem for anyone else. Also, no user, even if they manage to break the application level security can access any other user's data.
I keep all clients on their own databases (one mysql instance, multiple dbs), so I can do a complete sqldump (if using mysql, etc) or for another application which uses sqlite rather than mysql: copy the .sqlite database completely for backup.
Going this way means you can also easily set up a 'test' version of a client's site, as well as as live one. Then you can swap which one is actually live just by changing your nginx settings. Say for doing upgrades, you can upgrade the testing one, check it's all OK, then swap. (Also, for some applications, the client may like having their own 'testing' version, which they can break to their hearts content, and know they (or you) can reinstance it in moments, without harming their 'real' data).
Going with application dispatching, you cannot easily get nginx to serve separate client upload directories, without having a separate nginx config per client (and if you're doing that, then why not go for individual uWSGI instances anyway). Likewise for individual SSL certificates (if you want that...).
Each subdomain (or separate domain entirely for some) has it's own logging, so if a certain client is being DOS'd, or hacked otherwise, it's easy to see which one.
You can set up file-system level size quotas per user, so that if one client starts uploading gBs of video, your server doesn't get filled up as well.
The way I'm working is using ansible to provision and set up the server how I want it, with the client specific details kept in a separate host_var file. So my inventory is:
[servers]
myapp.com #or whatever...
[application_clients]
apples
pears
elephants
[application_clients:vars]
ansible_address=myapp.com
then host_vars/apples:
url=apples.myapp.com
db_user=apples
db_pass=secret
then in the provisioning, I set up a two new users & one group for each client. For instance: apples, web.apples as the two users, and the group simply as apples (which both are in).
This way, all the application code is owned by apples user, but the PHP-FPM instance (or uWSGI instance in your case) is run by web.apples. The permissions of all the code is rwXr-X---, and the permissions of uploads & static directories is rwXrwXr-X. Nginx runs as it's own user, so it can access ONLY the upload/static directories, which it can serve as straight files. Any private files which you want to be served by the uWSGI app can be set that way easily. The web user can read the code, and execute it, but cannot edit it. The actual user itself can read and write to the code, but isn't normally used, except for updates, installing plugins, etc.
I can give out a SFTP user to a client which is chroot'd to their uploads directory if they want to upload outside of the application interface.
Using ansible, or another provisioning system, means there's very little work needed to create a new client setup, and if a client (for whatever reason) wants to move over to their own server, it's just a couple of lines to change in the provisioning details, and re-run the scripts. It also means I can keep a development server installed with the exact same provisioning as the main server, and also I can keep a backup amazon instance on standby which is ready to take over if ever I need it to.
I realise this doesn't exactly answer your question about pros and cons each way, but it may be helpful anyway. Multiple instances of uWSGI or any other WSGI server (mainly I use waitress, but there are plenty of good ones) are very simple to set up and if done logically, with a good provisioning system, easy to administrate.
What are the best practices and solutions for managing dynamic subdomains in different technologies and frameworks? I am searching for something to implement in my Django project but those solutions that I saw, don't work. I also tried to use Apache rewrite mod to send requests from subdomain.domain.com to domain.com/subdomain but couldn't realize how to do it with Django.
UPDATE: What I need is to create virtual subdomains for my main domain using usernames from the site. So, if I have a new registered user that is called jack, when I go to jack.domain.com, it would operate make some operations. Like if I just went to domain.com/users/jack. But I don't want to create an actual subdomain for each user.
You may be able to do what you need with apache mod_rewrite.
Obviously I didn't read the question clearly enough.
As for how to do it in django: you could have some middleware that looks at the server name, and redirects according to that (or even sets a variable). You can't do it with the bare url routing system, as that only has path information, not hostname info.
Portal Technology Assessment in which we will be creating a placement portal for the campuses and industry to help place students. The portal will handle large volumes of data and people logging in, approximately 1000 users/day in a concurrent mode.
What technology should i use? PHP with CakePHP as a framework, Ruby on Rails, ASP.NET, Python, or should I opt for cloud computing? Which of those are the most cost beneficial?
Any of those will do, it really depends on what you know. If you're comfortable with Python, use Django. If you like Ruby go with ROR. These modern frameworks are built to scale, assuming you're not going to be developing something on the scale of facebook then they should suffice.
I personally recommend nginx as your main server to host static content and possibly reverse-proxy to Django/mod_wsgi/Apache2.
Another important aspect is caching, make sure to use something like memcached and make sure the framework has some sort of plugin or it's easily attachable.
Language choice is important as you must choose language that you and your team feel the most comfortable with as you must develop mid-large size application. Of course use framework with Python it must be Django, with ASP.NET .NET or MVC.NET whatever you feel better with with Ruby ROR and with PHP there are too large amount of frameworks.
1000 concurrent users is not that much, especially it depends what users will do. Places where users will get large amount of data are better to Cache with with any caching engine you want. You need to design application this what so you can easily swap between real DB calls and calls to cache. For that use Data Objects like for Logins create an Object array of course if you need it. Save some information in cookies when user logins for example his last login, password in case he wants to change it, email and such so you will make less calls to DB in read mode ( select queries ).
use cookie less domain for static content like images, js and css files. setup on this domain the fastest system you can with simplest server you can, probably something based on Linux.
For servers, best advice is to either get large machine and set Virtual Boxes on it with vmware or other Linux based solution or to get few servers which is better because if on big server down you lost everything if one of 1 is down you still can do some stuff. Especially if you set railroad mode. Railroad mode is simple you set up Application server (IIS or Apache) on one server and make it master while you set up SQL on the same server and make it slave. On other server you set up SQL as master and Application server as slave. So server one serves IIS/Apache and Other one SQL, if one down you just need to change line in host.etc in order to set something somewhere else ( i don't know how to do that in Linux ).
last server for static content.
Cloud Computing, you will use if you want it or not. You will share resources with some applications as Google API for jquery and jqueryUI for instance but you create unique application and i don't believe making core of application based on cloud computing will do any good. Use large site's CDNs for good.