I am trying to design a SaaS app using pyramid. Clients will be able to design their facet of the app and their facet should be able to be served via the app.
To make things simpler to understand, let's say for example that we want to create an e-commerce SaaS app. The users should be able to create their own e-commerce site within the app and then access it with their own fqdn from the app if they wish.
I understand that we will need to address at least two issues and probably a third one:
Which fqdn the specific request is about.
How to distinguish static assets for each different fqdn.
How to connect to a different database for each fqdn.
Issue (1) can be easily addressed using request.domain (or request.host, request.host_url, etc.) as far as views and routes are concerned (I think). I am not sure, though, how I can address (2) and (3). The reason I am mentioning (3) is because I think that it's better to scale horizontally (having separate databases for each different fqdn) than vertically (having one database with huge tables/collections/whatevers containing data for all fqdns).
Moreover, let's keep in mind that our app should be able to live on a multiple-application-servers setup, which means that if configuration functions are needed to be called on one application server to e.g. update views dynamically, then the rest of the application servers hosting the same app should be able to be somehow informed about this configuration change as well.
Any ideas on how to resolve these issues (with an example, please) or any suggestions on how the architecture of this app could change to be more scalable and easy to develop are more than appreciated!
Thanks all in advance!
EDIT:
OK, I thought of a "hack" that generates different views for each different fqdn without using any static views in my configuration. I won't post the hack as a self reply yet, because it doesn't seem very elegant to me; but if I come with no better solutions and nobody proposes a better solution I'll post it :).
The general idea is to create a view (not a static view) that will serve our static paths with a *stararg (eg config.add_route('my_static_view', '/static/*subpaths') that will handle the static assets along with its route (eg. config.add_route('my_static_view', '/static')).
We will also need a static_view object within the same scope (let's call it static_view_obj) that will be used to serve our static content (we don't need to reinvent the wheel, we'll use pyramid's satic_view's functionality to serve our static assets). Within our my_static_view function we can alter static_view_obj's content to serve a different static path for each different request.domain.
Related
I currently have a Django project running on a server behind Cloudflare. However, a number of the apps contain functionality that requires synchronizing data with certain web services. This is a security risk, because these web services may reveal the IP address of my server. Therefore I need a solution to prevent this.
So far I came up two alternatives: using a proxy or splitting the project to two servers. One server responsible for responding to requests through Cloudflare and one server responsible for synchronizing data with other web services. The IP address of the latter server will be exposed to the public, however attacks on this server will not cause the website to be offline. I prefer the second solution, because this will also split the load between two servers.
The problem is that I do not know how I should do this with Django without duplicating code. I know I can re-use apps, but for most of them counts that I, for instance, only need the models and the serializers and not the views etc. How should I solve this? What is the best approach to take? In addition, what is an appropriate naming for the two servers? Thanks
This sounds like a single project that is being split as part of the deployment strategy. So it makes sense to use just a single codebase for it, rather than splitting it into two projects. If that's the case, re-use is a non-issue since both servers are using the same code.
To support multiple deployments, then, you just create two settings files and load the appropriate one on the appropriate server. The degree to which they are different is up to you. If you want to support different views you can use different ROOT_URLCONF settings. The INSTALLED_APPS can be different, and so forth.
I am considering a multi-tenant environment where I can have each tenant access a different subdomain and then potentially allocate a namespace based on that domain.
For instance,
tenantA.mydomain.com
tenantB.mydomain.com
Then I would want to have namespace tenantA for all tenantA data and tenantB for all tenantB data.
From the docs, it sounds like I would accomplish this in my appengine_config.py file and do something like this:
from google.appengine.api import namespace_manager
def namespace_manager_default_namespace_for_request():
this_namespace = get_namespace_from_subdomain()
return this_namespace
First question, is this a reasonable/good approach?
Second question, it's unclear what variables are available in this scope - any pointers on how to implement the get_namepsace_from_subdomain() function?
Finally, if there were some functionality I wanted to provide that would cross namespaces could this still be achieved with a global namespace? For instance, say a user has an account in multiple tenants and I want to give a view of his activity across all tenants.
It is possible and reasonable to use multi-tenancy in your app based on sub-domains, though from my experience you should also allow overriding the namespace by using a url param.
e.g.
tenantB.mydomain.com/?tenant=tenantA => namespace=tenantA
This will make you life a lot easier and will enable you to test your newest appengine versions on *.appspot.com before moving them to production (especially if you are planning on SSL access).
Once you set the namespace than only the entities under that namespace will be available, you can change the namespace via code whenever you want - the scope doesn't matter.
for the sub-domain - you can parse it out from the one of the client's request headers.
you can write whatever you want to the global namespace and access it whenever you want via code. for the scenario you described, you need to save the user activity on the global namespace.
also, take a loot at the official python example for using namespaces from the GAE team.
https://github.com/GoogleCloudPlatform/appengine-guestbook-namespaces-python
It gives you everything you need to get started.
I don't think this is the best approach to take. The problem with this approach is that you're tightly coupling you application with the infrastructure in a way. Domain and subdomain are just an easier way to access a machine bound to a specific ip address. I would classify domain names to be part of the infrastructure, not really part of the application. If you go with the above mentioned approach you're introducing some knowledge about the infrastructure into the application and thus making your application less flexible. What happens if you, for some reason, sometime in the future decide that your client A should use clientA.mydomain.com? or how about keyClientA.myotherdomain.com? Or how about you want to allow your client A to use their domain name, i.e. support.clientA.com? If your application does not know anything about domains and infrastructure setup then it's a lot easier to just reconfigure DNS server and get the portability.
If I had this scenario, I would have some kind of mapping of URLs to a tenant id, and then use that tenant id as a namespace name. This way you can easily map different URLs to a tenant id. You can even map multiple URLs to the same tenant id and expose the same application on multiple URLs. Your mapping can be stored in a simple config file or even in the AppEngine datastore itself within global namespace. If the config is stored in the AppEngine datastore, you can have your admin section of the application (or even another AppEngine module) which you can use to update config in real time.
After reading through How do you divide your project into applications in Django? and Django documentation about applications I am still not sure whether I should make or not a new application for certain stuff.
Let's imagine I have a Website with the following sections: <Home> <Login> <Register> <My account>.
Should all them be different applications? or should they be just one?
Also, imagine I include a section <Wiki> but it is not very linked with the page (I mean, not the design but the content makes relation to it).
Would this be a new project or application?
The individual sets of functionality (such as the ones you listed) should be applications.
When you group several applications together, you form a project.
However, the individual applications are usually self-contained enough that they could be picked up and dropped into another application. In this manner, you can re-use your 'login' application in several projects.
If it's part of the same site or webapp you're building, than don't make another project, one project shares the same settings.py, for example, so you can imagine that everything that relates to it belongs to the same project.
About the apps, you can create an entire project with just one app, but that's not advisable, you can separate your models in apps in a way that you fell confortable with it, for your own organization.
About login/register, take a look on User authentication documentation, that can clarify a little how one part of the site, in this case, authentication, can work with it's own app.
You should also do the Tutorial which could help you to understant how django basically works.
I'm planning an iOS app that requires a server backend capable of efficiently serving image files and performing some dynamic operations based on the requests it gets (like reading and writing into a data store, such as Redis). I'm most comfortable with, and would thus prefer to write the backend in Python.
I've looked at a lot of Python web framework/server options, Flask, Bottle, static and Tornado among them. The common thread seems to be that either they support serving static files as a development-time convenience only, discouraging it in production, or are efficient static file servers but not really geared towards the dynamic framework-like side of things. This is not to say they couldn't function as the backend, but at a quick glance they all seem a bit awkward at it.
In short, I need a web framework that specializes in serving JPEGs instead of generating HTML. I'm pretty certain no such thing exists, but right now I'm hoping that someone could suggest a solution that works without bending the used Python applications in ways they are not meant for.
Specifications and practical requirements
The images I'd be serving to the clients live in the file system in a shallow directory hierarchy. The actual file names would be invisible to the clients. The server would essentially read the directory hierarchy at startup, assigning a numeric ID for each file, and would route the requests to controller methods that then actually serve the image files. Here are a few examples of ways the client would want to access the images in different circumstances:
Randomly (example URL path: /image/random)
Randomly, each file only once (/image/random_unique), produces some suitable non-200 HTTP status code when the files are exhausted
Sequentially in either direction (/image/0, /image/1, /image/2 etc.)
and so on. In addition, there would be URL endpoints for things like ratings, image info and other metadata, some client-specific information as well (the client would "register" with the server, so that needs some logic, too). This data would live in a Redis datastore, most likely.
All in all, the backend needs to be good at serving image/jpeg and application/json (which it would also generate). The scalability and concurrency requirements are modest, at least to start with (this is not an App Store app, going for ad-hoc or enterprise distribution).
I don't want the app to rely on redirects. That is, I don't want a model where a request to a URL would return a redirect to another URL that is backed by, say, nginx as a separate static file server, leaving only the image selection logic for the Python backend. Instead, a request to a URL from the client should always return image/jpeg, with metadata in custom HTTP headers where necessary. I specify this because it is a way of avoiding serving static files from Python that I thought of, and someone else might think of too ;-)
Given this information, what sort of solution would you consider a good choice, and why? Or is this something for which I need to code non-trivial extensions to existing projects?
EDIT: I've been thinking about this a bit more. I don't want redirects due to the delay inherent in the multiple requests they entail, plus I'd like to abstract out the file names from the client, but I was wondering if something like this would be possible:
It's pretty self-explanatory, but the idea is that the Python program is given the request info by nginx (or whatever serves the role), mulls it over and then tells nginx to respond to the client's request with a specific file from the file system. It does so. The client is none the wiser about how the request was fulfilled, it just receives a response with the correct content type.
This would be pretty optimal in my view, but is it possible? If not with nginx, perhaps something else?
I've been using Django for well over a year now, and it is the hammer I use for all my nails. You could probably do this with a bit of database-image storage and django's builtin orm and url routing (with regex). If you store the images in the database, you will automatically get the unique-id's set. According to this stackoverflow answer, you can use redis with django.
I don't want a model where a request to a URL would return a redirect to another URL that is backed by, say, nginx as a separate static file server, leaving only the image selection logic for the Python backend.
I think Nginx for serving static and python for figuring out the image url is the better solution.
Still if you do not want to do that I would suggest you use any Python web framework (like Django) and write your models and convert them into REST resources (Eg. Using django-tastypie) and/or return a base64 encoded image which you can then decode in your iOS client.
Refs:
Decoding a Base64 image
TastyPie returns the path as default, you might have to do extra work to either store the image blob in the table or write more code to return a base64 encoded image string
You might want to look at one of the async servers like Tornado or Twisted.
What are the best practices and solutions for managing dynamic subdomains in different technologies and frameworks? I am searching for something to implement in my Django project but those solutions that I saw, don't work. I also tried to use Apache rewrite mod to send requests from subdomain.domain.com to domain.com/subdomain but couldn't realize how to do it with Django.
UPDATE: What I need is to create virtual subdomains for my main domain using usernames from the site. So, if I have a new registered user that is called jack, when I go to jack.domain.com, it would operate make some operations. Like if I just went to domain.com/users/jack. But I don't want to create an actual subdomain for each user.
You may be able to do what you need with apache mod_rewrite.
Obviously I didn't read the question clearly enough.
As for how to do it in django: you could have some middleware that looks at the server name, and redirects according to that (or even sets a variable). You can't do it with the bare url routing system, as that only has path information, not hostname info.