I am new to pyramid and even though I've read a lot of its documentation (tutorial + documentation), it's still not clear to me how to write modular, reusable code.
To be more specific, let's present an example. Let's say that I want to create a site that handles a forum, a site that handles customers and their orders where people will also be able to participate in a forum, and a site that handles polls. Let's say that when designing these sites, I decided to write the following "modules" (apps, plugins, whatever their name should be): A Users module, a Forum module, a Customers module and a polls module. I'd like to maintain each module separately, and be able use them as follows:
When creating the Forum site I'd like to use the Forum module and the Users module.
When creating the Customers site I'd like to use the Users module, the Forum module and the Customers module.
When creating the Pools site I'd like to use the Users module and the Polls module.
Moreover, the Forums module, the Customers module and the Polls module should depend on the Users module, and their models.py should be referencing the Users module's users-schema (assuming that we are working with relational databases and I we are using sqlalchemy for our ORM representation). Each module should have their own static content, models, forms, etc.
I know that the example I've given could have been solved with different designs, but please, this was the first example that came to my mind that indeed fits my questions, so let's -please- pretend that it is indeed a requirement.
That said: How could I achieve this functionality with pyramid? I think that configuration wise, config.include() is my probable candidate, but the implementation is very unclear to me, so any help would be really appreciated!
One way to go is to distribute each application as a separate pyramid application using setuptools and install with pip. For example if you're developing new application that will require users and forum apps, you could just pip install my-users==0.1 my-forum==0.5 in your virtualenv and include both apps in your new app config:
config.include('my_users')
config.include('my_forum')
Both apps should implement an includeme function.
Check how other applications provide reusability:
https://github.com/Pylons/pyramid_debugtoolbar
https://github.com/ITCase/pyramid_sacrud
Related
I've googled around for this, but I still have trouble relating to what Django defines as "apps".
Should I create a new app for each piece of functionality in a site, even though it uses models from the main project?
Do you guys have good rule of thumb of when to split off a new app, and when to keep functionality together with the "main project" or other apps?
James Bennett has a wonderful set of slides on how to organize reusable apps in Django.
I prefer to think of Django applications as reusable modules or components than as "applications".
This helps me encapsulate and decouple certain features from one another, improving re-usability should I decide to share a particular "app" with the community at large, and maintainability.
My general approach is to bucket up specific features or feature sets into "apps" as though I were going to release them publicly. The hard part here is figuring out how big each bucket is.
A good trick I use is to imagine how my apps would be used if they were released publicly. This often encourages me to shrink the buckets and more clearly define its "purpose".
Here is the updated presentation on 6 September 2008.
DjangoCon 2008: Reusable Apps #7:53
Slide: Reusable_apps.pdf
Taken from the slide
Should this be its own application?
Is it completely unrelated to the app’s focus?
Is it orthogonal to whatever else I’m doing?
Will I need similar functionality on other sites?
If any of them is "Yes"? Then best to break it into a
separate application.
I tend to create new applications for each logically separate set of models. e.g.:
User Profiles
Forum Posts
Blog posts
The two best answers to this question I've found around the web are:
The Reusable Apps Talk (slides)(video) also mentioned in other answers. Bennett, the author and Django contributor, regularly publishes apps for others to use and has a strong viewpoint towards many small apps.
Doordash's Tips for Django at Scale which gives the opposite advice and says in their case they migrated to one single app after starting with many separate apps. They ran into problems with the migration dependency graph between apps.
Both sources agree that you should create a separate app in the following situations:
If you plan to reuse your app in another Django project (especially if you plan to publish it for others to reuse).
If the app has few or no dependencies between it and another app. Here you might be able to imagine an app running as its own microservice in the future.
The rule I follow is it should be a new app if I want to reuse the functionality in a different project.
If it needs deep understanding of the models in your project, it's probably more cohesive to stick it with the models.
The best answer to this question is given by Andrew Godwin (Django core developer):
The main purpose of apps is, in my eyes, to provide logical separation of reusable components - specifically, a first-class namespace for models/admin/etc. - and to provide an easy way to turn things “on” or “off”.
In some ways, it’s a relic of the time when Django was created - when Python packaging and modules were much less developed and you basically had to have your own solution to the problem. That said, it’s still a core part of Django’s mental model, and I think INSTALLED_APPS is still a cleaner, easier solution than Python’s replacement offering of entrypoints (which makes it quite hard to disable a package that is installed in an environment but which you don’t want to use).
Is there anything specifically you think could be decoupled from the app concept today? Models and admin need it for autodiscovery and a unique namespace prefix, so that’s hard to undo, and I’m struggling to think of other features you need it for (in fact, if all you want is just a library, you can make it a normal Python one - no need for the app wrapping unless you’re shipping models, templates or admin code IIRC)
An 'app' could be many different things, it all really comes down to taste. For example, let's say you are building a blog. Your app could be the entire blog, or you could have an 'admin' app, a 'site' app for all of the public views, an 'rss' app, a 'services' app so developers can interface with the blog in their own ways, etc.
I personally would make the blog itself the app, and break out the functionality within it. The blog could then be reused rather easily in other websites.
The nice thing about Django is that it will recognize any models.py file within any level of your directory tree as a file containing Django models. So breaking your functionality out into smaller 'sub apps' within an 'app' itself won't make anything more difficult.
I'm currently developing a project to store all of my old MP3 files. This project contains multiple applications(Artist, Album, Song) and I'm simply curious to know if it is common for developers to create a separate application which may be named something like 'Dashboard' and use this to host their homepage, login/logout functionality, search features, recently uploaded items, etc. Or is it more common to just stick your homepage into an existing app?
I wouldn't call an approach best practice, however certain practices are commonly used-
core app is created where it's a small project (i.e. probably three or less models and around five views). Dashboard can be a template in this app. You're correct, usually it's common to stick your homepage into an existing app.
account app is created to keep the authentication system separate.
Other approaches are also good.
I could use some help creating a plan of attack for a project I'm working on.
Imagine that the site is for a group that oversees regional sales offices, distributed around the world. The purpose of this project is to let superusers spin up a new sub-site specific to each and every office, at a fast pace -- sites are added on a frequent basis. The office sub-sites should be wholly contained with "admin" users specific to that sub-site and should be user-friendly CMS. A superuser should be able to step in and manage all of these office sub-sites.
In addition to the self-contained office sub-site instance, there is also a need for each sub-site to manage contacts, leads, etc and store this in one central area for the superusers.
I've done a few sites using Django, but never anything multi-tenant. I'd like suggestions for technologies to use or tutorials/documentation that might be helpful.
Requirements:
Each sub-site uses the same source (templates, JS, available features, etc), but can be modified to reflect custom content within the templates.
Assigned subdomains (with an option of using a fully qualified domain) per sub-site, configured within the project, not in a hardcoded settings file.
Sub-site specific user access controls, in addition to superusers who can access all sub-sites.
The ability to provide an "independent" CMS for each sub-site. i.e., A sub-site admin only sees their content. My preference for this project would be django-cms, but I'm open to suggestions.
Support for apps that pool the data from all the sub-sites, but limit sub-site "admins" to only viewing their records into that app.
Considering the above, what approach would you recommend? I am open to reconsidering technologies, but I would like to stick with Python.
There is a great app called django-tenant-schemas that uses PostgreSQL schemas mechanism to create multi-tenancy.
What you get is specyfing SHARED_APPS that contain objects shared across all the schemas (sub-sites), and TENANT_APPS that contain objects specific for a sub-site, i.e. users, records etc. The schemas are completely isolated from each other.
Each PostgreSQL schema is tied to a domain url, so that middleware checks the HOST part of the request and sets the db connection's schema to appriopriate one.
In addition, it allows you to define a PUBLIC_SCHEMA_URLCONF which allows you to specify urlconf file for public schema - the meta site that is not tied to any sub-site.
Sorry for quick and dirty answer, i just share what i've done to achieve multi tenancy:
django-tenancy I like the author's approach of using "dynamic model"
django-dynamicsite This is where dynamic SITE_ID based on domain will be linked to a tenant
Both libraries above, when combined, is able to serve a django instance which is multi-tenant, and flexible. What i mean flexible here is: you can define any model whether is it "tenant" or "global". So, you can have a site with global user but per tenant product catalogue, or per tenant + product. From many django app i've tried, this is the most flexible way to achieve multi tenancy
The Django based CMS Mezzanine also has multi-tenancy support.
It has most of the features you requested except the sub-site user controls I think. The admin page can be separated by site for admin users, but the normal users not.
However, if you dont need a CMS this might be an overkill for your use-case, But I wanted to mention it here for completeness.
I have been trying to use django-tenants for a while along with Wagtail but this combination didn't work very well, or let me say, despite of a lot of try I was not able to get wagtail admin-page working correctly. I think will try to switch to django-tenant-schemas which I more widely used .
NOTE: django-tenant-schemas is not maintained now.
EDIT: I'm new to this site but if you are going to down vote me, could you perhaps explain why? I've searched Google, this site and others but have not found anything that makes any sense and I thought this was a site to ask questions and get some help.
I've got a Custom PHP Forum that I am trying to convert to Python/Django as a learning experience and I'm having some problems. I've been reading up on Django and it is encouraged that our application is split into multiple apps. I went through the 6 part tutorial and many other parts of the documentation but I'm left with some questions.
Let's assume that I have about 30 tables.
Tables such as:
posts_index, posts, users, user_groups, user_activity, user_sessions, forums, payment_gateways, payment_logs, etc for a basic forum
I'm having issues structuring my models. With PHP all I needed was index.php, /admin/index.php, view_forum.php, view_thread.php and a few others, everything could pull directly from the database and I had no issues but now I have to deal with apps/modules.
I'm thinking I'd need to structure my apps in a manner similar to this:
/admin/ app
/forums/ app
/view_forum/ app
/view_thread/ app
/forums/view_forum/ app (instead of just /view_forum/, could be a sub app)
/forums/view_thread/ app (instead of /view_thread/, could be a sub app)
My problem and only question here is dealing with global state. For example Users/Group/Session/Logging/Permission information is going to need to be shared across multiple apps through importing in the other apps models file. To do this I need to reference their model information, what is the correct way to handle this?
Would either of these be acceptable?
Create a ton of different apps such as /users/ which would model my users_groups, users, user_sessions, another app for /posts/ that would include models for posts_index, forums, and so forth with these models existing but not actually being used publicly, they would be used in other apps only. They would be imported in areas such as the /view_forum/ app since when viewing a forum I might need to determine if the user is logged in, is a member of a particular group, etc and because of that would need access to a number of the hidden apps and hence would be imported from the hidden app.
What if I just had one single app, instead of it being an app it would just be my entire project. This sounds like the best solution to me but it seems to be suggested if we cannot summarize the entire application into a sentence it needs to be broken up. If I went with one single app being used as my entire project, my models file will have 30+ different models, is this acceptable? I assume not but figured I'd ask.
Do either of the above make any sense? If not what would you do fix it? I'll admit I'm lost so any feedback would mean a lot.
I'm new to Python/Django and am trying to figure things out. I hope I am clear on what I am trying to do. I'm more than welcome to any advice. I've been trying to play around with things but I figure it would be better to ask for advice from more experienced developers. I'm not a professional programmer and am still learning so please be nice :).
I have voted this up... I had similar questions when I first moved to Django (also coming from PHP)
Try not to think of apps in terms of db tables (or url paths), you want to create apps for independent pieces of functionality.
I would say most of your code will be in a single forum app with Forum and Post models in it and all your forum-related urls like /view_forum/ and /view_thread/. Note there's not really such thing as a sub-app in Django... these are different views which all belong to one related set of functionality in a single app.
The 'users' stuff... you normally want to hook in to the Django auth system (https://docs.djangoproject.com/en/dev/topics/auth/default/#user-objects) though if you are trying to keep the legacy database structure this may be harder... you may end up needing your own users app.
The payment gateways stuff sounds like another app again.
For the admin, you get this (almost) for free with Django:
https://docs.djangoproject.com/en/1.5/ref/contrib/admin/
You need an admin.py inside each app, where you register the models that you want to expose to the admin site.
You might find it's best to follow a tutorial and build a simple blog (or try and build a very simple minimal forum from scratch) to get a feel for Django before tackling your re-write. Because trying to exactly recreate the old functionality with legacy db tables will be a bit harder and may lead you into patterns which are not 'good Django'.
You might also find it useful to look at the source code of an existing Django forum app, such as this one:
https://bitbucket.org/slav0nic/djangobb/src/
They have just a single djangobb_forum app that does everything. I think this is entirely justified as a forum is a single complicated piece of functionality.
Note how their Profile model effectively extends the built-in Django auth.User model via a OneToOneField. It makes it easier to integrate with the built-in login and authentication system that way.
Start by integrating your legacy database and then build the Admin site. You'll see how the model information is available to the Admin app and every other app you write. The key is to import models in your apps.
I have a fairly complex "product" I'm getting ready to build using Django. I'm going to avoid using the terms "project" and "application" in this context, because I'm not clear on their specific meaning in Django.
Projects can have many apps. Apps can be shared among many projects. Fine.
I'm not reinventing the blog or forum - I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
If so... in terms of Django's project.app namespace, my inclination is to use myproduct.myproduct, but of course this isn't allowed (but the application I'm building is my project, and my project is an application!). I'm therefore lead to believe that perhaps I'm supposed to approach Django by building one app per "significant" model, but I don't know where to draw the boundaries in my schema to separate it into apps - I have a lot of models with relatively complex relationships.
I'm hoping there's a common solution to this...
Once you graduate from using startproject and startapp, there's nothing to stop you from combining a "project" and "app" in the same Python package. A project is really nothing more than a settings module, and an app is really nothing more than a models module—everything else is optional.
For small sites, it's entirely reasonable to have something like:
site/
models.py
settings.py
tests.py
urls.py
views.py
Try to answer question: "What does my
application do?". If you cannot answer
in a single sentence, then maybe you can
split it into several apps with cleaner
logic.
I read this thought somewhere soon after I've started to work with django and I find that I ask this question of myself quite often and it helps me.
Your apps don't have to be reusable, they can depend on each other, but they should do one thing.
What is to stop you using myproduct.myproduct? What you need to achieve that roughly consists of doing this:
django-admin.py startproject myproduct
cd myproduct
mkdir myproduct
touch myproduct/__init__.py
touch myproduct/models.py
touch myproduct/views.py
and so on. Would it help if I said views.py doesn't have to be called views.py? Provided you can name, on the python path, a function (usually package.package.views.function_name) it will get handled. Simple as that. All this "project"/"app" stuff is just python packages.
Now, how are you supposed to do it? Or rather, how might I do it? Well, if you create a significant piece of reusable functionality, like say a markup editor, that's when you create a "top level app" which might contain widgets.py, fields.py, context_processors.py etc - all things you might want to import.
Similarly, if you can create something like a blog in a format that is pretty generic across installs, you can wrap it up in an app, with its own template, static content folder etc, and configure an instance of a django project to use that app's content.
There are no hard and fast rules saying you must do this, but it is one of the goals of the framework. The fact that everything, templates included, allows you to include from some common base means your blog should fit snugly into any other setup, simply by looking after its own part.
However, to address your actual concern, yes, nothing says you can't work with the top level project folder. That's what apps do and you can do it if you really want to. I tend not to, however, for several reasons:
Django's default setup doesn't do it.
Often, I want to create a main app, so I create one, usually called website. However, at a later date I might want to develop original functionality just for this site. With a view to making it removable (whether or not I ever do) I tend to then create a separate directory. This also means I can drop said functionality just by unlinking that package from the config and removing the folder, rather than a complex delete the right urls from a global urls.py folder.
Very often, even when I want to make something independent, it needs somewhere to live whilst I look after it / make it independent. Basically the above case, but for stuff I do intend to make generic.
My top level folder often contains a few other things, including but not limited to wsgi scripts, sql scripts etc.
django's management extensions rely on subdirectories. So it makes sense to name packages appropriately.
In short, the reason there is a convention is the same as any other convention - it helps when it comes to others working with your project. If I see fields.py I immediately expect code in it to subclass django's field, whereas if I see inputtypes.py I might not be so clear on what that means without looking at it.
I've found the following blog posts very useful about django applications and projects:
http://www.b-list.org/weblog/2006/sep/10/django-tips-laying-out-application/
http://web.archive.org/web/20080302205555/www.pointy-stick.com/blog/2007/11/09/django-tip-developing-without-projects/
In principle, you have a lot of freedom with django for organizing the source code of your product.
If so... in terms of Django's project.app namespace, my inclination is to usemyproduct.myproduct, but of course this isn't allowed
There is nothing like not allowed. Its your project, no one is restricting you. It is advisable to keep a reasonable name.
I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
In a general django project there are many apps (contrib apps) which are used really in every project.
Let us say that your project does only one task and has only a single app (I name it main as thethe project revolves around it and is hardly pluggable). This project too still uses some other apps generally.
Now if you say that your project is using just the one app (INSTALLED_APPS='myproduct') so what is use of project defining the project as project.app, I think you should consider some points:
There are many other things that the code other than the app in a project handles (base static files, base templates, settings....i.e. provides the base).
In the general project.app approach django automatically defines sql schema from models.
Your project would be much easier to be built with the conventional approach.
You may define some different names for urls, views and other files as you wish, but I don't see the need.
You might need to add some applications in future which would be real easy with the conventional django projects which otherwise it may become equally or more difficult and tedious to do.
As far as most of the work being done in the app is concerned, I think that is the case with most of django projects.
Here Django creators points out that difference themselves.
I think that thinking about Apps as they have to be reusable in other projects is good. Also a good way of thinking about Apps in Django provide modern web applications.
Imagine that you are creating big dynamic web app basing on JavaScript.
You can create then in django App named e.g "FrontEnd" <-- in thins app you will display content.
Then you create some backend Apps. E.g App named "Comments" that will store user comments. And "Comments" App will not display anything itself. It will be just API for AJAX requests of your dynamic JS website.
In this way you can always reuse your "Comments" app. You can make it open source without opening source of whole project. And you keep clean logic of your project.