I am a Django developer with some experience. I am familiar with different best practices and controversial points of view.
Recently, while working on one project, I struggled for a while when I came upon one giant existing app. The issue isn't resolved and I'm still in progress of thinking.
The issue itself: project has a giant application, let's name it "Directory". The application is big and serves to multiple purposes:
1) Provides basic content types for the whole project
2) Provides some helping structure entities (like, Categories, Attributes etc)
3) Handles User Area dashboard views.
4) Handles basic front-end public views.
5) Handles search.
I really don't like how big it is right now. While thinking on possible way of decomposition, I came up with multiple variants. Most likely it'll be the best to do it like this:
I can create a dashboard app, a search app, a frontend functionality app (serves for displaying public content).
The biggest stopper right now is the model layer. The thing is, all of the possible apps mentioned above depend literally on the same models. In my opinion, it'll be wrong to put these models to some of the apps above and leave others without models, as it'll be non-obvious where the mentioned models should live.
The possible solution, in my opinion, will be to create a separate app like "Content Types" or something like that, where I should put these models. But if I do it, I'll come up with apps that:
1) Do nothing but only provide models and admin. 2) Apps that don't have models and only provide functionality (views etc)
So, basically, I'm lost as I'm not sure that app that does nothing has a right to live separately (while the second point is more or less imaginable).
Can anyone please give me possible advice on how should I decompose my app to still follow the django-way of development and have a nice structure? I suspect that the issue I have appeared due to wrong design of the db-layer as well. But, unfortunately, it's too late to save it at the moment and I have to do my best at least with other parts.
Related
I've googled around for this, but I still have trouble relating to what Django defines as "apps".
Should I create a new app for each piece of functionality in a site, even though it uses models from the main project?
Do you guys have good rule of thumb of when to split off a new app, and when to keep functionality together with the "main project" or other apps?
James Bennett has a wonderful set of slides on how to organize reusable apps in Django.
I prefer to think of Django applications as reusable modules or components than as "applications".
This helps me encapsulate and decouple certain features from one another, improving re-usability should I decide to share a particular "app" with the community at large, and maintainability.
My general approach is to bucket up specific features or feature sets into "apps" as though I were going to release them publicly. The hard part here is figuring out how big each bucket is.
A good trick I use is to imagine how my apps would be used if they were released publicly. This often encourages me to shrink the buckets and more clearly define its "purpose".
Here is the updated presentation on 6 September 2008.
DjangoCon 2008: Reusable Apps #7:53
Slide: Reusable_apps.pdf
Taken from the slide
Should this be its own application?
Is it completely unrelated to the app’s focus?
Is it orthogonal to whatever else I’m doing?
Will I need similar functionality on other sites?
If any of them is "Yes"? Then best to break it into a
separate application.
I tend to create new applications for each logically separate set of models. e.g.:
User Profiles
Forum Posts
Blog posts
The two best answers to this question I've found around the web are:
The Reusable Apps Talk (slides)(video) also mentioned in other answers. Bennett, the author and Django contributor, regularly publishes apps for others to use and has a strong viewpoint towards many small apps.
Doordash's Tips for Django at Scale which gives the opposite advice and says in their case they migrated to one single app after starting with many separate apps. They ran into problems with the migration dependency graph between apps.
Both sources agree that you should create a separate app in the following situations:
If you plan to reuse your app in another Django project (especially if you plan to publish it for others to reuse).
If the app has few or no dependencies between it and another app. Here you might be able to imagine an app running as its own microservice in the future.
The rule I follow is it should be a new app if I want to reuse the functionality in a different project.
If it needs deep understanding of the models in your project, it's probably more cohesive to stick it with the models.
The best answer to this question is given by Andrew Godwin (Django core developer):
The main purpose of apps is, in my eyes, to provide logical separation of reusable components - specifically, a first-class namespace for models/admin/etc. - and to provide an easy way to turn things “on” or “off”.
In some ways, it’s a relic of the time when Django was created - when Python packaging and modules were much less developed and you basically had to have your own solution to the problem. That said, it’s still a core part of Django’s mental model, and I think INSTALLED_APPS is still a cleaner, easier solution than Python’s replacement offering of entrypoints (which makes it quite hard to disable a package that is installed in an environment but which you don’t want to use).
Is there anything specifically you think could be decoupled from the app concept today? Models and admin need it for autodiscovery and a unique namespace prefix, so that’s hard to undo, and I’m struggling to think of other features you need it for (in fact, if all you want is just a library, you can make it a normal Python one - no need for the app wrapping unless you’re shipping models, templates or admin code IIRC)
An 'app' could be many different things, it all really comes down to taste. For example, let's say you are building a blog. Your app could be the entire blog, or you could have an 'admin' app, a 'site' app for all of the public views, an 'rss' app, a 'services' app so developers can interface with the blog in their own ways, etc.
I personally would make the blog itself the app, and break out the functionality within it. The blog could then be reused rather easily in other websites.
The nice thing about Django is that it will recognize any models.py file within any level of your directory tree as a file containing Django models. So breaking your functionality out into smaller 'sub apps' within an 'app' itself won't make anything more difficult.
EDIT: I'm new to this site but if you are going to down vote me, could you perhaps explain why? I've searched Google, this site and others but have not found anything that makes any sense and I thought this was a site to ask questions and get some help.
I've got a Custom PHP Forum that I am trying to convert to Python/Django as a learning experience and I'm having some problems. I've been reading up on Django and it is encouraged that our application is split into multiple apps. I went through the 6 part tutorial and many other parts of the documentation but I'm left with some questions.
Let's assume that I have about 30 tables.
Tables such as:
posts_index, posts, users, user_groups, user_activity, user_sessions, forums, payment_gateways, payment_logs, etc for a basic forum
I'm having issues structuring my models. With PHP all I needed was index.php, /admin/index.php, view_forum.php, view_thread.php and a few others, everything could pull directly from the database and I had no issues but now I have to deal with apps/modules.
I'm thinking I'd need to structure my apps in a manner similar to this:
/admin/ app
/forums/ app
/view_forum/ app
/view_thread/ app
/forums/view_forum/ app (instead of just /view_forum/, could be a sub app)
/forums/view_thread/ app (instead of /view_thread/, could be a sub app)
My problem and only question here is dealing with global state. For example Users/Group/Session/Logging/Permission information is going to need to be shared across multiple apps through importing in the other apps models file. To do this I need to reference their model information, what is the correct way to handle this?
Would either of these be acceptable?
Create a ton of different apps such as /users/ which would model my users_groups, users, user_sessions, another app for /posts/ that would include models for posts_index, forums, and so forth with these models existing but not actually being used publicly, they would be used in other apps only. They would be imported in areas such as the /view_forum/ app since when viewing a forum I might need to determine if the user is logged in, is a member of a particular group, etc and because of that would need access to a number of the hidden apps and hence would be imported from the hidden app.
What if I just had one single app, instead of it being an app it would just be my entire project. This sounds like the best solution to me but it seems to be suggested if we cannot summarize the entire application into a sentence it needs to be broken up. If I went with one single app being used as my entire project, my models file will have 30+ different models, is this acceptable? I assume not but figured I'd ask.
Do either of the above make any sense? If not what would you do fix it? I'll admit I'm lost so any feedback would mean a lot.
I'm new to Python/Django and am trying to figure things out. I hope I am clear on what I am trying to do. I'm more than welcome to any advice. I've been trying to play around with things but I figure it would be better to ask for advice from more experienced developers. I'm not a professional programmer and am still learning so please be nice :).
I have voted this up... I had similar questions when I first moved to Django (also coming from PHP)
Try not to think of apps in terms of db tables (or url paths), you want to create apps for independent pieces of functionality.
I would say most of your code will be in a single forum app with Forum and Post models in it and all your forum-related urls like /view_forum/ and /view_thread/. Note there's not really such thing as a sub-app in Django... these are different views which all belong to one related set of functionality in a single app.
The 'users' stuff... you normally want to hook in to the Django auth system (https://docs.djangoproject.com/en/dev/topics/auth/default/#user-objects) though if you are trying to keep the legacy database structure this may be harder... you may end up needing your own users app.
The payment gateways stuff sounds like another app again.
For the admin, you get this (almost) for free with Django:
https://docs.djangoproject.com/en/1.5/ref/contrib/admin/
You need an admin.py inside each app, where you register the models that you want to expose to the admin site.
You might find it's best to follow a tutorial and build a simple blog (or try and build a very simple minimal forum from scratch) to get a feel for Django before tackling your re-write. Because trying to exactly recreate the old functionality with legacy db tables will be a bit harder and may lead you into patterns which are not 'good Django'.
You might also find it useful to look at the source code of an existing Django forum app, such as this one:
https://bitbucket.org/slav0nic/djangobb/src/
They have just a single djangobb_forum app that does everything. I think this is entirely justified as a forum is a single complicated piece of functionality.
Note how their Profile model effectively extends the built-in Django auth.User model via a OneToOneField. It makes it easier to integrate with the built-in login and authentication system that way.
Start by integrating your legacy database and then build the Admin site. You'll see how the model information is available to the Admin app and every other app you write. The key is to import models in your apps.
I'm a beginner to Python and Django.
When starting a new project what do you do first before diving into the code?
For example, one could take the following steps:
Configure the settings.py file first
Configure models.py to lay out data structure
Create template files
Define the views/pages
Syncdb
etc
So my question is, what is a good workflow to get through the required steps for a Django application? This also serves as a checklist of things to do. In the definitive guide to Django, the author talks about approaching top down or bottom up. Can anyone expand further on this and perhaps share their process?
Thanks.
Follow the Agile approach. Finish one small case, from the start to the end. From the models to the tests to user experience. Then build on it. Iterate.
Thats the right way to software development.
To do it efficiently, you need: (don't bother right away, you will need it.)
Automated schema migration, automated build system, auto updating and deployment. - None of these, django has got anything to do with. Use pip, fabric, hudson, twill and south appropriately.
Take care not to over burden yourself with all these right away, particularly since you say, you are beginning.
the required steps for a Django application?
There are two required steps.
Write the settings. Write the urls.py
The rest of the steps are optional.
This also serves as a checklist of things to do.
Bad policy. You don't need a checklist of Django features. You need a collection of use cases or user stories which you must implement.
For some reason, you've omitted the two most important and valuable features of Django. Configure the default admin interface and write unit tests. The default admin interface is very high value. Unit testing is absolutely central.
You do it like this.
Gather use cases.
Prioritize the use cases.
Define the actors. The classes of actors becomes groups in the security model.
Define enough "applications" to satisfy the first release of use cases. Define the url structure. Cool URL's don't change.
Build the first use case: models (including security), admin, urls, tests, forms, views and templates. Note that these are the file names (models.py, admin.py, ...) except for templates. Also note that forms and admin should be defined in separate modules even though this isn't required. Also note that templates will be split between a generic templates directory for top-level stuff and application-specific templates.
Build the second use case: models (including security), admin, urls, tests, forms, views and templates.
...
n. Package for release. Tweak up the settings. Configure database and mod-wsgi
I personally can't make a template without writing the views (unless it's a photoshop draft) but in general that's the way I go after I have a plan.
What's extremely important for me is that I don't dive head-first into the code, and that I spend time mocking up the model structure based on the "screens" or "pages" that the user will see.
Once I have a user experience defined, I make sure the backend is robust enough to handle that experience. If I don't visualize the user experience, details get left out that are certainly accomplishable in the shell but not ideal for the website, default django admin, etc.
There are always tradeoffs between agile development and a huge spec: I think there's an important balance. Agile is good: there's no point planning every detail before writing your first line of code, as your needs will change by the time you get to the end. You don't know how your users will really use the site.
On the other hand, without a plan, you can end up with a messy foundation that affects all future code.
An educated guess is a good start. Don't think or assume too much, but definitely have a clear idea how your users will interact with your site for stage 1.
Always try to remember about a DRY rule. For example, why to write RequestContext every new view is defined, where you can simply write a function once, which will add it for you. Good description is here in another topic.
Try to keep a code written one way. Each time you upgrade a schema of your view, edit it in all already written views. That will help keep your code clear and save a lot time for you in future.
Generally good rule, and how do I write my applications is the rule of small steps. Start with writing a settings and urls, then add one model and one view. When it works, modify - add another models or another views. You won't even notice, when your project becomes bigger and bigger.
And the last useful rule for clarity of all the source. Keep files in folders. If you have two subsites based one one (for example "accounts" and "blogs") create two directories names the same. Remeber to put init.py file in each directory. It's really easy to forget. With this practice it's easy to write models and views dedicated to each category. By the way it's a good practice to keep urls like in a tree structure. Main urls.py should contain only links like this one:
(r'^accounts/', include('your_main_name.accounts.urls')),
and of course all media, static, css and so on. In accounts directory urls keep:
urlpatterns = patterns('your_main_name.accounts.views',
url(r'^$', 'index', name='index'),
)
with all views subdirectories.
Last one - keep code clear with actuall django version. Remeber, that the 3.0 release is comming soon.
Hope this will help.
I find that my process varies depending on a lot of variables, mainly whether I know something will work or if I'm experimenting and also whether I'm developing on my production server or in a development environment.
For example, I often do my development directly on the deployment server (most of my work is for intranet projects so there isn't any security risk, etc). But when I do this I really need to make sure the settings and urls are setup first and that gunicorn and nginx are configured.
If I know something should work, or am setting up a generic base set of code, sometimes I'll do all that coding for views and models before I even get enough setup to even run the development server. But when experimenting with new code I find it's good to be able to test every step of the way, so in that case you need your servers running.
In general I do settings, models, syncdb, views, urls, templates, collectstatic, graphics/aesthetics
In general I leave my base.html very plain until the everything else is working, then I add css/js etc.
I guess my point here is that there isn't really a wrong answer for how you do it, and there isn't even only one best practice (as far as I'm concerned). When you do more work, you'll find what you are comfortable with and it'll even vary from project to project.
Good luck, hopefully you learn to love django!
here is something I do in general,
configure basic settings
configure root url.py
configure settings, url.py for static (media) files
create model
sync db
write views (use simple template, if needed)
once you are done with back end implementation
think about UI
prepare styles, scripts
start working on template implementation
Are there any Django apps equivalent to Drupal's Views and CCK modules?
I find Django much more flexible and logically organized than Drupal. But I think Drupal's Views and CCK modules are killer apps. They let the webmaster very rapidly to build new data models and queries through GUI without touching the code. These modules are very useful for rapid application development. Do you know any similar apps in Django?
Django is a framework, this had been said, but if you look for functionality close to CCK, PINAX makes for python/Django, the equivalent of modules in Drupal, sort of ready to go modules, login/pass check, listing, input output, or CRUDs ect. But in no ways radio buttons and check boxes action programming. You will have to put your building blocks together and indulge some python programming. I found myself spending more time loading extra modules and themes tweaking in Drupal, than putting together a full blown site in Django, maybe because, I own many libraries wrote several times and improved over time, for cases encountered over and over. I focus only on new or cutting edge things. Both approaches are ok as long as you know what you want to go. The hard fact is that for a CMS that want to attract non programmers peoples if you want to build great sites, you have to be good in php/mysql and a good grip on css, and it kind of defeats the purpose. In France we say, "the best tool is the one you use every day". If you are frustrated with Drupal, learn Django, and in the same time needed to master Drupal, you will have a skill to write your own Drupal and others CMS, maybe. Good luck in your endeavor.
I don't think there is something similar and for a good reason. Django is a framework, while Drupal is a full scale CMS. One of the powerful things about Drupal, is how it handles content. Every piece of content is a node, and it lets developers make modules that can add to a node's functionality.
Django is a great tool to, but it's strength is more the ease of development, that lets you create applications very fast. That is what it was built for after all. It would be hard to make something CCK and Views like with Django, and I don't think it would make much sense either. I find that when developing with Django, you can very quickly create most CCK and views things, withe Django models and the orm. You can't do it in a GUI, but that was never the goal of Django. The admin interface also reflects that, as it is good mainly for one thing. Handling content, CRUD style operations. I guess that is why Satchmo created their own settings system for shops.
http://docs.djangoproject.com/en/dev/faq/general/#is-django-a-content-management-system-cms
...it doesn’t make much sense to compare Django to something like Drupal, because Django is something you use to create things like Drupal.
The lack of flexibility that you refer to is the price you pay for CCK and Views. I've used both Drupal and Django to complete major projects. You can use Drupal as a framework too, so in my opinion the two are absolutely comparable.
Django has a way better database abstraction than Drupal, follows more modern programming paradigms like OOP, MVC etc, is more flexible, and Python is just straight up superior to PHP.
...but I still usually use Drupal if either will do. It just gets the job done with less time spent, and works and performs well. Django has nothing like Views, and Drupal's form api is just light years ahead of Django's. Creating multi step ajax forms can be done without ever touching the markup or writing a single line of javascript in Drupal, and presenting dynamic lists to the user can be achieved without even leaving your browser.
Drupal has a much greater deployment rate than Django, not just because of PHP's popularity, but because it actually does some things really well.
My time is precious, and the end user doesn't give a damn as long as my system works.
Pretty new to this scene and trying to find some documentation to adopt best practices. We're building a fairly large content site which will consist of various media catalogs and I'm trying to find some comparable data / architectural models so that we can get a better idea of the approach we should use using a framework we've never made use of before. Any insight / help would be greatly appreciated!
"data / architectural models so that we can get a better idea of the approach we should use using a framework we've never made use of before"
Django imposes best practices on you. You don't have a lot of choices and can't make a lot of mistakes.
MVC (while a noble aspiration) is implemented as follows:
Data is defined in "models.py" files using the Django ORM models.
urls.py file maps URL to view function. Pick your URL's wisely.
View function does all processing, making use of models and methods in models
Presentation (via HTML templates) invoked by View function. Essentially no processing can be done in presentation, just lightweight iteration and decision-making
The model is defined for you. Just stick to what Django does naturally and you'll be happy.
Architecturally, you usually have a stack like this.
Apache does two things.
serves static content directly and immediately
hands dynamic URL to Django (via mod_python, mod_wsgi or mod_fastcgi). Django apps map URL to view functions (which access to database (via ORM/model) and display via templates.
Database used by Django view functions.
The architecture is well-defined for you. Just stick to what Django does naturally and you'll be happy.
Feel free to read the Django documentation. It's excellent; perhaps the best there is.
first, forget all MVC mantra. it's important to have a good layered structure, but MVC (as defined originally) isn't one, it was a modular structure, where each GUI module is split in these tree submodules. nothing to use on the web here.
in web development, it really pays to have a layered structure, where the most important layer is the storage/modelling one, which came to be called model layer. on top of that, you need a few other layers but they're really not anything like views and controllers in the GUI world.
the Django layers are roughly:
storage/modelling: models.py, obviously. try to put most of the 'working' concepts there. all the relationships, all the operations should be implemented here.
dispatching: mostly in urls.py. here you turn your URL scheme into code paths. think of it like a big switch() statement. try hard to have readable URLs, which map into user intentions. it will help a lot to add new functionality, or new ways to do the same things (like an AJAX UI later).
gathering: mostly the view functions, both yours and the prebuilt generic views. here you simply gather all the from the models to satisfy a user request. in surprisingly many cases, it just have to pick a single model instance, and everything else can be retrieved from relationships. for these URLs, a generic view is enough.
presentation: the templates. if the view gives you the data you need, it's simple enough to turn it into a webpage. it's here where you'll thank that the model classes have good accessors to get any kind of relevant data from any given instance.
To understand django fundementals and the django take on MVC, consult the following:
http://www.djangobook.com/
As a starting point to getting your hands dirty with ...
"...trying to find some comparable data / architectural models"
Here is a quick and dirty way to reverse engineer a database to get a models.py file,
which you can then inspect to see how django would handle it.
1.) get an er diagram that closely matches your target. For example something like this
http://www.databaseanswers.org/data_models/product_catalogs/index.htm
2.) create an sql script from the er diagram and create the database,
I suggest Postgre, as some MySQL
table type will not have forgien key constraints, but in a pinch MySQL or SQLITE
will do
3.) create and configure a django app to use that database. Then run:
python manage.py inspectdb
This will at least give you a models.py file which you can read to see how django attempts
to model it.
Note that the inspect command is intended to be a shortcut for dealing with legacy
database when developing in django, and as such is not perfect. Be sure to read the
following before attempting this:
http://docs.djangoproject.com/en/dev/ref/django-admin/#ref-django-admin