Where should utility functions live in Django? Functions like custom encrypting/decrypting a number, sending tweets, sending email, verifying object ownership, custom input validation, etc. Repetitive and custom stuff that I use in a number of places in my app. I'm definitely breaking DRY right now.
I saw some demos where functions were defined in models.py, although that didn't seem conceptually right to me. Should they go in a "utilities" app that gets imported into my project? If so, where do they go in the utilities app? The models.py file there?
Thanks for helping this n00b out.
UPDATE: Let me be even more specific. Say I need a function "light_encrypt(number)" which takes the param "number", multiplies it by 7, adds 10 and returns the result, and another function "light_decrypt(encr_number) which takes the param "encr_number", subtracts 10, divides by 7 and returns the results. Where in my Django tree would I put this? This is not middleware, right? As Felix suggests, do I create a python package and import it into the view where I need these functions?
Different question but same answer:
My usual layout for a django site is:
projects/
templates/
common/
local/
Where:
projects contains your main project and any others
common contains things you may share across sites, or are at least not project-specific, like if you need to download django-profile and django-registration rather than having it directly in python/site-packages
templates contains just that
local contains things that are going to be specific to the current machine, so that you can have properly separated data, like database location and password - I then soft-link the machine-specific versions (say "machine1-localconfig.py") to local/localconfig.py and then can "import localconfig" in settings.py
I generally put middleware that's project-specific inside a project, and middleware that's not project-specific in common/middleware/
make sure to add the templates directory to the right place in settings (or most probably, localconfig.py and then import it in settings), and makse sure to add the projects, common, and local directories to your PYTHONPATH.
OK, after reading the comments and answer here I've decided to create a directory called "common/util/" inside my project directory. Within that I have a file "__ init__.py" where I have my little helper functions.
I guess if the file gets too big, I'll then split out the functions into individual .py files in common. So now, my project structure looks like this. Please correct if I'm making any poor choices, I'm early enough in development that I can fix it now while it is still easy to do so!
myproject/ (Django project)
common/
util/
__init__.py (helper functions)
middleware/ (custom middleware)
templates/ (project templates)
myapp/
fixtures/ (initial data to load)
migrations/ (south migrations)
urls/
views/
admin.py
forms.py
models.py
test.py
public/ (static stuff not served by Django)
media/
css/
img/
js/
lib/
Here is another way to do it:
If the utility functions can be a stand-alone module
and you are using a virtualenv environment for your django app
then you can bundle the functionality as a package and install it in your virtualenv.
This makes it easy to import where ever you need it in your django app.
It depends whether the functions are project or app specific.
The other answers are already addressing where to place it for project specific functions. More precisely, in a folder called common in the root of the project.
If we are talking about app specific, then I'd simply place it inside of the app in a file named utils.py, like
myproject/ (Django project)
common/
util/
__init.py__ (project app specific functions)
myapp/
migrations/ (myapp migrations)
utils.py (myapp specific functions)
views.py
admin.py
forms.py
models.py
test.py
Related
Where should utility functions live in Django? Functions like custom encrypting/decrypting a number, sending tweets, sending email, verifying object ownership, custom input validation, etc. Repetitive and custom stuff that I use in a number of places in my app. I'm definitely breaking DRY right now.
I saw some demos where functions were defined in models.py, although that didn't seem conceptually right to me. Should they go in a "utilities" app that gets imported into my project? If so, where do they go in the utilities app? The models.py file there?
Thanks for helping this n00b out.
UPDATE: Let me be even more specific. Say I need a function "light_encrypt(number)" which takes the param "number", multiplies it by 7, adds 10 and returns the result, and another function "light_decrypt(encr_number) which takes the param "encr_number", subtracts 10, divides by 7 and returns the results. Where in my Django tree would I put this? This is not middleware, right? As Felix suggests, do I create a python package and import it into the view where I need these functions?
Different question but same answer:
My usual layout for a django site is:
projects/
templates/
common/
local/
Where:
projects contains your main project and any others
common contains things you may share across sites, or are at least not project-specific, like if you need to download django-profile and django-registration rather than having it directly in python/site-packages
templates contains just that
local contains things that are going to be specific to the current machine, so that you can have properly separated data, like database location and password - I then soft-link the machine-specific versions (say "machine1-localconfig.py") to local/localconfig.py and then can "import localconfig" in settings.py
I generally put middleware that's project-specific inside a project, and middleware that's not project-specific in common/middleware/
make sure to add the templates directory to the right place in settings (or most probably, localconfig.py and then import it in settings), and makse sure to add the projects, common, and local directories to your PYTHONPATH.
OK, after reading the comments and answer here I've decided to create a directory called "common/util/" inside my project directory. Within that I have a file "__ init__.py" where I have my little helper functions.
I guess if the file gets too big, I'll then split out the functions into individual .py files in common. So now, my project structure looks like this. Please correct if I'm making any poor choices, I'm early enough in development that I can fix it now while it is still easy to do so!
myproject/ (Django project)
common/
util/
__init__.py (helper functions)
middleware/ (custom middleware)
templates/ (project templates)
myapp/
fixtures/ (initial data to load)
migrations/ (south migrations)
urls/
views/
admin.py
forms.py
models.py
test.py
public/ (static stuff not served by Django)
media/
css/
img/
js/
lib/
Here is another way to do it:
If the utility functions can be a stand-alone module
and you are using a virtualenv environment for your django app
then you can bundle the functionality as a package and install it in your virtualenv.
This makes it easy to import where ever you need it in your django app.
It depends whether the functions are project or app specific.
The other answers are already addressing where to place it for project specific functions. More precisely, in a folder called common in the root of the project.
If we are talking about app specific, then I'd simply place it inside of the app in a file named utils.py, like
myproject/ (Django project)
common/
util/
__init.py__ (project app specific functions)
myapp/
migrations/ (myapp migrations)
utils.py (myapp specific functions)
views.py
admin.py
forms.py
models.py
test.py
I know this question has been asked in a lot of places, but none of them really seem to answer my question.
I am creating a standalone application in python, my current project structure is like so:
Project/
utils/
log.py
api/
get_data_from_api.py
main.py
I would like to have a set up similar to django, in which I can refer to any file by using a Project.<package> syntax.
For example, if I wanted to access my log module from my get_data_from_api module, I could do something like this:
get_data_from_api.py
import Project.utils.log
# Rest of code goes here...
However, I can't seem to get that to work, even when I added an __init__.py file in the root directory.
I read somewhere that I should modify my PYTHONPATH, but I would like to prevent that, if possible. Additionally, django seemed to pull it off, as I couldn't find any PYTHONPATH modification code in there.
I really appreciate the help!
Post Note: Where would tests fit in this file structure? I would like them to be separate, but also have access to the entire project really easily.
You need to add __init__.py to every directory that you want to treat as a Python package, not just to the root directory.
Post Note: Where would tests fit in this file structure? I would like
them to be separate, but also have access to the entire project really
easily.
You could place your tests in the Project directory itself, parallel to api and utils.
I ended up using a file structure like the following:
src/
main.py
project/
api/
get_data_from_api.py
util/
log.py
test/
That way, because I am running the main.py file, from anywhere in the program, I can just do import project.<package>.<module>. For example, I can just do this:
get_data_from_api.py
import project.util.log
# Rest of code goes here...
And everything works!
If I'm completely shooting in the dark please let me know, I'd much rather be wrong now then hundreds of hours into the project!
It may be helpful if you look at these three tutorials. It may not completely solve what you imagined, but it will certainly give you an idea or background on how to do it.
How to structure a Flask-RESTPlus web service for production builds - GitHub Code
Structuring Your Project - GitHub Code
Clean architectures in Python GitHub Code
They have already written about Django, so you can see the details at: Best practice for Django project working directory structure.
In Django 1.4 and above :
There is a new file called app.py in every django application. It defines the scope of the app and some initials required when loaded.
Why don't they use __init__.py for the purpose? Any advantage over __init__.py approach? Can you link to some official documentation for the same?
Even though the details are wrong (there's no app.py in new Django projects), the question is still valid.
__init__.py is imported implicitly when importing a sub-module. So if something in __init__.py executes automatically with side effects, you might run into unintended consequences. Doing everything in app.py incurs a longer import, but separates package init from app init logic.
As all the links you have provided clearly show, this is nothing at all to do with Django itself, but a convention applied by the third-party app django-oscar.
Sometimes when people are learning a new technology they copy the file names as they saw on a tutorial. Some even continue using those either out of habit, out of thinking that is what they need to do, or they just like it. That might be why you see app.py so often.
Here is an example of one such tutorial https://realpython.com/blog/python/flask-by-example-part-1-project-setup/
there they are building "flask by example" and the instructions give you this command
touch app.py .gitignore README.md requirements.txt
So of course now you have that file. This is the same for javascript projects where they might list scripts.js or css tutorials where they use styles.css.
I'm running Satchmo. There are many apps and I've changed some of the source in the Product app.
So my question is how can I override this properly because the change is site specific. Do I have to copy over the whole Satchmo framework and put it into my project or can I just copy one of the apps out and place it in say Satchmo>App>Products? (Kinda like with templates)
Thanks
What I have done which works is to copy the application that I have changed. In this case satchmo\apps\product.
I copied the app into my project folder
Amended my setting.py INSTALLED_APPS from 'product', to 'myproject.product',
This now carries the changes I've made to this app for this project only and leaves the original product app untouched and still able to be read normally from other projects.
When you add a 'Django App' to INSTALLED_APPS in your settings.py file, you're telling Django that there exists an importable python module with that name on your "python path". You can view your python path by viewing the contents of the list stored at sys.path.
Whenever Python (and in this case Django) attempts to import a module it checks each of the directories listed in sys.path in order, when it finds a module matching the given name it stops.
The solution to your problem then is too place your customized Django Apps, e.g., the Satchmo product module, into a location in your python path which will be checked before the "real" Satchmo product module.
Because I don't know how you have your directory structure laid out I'm basically making a guess here, but in your case, it sounds like you have the Satchmo apps living somewhere like /satchmo/apps/ and your project at /my_custom_path/my_project/. In which case you might want to add your customized product module to /my_custom_path/my_project/product/. Because the path at which your Django settings.py file lives is always checked first, that should mean that your customized product module will be found first and imported instead of the built in Satchmo one.
FYI: To check and see the order in which your Satchmo installation is checking directories for modules run python manage.py shell and then in the prompt do import sys; print sys.path.
Normally, I'd say the best thing is to fork Satchmo and keep a copy with your changes.
If you are willing to play with the python path, make sure that your app's directory appears before the other (default) directory. From my tests, if you have two apps/modules with identical names, the first one found is used.
Am new to django and was looking for advice where to place my shared library. Am planning on creating classes that I want to use across all my apps within the project. Where would be the best location to place them?
e.x abstract models
regards,
We usually set our projects up like this:
/site/
__init__.py
manage.py
settings.py
urls.py
/apps/
__init__.py
/appA/
__init__.py
/appB/
__init__.py
/lib/
__init__.py
/django-lib/
__init__.py
/shared-lib/
__init__.py
Just make sure your site directory is on your python path:
import sys
sys.path.append('/path/to/site/')
Also, ensure that an init.py exists in site, apps, and lib so they can be treated as modules using dot notation imports (import site.lib.shared-lib)
Edit:
In answer to your question regarding your python path, it all has to do with where your 'manage.py' or equivalent file resides. If it is under the /site/ directory (next to apps and lib) then the PYTHONPATH should be fine.
You need to make sure that every directory contains an empty file called __init__.py. This tells Python to treat that directory as a module. See the new and improved ASCII art above.
What I learned from Two Scoops of Django is that you put the shared code inside a Django app that you create on your own called core.
To use the core app then is similar to how you cross-use classes from other apps.
To learn more, go to Chapter 28 of the book titled: "What About Those Random Utilities?"
If the shared library is going to be used in others projects as well you can make a installer using distutils.
But if it's only for the apps in the project you can take AntonioP's answer.
Remember that the root of your project (folder that contains manage.py) is always in your PYTHON_PATH when you run your django project, so you can create a folder deps or dependencies or extra or anything you like it that contains your shared libraries.
Wherever you want, you can import them if they are in the PYTHON_PATH.