Am new to django and was looking for advice where to place my shared library. Am planning on creating classes that I want to use across all my apps within the project. Where would be the best location to place them?
e.x abstract models
regards,
We usually set our projects up like this:
/site/
__init__.py
manage.py
settings.py
urls.py
/apps/
__init__.py
/appA/
__init__.py
/appB/
__init__.py
/lib/
__init__.py
/django-lib/
__init__.py
/shared-lib/
__init__.py
Just make sure your site directory is on your python path:
import sys
sys.path.append('/path/to/site/')
Also, ensure that an init.py exists in site, apps, and lib so they can be treated as modules using dot notation imports (import site.lib.shared-lib)
Edit:
In answer to your question regarding your python path, it all has to do with where your 'manage.py' or equivalent file resides. If it is under the /site/ directory (next to apps and lib) then the PYTHONPATH should be fine.
You need to make sure that every directory contains an empty file called __init__.py. This tells Python to treat that directory as a module. See the new and improved ASCII art above.
What I learned from Two Scoops of Django is that you put the shared code inside a Django app that you create on your own called core.
To use the core app then is similar to how you cross-use classes from other apps.
To learn more, go to Chapter 28 of the book titled: "What About Those Random Utilities?"
If the shared library is going to be used in others projects as well you can make a installer using distutils.
But if it's only for the apps in the project you can take AntonioP's answer.
Remember that the root of your project (folder that contains manage.py) is always in your PYTHON_PATH when you run your django project, so you can create a folder deps or dependencies or extra or anything you like it that contains your shared libraries.
Wherever you want, you can import them if they are in the PYTHON_PATH.
Related
Where should utility functions live in Django? Functions like custom encrypting/decrypting a number, sending tweets, sending email, verifying object ownership, custom input validation, etc. Repetitive and custom stuff that I use in a number of places in my app. I'm definitely breaking DRY right now.
I saw some demos where functions were defined in models.py, although that didn't seem conceptually right to me. Should they go in a "utilities" app that gets imported into my project? If so, where do they go in the utilities app? The models.py file there?
Thanks for helping this n00b out.
UPDATE: Let me be even more specific. Say I need a function "light_encrypt(number)" which takes the param "number", multiplies it by 7, adds 10 and returns the result, and another function "light_decrypt(encr_number) which takes the param "encr_number", subtracts 10, divides by 7 and returns the results. Where in my Django tree would I put this? This is not middleware, right? As Felix suggests, do I create a python package and import it into the view where I need these functions?
Different question but same answer:
My usual layout for a django site is:
projects/
templates/
common/
local/
Where:
projects contains your main project and any others
common contains things you may share across sites, or are at least not project-specific, like if you need to download django-profile and django-registration rather than having it directly in python/site-packages
templates contains just that
local contains things that are going to be specific to the current machine, so that you can have properly separated data, like database location and password - I then soft-link the machine-specific versions (say "machine1-localconfig.py") to local/localconfig.py and then can "import localconfig" in settings.py
I generally put middleware that's project-specific inside a project, and middleware that's not project-specific in common/middleware/
make sure to add the templates directory to the right place in settings (or most probably, localconfig.py and then import it in settings), and makse sure to add the projects, common, and local directories to your PYTHONPATH.
OK, after reading the comments and answer here I've decided to create a directory called "common/util/" inside my project directory. Within that I have a file "__ init__.py" where I have my little helper functions.
I guess if the file gets too big, I'll then split out the functions into individual .py files in common. So now, my project structure looks like this. Please correct if I'm making any poor choices, I'm early enough in development that I can fix it now while it is still easy to do so!
myproject/ (Django project)
common/
util/
__init__.py (helper functions)
middleware/ (custom middleware)
templates/ (project templates)
myapp/
fixtures/ (initial data to load)
migrations/ (south migrations)
urls/
views/
admin.py
forms.py
models.py
test.py
public/ (static stuff not served by Django)
media/
css/
img/
js/
lib/
Here is another way to do it:
If the utility functions can be a stand-alone module
and you are using a virtualenv environment for your django app
then you can bundle the functionality as a package and install it in your virtualenv.
This makes it easy to import where ever you need it in your django app.
It depends whether the functions are project or app specific.
The other answers are already addressing where to place it for project specific functions. More precisely, in a folder called common in the root of the project.
If we are talking about app specific, then I'd simply place it inside of the app in a file named utils.py, like
myproject/ (Django project)
common/
util/
__init.py__ (project app specific functions)
myapp/
migrations/ (myapp migrations)
utils.py (myapp specific functions)
views.py
admin.py
forms.py
models.py
test.py
tl;dr I have a directory of common files outside of my various project directories. What is the pythonic way of using/importing these common files inside my projects, and for building them into an output directory.
Background:
I'm in school and taking a data structures class that uses Python as the language. I'm learning the languages as I take the class but have having some issues trying to maintain a shared code base.
In all of the other languages I've used, both compiled and interpreted, there has been a fairly intuitive way of being able to keep shared modules separate from the code that is using them so that updating a shared module doesn't require updates to the calling code.
This is how I initially had my directory structure organized.
/.../Projects
Assignment_1
__init__.py
classA.py
classB.py
Assignment_2
__init__.pu
classC.py
(etc)
After realizing that much of the functionality of classA and classB would be required later on, I reorganized to this:
/.../Projects
Common
Sorters
__init__.py
BubbleSort.py
MergeSort.py
__init__.py
SimpleProfiler.py
Assignment_1
__init__.py
main.py
Assignment_2
__init__.py
main.py
My issue is that I can't find a good way of importing things like SimpleProfiler or MergeSort from main,py. Right now I'm manually coping all of the Common files into each assignment, which is bad.
I understand that one possible solution is to update the path to include the common folder form within each main.py file, but I've also read that this is very hacky and isn't encouraged.
Another Stackoverflow answer to a similar question suggested that the user structure everything under one large project. I tried this but still couldn't import modules from one sibling into another sibling.
My other issue is how to package everything together when submitting the assignment. In other languages it was easy to implement a build script that would scan the main project for any imports, then copy (flatten) those imported files into a single output directory which I could then compress and submit for grading. I'm using PyCharm, but can't seem to find a way to reference the imports as part of the build process. Is there any kind of script for this? Whatever the solution is, I need to be able to submit the project in such a way that all the instructor has to do is call a single python file (such as main.py)
This issue isn't unique to a school setting, but seems universal to most programming projects. So, what is the pythonic way of managing a shared code base and for building that shared code into a final project?
[Disclaimer: I think it is better to use PYTHONPATH environment variable]
I think of two very similar alternatives:
/.../Projects
Common
Sorters
__init__.py
BubbleSort.py
MergeSort.py
__init__.py
SimpleProfiler.py
assignment_1.py
assignment_2.py
If you use, from assignment_1.py, the following import: from Common.Sorters.BubbleSort import bubble_sort. This is because, by default, PYTHONPATH considers the current path as a valid PYTHONPATH. This is assuming that you are invoking the scripts assignment_* directly.
The other alternative would be:
/.../Projects
Common
Sorters
__init__.py
BubbleSort.py
MergeSort.py
__init__.py
SimpleProfiler.py
Assignment_1
__init__.py
__main__.py
Assignment_2
__init__.py
__main__.py
And invoking the assignments like so: python -m Assignment_1 (from the Projects folder). By default, "executing" a module like that will load its __main__.py code. (This is not a rigurous explanation, although the official one is a bit short).
It works for the same reasons as before: Python interpreter will consider the current path as a valid PYTHONPATH.
Try setting PYTHONPATH environment variable to your directory.
Python first searches for files being imported in sys.path, and the first directory in sys.path is the current directory. PYTHONPATH is the next where python will look for files.
On the minimum end, make a PyCharm run configuration that sets your PYTHONPATH before executing and include the other directory. That way you don't need to do a sys.path call in your code.
Closer to the "perfect" end, make your other directory into a Python package with a setup.py. Then, using the interpreter from your project, do "python path/to/other/dir/setup.py develop" to bring your separately-developed package into the consuming project.
As a beginning programmer (I'm rather doing scripting), I'm struggling with my growing collection of scripts and modules.
Currently, small scripts are in a general folder that is added to my pythonpath. Specific projects get their own subfolder, but more and more I try to write the general parts of those projects as modules that can be used by other projects. But I cannot just import them unless this subfolder is also in my pythonpath.
I don't know how to organise all this. I will be happy to get your hints and recommendations on organising (python) code. Thanks!
Be sure to read about Packages, specifically the part about creating a file named __init__.py in directories you want to import from.
I have a git repository named MyCompany in my home directory. There's a link from /usr/local/lib/python2.7/site-packages/ to it. Inside MyCompany are package1, package2, package3, etc. In my code, I write import MyCompany.package1.modulefoo. Python looks in site-packages and finds MyCompany. Then it finds the package1 subdirectory with an __init__.py file in it - yay, a package! Then it imports the modulefoo.py file in that directory, and I'm off and running.
General-purpose custom modules should go to ~/.local/lib/pythonX.Y/site-packages, or to /usr/local/lib/pythonX.Y/site-packages if they should be available to everyone. Both paths are automatically available in your $PYTHONPATH. (The former is available since Python 2.6 – see PEP 370 – Per user site-packages directory.)
Where should utility functions live in Django? Functions like custom encrypting/decrypting a number, sending tweets, sending email, verifying object ownership, custom input validation, etc. Repetitive and custom stuff that I use in a number of places in my app. I'm definitely breaking DRY right now.
I saw some demos where functions were defined in models.py, although that didn't seem conceptually right to me. Should they go in a "utilities" app that gets imported into my project? If so, where do they go in the utilities app? The models.py file there?
Thanks for helping this n00b out.
UPDATE: Let me be even more specific. Say I need a function "light_encrypt(number)" which takes the param "number", multiplies it by 7, adds 10 and returns the result, and another function "light_decrypt(encr_number) which takes the param "encr_number", subtracts 10, divides by 7 and returns the results. Where in my Django tree would I put this? This is not middleware, right? As Felix suggests, do I create a python package and import it into the view where I need these functions?
Different question but same answer:
My usual layout for a django site is:
projects/
templates/
common/
local/
Where:
projects contains your main project and any others
common contains things you may share across sites, or are at least not project-specific, like if you need to download django-profile and django-registration rather than having it directly in python/site-packages
templates contains just that
local contains things that are going to be specific to the current machine, so that you can have properly separated data, like database location and password - I then soft-link the machine-specific versions (say "machine1-localconfig.py") to local/localconfig.py and then can "import localconfig" in settings.py
I generally put middleware that's project-specific inside a project, and middleware that's not project-specific in common/middleware/
make sure to add the templates directory to the right place in settings (or most probably, localconfig.py and then import it in settings), and makse sure to add the projects, common, and local directories to your PYTHONPATH.
OK, after reading the comments and answer here I've decided to create a directory called "common/util/" inside my project directory. Within that I have a file "__ init__.py" where I have my little helper functions.
I guess if the file gets too big, I'll then split out the functions into individual .py files in common. So now, my project structure looks like this. Please correct if I'm making any poor choices, I'm early enough in development that I can fix it now while it is still easy to do so!
myproject/ (Django project)
common/
util/
__init__.py (helper functions)
middleware/ (custom middleware)
templates/ (project templates)
myapp/
fixtures/ (initial data to load)
migrations/ (south migrations)
urls/
views/
admin.py
forms.py
models.py
test.py
public/ (static stuff not served by Django)
media/
css/
img/
js/
lib/
Here is another way to do it:
If the utility functions can be a stand-alone module
and you are using a virtualenv environment for your django app
then you can bundle the functionality as a package and install it in your virtualenv.
This makes it easy to import where ever you need it in your django app.
It depends whether the functions are project or app specific.
The other answers are already addressing where to place it for project specific functions. More precisely, in a folder called common in the root of the project.
If we are talking about app specific, then I'd simply place it inside of the app in a file named utils.py, like
myproject/ (Django project)
common/
util/
__init.py__ (project app specific functions)
myapp/
migrations/ (myapp migrations)
utils.py (myapp specific functions)
views.py
admin.py
forms.py
models.py
test.py
Why aren't they simply directories? Any good advice says to keep as much as possible in the apps and not to couple them to the project. The very ability to import an app as project.application discourages this. Why does django-admin.py create the __init__.py at all? The project is perfectly useful without it. What is the justification?
We have a single project that we "subclass" of sorts for other projects. So we have other projects that import stuff from the main project. I guess for us it provides the common namespace that contains all the other apps.
We could move to a package with all our apps in it separate from the projects i guess. Our system has grown rather than been planned.
So I guess my answer is, it provides a good root namespace. (for our needs) :)
The core of a project is a settings.py and a root urls.py. Both of those are Python modules, thus they need to be importable somehow. You can put the project directory directly on the Python path and thus make them importable as top-level modules, but that's arguably even worse practice. Better to have the project be a package and the settings and urls be modules within it.
There isn't a requirement that apps be inside the project's namespace, to my knowledge. Just that they be on the $PYTHONPATH. As such, they are usable by any other code on the system which shares the same PYTHONPATH.
I think the idea is that you can reuse the applications but you don't need to move them from the project where they were initially created. If the project weren't a package you would need to copy/move the application you want to reuse to a python package. Because of that being the project itself a proper python package you can reuse the applications in it without moving the applications to other place.