I want to use a couple of third party django packages for my application. So, I want to install each package locally for that application (or my whole project).
The python custom installation instructions look a bit scary - how do I do this as simply as possible?
You can just put the library module into your project folder because your project folder will be in the PYTHONPATH automatically when running via manage.py runserver or your wsgi script will point to it when running on production.
Usually all python packages are packaged like this:
package directory
module directory
... other files/dirs like README, Manifest and so on
What need to be in your Django project folder is only the module directory part from the example above, not the rest of the package.
Use virtualenv.
That is by far the best solution for installing custom packages for each project.
Just use --home - nothing scary about it.
Don't forget to make your PYTHONPATH point there, though.
Related
What is my goal
I want to publish a PyPI package where the code within this package will be a Django application (migrations + templates folder, views, models, etc...)
I created the project with setup.py which includes the necessary data so that the build module of python will build the PyPI package smoothly, with version management.
This is the repository for further look: github-repo
What is the problem
The python -m build command does not take the templates directory within the package.
It leads to a broken package of a Django application, because when you attempt to install the (package), it does not bring the templates. It is testable with pip install rlocker-expiryaddon
What did I try
I know that mentioning a file inside MANIFEST.in will be responsible to grab the files in the build process. (?)
So I tried to inject the following line:
recursive-include templates *.html schema.js
Yet, no luck.
I know that there are a lot of open-source libraries like the djangorestframework that is easy downloadable via pip and it brings it's templates directory to the system interpreter as expected. But for me, I am not sure what is wrong.
Thanks for your help!
I am currently in the process of developing a module that I am using as a library to be imported in another project. I need a sane way that I can install this module in the python site packages in such a way that I don't need to reinstall it everytime I make changes to it. Currently, I am using sudo pip install --force-reinstall {BASE_FOLDER_FOR_MODULE}, but need to re run this command everytime I make any changes to the module code.
The little bit of info I've been able to find on the subject seems to indicate that while I can symlink the base folder for the module in the site-packages folder for my other project, that this may not necessarily be a good way to do it. Is symlinking the folder bad, and if so why? Is there an alternate (better) option?
Thanks
If the library is still under active development then consider adding it to environment variable PYTHONPATH. Directories in PYTHONPATH are appended to sys.path and are searched last when trying to find a module. Using PYTHONPATH means you only have to make minimal changes (set it in a config file source file or .bashrc file) for things to work. Once the library is finalised you can install it to the site-packages directory and remove it from PYTHONPATH.
I have a django project that uses a lot of 3rd party apps, so wanted to decide out of the two approaches to manage my situation :
I can use [ virtualenv + pip ] along with pip freeze as requirements file to manage my project dependencies.
I don't have to worry about the apps, but can't have that committed with my code to svn.
I can have a lib folder in my svn structure and have my apps sit there and add that to sys.path
This way, my dependencies can be committed to svn, but I have to manage sys.path
Which way should I proceed ?
What are the pros and cons of each approach ?
Update:
Method1 Disadvantage : Difficult to work with appengine.
This has been unanswered question (at least to me) so far. There're some discussion on this recently:-
https://plus.google.com/u/0/104537541227697934010/posts/a4kJ9e1UUqE
Ian Bicking said this in the comment:-
I do think we can do something that incorporates both systems. I
posted a recipe for handling this earlier, for instance (I suppose I
should clean it up and repost it). You can handle libraries in a very
similar way in Python, while still using the tools we have to manage
those libraries. You can do that, but it's not at all obvious how to
do that, so people tend to rely on things like reinstalling packages
at deploy time.
http://tarekziade.wordpress.com/2012/02/10/defining-a-wsgi-app-deployment-standard/
The first approach seem the most common among python devs. When I first started doing development in Django, it feels a bit weird since when doing PHP, it quite common to check third party lib into the project repo but as Ian Bicking said in the linked post, PHP style deployment leaves out thing such non-portable library. You don't want to package things such as mysqldb or PIL into your project which better being handled by tools like Pip or distribute.
So this is what I'm using currently.
All projects will have virtualenv directory at the project root. We name it as .env and ignore it in vcs. The first thing dev did when to start doing development is to initialize this virtualenv and install all requirements specified in requirements.txt file. I prefer having virtualenv inside project dir so that it obvious to developer rather than having it in some other place such as $HOME/.virtualenv and then doing source $HOME/virtualenv/project_name/bin/activate to activate the environment. Instead developer interact with the virtualenv by invoking the env executable directly from project root, such as:-
.env/bin/python
.env/bin/python manage.py runserver
To deploy, we have a fabric script that will first export our project directory together with the .env directory into a tarball, then copy the tarball to live server, untar it deployment dir and do some other tasks like restarting the server etc. When we untar the tarball on live server, the fabric script make sure to run virtualenv once again so that all the shebang path in .env/bin get fixed. This mean we don't have to reinstall dependencies again on live server. The fabric workflow for deployment will look like:-
fab create_release:1.1 # create release-1.1.tar.gz
fab deploy:1.1 # copy release-1.1.tar.gz to live server and do the deployment tasks
fab deploy:1.1,reset_env=1 # same as above but recreate virtualenv and re-install all dependencies
fab deploy:1.1,update_pkg=1 # only reinstall deps but do not destroy previous virtualenv like above
We also do not install project src into virtualenv using setup.py but instead add path to it to sys.path. So when deploying under mod_wsgi, we have to specify 2 paths in our vhost config for mod_wsgi, something like:-
WSGIDaemonProcess project1 user=joe group=joe processes=1 threads=25 python-path=/path/to/project1/.env/lib/python2.6/site-packages:/path/to/project1/src
In short:
We still use pip+virtualenv to manage dependencies.
We don't have to reinstall requirements when deploying.
We have to maintain path into sys.path a bit.
Virtualenv and pip are fantastic for working on multiple django projects on one machine. However, if you only have one project that you are editing, it is not necessary to use virtualenv.
Is there a way to import all the modules in the django project itself instead of setting up again and again in all the systems.
I would have used gem freeze or something like that in a rails project.
There's a bit of a terminology confusion here: "modules" refers to individual .py files within a Python package. And importing is what you do within code, to bring modules into the current namespace.
I think what you're asking is how to install Python packages on deployment. The answer to that is, use pip with the freeze command, in conjunction with virtualenv.
First of all you should be using virtualenv. This way the python path of your django app only has stuff relevant to it. This also allows you to run several separate django / python apps on the same server without them bumping heads.
When you have a virtualenv with your django app running in it you need to generate a requirements file.
pip freeze -E virtualenv_path > stable-req.txt
You can then use this file to generate a bundle.
pip bundle mybundle.bundle -r stable-req.txt
This bundle can then be used to deploy with.
Instead of
gem freeze
Try to use
pip bundle
I found this solution here: Django equivalent to "rake rails:freeze:gems" and "rake gems:unpack"
Is there a reason you want to import all modules?
It's a good practice to import only those modules, classes etc. which are needed...
I want to distribute some python code, with a few external dependencies, to machines with only core python installed (and users that unfamiliar with easy_install etc.).
I was wondering if perhaps virtualenv can be used for this purpose? I should be able to write some bash scripts that trigger the virtualenv (with the suitable packages) and then run my code.. but this seems somewhat messy, and I'm wondering if I'm re-inventing the wheel?
Are there any simple solutions to distributing python code with dependencies, that ideally doesn't require sudo on client machines?
Buildout - http://pypi.python.org/pypi/zc.buildout
As sample look at my clean project: http://hg.jackleo.info/hyde-0.5.3-buildout-enviroment/src its only 2 files that do the magic, more over Makefile is optional but then you'll need bootstrap.py (Make file downloads it, but it runs only on Linux). buildout.cfg is the main file where you write dependency's and configuration how project is laid down.
To get bootstrap.py just download from http://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py
Then run python bootstap.py and bin/buildout. I do not recommend to install buildout locally although it is possible, just use the one bootstrap downloads.
I must admit that buildout is not the easiest solution but its really powerful. So learning is worth time.
UPDATE 2014-05-30
Since It was recently up-voted and used as an answer (probably), I wan to notify of few changes.
First of - buildout is now downloaded from github https://raw.githubusercontent.com/buildout/buildout/master/bootstrap/bootstrap.py
That hyde project would probably fail due to buildout 2 breaking changes.
Here you can find better samples http://www.buildout.org/en/latest/docs/index.html also I want to suggest to look at "collection of links related to Buildout" part, it might contain info for your project.
Secondly I am personally more in favor of setup.py script that can be installed using python. More about the egg structure can be found here http://peak.telecommunity.com/DevCenter/PythonEggs and if that looks too scary - look up google (query for python egg). It's actually more simple in my opinion than buildout (definitely easier to debug) as well as it is probably more useful since it can be distributed more easily and installed anywhere with a help of virtualenv or globally where with buildout you have to provide all of the building scripts with the source all of the time.
You can use a tool like PyInstaller for this purpose. Your application will appear as a single executable on all platforms, and include dependencies. The user doesn't even need Python installed!
See as an example my logview package, which has dependencies on PyQt4 and ZeroMQ and includes distributions for Linux, Mac OSX and Windows all created using PyInstaller.
You don't want to distribute your virtualenv, if that's what you're asking. But you can use pip to create a requirements file - typically called requirements.txt - and tell your users to create a virtualenv then run pip install -r requirements.txt, which will install all the dependencies for them.
See the pip docs for a description of the requirements file format, and the Pinax project for an example of a project that does this very well.