Is there a way to import all the modules in the django project itself instead of setting up again and again in all the systems.
I would have used gem freeze or something like that in a rails project.
There's a bit of a terminology confusion here: "modules" refers to individual .py files within a Python package. And importing is what you do within code, to bring modules into the current namespace.
I think what you're asking is how to install Python packages on deployment. The answer to that is, use pip with the freeze command, in conjunction with virtualenv.
First of all you should be using virtualenv. This way the python path of your django app only has stuff relevant to it. This also allows you to run several separate django / python apps on the same server without them bumping heads.
When you have a virtualenv with your django app running in it you need to generate a requirements file.
pip freeze -E virtualenv_path > stable-req.txt
You can then use this file to generate a bundle.
pip bundle mybundle.bundle -r stable-req.txt
This bundle can then be used to deploy with.
Instead of
gem freeze
Try to use
pip bundle
I found this solution here: Django equivalent to "rake rails:freeze:gems" and "rake gems:unpack"
Is there a reason you want to import all modules?
It's a good practice to import only those modules, classes etc. which are needed...
Related
I'm new to Python, so I think my question is very fundamental and is asked a few times before but I cannot really find something (maybe because I do not really know how to search for that problem).
I installed a module in Python (reportlab). Now I wanted to modify a python script in that module but it seems that the python interpreter does not notice the updates in the script. Ironically the import is successful although Python actually should not find that package because I deleted it before. Does Python uses something like a Cache or any other storage for the modules? How can I edit modules and use those updated scripts?
From what you are saying, you downloaded a package and installed it using either a local pip or setup.py. When you do so, it copies all the files into your python package directory. So after an install, you can delete the source folder because python is not looking here.
If you want to be able to modify, edit, something and see changes, you have to install it in editable mode. Inside the main folder do:
python setup.py develop
or
pip install -e .
This will create a symbolic link to you python package repository. You will be able to modify sources.
Careful for the changes to be effective, you have to restart your python interpreter. You cannot just import again the module or whatever else.
I just started learning Pyramid using the official documentation and I found it very cool so far.
Unfortunately, while the basic one-file app is really simple and straight, I'm having an hard time trying to understand how a "serious app", generated using the pcreate scaffolding command (alchemy in my case), is supposed to be handled.
For example:
is setup.py mandatory or can I just use requirements.txt as I'm used to do with Django in order to install dependencies?
if I have to rely on setup.py am I supposed to execute python setup.py develop each time I create/delete a new file (since I saw them listed in SOURCES.txt)?
In settings.ini how "use" (under [app:main]) works? (can I "bypass" the egg-info which is it pointing to and "bootrsapping" the app in an alternative way?)
There are several tutorials that addresses all these topics, and provide references to further relevant reading for each step along the way. I suggest starting with the Quick Tutorial.
To answer your bullet points in order:
setup.py is a standard for installing Python packages and dependencies and testing your app.
If you need to install more packages as a result of your changes, then yes. EDIT: It is now recommended to use pip install -e . for installing more packages. Also while developing, you can use pserve development.ini --reload which will monitor file changes and restart the server for you.
For more information about the meaning of the use = egg:MyProject, see Entry Points and PasteDeploy .ini Files. There are many ways to configure Pyramid apps including Application Configuration and Advanced Configuration.
You chose Pyramid, the best Python web microframework, a very choice! Here are some pointers for further insight. Actually your question is not specific to Pyramid, but generally how Python packages and applications generally work.
is setup.py mandatory or can I just use requirements.txt as I'm used to do with Django in order to install dependencies?
It is not. You can use requirement.txt. setup.py install_dependencies is mainly for libraries. For more information read blog post Declaring dependencies in Python.
if I have to rely on setup.py am I supposed to execute python setup.py develop each time I create/delete a new file (since I saw them listed in SOURCES.txt)?
It's not necessary.
In settings.ini how "use" (under [app:main]) works? (can I "bypass" the egg-info which is it pointing to and "bootrsapping" the app in an alternative way?)
Please see the other answer regarding Paster and Entry points.
I want to use a couple of third party django packages for my application. So, I want to install each package locally for that application (or my whole project).
The python custom installation instructions look a bit scary - how do I do this as simply as possible?
You can just put the library module into your project folder because your project folder will be in the PYTHONPATH automatically when running via manage.py runserver or your wsgi script will point to it when running on production.
Usually all python packages are packaged like this:
package directory
module directory
... other files/dirs like README, Manifest and so on
What need to be in your Django project folder is only the module directory part from the example above, not the rest of the package.
Use virtualenv.
That is by far the best solution for installing custom packages for each project.
Just use --home - nothing scary about it.
Don't forget to make your PYTHONPATH point there, though.
I want to distribute some python code, with a few external dependencies, to machines with only core python installed (and users that unfamiliar with easy_install etc.).
I was wondering if perhaps virtualenv can be used for this purpose? I should be able to write some bash scripts that trigger the virtualenv (with the suitable packages) and then run my code.. but this seems somewhat messy, and I'm wondering if I'm re-inventing the wheel?
Are there any simple solutions to distributing python code with dependencies, that ideally doesn't require sudo on client machines?
Buildout - http://pypi.python.org/pypi/zc.buildout
As sample look at my clean project: http://hg.jackleo.info/hyde-0.5.3-buildout-enviroment/src its only 2 files that do the magic, more over Makefile is optional but then you'll need bootstrap.py (Make file downloads it, but it runs only on Linux). buildout.cfg is the main file where you write dependency's and configuration how project is laid down.
To get bootstrap.py just download from http://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py
Then run python bootstap.py and bin/buildout. I do not recommend to install buildout locally although it is possible, just use the one bootstrap downloads.
I must admit that buildout is not the easiest solution but its really powerful. So learning is worth time.
UPDATE 2014-05-30
Since It was recently up-voted and used as an answer (probably), I wan to notify of few changes.
First of - buildout is now downloaded from github https://raw.githubusercontent.com/buildout/buildout/master/bootstrap/bootstrap.py
That hyde project would probably fail due to buildout 2 breaking changes.
Here you can find better samples http://www.buildout.org/en/latest/docs/index.html also I want to suggest to look at "collection of links related to Buildout" part, it might contain info for your project.
Secondly I am personally more in favor of setup.py script that can be installed using python. More about the egg structure can be found here http://peak.telecommunity.com/DevCenter/PythonEggs and if that looks too scary - look up google (query for python egg). It's actually more simple in my opinion than buildout (definitely easier to debug) as well as it is probably more useful since it can be distributed more easily and installed anywhere with a help of virtualenv or globally where with buildout you have to provide all of the building scripts with the source all of the time.
You can use a tool like PyInstaller for this purpose. Your application will appear as a single executable on all platforms, and include dependencies. The user doesn't even need Python installed!
See as an example my logview package, which has dependencies on PyQt4 and ZeroMQ and includes distributions for Linux, Mac OSX and Windows all created using PyInstaller.
You don't want to distribute your virtualenv, if that's what you're asking. But you can use pip to create a requirements file - typically called requirements.txt - and tell your users to create a virtualenv then run pip install -r requirements.txt, which will install all the dependencies for them.
See the pip docs for a description of the requirements file format, and the Pinax project for an example of a project that does this very well.
How to pack python libs I'm using so I can distribute them with my app and have as few dependencies as possible and also not to conflict with different lib/version that is already on my system.
L.E.: Sorry i forgot to specify. I will be doing this on linux. And I'm not referring in making my app a installable file like deb/rpm, etc but how to organize my files so like for example I'll be using cherrypy and sqlalchemy I'll ship those with my app and not put the user through the pain of installing all the dependencies by himself.
You could try freeze.py, see http://wiki.python.org/moin/Freeze for more details.
Try py2exe.
You can have your users run the system from a startup script, and that script can fix the pythonpath ahead of time to put your versions first. For example if you put CherryPy, SQLAlchemy, etc. in an "external" subdirectory, you could try:
# startproj.sh
script_path=`dirname $0`
export PYTHONPATH=${script_path}/external;${PYTHONPATH}
exec ${script_path}/projstartup.py
But if you make a deb with the correct dependencies listed the installer will download them for the user. That's the best way, as it's non redundant.
Maybe you could make a tar or zip with your deb and all the third-party deb's and an install script that just install all the debs in the correct order. This way, if the user already has some package it wouldn't be installed again.