A lot of software packages require users of the system to install a set of dependencies first before they can use the software. (This is a general question not specific to python, but I'll speak in context of python.). Can't we make the python module install its own dependencies before executing the code if the dependencies are not already installed. This should be doable with the help of system or subprocess calls but I rarely see people doing this.
For example lets say lib is a python library that needs to be used in the python file main.py:
import os
try:
import lib
except:
os.system('pip install pdir')
import lib
# Can make use of lib now
Is there anything potentially wrong with this approach? Could doing something like this cause problems for big projects?
Note: The advantage here is that a user using the file does not have to install the dependencies separately, he can simply run python main.py. And the second thing that I realize is that such approach makes sense only when virtualenv is being used.
You should never do this - pip doesn't have dependency resolution so there's no guarantee that you'll get a certain version. Dependencies should be installed using setup.py, requirements.txt or a different approach.
You also shouldn't need user permissions or sudo to install the packages just for running the code. The user should be aware of the packages that are needed for installing your package as they may come from PyPI or the OS's package system or an internal company PyPI mirror - and silently installing dependencies is not a good idea in that case.
You could always consider using a more advanced print statement to inform the user.
try:
import ConfigParser
except ImportError as err:
print '\n'.join([i + ''.join(str(err).split(' ')[-1:]) for i in ['$ pip install ', '$ easy_install ']])
You can recommend them installing the missing package this way.
$ pip install ConfigParser
$ easy_install ConfigParser
Additionally you could consider having a ImportError be reason to recommend the user to run their ./setup.py or ./INSTALL instead of just the missing module.
Related
This weekend I've been reading up on conda and the python packaging user guide because I have a simple pure python project that depends on numpy. It seemed to me that distributing/installing this project via conda was better than pip due to this dependency.
One thing on which I'm still not clear: conda will install a python package from a recipe in build.sh, but it seems like build.sh just ends up calling python setup.py install for most python packages.
So even if I want to distribute/install my python package with conda, I still end up depending on setuptools (or distutils) for the actual installation, correct? I was unable to find a conda utility analogous to setuptools; am I missing something?
FWIW, I posted this question on the conda issue tracker.
Thanks!
Typically you will still be using distutils (or setuptools if the library requires it) to install things, yes. It is not technically required. The build.sh can be anything. If you wanted to, you could just copy the code into site-packages. Using setup.py install is recommended, though, as libraries will already have setup.py working, it will install metadata that can be read by pip, and it will compile any extension modules and install any data files.
The following works:
pip install git+git://github.com/pydata/pandas#master
But the following doesn't:
pip install -e git+git://github.com/pydata/pandas#master
The error is:
--editable=git+git://github.com/pydata/pandas#master is not the right format; it must have #egg=Package
Why?
Also, I read that the -e does the following:
--egg
Install as self contained egg file, like easy_install does.
what is the value of this? When would this be helpful? (I always work on a virtualenv and install through pip)
Generally, you don't want to install as a .egg file. However, there are a few rare cases where you might. For example:
It's one of a handful of packages that needs to override a built-in package, and knows how to do so when installed as a .egg. With Apple Python, readline is such a package. I don't know of any other common exceptions.
The egg has binary dependencies that point to other eggs on PyPI, and can serve as a binary dependency for yet other eggs on PyPI. This is pretty rare nowadays, because it doesn't actually work in many important cases.
You want a package embedded in a single file that you can copy-and-paste, FTP, etc. from one installation to another.
You want a package that you can install into another installation straight out of site-packages.
The package is badly broken (and you can't fix it, for whatever reason), so that setup.py install does not work, but it can properly build an egg and run out of an egg.
Meanwhile, if you want to use editable mode, the package, and all other packages it depends on, have to be egg-compatible, whether or not you install them as eggs; pip will add #egg=<project name> to the VCS URL for each one, and if any of them don't understand that, it will fail.
I'm wondering if there's a way to "install" single-file python modules using pip (i.e. just have pip download the specified version of the file and copy it to site-packages).
I have a Django project that uses several 3rd-party modules which aren't proper distributions (django-thumbs and a couple others) and I want to pip freeze everything so the project can be easily installed elsewhere. I've tried just doing
pip install git+https://github.com/path/to/file.git
(and tried with the -e tag too) but pip complains that there's no setup.py file.
Edit: I should have mentioned - the reason I want to do this is so I can include the required module in a requirements.txt file, to make setting up the project on a new machine or new virtualenv easier.
pip requires a valid setup.py to install a python package. By definition every python package has a setup.py... What you are trying to install isn't a package but rather a single file module... what's wrong with doing something like:
git clone git+https://github.com/path/to/file.git /path/to/python/install/lib
I don't quite understand the logic behind wanting to install something that isn't a package with a package manager...
I am trying to install MySQLdb package. I found the source code here.
I did the following:
gunzip MySQL-python-1.2.3c1.tar.gz
tar xvf MySQL-python-1.2.3c1.tar
cd MySQL-python-1.2.3c1
python setup.py build
As the result I got the following:
Traceback (most recent call last):
File "setup.py", line 5, in ?
from setuptools import setup, Extension
ImportError: No module named setuptools
Does anybody knows how to solve this problem?
By the way, if I am able to do the described step, I will need to do the following:
sudo python setup.py install
And I have no system-administrator-rights. Do I still have a chance to install MySQLdb?
Thank you.
After trying many suggestions, simply using sudo apt-get install python-mysqldb worked for me.
More info:
Getting "Error loading MySQLdb module: No module named MySQLdb" - have tried previously posted solutions
If MySQLdb's now distributed in a way that requires setuptools, your choices are either to download the latter (e.g. from here) or refactor MySQLdb's setup.py to bypass setuptools (maybe just importing setup and Extension from plain distutils instead might work, but you may also need to edit some of the setup_*.py files in the same directory).
Depending on how your site's Python installation is configured, installing extensions for your own individual use without requiring sysadm rights may be hard, but it's never truly impossible if you have shell access. You'll need to tweak your Python's sys.path to start with a directory of your own that's your personal equivalent of the system-wide site pacages directory, e.g. by setting PYTHONPATH persistently in your own environment, and then manually place in said personal directory what normal installs would normally place in site-packages (and/or subdirectories thereof).
I resolved this issue on centos5.4 by running the following command to install setuptools
yum install python-setuptools
I hope that helps.
This was sort of tricky for me too, I did the following which worked pretty well.
Download the appropriate Python .egg for setuptools (ie, for Python 2.6, you can get it here. Grab the correct one from the PyPI site here.)
chmod the egg to be executable: chmod a+x [egg] (ie, for Python 2.6, chmod a+x setuptools-0.6c9-py2.6.egg)
Run ./[egg] (ie, for Python 2.6, ./setuptools-0.6c9-py2.6.egg)
Not sure if you'll need to use sudo if you're just installing it for you current user. You'd definitely need it to install it for all users.
#main:
$ su
$ yum install MySQL-python
and it will be installed (MySQLdb).
#!/usr/bin/env python
import os
import sys
from **distutils.core** import setup, Extension
if sys.version_info < (2, 3):
raise Error("Python-2.3 or newer is required")
if os.name == "posix":
from setup_posix import get_config
else: # assume windows
from setup_windows import get_config
metadata, options = get_config()
metadata['ext_modules'] = [Extension(sources=['_mysql.c'], **options)]
metadata['long_description'] = metadata['long_description'].replace(r'\n', '')
setup(**metadata)
For Python 2.7, one can easily install using this
apt-get install python2.7-mysqldb
Also, you can see the build dependencies in the file setup.cfg
I am experiencing the same problem right now. According to this post you need to have a C Compiler or GCC. I'll try to fix the problem by installing C compiler. I'll inform you if it works (we'll I guess you don't need it anymore, but I'll post the result anyway) :)
well installing C compiler or GCC didn't work but I found a way to successfully install mysqldb package
kindly follow Mike schrieb's (Thanks to him) instructions here . In my case, I used setuptools-0.6c11-py2.7.egg and setuptools-0.6c11 . Then download the executable file here then install that file. hope it helps :)
When you need to install modules in Linux/Unix and you lack sudo / admin rights, one simple way around it is to use the user scheme installation, basically run
"python setup.py install --user" from the command line in the folder of the module / library to be installed
(see http://docs.python.org/install/index.html for further details)
I'm working on a buildout script that needs to install a distutils package on remote server.
On PyPi there are 2 recipes for doing this
collective.recipe.distutils 0.1 and zerokspot.recipe.distutils 0.1.1.
The later module a derivative of the former, and is a little more convenient then the first, but the both suffer from the same problem, which I will describe now.
When bootstrap.py is executed, it downloads zc.buildout package and puts it into buildout's eggs directory. This gives ./bin/buildout access to zc.buildout code, but /usr/local/python does not know anything about zc.buildout at this point.
Buildout attepts to install the package by running 'python setup.py install' inside of a subprocess. This produces an ImportError because zc.buildout is not installed for /usr/local/python.
So, I have several solutions.
Install zc.buildout using easy_install on the remote server. I don't like this option at all, it makes a special case for a module that is very insignificant.
Modify zerokspot.recipe.distutils to put try block around 'import zc.buildout' this way, it will install even if zc.buildout is not installed. It's an ok solution, but somewhat hackish.
Replace subprocess with code that will install the package using distutils api or setuptools api. This would be the best solution in my opinion.
The question is how would i do #3?
Thank you,
Taras
PS: I solved the problem by creating another package that does not have dependancy on zc.buildout. My package is called taras.recipe.distutils and it's available on pypi.
You can call a command line program within your Python program using the subprocess module:
import subprocess
subprocess.call('python setup.py install')
However, how much control do you have over the environment that this install will be run? If it is a package that you are distributing, you will likely have issues no matter what solution people propose. How will you handle cases of needing root access (e.g. sudo python setup.py install)?
You may consider looking into Paver since it provides an API that is in some ways an extension of setuptools.
zerokspot.recipe.distutils is fundamentally broken in that it adds a dependency on zc.buildout in it's setup.py, as follows:
setup.py imports get_version from zerokspot.recipe.distutils
All of zerokspot.recipe.distutils is defined in it's __init__.py, including get_version
__init__.py in zerokspot.recipe.distutils imports zc.buildout
Why the author defines get_version is a mystery to me; best practice keeps a simple version string in setup.py itself and lets setuptools deal with dev versions (through setup.cfg), and distutils for version metadata extraction.
Generally it is not a good idea to import the whole package in setup.py as that would require all the package dependencies to be present at install time. Obviously the author of the package has zc.buildout installed as a site-wide package and didn't notice his oversight.
Your best bet is to fork the package on github, remove the get_version dependency, and propose the change to the original author while you use your fork instead.
Are you sure you don't want to just generate a bdist?