I use data_files in my setup.py for storing some files in my package.
And I also use pkg_resources module to get access to it. It works fine when I use install the package regularly. but when I install it via virtualenv the data files goes to the root directory of virtual environment. but the pkg_resources detects the lib/python3.5/site-packages.
P.S sys.prefix is root directory of virtualenv
why is that and how can I use pkg_resources correctly
Related
I'm building a python package to use for 'global' functions (i.e. stuff that I will use in multiple other projects). I have built the package using py -m build and it then puts the MyPackage-0.1.0.tar.gz into the dist directory in my folder.
My goal is to be able to run pip install MyPackage from within any other projects, and it will install the latest build of my package. In other words, I do not want to use something like --find-links. This way, I could also include the package in a requirements.txt file.
I have tried putting the tarball in a directory which is on my system's PATH, and into a subfolder within there (e.g. PathDir/MyPackage/MyPackage-0.1.0.tar.gz), but I keep getting the same 'No matching distribution found' error.
The documentation for pip install says:
pip looks for packages in a number of places: on PyPI (if not disabled via --no-index), in the local filesystem, and in any additional repositories specified via --find-links or --index-url.
When it says 'in the local filesystem' where does it begin it's search? Is there a way to change this (e.g. set some environment variable)
When looking for files in the local filesystem, pip has no notion of search path. You must give a path accessible from the current working directory. It can be an absolute path:
pip install /path/to/MyPackage-0.1.0.tar.gz
a relative path:
cd /path
pip install to/MyPackage-0.1.0.tar.gz
or a simple name if the package file is inside the current working directory:
cd /path/to
pip install MyPackage-0.1.0.tar.gz
I found the answer after a lot of searching, and so here is the solution:
pip uses configuration files to define its internal settings. In these configuration files, you can specify default values for find-links. This means that python will look here for compatible packages, as well as online.
You can check what configurations have been set, and what files they will be searched in by running pip config list -v. You just need to edit/create one of the files listed and add your configuration as pip.ini with the following:
[install]
find-links=file://C:/Users/.../PathDir/MyPackage/
By creating this at the User/Global level (rather than the site level), this installation also works when inside a virtual environment.
Source: https://pip.pypa.io/en/stable/topics/configuration/
pip can install packages from the local file system. They do not need to be even package files, they can be just working directories or git checkouts.
Usually I use pip --editable:
pip --editable /path/to/my/python/package
With --editable changes in .py files in the folder are automatically reflected to your application.
You can use --editable in requirements.txt file as well.
I have a question regarding python modules.
For example, I have a script which uses netaddr/paramiko module and now I want to start the script on another linux os where I have no root access. Pip is not installed and the user has no homedir. Virtuelenv is no option.
So is there a possibility/way to 'make' python modules 'portable' and add it to a folder where I can load it in my script??
You could use pip install --no-install to list the dependencies of the library you need, then copy the modules / packages of all of these libs in your project directory, for example in an /ext subdirectory.
# echo $PYTHONPATH
/usr/lib/python3.6/site-packages
# whoami
root
# easy_install --prefix=/usr/lib/python3.6/site-packages django==1.9
TEST FAILED: /usr/lib/python3.6/site-packages/lib/python2.7/site-packages does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/lib/python3.6/site-packages/lib/python2.7/site-packages
and your PYTHONPATH environment variable currently contains:
'/usr/lib/python3.6/site-packages'
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://pythonhosted.org/setuptools/easy_install.html#custom-installation-locations
Please make the appropriate changes for your system and try again.
# ls /usr/lib/python3.6/site-packages/lib/python2.7/site-packages
#
After reading answer, I couldn't find .pydistutils.cfg file in the file system
easy_install command tries to install in install path/usr/lib/python3.6/site-packages/lib/python2.7/site-packages which is invalid. This install path is getting created, amidst easy_install
Question:
How to resolve the install path using easy_install?
1.Are you using easy_install that works with Python2? Try to use easy_install that works with Python3.
Look at this.
2.--prefix set installation prefix.
When you use easy_install --prefix=/usr/lib/python3.6/site-packages,
the dir is :
'/usr/lib/python3.6/site-packages/' + 'lib/python2.7/site-packages'
Maybe you should use --install-dir
import nest gives the 'no module named nest' error when it is in the $PATH, which means that there is a /opt/nest/lib/python2.7/site-packages: in my system $PATH. At this location there is a directory named nest and the structure inside the nest directory is looks like:
, where there is an __init__.py obviously. So why can't python find nest?
more information:
I am sure that I installed nest with python2.7, and run it with the same python2.7.
According to the docs, there are several ways to install python packages:
using distutils - running python setup.py install installs the package to site-packages of your current distribution;
passing --user to setup.py install installs module to ~/.local/lib/python2.7/site-packages on Unix, and this directory in under normal conditions always included in sys.path;
passing --home=$HOME to setup.py install installs module under $HOME directory. This directory should be included to sys.path explicitly;
modifying python search path:
you can do either
import sys
sys.path.append('/opt/nest/lib/python2.7/site-packages')
in the beginning of your script; or your can add
PYTHONPATH=/opt/nest/lib/python2.7/site-packages
export PYTHONPATH
in the end of your ~/.bash_profile file.
UPDATE:
Just tried to install nest and found that it comes in two flavours - 32bit (under /opt/nest/lib) and 64bit (under /opt/nest/lib64). You might have tried to use 32-bit python package with 64-bit python distribution. Try to change the string in ./zshrc to
PYTHONPATH=/opt/nest/lib64/python2.7/site-packages and see if it works. It works for me at least.
To set relevant environment variables correctly, NEST installs have a config that you can just source
source <installpath>/bin/nest_vars.sh
That sets the PATH and PYTHONPATH correctly and points NEST to the right directories for dynamically loaded content (modules, help pages, …)
So I decided to install python packages (technically Django apps) directly from the downloaded tar file, by extracting it and using the following command:
python setup.py install
However, inside my site-packages directory, I find that the package was installed inside a .egg directory that also has version numbers. The directories look annoyingly like this:
site-packages/django_cms-2.1.3-py2.7.egg/cms
site-packages/django_cms-2.1.3-py2.7.egg/mptt
I need the packages to install as a directory with the package name with no .egg or version number, otherwise, Django can't find the package. It should be like this:
site-packages/cms
site-packages/mptt
Attempting to install the same package from pip, and it works fine. This is frustrating, so some help would be appreciated.
I tracked down a thread that discusses something similar, but it didn't give a solution that worked.
The Django project is unable to locate the packages I installed because the packages aren't installed at the root of site-packages dir. Instead, it resides inside the .egg directories. What I had to do is manually move the package to the root, but how do you correctly install python packages?
Also, I didn't suppress the easy-install.pth file, but it isn't picked up by PyDev.