I'm finding that my pythonpath environment variable is ignored. I'm using python 2.6 on ubuntu. I have in my .bashrc the following:
export PTYHONPATH=/my/home/mylibs/lib/python2.6/site-packages/:$PYTHONPATH
Then I install a new version of numpy using:
python setup.py install --prefix=/my/home/mylibs/
and it gets correctly installed locally. However, when I try to install other packages (also using setup.py) that depend on the new version of numpy, they cannot find it, because by default the loaded numpy is the one in /usr/llib, and not the one specified in my PYTHONPATH. My PYTHONPATH gets correctly set but the system-wide directory is still overruling it.
How can this be fixed? I just want my local version of numpy to be accessed when I do import numpy. I saw other posts related to this with python 2.4 but as far as I can tell it never got resolved. Also, i'd like to do this without installing pip or virtualenv for now. It seems like it should be possible using --prefix or --home options passed to setup.py and then alteration of PYTHONPATH but this does not work for me... the system wide lib dirs are read first.
edit: I try to follow the suggestions and use pip. I have a system wide install of an old pip that does not recognize --user (ver 0.3). I tried to upgrade pip with pip itself but of course that failed because I cannot install it locally, so pip install pip --upgrade --user is not an option. I downloaded a new version of pip and installed locally in my home directory but the system wide old one is still used when I type pip at the prompt. I looked into the pip package and found runner.py so I tried to use it to install packages using:
runner.py install --user numpy --upgrade
That still fails with permission denied:
OSError: [Errno 13] Permission denied: '/usr/bin/f2py2.6'
It looks like --user is broken. I also am not sure how this would solve the fact that the system wide python uses the system wide packages in /usr/lib... is there a solution to this? It seems like it's virtually impossible to install local packages in python nowadays.
Ok, Python will use the first package it finds. The PYTHONPATH gets appended to sys.path, after the system one. So it will normally find the system one first. But the "official" per-user packages directory seems to be placed before that. So create your personal site-packages directory:
mkdir -p $HOME/.local/lib64/python2.7/site-packages
mkdir $HOME/bin
(You may have to change "lib64" to "lib32" or just "lib")
This directory gets placed before the system one on my system. But you should verify it by printing out sys.path.
Then install your packages into there. However, the --user option in the latest pip version should already place it there.
As a list resort you can manipulate sys.path. You can insert your directory into sys.path before the system site-packages, then import numpy.
You are getting permissions errors from the scripts installation, trying to put that in the system location. You can pass additional options to install scripts in your $HOME/bin directory.
Install like this:
pip install --user --install-option="--install-scripts=$HOME/bin"
Related
I just upgraded to python 3.7 and I realized that all my modules stuck with the previous version. Even Django is not recognised anymore. How can I do to transfer everything to the new version? I am a little lost right now, don't even know where the new version has been installed.
Edit:
When I do $ which python3.6 the terminal tells me it doesn't exist, but I have a python3.6 directory in /usr/local/lib/, where all modules are installed.
In the same directory /usr/local/lib/ I also have a python3.7 directory with some modules installed but many are missing. However when I search for the file python3.7 in my finder it doesn't appear. when I do $ which python3.7 the path is /usr/local/bin so not the same path as the directory.
Anyone sees what happened and knows how I can transfer all modules to python3.7?
Even if the old python version has been removed, it is possible to use the pip of the current python version with the --path option to list all the modules installed in the previous version.
For example, migrating all my user installed python modules from 3.7 to 3.8
pip freeze --path ~/.local/lib/python3.7/site-packages > requirements.txt
pip install --user -r requirements.txt
Incidentally, I always use pip install with --user and leave the system wide installations to the package manager of my linux distro.
It is safer to re-install all packages due to possible compatibility issues:
pip3.6 list | awk '{print $1}' | xargs -I{} pip3.7 install {}
in older version of Python --run the command
pip freeze > requirements.txt
download and install newer version on python.. change the PATH variable to the new version
and run the command
pip install -r requirements.txt
I'm not sure about all modules...but if you want to install a module specifically in python3.7, try this:
python3.7 -m pip install *module_name*
In some cases, we don't have the opportunity to pip freeze in old version--because I've already updated and old version have been purged! There are some measures I've taken to recover some of the packages but I'm NOT sure every package would work with this fix.(e.g. the packages built with wheels)
mv /your/path/to/python3.{6,7}/site-packages/
If the case is packages installed outside venv (in /usr/local/ or ~/.local), reinstall pip with get-pip.py, just to be safe.
If you are recovering a virtualenv. Activate your virtualenv and use my script
Most of your packages should work by now. If anything malfunctions, pip reinstall would works. If you still want it 100% works, pip freeze now.😉
I have an alternative
(Not sure if works outside Windows 10)
I'm currently migrating from 3.7 to 3.8 and the way I found to re-install my previous libraries was by using a script I had that updates all packages via pip install. (Assuming you installed your new Python version as your main version) This checks for all the packages I had and updates/install them in the new Python version.
Note: I prefer to run the script from the command line
Use the file explorer to go to the folder where you have the script;
Click on the path box, write "cmd" and press enter to open a command line from the folder where you are;
Write "python name_of_your_script.py" and press enter to run the command.
The script (adapted from this solution):
import pkg_resources
from subprocess import call
packages = [dist.project_name for dist in pkg_resources.working_set]
[call("pip install " + name + " --upgrade") for name in packages]
I faced a similar problem, now that I upgraded from python 3.7 to python 3.8 (new)
I installed Python 3.8, but the system kept the python37 subfolder with the already installed packages(...\Python37-32\Lib\site-packages) even with the Pyhton38 subfolder created, with the new python.exe.
Usually, it's possible to keep on using the old libraries on your new Python version, because the existent libraries installation folder are already registered in your local computer system path (*).
Even though, I've had problems to use some libraries (some worked in Jupyter Notebook but not in Spyder). I tried the alternatives others proposed to migrate libraries but it did not worked (maybe I did not
So I used brutal force solution.. Not elegant at all, but it worked:
remove the OLD python version folders from the system path or even remove the folder itself for good..
Folders: C:\Users\USERNAME\AppData\Roaming\Python\Python37
C:\Users\USERNAME\AppData\Local\Programs\Python\Python37
Reinstall the packages you need, preferably via Anaconda prompt.
python -mpip install library_name
OR
pip install --user --force-reinstall package_name
OR
pip install --user --force-reinstall package_name == specify_package_version
The libraries will be installed at c:\users\USERNAME\anaconda3\lib\site-packages and recognized by your new python version.
(*) to add the folder to the PATH: System properties --> environment variables --> click "Path"--> edit --> add folder name)
I have been trying to install python packages and change the permissions of a folder using chmod. I can't remember which folder as it was just the one which the terminal said I did not have permissions for. I cannot even open it from x-term due to a python error(using python 2 print instead of 3?)
Since doing this I have been unable to open the gnome-terminal using the icon, the cursor becomes a laoding icon for a moment and then dissapears without opening the program.
I am also unable to use pip to install programs without using the program in sudo mode
Furthermore I cannot use the import command in python. Note that I have intalled scipy to my machine
Does anyone know what I have done and how I can revert it?
wrt to the import failing, pip installs modules into the /site-packages directory of your python directory. So I think the issue is that pip installed it in a different python directory than the directory that the system is using when you type python3. I would highly recommend using virtualenv when installing packages - this way you can keep your python executable as well as the dependencies all in one place. However, some packages are required to be installed in the system's python directory. wrt pip requiring you to use sudo, how did you install pip? Usually I do sudo apt-get install python3-pip
I want to set up a Python environment for a whole team and I don't have root access to the server.
I have done a similar thing with Perl and expected to be able to do this for Python in a similar way but I keep running into a problem.
Basically, I want to be able to install a package into /SOME/DIR on the system while ignoring any system wide versions of that package.
However, when I run
pip install --install-option="--prefix=/SOME/DIR/" --up --ignore-installed SOME-MODULE
I keep getting a "permission denied" error because pip keeps trying to remove system-wide packages when upgrading.
What does work is this
pip install --user --up --ignore-installed SOME-MODULE
Which does not try to touch the system-wide packages but it installs the module into a directory in $HOME/.lib, which is not what I need.
It seems impossible to combine --user and a "--prefix" option, so it sems that I can either install into an arbitrary path but then get conflicts with already install system-wide packages or install into my home directory. Neither of them are what I need.
For now I have been using the --user option and then moved the installed files across to /SOME/DIR which works but seems odd.
Am I missing something? I have read up on virtualenv but this also doesn't quite sound like what I need. Thanks for your help!
Note that --install-options is passed directly to the packages setup.py install command - this requires the installation directory to be in your python path.
add it to your PYTHONPATH i.e.
set -gx PYTHONPATH $PYTHONPATH '/home/user/temp/lib/python3.4/site-packages'
and run pip
pip install django==1.6 --ignore-installed --install-options="--prefix=/home/user/temp"
Mostly this is a pain in the ass if you have to do this for each library (note that you will still have potential conflicts with imports if you want to use certain standard libraries from the default site-packages dir, and others from your custom dir). And the best choice is probably, as the comment says, to install virtualenv and virtualenvwrapper
I had a similar problem (permission denied + no root access), --build option made it work: pip install --install-option="--prefix=/path/to/local/lib" --build=/tmp wget
Hello I'm trying to run twisted along with python but python cannot find twisted.
I did run $pip install twisted successfully but it is still not available.
ImportError: No module named twisted.internet.protocol
It seems that most people have $which python at /usr/local/bin/python
but I get /Library/Frameworks/Python.framework/Versions/2.7/bin/python
May this be the issue? If so, how can I change the PATH env?
It is just fine.
Python may be installed in multiple places in your computer.
When you get a new Mac, the default python directory may be
'usr/bin/python2.7'
You may also have a directory
'System/Library/Frameworks/Python.framework/Versions/2.7/bin/python'
The first one is the symlink of the second one.
If you use HomeBrew to install python, you may get a directory in
'usr/local/bin/python2.7'
You may also have a directory as
'Library/Frameworks/Python.framework/Versions/2.7/bin/python'
which is exactly where my directory is.
The difference between the second one and the fourth one, you may find it here
Installing Your Framework
In your question, as you mentioned pip install is successful, but the installed packages still not available. I may guess your pip directory is not in your default python directory, and the packages are installed where your pip directory is. (Please use 'which pip' to check it out)
For example, in my computer, the default pip directory is
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip
though, I have also pip in usr/local/bin.
So, all my packages installed via 'pip install' are stored in
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Hope that resolves your doubt. Similar things have happened to me, and it took me a whole night to figure out.
Here is the solution:
Use PYTHONPATH="/usr/local/lib/python2.7/site-packages:$PYTHONPATH" to modify your python directory, or modify your pip directory.
However, I would recommend a better way, use virtualenv. This can isolates Python environments, and can help you easily set up packages for each project.
By the path your giving for OS X python I'm guessing your a rev-or-so old on your OS X (leopard?) so I can't directly compare with my machine.
But, adding packages to the base OS X install is always a touchy thing, one check I would recommend is the permissions on any packages you add. Do a ls -l /Library/Python/2.7/site-packages/ and make sure everything has r rights (and x rights for directories) (I.E. -rwxr-xr-x or drwxr-xr-x).
I had a recent case where a sudo pip wouldn't set user read rights on installed packages, and I believe "No module" was the error I was getting when I tried to use them
Because adding packages is so touchy on OS X, there are tons of guide on the net to doing hand installs of python. The first one I matched on a google is Installing / Updateing Python on OS X (use at your own risk, I personally haven't followed that guide)
(... the 3rd part install system Brew is a very common method for people to do automated installs of python as well)
Okay well in the terminal I finally found out:
open .bash_profile located at your user root (simply do a $cd in terminal to go there) and add where the path is the location of twisted
PYTHONPATH="/usr/local/lib/python2.7/site-packages:$PYTHONPATH"
export PYTHONPATH
I too was getting a ImportError: No module named xxxeven though I did a pip install xxx and pip2 install xxx.
pip2.7 install xxx worked for me. This installed it in the python 2.7 directory.
I want to install Sphinx 1.1.3 for python 2.6. However, I don't have sudo rights. So instead of installing it in the default place, I want to set a different location, using --prefix. Doing the following:
-bash-3.2$ easy_install Sphinx-1.1.3-py2.6.egg --prefix=/homes/ndeklein/python2.6/site-packages/
gives me:
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/usr/lib/python2.4/site-packages/test-easy-install-18534.write-test'
The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/lib/python2.4/site-packages/
Am I typing something wrong with the prefix? Also, what I could use instead (which I've used with other packages):
python setup.py install --home=/homes/ndeklein/python2.6/site-packages/
but I can't find the setup.py script. I'm guessing that EGGs don't have a setup.py script, is that true?
You need to specify options before the package, so the command should be:
easy_install --prefix=/homes/ndeklein/python2.6/site-packages/ Sphinx-1.1.3-py2.6.egg
This website discusses non-root python installs. It might be useful to you...
http://www.astropython.org/tutorials/user-rootsudo-free-installation-of-python-modules7/
To quote a little bit of it:
A user configuration file, ~/.pydistutils.cfg, will override the internal system path for python package installations, redirecting the built libraries (lib), scripts (bin) and data (share) into user owned and specified directories. You must simply tell the python installer where theses directories are located.
The user file, ~/.pydistutils.cfg, has the following lines, using a pretty obvious syntax:
[install]
install_scripts = ~/usr/bin
install_data = ~/usr/share
install_lib = ~/usr/lib/python2.4/site-packages
Of course, whatever directories you specify there should probably exist and you should put them at the front of your PYTHONPATH:
export PYTHONPATH=~/usr/lib/python2.4/site-packages:${PYTHONPATH}
It also looks like more modern python installations (compared to the things in the link) should be able to use the ~/.local directory:
easy_install --prefix=~/.local ...
There is also:
easy_install --user ...
which will install to a user-specific site directory.
You could try using pip install of easy_install(pip is recommended over easy_install these days)
Then you can just use
pip install --user Sphinx
see http://www.pip-installer.org/en/latest/installing.html on how to install pip if needed
You may also want to pip install virtualenv and work inside virtualenv(where pip will install all packages in a local site packages folder). see http://pypi.python.org/pypi/virtualenv for more info.