Difference between sudo -H pip install and pip --user install - python

I am wondering what is the difference between these two commands (I have the feeling that they are identical):
sudo -H pip install <package>
pip --user install <package>
More informations:
From the sudo manpage:
-H, --set-home
Request that the security policy set the HOME environment
variable to the home directory specified by the target user's
password database entry. Depending on the policy, this may be
the default behavior.
And the pip user guide: https://pip.pypa.io/en/stable/user_guide/
Related questions:
What is the difference between pip install and sudo pip install?
What is the purpose of "pip install --user ..."? and
sudo pip install VS pip install --user
But none of them talk about the sudo -H option or the precise difference between the two.

sudo is short for 'superuser do'. It simply runs the command with root privileges, which may be useful if you are installing to a directory you normally wouldn't have access to.
However, in the examples you have given the two commands would function identically, as you don't need root privileges to pip install --user

The difference comes down to the permissions that are given to the package, and the location where the package is installed. When you run a command as root, the package will be installed with root permissions.
Here's an example:
Running sudo -H pip3 install coloredlogs results in the following:
$ sudo pip3 show coloredlogs | grep Location
Location: /usr/local/lib/python3.8/dist-packages
$ ls -l /usr/local/lib/python3.8/dist-packages
drwxr-sr-x 4 root staff 4096 Feb 25 01:14 coloredlogs
$ which coloredlogs
/usr/local/bin/coloredlogs
Running pip3 install --user <package> results in the following:
$ pip3 show coloredlogs | grep Location
Location: /home/josh/.local/lib/python3.8/site-packages
$ ls -l /home/josh/.local/lib/python3.8/site-packages
drwxrwxr-x 4 josh josh 4096 Feb 25 01:14 coloredlogs
$ which coloredlogs
coloredlogs not found
Notice the location differences between the two, and also notice that the package isn't installed on PATH when installed using the --user flag. If for some reason I wanted to directly call the package, I would need to add /home/josh/.local/bin to my PATH.

Related

python pip module installations produces WARNING Target directory /python/bin already exists

In a docker container, I have python installed and virtualenv. I am installing multiple pip modules but eventually receive this error when trying to install multiple modules:
WARNING: Target directory /python/bin already exists. Specify --upgrade to force replacement.
Here's how I'm installing the pip modules:
python3 -m pip install virtualenv
python3 -m venv cer
source cer/bin/activate
pip3 install pandas -t ./python
pip3 install numpy-t ./python
pip3 install requests -t ./python # WARNING HAPPENS HERE
pip3 install xlsxwriter -t ./python # WARNING HAPPENS HERE
Contents of ./python/bin/ after pandas installation;
ls -lh
-rwxr-xr-x 1 root root 216 Dec 8 17:50 f2py
-rwxr-xr-x 1 root root 216 Dec 8 17:50 f2py3
-rwxr-xr-x 1 root root 216 Dec 8 17:50 f2py3.7
Contents of ./python/bin/ after numpy installation is the same as above. Contents of ./python/bin/ after requests installation is the same but that's when the Target directory warning appears.
If I run pip3 install requests -t ./python --upgrade like the WARNING suggests the contents of ./python/bin/ are overwritten with this:
-rwxr-xr-x 1 root root 244 Dec 8 17:56 normalizer
but in reality I would want the previous contents not to get overwritten & rather they get merged instead so all the pip modules installed have what they need.
How would I go about achieving this?
Or should these modules be installed in another way to avoid this problem?
Ultimately I need to upload these modules / dependencies for a lambda function I'm creating.
The solution was to install the pip modules simultaneously;
python3 -m pip install virtualenv
python3 -m venv cer
source cer/bin/activate
pip3 install pandas numpy requests xlsxwriter -t ./python

Fixing python packages mis-installed by `sudo pip` / `sudo pip3` [duplicate]

How do I uninstall all packages installed by pip from my currently activated virtual environment?
I've found this snippet as an alternative solution. It's a more graceful removal of libraries than remaking the virtualenv:
pip freeze | xargs pip uninstall -y
In case you have packages installed via VCS, you need to exclude those lines and remove the packages manually (elevated from the comments below):
pip freeze | grep -v "^-e" | xargs pip uninstall -y
If you have packages installed directly from github/gitlab, those will have #.
Like:
django # git+https://github.com/django.git#<sha>
You can add cut -d "#" -f1 to get just the package name that is required to uninstall it.
pip freeze | cut -d "#" -f1 | xargs pip uninstall -y
This will work for all Mac, Windows, and Linux systems.
To get the list of all pip packages in the requirements.txt file (Note: This will overwrite requirements.txt if exist else will create the new one, also if you don't want to replace old requirements.txt then give different file name in the all following command in place requirements.txt).
pip freeze > requirements.txt
Now to remove one by one
pip uninstall -r requirements.txt
If we want to remove all at once then
pip uninstall -r requirements.txt -y
If you're working on an existing project that has a requirements.txt file and your environment has diverged, simply replace requirements.txt from the above examples with toberemoved.txt. Then, once you have gone through the steps above, you can use the requirements.txt to update your now clean environment.
And For single command without creating any file (As #joeb suggested).
pip uninstall -y -r <(pip freeze)
I wanted to elevate this answer out of a comment section because it's one of the most elegant solutions in the thread. Full credit for this answer goes to #joeb.
pip uninstall -y -r <(pip freeze)
This worked great for me for the use case of clearing my user packages folder outside the context of a virtualenv which many of the above answers don't handle.
Edit: Anyone know how to make this command work in a Makefile?
Bonus: A bash alias
I add this to my bash profile for convenience:
alias pipuninstallall="pip uninstall -y -r <(pip freeze)"
Then run:
pipuninstallall
Alternative for Pipenv
If you are using pipenv, you can run:
pipenv uninstall --all
Alternative for Poetry
If you are using Poetry, run:
poetry env remove --python3.9
(Note that you need to change the version number there to match whatever your Python version is.)
This works with the latest. I think it's the shortest and most declarative way to do it.
virtualenv --clear MYENV
But why not just delete and recreate the virtualenv?
Immutability rules. Besides it's hard to remember all those piping and grepping the other solutions use.
Other answers that use pip list or pip freeze must include --local else it will also uninstall packages that are found in the common namespaces.
So here are the snippet I regularly use
pip freeze --local | xargs pip uninstall -y
Ref: pip freeze --help
I managed it by doing the following:
Create the requirements file called reqs.txt with currently installed packages list
pip freeze > reqs.txt
Then uninstall all the packages from reqs.txt
# -y means remove the package without prompting for confirmation
pip uninstall -y -r reqs.txt
I like this method as you always have a pip requirements file to fall back on should you make a mistake. It's also repeatable, and it's cross-platform (Windows, Linux, MacOs).
Method 1 (with pip freeze)
pip freeze | xargs pip uninstall -y
Method 2 (with pip list)
pip list | awk '{print $1}' | xargs pip uninstall -y
Method 3 (with virtualenv)
virtualenv --clear MYENV
On Windows if your path is configured correctly, you can use:
pip freeze > unins && pip uninstall -y -r unins && del unins
It should be a similar case for Unix-like systems:
pip freeze > unins && pip uninstall -y -r unins && rm unins
Just a warning that this isn't completely solid as you may run into issues such as 'File not found' but it may work in some cases nonetheless
EDIT: For clarity: unins is an arbitrary file which has data written out to it when this command executes: pip freeze > unins
That file that it written in turn is then used to uninstall the aforementioned packages with implied consent/prior approval via pip uninstall -y -r unins
The file is finally deleted upon completion.
I use the --user option to uninstall all the packages installed in the user site.
pip3 freeze --user | xargs pip3 uninstall -y
For Windows users, this is what I use on Windows PowerShell
pip uninstall -y (pip freeze)
First, add all package to requirements.txt
pip freeze > requirements.txt
Then remove all
pip uninstall -y -r requirements.txt
The quickest way is to remake the virtualenv completely. I'm assuming you have a requirements.txt file that matches production, if not:
# On production:
pip freeze > reqs.txt
# On your machine:
rm $VIRTUALENV_DIRECTORY
mkdir $VIRTUALENV_DIRECTORY
pip install -r reqs.txt
Using virtualenvwrapper function:
wipeenv
See wipeenv documentation
Its an old question I know but I did stumble across it so for future reference you can now do this:
pip uninstall [options] <package> ...
pip uninstall [options] -r <requirements file> ...
-r, --requirement file
Uninstall all the packages listed in the given requirements file. This option can be used multiple times.
from the pip documentation version 8.1
(adding this as an answer, because I do not have enough reputation to comment on #blueberryfields 's answer)
#blueberryfields 's answer works well, but fails if there is no package to uninstall (which can be a problem if this "uninstall all" is part of a script or makefile). This can be solved with xargs -r when using GNU's version of xargs:
pip freeze --exclude-editable | xargs -r pip uninstall -y
from man xargs:
-r, --no-run-if-empty
If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if there
is no input. This option is a GNU extension.
pip3 freeze --local | xargs pip3 uninstall -y
The case might be that one has to run this command several times to get an empty pip3 freeze --local.
Best way to remove all packages from the virtual environment.
Windows PowerShell:
pip freeze > unins ; pip uninstall -y -r unins ; del unins
Windows Command Prompt:
pip freeze > unins && pip uninstall -y -r unins && del unins
Linux:
pip3 freeze > unins ; pip3 uninstall -y -r unins ; rm unins
This was the easiest way for me to uninstall all python packages.
from pip import get_installed_distributions
from os import system
for i in get_installed_distributions():
system("pip3 uninstall {} -y -q".format(i.key))
the easy robust way
cross-platform
and work in pipenv as well is:
pip freeze
pip uninstall -r requirement
by pipenv:
pipenv run pip freeze
pipenv run pip uninstall -r requirement
but won't update piplock or pipfile so be aware
Cross-platform support by using only pip:
#!/usr/bin/env python
from sys import stderr
from pip.commands.uninstall import UninstallCommand
from pip import get_installed_distributions
pip_uninstall = UninstallCommand()
options, args = pip_uninstall.parse_args([
package.project_name
for package in
get_installed_distributions()
if not package.location.endswith('dist-packages')
])
options.yes = True # Don't confirm before uninstall
# set `options.require_venv` to True for virtualenv restriction
try:
print pip_uninstall.run(options, args)
except OSError as e:
if e.errno != 13:
raise e
print >> stderr, "You lack permissions to uninstall this package.
Perhaps run with sudo? Exiting."
exit(13)
# Plenty of other exceptions can be thrown, e.g.: `InstallationError`
# handle them if you want to.
On Windows if your path is configured correctly, you can use:
pip freeze > unins && pip uninstall -y -r unins && del unins
This works on my windows system
pip freeze > packages.txt && pip uninstall -y -r packages.txt && del packages.txt
The first part pip freeze > packages.txt creates a text file with list of packages installed using pip along with the version number
The second part pip uninstall -y -r packages.txt deletes all the packages installed without asking for a confirmation prompt.
The third part del packages.txt deletes the just now created packages.txt.
This is the command that works for me:
pip list | awk '{print $1}' | xargs pip uninstall -y
If you're running virtualenv:
virtualenv --clear </path/to/your/virtualenv>
for example, if your virtualenv is /Users/you/.virtualenvs/projectx, then you'd run:
virtualenv --clear /Users/you/.virtualenvs/projectx
if you don't know where your virtual env is located, you can run which python from within an activated virtual env to get the path
In Command Shell of Windows, the command pip freeze | xargs pip uninstall -y won't work. So for those of you using Windows, I've figured out an alternative way to do so.
Copy all the names of the installed packages of pip from the pip freeze command to a .txt file.
Then, go the location of your .txt file and run the command pip uninstall -r *textfile.txt*
If you are using pew, you can use the wipeenv command:
pew wipeenv [env]
I simply wanted to remove packages installed by the project, and not other packages I've installed (things like neovim, mypy and pudb which I use for local dev but are not included in the app requirements). So I did:
cat requirements.txt| sed 's/=.*//g' | xargs pip uninstall -y
which worked well for me.
Select Libraries To Delete From This Folder:
C:\Users\User\AppData\Local\Programs\Python\Python310\Lib\site-packages
Why not just rm -r .venv and start over?
pip uninstall `pip freeze --user`
The --user option prevents system-installed packages from being included in the listing, thereby avoiding /usr/lib and distutils permission errors.

Clean Uninstallation of Python Packages after Ubuntu Upgrade

I recently upgraded a machine from Ubuntu Server 16.04 LTS to 18.04 LTS using command line. As a result, the built-in Python installation is upgraded from Python 3.5 to 3.6. And it seems that all the Python packages previously installed using pip3 are no longer accessible. I plan to clean up all those packages and use conda for Python package management. My question is, what is the best practice for a clean uninstallation of those inaccessible packages in this case?
The old packages installed via pip3 were primarily located under /usr/local/lib/python3.5/ and ~/.local/lib/python3.5/. But there could be other leftover files, e.g., under /usr/local/bin/. I would like to remove all of related files that came with pip3 install.
sudo pip install installs pip packages to/usr/local/lib/<python_version>/dist-packages, and apt packages to /usr/lib/<python_version>/dist-packages. Check these directories and remove the unwanted packages.
I ended up writing a bash script to call pip3 uninstall on each previously installed package iteratively.
#!/bin/bash
pypath_cmd="PYTHONPATH=$HOME/.local/lib/python3.5/site-packages"
export $pypath_cmd
echo "Uninstalling editable packages in $PYTHONPATH"
rm -f $PYTHONPATH/*.egg-link
rm -f $PYTHONPATH/easy-install.pth
pip3 freeze --all --local | cut --delimiter="=" -f 1 | while read pkg ; do
echo $pkg: $(pip3 show $pkg | grep "Location:")
pip3 uninstall -y $pkg
done
pypath_cmd="PYTHONPATH=/usr/local/lib/python3.5/dist-packages"
export $pypath_cmd
echo "Uninstalling editable packages in $PYTHONPATH"
sudo rm -f $PYTHONPATH/*.egg-link
sudo rm -f $PYTHONPATH/easy-install.pth
pip3 freeze --all --local | cut --delimiter="=" -f 1 | while read pkg ; do
echo $pkg: $(pip3 show $pkg | grep "Location:")
sudo $pypath_cmd pip3 uninstall -y --no-cache-dir $pkg
done

pip freeze reports version number instead of git sha for certain package

Typically, when I pip install -e from a git repo, and then do "pip freeze", it gives me git version info. For example:
$ pip install -e git://github.com/kennethreitz/requests.git#27b55a74d7b9bd2f8c60fd0ee342bcbbf40e0a66#egg=requests-dev
$ pip freeze | grep requests
-e git://github.com/kennethreitz/requests.git#27b55a74d7b9bd2f8c60fd0ee342bcbbf40e0a66#egg=requests-dev
Why is it that when I do this with the piplint package, "pip freeze" reports the version number instead of the git details?
$ pip install -e git://github.com/dcramer/piplint.git#cb2752e0c9692e4df2b2b03dec3087699e90f4da#egg=piplint
$ pip freeze | grep piplint
piplint==0.1.1
When "pip freeze" tries get vcs backend to piplint, It doesn't finds it, then it marks the package as not editable.
See:
https://github.com/pypa/pip/blob/develop/pip/init.py#L135
https://github.com/pypa/pip/blob/develop/pip/vcs/init.py#L60
Maybe it's a bug in pip or in piplint setup.py.

pip freeze lists uninstalled packages

On OS X 10.6.8, I uninstalled a package using (at least pip tells me so)
sudo pip uninstall pkg_name
but the package still shows up when I do
pip freeze
I try to do the uninstall command above again, and pip tells me the package is not installed.
What is the problem here? How do I verify whether the package is uninstalled or not? If so, can I refresh some sort of index of pip to get it corrected?
I thought you may have two pip binaries, and when you run as sudo, your shell chooses the wrong one, at first. But it does not make any sense if you run it again as sudo and pip removed the package. Did you do exactly this?
If you did not run the same commands twice, you may have different pip binaries running the uninstall and freeze. Check if the following two commands result in the same output:
$ sudo pip freeze
# ... sudo output
$ pip freeze
# ... normal output
Anyway, you can check if the package is installed using:
$ python -c 'import pkg_name' &> /dev/null && echo installed || echo not installed
There is no sort of refresh feature in pip.
I had this same problem and it was due to broken symlinks from homebrew once the file was uninstalled.
$ pip freeze | grep Magic
Magic-file-extensions==0.2
$ pip uninstall Magic-file-extensions
# say `y` at prompt / see it go through as success
$ pip freeze | grep Magic # still there :(
Magic-file-extensions==0.2
$ ll /usr/local/lib/python2.7/site-packages/ | grep Magic # symlink shows up red
├── [lrwxr-xr-x tomfuert 98 Feb 16 11:06] Magic_file_extensions-0.2-py2.7.egg-info -> ../../../Cellar/libmagic/5.17/lib/python2.7/site-packages/Magic_file_extensions-0.2-py2.7.egg-info
$ rm /usr/local/lib/python2.7/site-packages/Magic_file_extensions-0.2-py2.7.egg-info
$ pip freeze | grep Magic
# nothing!
If you use a virtual environment, try clean command. Don't forget sudo.
sudo pipenv clean

Categories

Resources