There's a git repo at https://github.com/Ivancic91/LammpsIO that I can install in "editable" mode by cloning and running pip install -e /path/to/LammpsIO. When I do this, I'm able to run import LammpsIO successfully in python.
However, when I pip install directly from github with pip install git+https://github.com/Ivancic91/LammpsIO.git I run into problems. The actual pip install command looks successful with the usual success messages:
Successfully built LammpsIO
Installing collected packages: LammpsIO
Successfully installed LammpsIO-0.0.1
But when I try to import LammpsIO, I get ModuleNotFoundError: No module named 'LammpsIO'.
I thought this might be some sort of path error, but which pip and which python both point to the same conda environment folder.
Here are a couple more outputs in case anyone needs them.
After pip install git+...:
conda list returns:
krb5 1.16.3 h24a3359_1000 conda-forge
lammpsio 0.0.1 pypi_0 pypi
libcurl 7.64.0 h76de61e_0 conda-forge
pip list returns:
kiwisolver 1.0.1
LammpsIO 0.0.1
MarkupSafe 1.1.0
After pip install -e ...:
conda list returns:
krb5 1.16.3 h24a3359_1000 conda-forge
lammpsio 0.0.1 dev_0 libcurl 7.64.0 h76de61e_0 conda-forge
pip list returns:
kiwisolver 1.0.1
LammpsIO 0.0.1 /path/to/LammpsIO/src
MarkupSafe 1.1.0
I'm guessing this has to do with the unorthodox structure of LammpsIO, whose setup.py file contains:
package_dir = {'': 'src'},
packages = find_packages(where='src'),
but I'm surprised that pip install from github fails where pip install -e /path works.
Your reasoning seems sound here. I'd think that what you're trying to would work. I created a new virtualenv and tried the github install myself. I then looked in site-packages. What I see is strange. There's a 'LammpsIO-0.0.1.dist-info' directory containing basic metadata, but no Lammps package itself. I did the find 'find . -iname "*mmps*"' in site-packages, and only got that info directory. So it seems like the package didn't actually install. - LammpsIO does show up when I do "pip list".
If you "install" the package by download a zip and doing "pip install [zip]", you get the same output from pip and end up with the same problem. So it has nothing to do with installing directly from github. It has to do with actually installing the package vs just linking its source into your enviroment .
My guess is that the Lammps installer is just broken - Since '-e' doesn't install anything, but just creates a link, that would explain why it works but a regular install does not.
Related
I am using Anaconda to build environments. I've been using pip install my_module the entire time, and I've recently realized this is incorrect. The environment I made is called dp_offfsets_environment and I have been able to run my flask app I wrote in IntelliJ using an SDK linked to this environment.
Why is this? Seen below, when calling conda list no packages are installed under my environment so I would expect my flask app to fail.
What is pip list displaying below? Is it showing packages installed in my base environment?
How do you recommend uninstalling these packages and moving them into the actual environment?
(dp_offsets_environment) C:\WINDOWS\system32>pip list
Package Version
---------------------- -------
click 8.0.3
colorama 0.4.4
cycler 0.10.0
Flask 2.0.2
fonttools 4.29.1
importlib-metadata 4.10.1
itsdangerous 2.0.1
Jinja2 3.0.3
kiwisolver 1.3.1
MarkupSafe 2.0.1
matlab 0.1
mysql-connector-python 8.0.26
packaging 21.3
pandas 1.3.0
Pillow 8.3.1
pip 22.0.2
pyparsing 2.4.7
python-dateutil 2.8.1
pytz 2021.1
setuptools 47.1.0
six 1.16.0
typing_extensions 4.0.1
Werkzeug 2.0.2
wheel 0.37.1
zipp 3.7.0
(dp_offsets_environment) C:\WINDOWS\system32>conda list
# packages in environment at C:\Users\ckurtz\.conda\envs\dp_offsets_environment:
#
# Name Version Build Channel
When you run conda create -n dp_offsets_environment, then conda creates an empty environment, i.e. with no packages. conda does not install python, pip or anything else in the environment. So when you run pip with that env activated, you will still get whatever pip is first located in your PATH (most likely from your base env).
So whenever you ran pip install nothing was installed to your currently active conda env and your flask app was probably not using that conda env, but also used the python that corresponds to your pip.
You need to install python/pip to your env and then install to your env.
See this as an example (I added annotation with ###):
(base) C:\Users\FlyingTeller>where pip
C:\Users\FlyingTeller\miniconda3\Scripts\pip.exe ### We are in base, so pip from base is found
(base) C:\Users\FlyingTeller>conda create -n someEnv
Collecting package metadata (current_repodata.json): done
Solving environment: done
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: C:\Users\FlyingTeller\miniconda3\envs\someEnv
### Note: No packages are being installed
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate someEnv
#
# To deactivate an active environment, use
#
# $ conda deactivate
(base) C:\Users\FlyingTeller>conda activate someEnv
(someEnv) C:\Users\FlyingTeller>where pip
C:\Users\FlyingTeller\miniconda3\Scripts\pip.exe ### <----- pip still from base env
(someEnv) C:\Users\FlyingTeller>conda install python pip
Collecting package metadata (current_repodata.json): done
Solving environment: done
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: C:\Users\FlyingTeller\miniconda3\envs\someEnv
added / updated specs:
- pip
- python
### Download and installation pruned
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(someEnv) C:\Users\FlyingTeller>where pip
C:\Users\FlyingTeller\miniconda3\envs\someEnv\Scripts\pip.exe ### Only now pip is in this env
C:\Users\FlyingTeller\miniconda3\Scripts\pip.exe
(someEnv) C:\Users\FlyingTeller>
Side Note
I've been using pip install my_module the entire time, and I've recently realized this is incorrect
This is a misconception of a commonly given advice "Do not use pip in a conda env". This advice is always a bit strong. There is no reason to refrain from using pip in a conda env all together. There is a good reason to be careful and aware of certain things that can easily break your env. There is also a good post on anaconda.org about this.
For anyone looking back on this, I discovered I had another instance of Anaconda on my machine. My company gives out laptops with Spyder installed which by default, has it's own version of Anaconda. I did not know Spyder came with Anaconda so I accidently installed two versions of Anaconda during my initial setup. To fix this I had to go through and uninstall both instances of Anaconda on my machine including their packages. Once completely removed, I reinstalled Anaconda, and my problems were alleviated.
When you use pipenv to install a package, all the dependent packages will also be installed with it. Uninstalling that package with pipenv uninstall will not remove the dependent packages automatically.
How to get the equivalent functionality of pip-autoremove within a pipenv?
For example:
$ cd testpipenv
$ pipenv install requests
$ pipenv shell
(testpipenv) $ pipenv graph
requests==2.24.0
- certifi [required: >=2017.4.17, installed: 2020.6.20]
- chardet [required: >=3.0.2,<4, installed: 3.0.4]
- idna [required: >=2.5,<3, installed: 2.10]
- urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.9]
(testpipenv) $ pipenv uninstall requests
(testpipenv) $ pip list
Package Version
---------- ---------
certifi 2020.6.20
chardet 3.0.4
idna 2.10
pip 20.1.1
setuptools 47.3.1
urllib3 1.25.9
wheel 0.34.2
The dependent packages of requests, such as urllib3 are still installed which can be verified by
(testpipenv) $ python
>>> import urllib3
This was also discussed here: Pipenv uninstall doesn't uninstall dependencies - issue #1470, and I have not found a more current set of instructions in regards to autoremoval of packages with pipenv.
Versions used:
pipenv, version 2020.5.28
Python 3.6.10
TL;DR
(testpipenv) $ pipenv uninstall requests && pipenv clean
To get similar functionality to pip-autoremove using just pipenvcommands I did the following, continuing with the example above:
(testpipenv) $ pipenv graph
certifi==2020.6.20
chardet==3.0.4
idna==2.10
urllib3==1.25.9
This shows that the dependent packages are still installed, but these have already been removed form Pipfile.lock. Therefore, using pipenv clean will remove them:
(testpipenv) $ pipenv clean
Uninstalling certifi…
Uninstalling idna…
Uninstalling urllib3…
Uninstalling chardet…
In summary...
(testpipenv) $ pipenv uninstall requests && pipenv clean
... will remove all the dependent packages and is the closest equivalent to pip-autoremove.
This may be more aggressive than intended as the clean command 'Uninstalls all packages not specified in Pipfile.lock' If packages have not been added to Pipfile.lock before using clean, this command might remove more than intended. I do not know if this is really a concern, as uninstall automatically updates Pipfile.lock unless additional options such as --skip-lock are specified.
Another alternative would be...
pipenv --rm && pipenv install
... which will remove everything and rebuild the virtual environment based on Pipfile.lock. This would work but is slow as it deletes the full virtual environment and reinstalls everything, except for the unneeded dependencies.
create virtual environment conda create test python=3.7
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /Users/mac/miniconda3/envs/test
added / updated specs:
- python=3.7
The following NEW packages will be INSTALLED:
ca-certificates pkgs/main/osx-64::ca-certificates-2019.10.16-0
certifi pkgs/main/osx-64::certifi-2019.9.11-py37_0
libcxx pkgs/main/osx-64::libcxx-4.0.1-hcfea43d_1
libcxxabi pkgs/main/osx-64::libcxxabi-4.0.1-hcfea43d_1
libedit pkgs/main/osx-64::libedit-3.1.20181209-hb402a30_0
libffi pkgs/main/osx-64::libffi-3.2.1-h475c297_4
ncurses pkgs/main/osx-64::ncurses-6.1-h0a44026_1
openssl pkgs/main/osx-64::openssl-1.1.1d-h1de35cc_3
pip pkgs/main/osx-64::pip-19.3.1-py37_0
python pkgs/main/osx-64::python-3.7.5-h359304d_0
readline pkgs/main/osx-64::readline-7.0-h1de35cc_5
setuptools pkgs/main/osx-64::setuptools-42.0.1-py37_0
sqlite pkgs/main/osx-64::sqlite-3.30.1-ha441bb4_0
tk pkgs/main/osx-64::tk-8.6.8-ha441bb4_0
wheel pkgs/main/osx-64::wheel-0.33.6-py37_0
xz pkgs/main/osx-64::xz-5.2.4-h1de35cc_4
zlib pkgs/main/osx-64::zlib-1.2.11-h1de35cc_3
check the reference of pip, which is correct.
(test) mac#mac-MBP ~ % which pip
/Users/mac/miniconda3/envs/test/bin/pip
(test) mac#mac-MBP ~ % pip --version
pip 19.3.1 from /Users/mac/miniconda3/envs/test/lib/python3.7/site-packages/pip (python 3.7)
install pip install jupyter
shows
Requirement already satisfied: jupyter in ./.local/lib/python3.7/site-packages (1.0.0)
I can use conda to uninstall and reinstall pip, which will solve the Requirement already satisfied issue, but it also upgrades python3.7 to 3.8 for some reason. I have to use 3.7 because tensorflow currently only supports 3.7.
Not sure what's wrong here. I have used the same commands to create environment and also worked smoothly. The only change I can think of is that I changed from bash to zsh because of Catalina. Any help would be appreciated.
The issue was I installed Jupyter with --user flag before, so the package was installed in the user directory outside of the conda env directory. pip list --user shows these packages.
The solution is to uninstall these packages first.
pip freeze --user > packages.txt
pip uninstall -r packages.txt
You can call the pip with the python version you want to install the package for. For more detail please go through the link am pasting below:
Install a module using pip for specific python version
and do not forget to restart your anaconda after installing.
Hope it'll solve your issue
I used pip to install the Resource module to the default conda environment on my laptop: (C:\Users\my_username\Anaconda2). I think it is called root. I installed pip to the conda environment and so I'm 90% sure the resource was installed within the environment. And indeed running conda list shows that the package is listed within the environment. Here is a section of the output:
# packages in environment at C:\Users\conna\Anaconda2:
#
qtpy 1.2.1 py27_0
requests 2.14.2 py27_0
Resource 0.2.0 <pip>
rope 0.9.4 py27_1
ruamel_yaml 0.11.14 py27_1
scandir 1.5 py27_0
scikit-image 0.13.0 np112py27_0
However when I run
conda update Resource
I get the following error:
PackageNotInstalledError: Package is not installed in prefix.
prefix: C:\Users\conna\Anaconda2
package name: Resource
How is it possible that conda list shows the module is present but conda update can't see them? I also noticed that conda update doesn't recognize any packages with <pip>. What is happening?
conda only manages the packages that are installed using a conda command. If you installed a package with pip (or using python setup.py install or develop) it will show up with conda list (because that shows all packages no matter how they were installed) but conda won't manage that package. Simply because it doesn't know how!
So if you installed a package with pip you also need to upgrade/update it with pip:
pip install [package_name] --upgrade
Try this;
pip install Resource --upgrade
I use Anaconda (because it is awesome), and the packages available through conda install are quite extensive. However now and then I do need to install a package that isn't available in the conda repositories, and so get it from pypi instead.
My question: when I run the command conda update --all, will conda also update these pypi packages? Or do I have to update them separately? The conda docs don't seem to contain an answer to this. This question and answer seems to indicate that no, conda does not manage pypi packages, but I'm still uncertain.
No, conda update and conda install don't update packages installed with pip (or install them using pip).
These conda commands only check your "default" anaconda-channels or the ones specified with -c, they ignore everything else. One exception is conda list which shows also the packages installed with pip, these are marked with <pip> and won't be updated.
One example using pip and six:
$ conda create -n testenv python=3.5
Fetching package metadata .................
Solving package specifications: .
Package plan for installation in environment testenv:
The following NEW packages will be INSTALLED:
pip: 9.0.1-py35_1
python: 3.5.3-3
setuptools: 27.2.0-py35_1
vs2015_runtime: 14.0.25123-0
wheel: 0.29.0-py35_0
Proceed ([y]/n)? y
$ activate testenv
Installing six with pip (old version):
(testenv) $ pip install six==1.6
Collecting six==1.6
Downloading six-1.6.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six-1.6.0
conda update doesn't update it (note that six isn't listed in the "all requested packages" but it's listed in conda list):
(testenv) $ conda update --all
Fetching package metadata .................
Solving package specifications: .
# All requested packages already installed.
# packages in environment at testenv:
#
pip 9.0.1 py35_1
python 3.5.3 3
setuptools 27.2.0 py35_1
vs2015_runtime 14.0.25123 0
wheel 0.29.0 py35_0
(testenv) $ conda list
# packages in environment at testenv:
#
pip 9.0.1 py35_1
python 3.5.3 3
setuptools 27.2.0 py35_1
six 1.6.0 <pip>
vs2015_runtime 14.0.25123 0
wheel 0.29.0 py35_0
But it can be upgraded with pip:
(testenv) $ pip install six --upgrade
Collecting six
Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
Found existing installation: six 1.6.0
Uninstalling six-1.6.0:
Successfully uninstalled six-1.6.0
Successfully installed six-1.10.0
Just to show that there is a newer version of six in the anaconda channel (which was ignored when I did conda update):
(testenv) $ conda install six
Fetching package metadata .................
Solving package specifications: .
Package plan for installation in environment testenv:
The following NEW packages will be INSTALLED:
six: 1.10.0-py35_0
Proceed ([y]/n)?
Conda 4.6 has an experimental feature to enable interoperability with pip-installed packages. Use conda config --set pip_interop_enabled true. Non-conda-installed python packages that can be "managed" by conda (i.e. removed) may be updated/changed to satisfy the current solve. Manageable packages were typically installed from wheels. Sdists installed with newer versions of pip are also typically manageable. However conda won't switch out the non-conda-installed package for a conda package if the versions are equivalent.
Non-conda-installed python packages that can't be managed will anchor the environment in place until they are removed by other means. An example of unmanageable packages are "editable" installs that used pip install -e.
All of this applies to conda update --all.
This question is old, but here's a batch script that might help with automating this process on Windows. It involves going through conda list and finding packages marked with the pypi tag, which are then subsequently upgraded with pip --upgrade en masse (assuming they are out-of-date; otherwise the standard Requirement already up-to-date message will be returned).
Place the following in a batch file (e.g., condapip.bat) and try it out:
#echo off
set packages=pip install --upgrade
for /f "tokens=1" %%i in ('conda list ^| findstr /R /C:"pypi"') do (call :join %%i)
#echo on
%packages%
#echo off
goto :eof
:join
set packages=%packages% %1
goto :eof