I'm trying to create a conda env (actually I'm using Mamba) whith PyPy as default interpreter and then installing Scikit-Learn to perform machine-learning tasks.
I succeded in the environment creation and setup with:
mamba create -n sklearn -c conda-forge pypy python
In fact, running python inside the shell after env activation returns:
Python 3.9.16 | packaged by conda-forge | (feeb267e, Jan 18 2023, 16:13:01)
[PyPy 7.3.11 with GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>>
Then i tried to install scikit-learn package with mamba install -n sklearn -c conda-forge scikit-learn and i get this output:
Looking for: ['scikit-learn']
conda-forge/linux-64 No change
conda-forge/noarch 11.1MB # 2.3MB/s 5.0s
Could not solve for environment specs
Encountered problems while solving:
- nothing provides python 3.6.9 1_73_pypy needed by pypy-7.3.1-h9f0ad1d_1
The environment can't be solved, aborting the operation
I tried both with conda and pip install, but I got other errors about package conflicts. I also tried to make an environment with mamba create -n myenv -c conda-forge scikit-learn, but this will install the standard CPython implementation as the interpreter.
There are ways to achieve this or should I get rid of PyPy?
The conda-forge recipe for scikit-learn has not yet been updated for pypy3.9. You could try to build from source.
Related
I have tensorflow listed as a requirement in the install_requires section of the setup.py of my project.
When I attempt to install my project into a new Anaconda environment I get the following error:
$ python setup.py install
...
Searching for tensorflow
Reading https://pypi.org/simple/tensorflow/
No local packages or working download links found for tensorflow
error: Could not find suitable distribution for Requirement.parse('tensorflow')
I can get past this by installing tensorflow "manually" via conda:
$ conda install tensorflow
Once I do this the install via setup.py works without a hitch.
Am I mistaken in assuming that something is amiss with my environment? If not then what is going on and how can I avoid this issue? (My concern is that users of my package will not be able to install from source using setup.py)
Another oddity that I assume is related or may provide a clue is that the version of TensorFlow listed in my Anaconda environment is 2.0 but if I import it when running Python it appears to instead be using version 1.15. For example:
$ conda list tensorflow
# packages in environment at /home/james/miniconda3/envs/cvdata_test:
#
# Name Version Build Channel
tensorflow 2.0.0 mkl_py37h66b46cc_0
tensorflow-base 2.0.0 mkl_py37h9204916_0
tensorflow-estimator 2.0.0 pyh2649769_0
$ python
Python 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 22:33:48)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.15.0-rc2'
This is on a Dell laptop running Ubuntu 18.04 without a GPU, so perhaps the version shown in the interpreter is akin to tensorflow-cpu? If I run pip freeze I see tensorflow==2.0.0 and tensorflow-cpu==1.15.0rc2, which is a bit confusing...
This appears to have been caused by using Python version 3.8, which is currently an unsupported version of Python. Once I created a new Anaconda environment with Python version 3.7 this issue went away.
The only remaining issue is this error that I see when I run pip install -e . for my project which includes tensorflow:
ERROR: tensorflow-cpu 1.15.0rc2 has requirement tensorboard<1.16.0,>=1.15.0, but you'll have tensorboard 2.1.0 which is incompatible.
ERROR: tensorflow-cpu 1.15.0rc2 has requirement tensorflow-estimator==1.15.1, but you'll have tensorflow-estimator 2.1.0 which is incompatible.
So the issue of tensorflow-cpu version 1.15.0rc2 actually being installed when version 2.1.0 shows as being the installed version is still a mystery. To wit:
$ conda list tensorflow
# packages in environment at /home/james/miniconda3/envs/cvd:
#
# Name Version Build Channel
tensorflow 2.1.0 pypi_0 pypi
tensorflow-estimator 2.1.0 pypi_0 pypi
$ python -c "import tensorflow as tf; print(tf.__version__)"
1.15.0-rc2
I installed Conda using the scripts from bootstrap-conda.sh and install-conda-env.sh
Conda was installed successfully. When I install the first package,
e.g., conda install --yes lxml=4.3.0
it worked fine. The log can be found here. But when I tried to install the second package,
e.g., 'conda install --yes numpy=1.15.4'
it throws this error:
Traceback (most recent call last):
File "/opt/conda/default/bin/conda", line 12, in <module>
from conda.cli import main
ModuleNotFoundError: No module named 'conda'
Tried every possible solution I could find, it did not help.
Before package installation, when I type python, it takes me to Python 3.6.7
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
python default import path:
/opt/conda/default/lib/python36.zip
/opt/conda/default/lib/python3.6
/opt/conda/default/lib/python3.6/lib-dynload
/opt/conda/default/lib/python3.6/site-packages
/usr/lib/spark/python
After package installation, when I type python, it takes me to Python 3.7.3
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
python default import path:
/opt/conda/default/lib/python37.zip
/opt/conda/default/lib/python3.7
/opt/conda/default/lib/python3.7/lib-dynload
/opt/conda/default/lib/python3.7/site-packages
I re-installed Conda. The same problem still persists.
Conda Version:
conda --version
conda 4.6.14
When I install the first package, the following log caught my attention. It upgraded python from 3.6 to 3.7. I am not sure if this is what causes the problem.
The following packages will be UPDATED:
certifi conda-forge::certifi-2019.3.9-py36_0 --> pkgs/main::certifi-2019.6.16-py37_0
libedit 3.1.20170329-h6b74fdf_2 --> 3.1.20181209-hc058e9b_0
ncurses 6.1-hf484d3e_0 --> 6.1-he6710b0_1
openssl conda-forge::openssl-1.1.1b-h14c3975_1 --> pkgs/main::openssl-1.1.1c-h7b6447c_1
pip 10.0.1-py36_0 --> 19.1.1-py37_0
python conda-forge::python-3.6.7-h381d211_10~ --> pkgs/main::python-3.7.3-h0371630_0
readline 7.0-ha6073c6_4 --> 7.0-h7b6447c_5
setuptools 39.2.0-py36_0 --> 41.0.1-py37_0
wheel 0.31.1-py36_0 --> 0.33.4-py37_0
zlib 1.2.11-ha838bed_2 --> 1.2.11-h7b6447c_3
You should create a dedicated environment for your lxml and numpy. That will keep the prerequisites for conda and for your code separate, which brings benefits in the long run.
About two weeks ago, conda started to aggressively update Python, and probably other packages too. You can try to prevent that by passing python=3.6 with every install command, or by pinning the Python version. This answer might help:
https://stackoverflow.com/a/48733093/5629418
Like you assumed, upgrading Python version is what making you an issue. If you try to downgrade it, the same thing can happen in some cases. This means that Conda will be broken if you do not explicitly say to keep existing Python version (not always, but in some cases). In your case you should do following:
conda install --yes lxml=4.3.0 python=3.6.7
conda install --yes numpy=1.15.4 python=3.6.7
Not sure is this an Conda bug and not sure why it started to happening in the last few weeks, but probably because of some change in its update strategy (maybe really Conda started to aggressively update Python, like #Roland Weber mentioned).
New python3 environment created with
conda create -n py3CVCaffe python=3.7
caffe installed with
conda install caffe-gpu=1.0=py37hdc87d0a_5
among the many packages installed are these two
libopencv-3.4.2 | hb342d67_1 40.4 MB
py-opencv-3.4.2 | py37hb342d67_1 1.2 MB
both packages are installed and seem to be functional
$ python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> import cv2
>>>
However certain video functions are absent as the openCV which is provided has not been compiled
cv2.error: OpenCV(3.4.2) /tmp/build/80754af9/opencv-suite_1535558553474/work/modules/highgui/src/window.cpp:632: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage'
This is a well known problem as discussed here
So I need to install an openCV that has been compiled with these libraries. Before I do that I have to remove the openCV that was pulled in with caffe.
Unfortunately, conda does not want to-it also wants to remove caffe!
$conda uninstall py-opencv
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: /home/stefan/miniconda3/envs/p37cu9CfeG
removed specs:
- py-opencv
The following packages will be REMOVED:
caffe-gpu-1.0-py37hdc87d0a_5
py-opencv-3.4.2-py37hb342d67_1
Proceed ([y]/n)?
If you only want to remove py-opencv and not any of its reverse dependencies, then use the --force flag. From the conda remove --help:
--force-remove, --force
Forces removal of a package without removing packages
that depend on it. Using this option will usually
leave your environment in a broken and inconsistent
state.
which should be read as "do so at your own risk".
I use sudo pip install tensorflow-1.7.0rc0-cp27-cp27mu-manylinux1_x86_64.whl in centos7,python version is 2.7.14
I get successfully installed tensorflow,but when I run import tensorflow as tf,I get a error:No module named tensorflow
what should I do? thanks!
As mentioned in the comments you best method for installing pyhton packages is to use a virtual environment. For a simple way to do this you can install Minicaonda for linux x86 using the following command:
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash ./Miniconda3-latest-Linux-x86_64.sh
Then it is worth reading the instructions on using conda, to set up a virtual environment called tensorflow. But the basic command should be:
conda create -n tensorflow python=3.6 tensorflow
provided you want to use tensorflow with python3.6. Then activate your tensorflow conda environment:
source activate tensorflow
and you should be able to import tensorflow without issue.
(tensorflow) user#machine:~$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>>
python3 is my local Anaconda version of python, while python3.4 is the system one. I can import gi module with python3.4 (probably because i installed it with sudo apt-get install python3-gi) but python3 doesn't see it:
$ python3 -c 'import gi'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named 'gi'
$ python3.4 -c 'import gi'
$ python3 --version
Python 3.5.1 :: Anaconda 4.0.0 (64-bit)
$ python3.4 --version
Python 3.4.3
$ which python3
/home/kossak/anaconda3/bin/python3
$ which python3.4
/usr/bin/python3.4
$
How should i install gi for Anaconda python? Or maybe i can somehow import sysem-wide modules?
My os:
System: Kernel: 3.19.0-32-generic x86_64 (64 bit gcc: 4.8.2) Desktop: Cinnamon 2.8.8 (Gtk 2.24.23) dm: mdm
Distro: Linux Mint 17.3 Rosa
If you're using conda virtualenv for python-3, you can use
$ conda install -c conda-forge pygobject
in your virtualenv
You can read more about this on:
https://anaconda.org/conda-forge/pygobject
This is how you do it: (example for Linux Mint and python3)
First install gi module using your distro package manager. For Linux Mint it would be:
sudo apt-get install python3-gi
Then run your distro python to check where the module is located:
$ /usr/bin/python3
Python 3.5.2 (default, Sep 10 2016, 08:21:44)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>> print(gi)
<module 'gi' from '/usr/lib/python3/dist-packages/gi/__init__.py'>
So in my case the module gi was installed to /usr/lib/python3/dist-packages/gi. Assuming you have your anaconda installed in /home/kossak/anaconda3, create a link to gi module in the proper folder:
ln -s /usr/lib/python3/dist-packages/gi /home/kossak/anaconda3/lib/python3.5/site-packages/
If you have conda virtual environment and want gi module to be available there, the path should be a bit different. Assuming the virtual env is called TEST:
ln -s /usr/lib/python3/dist-packages/gi /home/kossak/anaconda3/envs/TEST/lib/python3.5/site-packages/
and it works:
$ python3
Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>>
If you want to perform a proper install (without soft linking) inside a Linux anaconda environment. Keep in mind that the errors may vary if you have not installed gcc previously (I assumed it was installed by default when I posted, however not everyone does so). Install it if you don't know very well what you're doing to avoid missunderstandings
Create or open your conda environment.
Attemp to install pygobject (don't worry, it will most likely throw an error):
pip install pygobject
In linux, it will promp the usual installation progress followed by an error:
(...) Please, try executing the following in your system:
sudo apt install libgirepository1.0-dev
Depending on your operation sistem or installed dependencies, the command name or package name may vary. Just follow the instructions and allow the system to install your packets. This step doesn't change anything, is just to give you the precise info of the package you need on your system. If you run this on Windows, it will ask you to install a specific version of Visual Studio. If you are in Windows, download the required Visual Studio from MS website, install it, reboot your computer and go to step 5 (in my case I never needed step 4 in windows, however, I'm not a MS expert.
Close your conda environment
conda deactivate
Next you need to install pygobject from conda-forge repository in your conda environment. You can add the repo to your favourite conda package manager or simply run the following command as root (it is important to be root). I did it outside the project, but you may do it inside:
conda install -c conda-forge pygobject
In my case conda was not in the path. I had is installed in:
/opt/anaconda3/bin/
You can run the following command from your normal user to find out where conda is:
which conda
Open the conda environment
source activate <your env name>
or the corresponding anaconda activate syntax (I never use it so I cant remember precisely)
Repeat the first step and now the installation wont fail:
pip install pygobject
OR if you specifically want to install gi you can run:
pip install pgi
the correct package is "pgi" NOT "gi"
As gi is a dependency of pygobject, everything will get properly installed. You can check it by running
python
>>> import gi
You may find the following usefull for Windows, although you may need to work it out a bit:
GStreammer python bindings on Windows
for me
conda install -c pkgw/label/superseded gtk3
worked