Develop Reference

libtorch and pytorch cannot be installed simultaneously?

I am learning to develop with PyTorch as well as LibTorch. I have the following line in my ~/.bashrc for dynamic linking of libtorch libraries:
# libtorch linking path
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/.dev_libraries/libtorch/lib/
However, when this path is in LD_LIBRARY_PATH, importing torch in Python reports segmentation fault:
user#host:~$ $LD_LIBRARY_PATH
bash: /home/user/packages/embree-2.16.0.x86_64.linux/lib:/home/user/packages/embree-2.16.0.x86_64.linux/lib::/usr/local/lib/:/usr/local/cuda-11.1/lib64:/usr/local/lib/:/usr/local/cuda-11.1/lib64:/home/user/.dev_libraries/libtorch-cpu/libtorch/lib/: No such file or directory
user#host:~$ python
Python 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Segmentation fault (core dumped)
user#host:~$
As soon as I remove that path from the environment variable LD_LIBRARY_PATH, torch can be correctly imported in Python.
I am guessing the cause is that some shared libraries of PyTorch having the same names as the ones in LibTorch. Does this mean PyTorch and LibTorch cannot be installed simultaneously, or is my environment setting incorrect? I'd prefer not to reset LD_LIBRARY_PATH every time I switch between the two.
System specs:
Ubuntu 20.04 + CUDA 11.1 + python 3.8.10 + GCC 9.3.0
pytorch 1.9.0+cu111
libtorch is downloaded from here: https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip

I faced the same problem too.
You can type
import torch
print(torch.__version__)
to see the version of torch, and use the same version of libtorch, that would solve the problem probably.

Python - why would library version from pip freeze be different than when I actually run it?

I'm using conda and pip to manage my packages.
In my environment.yml, I have the following
- LOTS OF PACKAGES
- ...
- ...
- pip:
- pyarrow==0.16.0
So pyarrow should be a specific version - 0.16.
I conda activate into that environment. And when I do a pip freeze or pip show, the version agrees. It's supposed to be 0.16
(CONDA) $ pip show pyarrow
Name: pyarrow
Version: 0.16.0
Summary: Python library for Apache Arrow
Home-page: https://arrow.apache.org/
Author: None
Author-email: None
License: Apache License, Version 2.0
Location: /home/<USRER>/anaconda3/envs/CONDA/lib/python3.6/site-packages
Requires: numpy, six
But when I fire up python, import the library, and try get the version, it's a different version.
(CONDA) $ python
Python 3.6.6 |Anaconda, Inc.| (default, Oct 9 2018, 12:34:16)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow
>>> pyarrow.__version__
'0.12.0'
>>> pyarrow.__file__
'/home/<USER>/anaconda3/envs/CONDA/lib/python3.6/site-packages/pyarrow/__init__.py'
I don't understand how that's possible. I would expect the version to agree, but for some reason python insists that pyarrow is a different version.
Now I suspect my entire conda environment is bad. Shouldn't the version that I get in python agree with pip freeze?

pyarrow does some funny stuff with the __version__ variable. It is generated using setuptools-scm.
So maybe the conda release you installed is broken for some reason. Maybe try to install it manually by cloning it and installing it with pip from the cloned folder without passing through conda and see if you get a different result.
I beleive you can do this automagically by running:
pip install "git+https://github.com/apache/arrow#apache-arrow-0.16.0#egg=pyarrow&subdirectory=python"

Conda install package ModuleNotFoundError: No module named 'conda'

I installed Conda using the scripts from bootstrap-conda.sh and install-conda-env.sh
Conda was installed successfully. When I install the first package,
e.g., conda install --yes lxml=4.3.0
it worked fine. The log can be found here. But when I tried to install the second package,
e.g., 'conda install --yes numpy=1.15.4'
it throws this error:
Traceback (most recent call last):
File "/opt/conda/default/bin/conda", line 12, in <module>
from conda.cli import main
ModuleNotFoundError: No module named 'conda'
Tried every possible solution I could find, it did not help.
Before package installation, when I type python, it takes me to Python 3.6.7
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 09:07:38)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
python default import path:
/opt/conda/default/lib/python36.zip
/opt/conda/default/lib/python3.6
/opt/conda/default/lib/python3.6/lib-dynload
/opt/conda/default/lib/python3.6/site-packages
/usr/lib/spark/python
After package installation, when I type python, it takes me to Python 3.7.3
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
python default import path:
/opt/conda/default/lib/python37.zip
/opt/conda/default/lib/python3.7
/opt/conda/default/lib/python3.7/lib-dynload
/opt/conda/default/lib/python3.7/site-packages
I re-installed Conda. The same problem still persists.
Conda Version:
conda --version
conda 4.6.14
When I install the first package, the following log caught my attention. It upgraded python from 3.6 to 3.7. I am not sure if this is what causes the problem.
The following packages will be UPDATED:
certifi conda-forge::certifi-2019.3.9-py36_0 --> pkgs/main::certifi-2019.6.16-py37_0
libedit 3.1.20170329-h6b74fdf_2 --> 3.1.20181209-hc058e9b_0
ncurses 6.1-hf484d3e_0 --> 6.1-he6710b0_1
openssl conda-forge::openssl-1.1.1b-h14c3975_1 --> pkgs/main::openssl-1.1.1c-h7b6447c_1
pip 10.0.1-py36_0 --> 19.1.1-py37_0
python conda-forge::python-3.6.7-h381d211_10~ --> pkgs/main::python-3.7.3-h0371630_0
readline 7.0-ha6073c6_4 --> 7.0-h7b6447c_5
setuptools 39.2.0-py36_0 --> 41.0.1-py37_0
wheel 0.31.1-py36_0 --> 0.33.4-py37_0
zlib 1.2.11-ha838bed_2 --> 1.2.11-h7b6447c_3

You should create a dedicated environment for your lxml and numpy. That will keep the prerequisites for conda and for your code separate, which brings benefits in the long run.
About two weeks ago, conda started to aggressively update Python, and probably other packages too. You can try to prevent that by passing python=3.6 with every install command, or by pinning the Python version. This answer might help:
https://stackoverflow.com/a/48733093/5629418

Like you assumed, upgrading Python version is what making you an issue. If you try to downgrade it, the same thing can happen in some cases. This means that Conda will be broken if you do not explicitly say to keep existing Python version (not always, but in some cases). In your case you should do following:
conda install --yes lxml=4.3.0 python=3.6.7
conda install --yes numpy=1.15.4 python=3.6.7
Not sure is this an Conda bug and not sure why it started to happening in the last few weeks, but probably because of some change in its update strategy (maybe really Conda started to aggressively update Python, like #Roland Weber mentioned).

Pip Install Keras and TensorFlow missing packages?

I used pip to install keras and tensorflow, yet when I import subpackages from keras, my shell fails a check for PyBfloat16_Type.tp_base.
I tried uninstalling and reinstalling tensorflow, but I don't know for certain what is causing this error.
from keras.models import Sequential
from keras.layers import Dense
3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]
Python Type "help", "copyright", "credits" or "license" for more information.
>>>[evaluate machineLearning.py]
Using TensorFlow backend. 2019-04-21 00:31:22.995541: F tensorflow/python/lib/core/bfloat16.cc:675] Check failed: PyBfloat16_Type.tp_base != nullptr
aborted (disconnected)
>>>
Can someone help me solve this issue?

You may try to downgrade python to 3.6 (I know some people have troubles with tensorflow and keras using python 3.7). One simple way is to download anaconda, create a new environment with python 3.6, then install tensorflow and keras.
conda create -n myenv python=3.6
conda activate myenv
pip3 install tensorflow
pip3 install keras

You have a few options to try:
First, try to uninstall and re-install the TensorFlow and see whether the problem is resolved or not (replace tensorflow with tensorflow-gpu in the following commands if you have installed the GPU version):
pip uninstall tensorflow
pip install --no-cache-dir tensorflow
If the problem is not resolved, try to do the same thing with numpy:
pip uninstall numpy
pip install --no-cache-dir numpy
Hopefully, one of these two would resolve the problem.

package installed but not showing in the pycharm's available packages

I installed the PyAutoGUI package using pip install pyautogui.
I can tell it's installed since I can import it using the PyCharm terminal:
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyautogui
>>> print(pyautogui)
<module 'pyautogui' from 'C:\\python\\anaconda3\\lib\\site packages\\pyautogui\\__init__.py'>
>>>
My problem is: When I try to add it to my project virtual environment, I can't find it on the available packages. Is there any way to add it manualy ?
P.S.: This is how I'm trying to add it:
File>Settings>Project>Project interpreter>install(the green plus button)>available packages>install

Since you cannot find PyAutoGUI on available packages (for me it is there), you can try running
pip install pyautogui
in the pycharm terminal. If you have already installed it in the global site-packages, pip will use the cached files in installation instead of downloading it again.

I work around this problem by uninstalling anaconda python and pycharm then reintalling pycharm and python (not anaconda), now i can add modules to my venvs no problem.
Note: at first, the available packages list was empty until i added "https://pypi.python.org/" to my repositories using the manage repositories button.