Recently, I want to run some pytorch codes on centos6. However, no matter I perform either "pip install torch" or "conda install torch", the prompt shows:
>>> import torch
Traceback (most recent call last):
File "", line 1, in
File "XXX/anaconda3/envs/XXX/lib/python3.6/site-packages/torch/init.py", line 56, in
from torch._C import *
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by XXX/anaconda3/envs/XXX/lib/python3.6/site-packages/torch/_C.cpython-36m-x86_64-linux-gnu.so)
My enviroment:
OS: CentOS release 6.7
How you installed PyTorch (conda, pip, source): pip & conda
Python version: 3.6.5
I tried mannually compiling glibc-2.19, but when I put the library path into LD_LIBRARY_PATH, I can't use commands like "ls", "make", etc. And I got "segment fault". I also tried just copying the *.so to the lib directory under the python virtual environment, no wonder, python crashed with "segment fault". I understand plenty of components rely on a specific glibc library. I just wonder how I can run pytorch on centos6 without switching to a new linux distribution.
PS: I need to deploy such python environments on the product environment, whose OS is centos6.
It's a tough one. You can either downgrade to a very old version of torch ( v0.3.1 as I remember was running ok on Centos 6.5 ), or upgrade to Centos 7. Having 2 version of glibc is hell.
If you really need Centos 6 to live with the latest version of torch, try installing glibc into non standard location and compiling both Python and pytorch from source.
update
You can't replace system's glibc and but you can install it somewhere else, like /opt/myglibc.
Pytorch stopped supporting Centos 6 since v0.4.1. So you will have to build it using gcc v5+ and linking it to your glibc version. Here are the instructions. But since you don't invoke pytorch directly, you need to build Python also. Then you can run your program by setting glibc path specifically for your program.
LD_LIBRARY_PATH=/opt/myglibc python my_program.py
Related
I am using a MacBook Pro with M1 processor, macOS version 11.0.1, Python 3.8 in PyCharm, Tensorflow version 2.4.0rc4 (also tried 2.3.0, 2.3.1, 2.4.0rc0). I am trying to run the following code:
import tensorflow
This causes the error message:
Process finished with exit code 132 (interrupted by signal 4: SIGILL)
The code runs fine on my Windows and Linux machines.
What does the error message mean and how can I fix it?
Seems that this problem happens when you have multiple python interpreters installed, and some of them are for differente architectuers (x86_64 vs arm64). You need to make sure that the correct python interpreter is being used, if you installed Apple's version of tensorflow, then that probably requires an arm64 interpreter.
If you use rosetta (Apple's x86_64 emulator) then you need to use a x86_64 python interpreter, if you somehow load the arm64 python interpreter, you will get the illegal instruction error (which totally makes sense).
If you use any script that installs new python interpreters, then you need to make sure the correct interpreter for the architecture is installed (most likely arm64).
Overalll I think this problem happens because the python environment setup is not made for systems that can run multiple instruction sets/architectures, pip does check the architecture of packages and the host system but seems you can run a x86_64 interpreter to load a package meant for arm64 and this produces the problem.
For reference there is an issue in tensorflow_macos that people can check.
For M1 Macs, From Apple developer page the following worked:
First, download Conda Env from here and then follow these instructions (assuming the script is downloaded to ~/Downloads folder)
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
source ~/miniforge3/bin/activate
reload the shell and do
python -m pip uninstall tensorflow-macos
python -m pip uninstall tensorflow-metal
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
If the above doesn't work for some reason, there are some edge cases and additional information provided at the Apple developer page
Installing Tensorflow version 1.15 fixed this for me.
$ conda install tensorflow==1.15
I have been able to resolve this issue by using Miniforge instead of Anaconda as the Python environment. Anaconda doesn't support the arm64 architecture, yet.
I had the same issue
This is because of M1 chip. Now there is a pre-release that delivers hardware-accelerated TensorFlow and TensorFlow Addons for macOS 11.0+. Native hardware acceleration is supported on M1 Macs and Intel-based Macs through Apple’s ML Compute framework.
You need to install the TensorFlow that supports M1 chip Simply pull this tensorflow macos repository and run the ./scripts/download_and_install.sh
I'm installing vowpal wabbit python3 interface using OSX 10.13.2, python 3.5.1, Anaconda3 virtual env for Anaconda.
I've followed this step.
# edit Makefile
# change BOOST_INCLUDE to use anaconda env dir: /anaconda/envs/vw/include
# change BOOST_LIBRARY to use anaconda lib dir: /anaconda/envs/vw/lib
I've followed the step above.
https://gist.github.com/suisuiwudi/86f9657d785b3e4fdc02e11e3e35899b
I got the error, please check the details in the gist above.
ld: library not found for -lpython3.5m
I also followed their methods, but these didn't work.
https://github.com/JohnLangford/vowpal_wabbit/issues/1267
https://github.com/ContinuumIO/anaconda-issues/issues/2871
Any idea about how to install these on MAC?
Your configuration of the system might be a problem here it is a very known issue in VowpalWabbit's installation that it doesn't work with MacOSX because a dependency BoostC++(in which all the reductions are implemented i.e in C++) requires a file named libstdC++ and this file was discontinued by Apple 2 years ago. Also I wouldn't recommend using Anaconda along with it although users having installation problems with Anaconda was resolved in an issue but just to be on the safe side. This is due to the dynamic linking nature of the libraries used in it namely Cmake and boost-python.
Really there are two solutions to your problem
1) Find an older version of Xcode and copy that file into its required path
2) Using a virtual machine with Windows or linux installed and install VowpalWabbit there
Apple does sometimes annoy even its most loyal customers.
I installed Pytorch for PYthon 3.6 using pip as instructed on https://pytorch.org/.
Pytorch is installed succesfully, but when I run code, I get this:
File "C:\Users\\PycharmProjects\chatbot-light\pytorch\rnn_attention\seq2seq_translation_tutorial.py", line 93, in <module>
import torch
File "C:\Users\\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\__init__.py", line 78, in <module>
from torch._C import *
ImportError: DLL load failed: The specified module could not be found
Pytorch devs recommend installing Pytorch using Anaconda.
Since Anaconda deals with all the dependencies you shouldn't have any DLL-related problems after installing Pytorch with it.
Go https://anaconda.org/anaconda/intel-openmp/files
Download win-64/intel-openmp-2018.0.3-0.tar.bz2
Extract files under its Library/bin/ to any directory
Add the directory in Step 3 to your PATH environment
Restart
This solved ImportError on my computer with Windows 10 + Python 3.6.6 + PyTorch 0.4.0.
Hope it helps.
This is an open Windows-related issue, see https://github.com/pytorch/pytorch/issues/4518 .
As a work-around, I installed SUSE Linux in a VirtualBox on Win10, then used the Linux pip instructions on PyTorch.org. No import error. BTW, the MS Visual Studio Code editor can be installed in Linux if you go that route.
I remember getting an import error (I think it was the same error you show) when I installed Pytorch with CUDA.
Installing Pytorch without CUDA made the error go away.
conda install pytorch-cpu -c pytorch
pip3 install torchvision
The upshot is that you can only run it on your CPU, not a GPU, which is an issue if you're already dealing with complex models and huge datasets, but shouldn't matter too much if you're just starting to tinker with Pytorch.
I am trying to install Tensorflow on Windows.
I have Anaconda 4.2.0. I tried running
conda create -n tensorflow python=3.5
in my command prompt. This seemed to do something, but I'm not sure what this accomplished. It created a folder within the Anaconda3 program in my username folder.
This folder is filled with the following content:
Over the summer, I used mainly Jupyter Notebooks to do my python coding. Within this environment, there is a tab marked Condas
So it looks like I should be able to switch to the Tensorflow environment. But this doesn't work when I try to switch, there is no option to change my kernel to a Tensorflow one.
I tried running
conda search tensorflow
But nothing appears.
I'm not sure what to do. I asked a few grad students in my economics research group, but they weren't sure what to do either.
My Question
How do I properly install Tensorflow on Windows?
The syntax of the command is conda create -n <name_of_new_env> <packages>. As a result, you created a clean environment named tensorflow with only Python 3.5 installed. Since conda search tensorflow returned nothing, you will have to use pip or some other method of installing the package. Since there is spotty official support for Windows, the conda-forge package (CPU only) at https://github.com/conda-forge/tensorflow-feedstock is probably the best way.
People have also reported success installing Tensorflow with docker, if you have docker set up already.
I was able to run it under the Windows 10 linux subsystem (https://msdn.microsoft.com/en-us/commandline/wsl/install_guide)
Which is basically a linux environment within windows.
The latest tensorflow version (0.12) added windows support
https://www.tensorflow.org/get_started/os_setup#pip_installation_on_windows
just run:
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-0.12.1-cp35-cp35m-win_amd64.whl
(the url is for the specific version - you will need to change it for future versions or other setups)
An Anaconda environment isolates itself completely with the outer world, so all the packages you installed outside the virtualenv is nothing in the virtualenv, if you want to use Tensorflow in the environment(seems like the only way with Anaconda), use activate tensorflow command and install the packages you want seperately.
pip provides an easy method to install tensorflow on windows machine.
use the following pip command
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-0.12.0rc0-cp35-cp35m-win_amd64.whl
Tensorflow only support python3.5 x64 bit on windows machines and it requires that you install Visual C++ 2015 redistributable (x64 version) to be able to import tensorflow
I'm using ubuntu 12.10 with default python3.2. However I downloaded python 3.3 as its much more polished. Of course, since then I have a nightmare with installing modules for 3.3, as python3 packages from synaptic install to 3.2 dir. So, I installed pip using python 3.3. Now I have pip-3.3 command, great. But, when I tried "sudo pip-3.3 install PySide" I quickly got an error: "error: Failed to locate the Python library /usr/lib/libpython3.3m.so". What's more, when I run "sudo pip install PySide" (command for 2.7), it builded and installed and runs flawlessy. Pyvenv and virtualenv both fail too. But I wan't to do it non-env way,just install pyside to global 3.3 lib dir.
So how can I install that PySide to python 3.3? I just can't get it to work... It looks like I will have to stick with default ubuntu python 3 version (this one runs great, everything works both from pip and synaptic), but I like to use newest python as it gets better and better with every release.
Btw, if that matters I also got following warning before that error:
"package init file 'PySide/init.py' not found (or not a regular file)" and
"package init file 'pysideuic/init.py' not found (or not a regular file)"
Edit: I installed full python3.3 from synaptic, including pythoh3.3-dev, debug, lib etc. Everything.
And that dynamic python 3.3 lib exist in /usr/lib/x86_64-linux-gnu/libpython3.3m.so, maybe just python3.3 package for ubuntu is poor and installs it in wrong direction/doesn't do any symlinks tricks?
Please try this answer installing from source (PySide). Download pyside sources and install from source, follow the instructions on the link.