I'm having a mess of problems trying to install python modules with pip using a local version of python/pip installed in my home directory. First, this should prove my PATH variable is set correctly:
command -v python
/home/myuserid/PYTHON4/bin/python
command -v python
/home/myuserid/PYTHON4/bin/pip
I then try to install python modules with pip. Some modules install successfully. But one named matlabplot is giving me problems. Specifically, I run
pip install matplotlib --target=/home/myuserid/PYTHON4/PYTHON
The errors are
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-VwF2LG/subprocess32/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-pj2g8u-record/install-record.txt --single-version-externally-managed --compile --home=/tmp/tmplDhrIG:
....
gcc -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o
_posixsubprocess.c:16:10: fatal error: Python.h: No such file or directory
#include "Python.h"
There are two problems with that error message:
First, it is using /usr/local/bin/python. This is the wrong python (I previously installed python there). Where is that coming from? It's not on my path. If I print my entire environment with printenv, it's not listed anywhere.
Second, the gcc compile command explictly includes "-I/usr/include/python2.7" on the command line. That should not happen because I'm trying to use a local python that is installed in/p/home/myuserid/PYTHON4.
I'm guessing the cause is I'm somehow not specifying where python is installed. How can I fix these problems?
Avoid pip install, prefer python -m pip install. This way you're telling explicitly you want to use the python program for which you checked the path. And you can play many variations around it:
python3 -m pip install
/usr/bin/python3 -m pip install
python3.9 -m pip install
...
About the numpy issue, it's because pip don't find an already compiled numpy for your platform. It may be because you're running an old version or pip, so you can first try a python -m pip install --upgrade pip, and try installing numpy again.
You can also take a look at https://pypi.org/simple/numpy, you'll see a list of files like:
numpy-1.21.2-cp38-cp38-win_amd64.whl
numpy-1.21.2-cp39-cp39-macosx_11_0_arm64.whl
numpy-1.21.2-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
the whl files are pre-compiled ones, but for specific platforms and specific Python versions, here the 3 wheels I listed are for numpy 2.21.2, for cpython 3.9, for windows, macosx, and many linuxes.
Maybe your specific set of Python version / numpy version / platform is not pre-compiled here?
If you really need to let pip compile numpy you'll need some more dependencies, the missing Python.h one means you're missing the python3-dev package from your distrib.
Last hint: you should use a venv: a venv is created by a specific version of Python, and all installations are done inside the venv (so easily removable: just remove the directory, in case it goes badly).
Related
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DNPY_NO_DEPRECATED_API=0 -Ipandas/_libs/src/ujson/python -Ipandas/_libs/src/ujson/lib -Ipandas/_libs/src/datetime -I/Users/jesse/Desktop/ssc/pythonProject3/venv/lib/python3.8/site-packages/numpy/core/include -I/Users/jesse/Desktop/ssc/pythonProject3/venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c pandas/_libs/src/ujson/python/ujson.c -o build/temp.macosx-10.14.6-arm64-3.8/pandas/_libs/src/ujson/python/ujson.o -D_GNU_SOURCE -Wno-unused-function
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DNPY_NO_DEPRECATED_API=0 -Ipandas/_libs/src/ujson/python -Ipandas/_libs/src/ujson/lib -Ipandas/_libs/src/datetime -I/Users/jesse/Desktop/ssc/pythonProject3/venv/lib/python3.8/site-packages/numpy/core/include -I/Users/jesse/Desktop/ssc/pythonProject3/venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c pandas/_libs/src/ujson/python/objToJSON.c -o build/temp.macosx-10.14.6-arm64-3.8/pandas/_libs/src/ujson/python/objToJSON.o -D_GNU_SOURCE -Wno-unused-function
pandas/_libs/src/ujson/python/objToJSON.c:181:12: error: use of undeclared identifier 'NUMPY_IMPORT_ARRAY_RETVAL'
return NUMPY_IMPORT_ARRAY_RETVAL;
^
1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/jesse/Desktop/ssc/pythonProject3/venv/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/ky/s4qqls154f9c4j5lfcfspbpm0000gn/T/pip-install-t5zlzk09/pandas_67e63708ec954c589e697c9f10e3730c/setup.py'"'"'; __file__='"'"'/private/var/folders/ky/s4qqls154f9c4j5lfcfspbpm0000gn/T/pip-install-t5zlzk09/pandas_67e63708ec954c589e697c9f10e3730c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/ky/s4qqls154f9c4j5lfcfspbpm0000gn/T/pip-record-cnus0d48/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jesse/Desktop/ssc/pythonProject3/venv/include/site/python3.8/pandas Check the logs for full command output.
(base)
I want to download pandas on Mac mini using PyCharm.
What should I do?
This was answered on:
Trouble installing Pandas on new MacBook Air M1
python3 -m pip install virtualenv
virtualenv -p python3.8 venv
source venv/bin/activate
pip install --upgrade pip
pip install numpy cython
git clone https://github.com/pandas-dev/pandas.git
cd pandas
python3 setup.py install
You have to have numpy installed in your environment to build pandas from source for all versions of pandas that come without a pyproject.toml. Please install numpy first and try again.
If numpy is installed on your machine you may need to update the version using pip install numpy -U or an equivalent command executed via PyCharm GUI.
While caeneb's trick worked like a charm, I found that upgrading your python to a later version and updating pip worked as well.
for me these where the commands that did the trick (I manage my python installation with pyenv)
pyenv install 3.9.9
pyenv global 3.9.9 # use this python version as the default
pip install pandas # just works
I guess that without pyenv you can simply run the following commands (untested)
brew install python#3.9.9
$(brew --prefix)/bin/python3.9 -m pip install pandas # makes sure to use the right pip
From manually reviewing the numpy git log, downgrading numpy using
pip install numpy==1.18.5 and then
OPENBLAS="$(brew --prefix openblas)" MACOSX_DEPLOYMENT_TARGET=12.4 pip3 install pandas==0.25.3 --no-use-pep517
After installation of pandas, I was able to upgrade numpy again
I recommend you to use conda, so that you can install virtual environments, also with different python version and eventually with different R (and also node.js). Much simpler and programs are pre-compiled (and conda was created for such cases).
Else, are you sure you have xcode installed? Check on Apple store. Xcode is the "SDK" (developer toolkit) of Apple with the compiler: the clang: you doesn't have it, so the error. It gives you also other tools which are generally needed for pip. Note: you should also execute manually also one xcode command, to be able to accept the xcode condition (else you may have problem on scripts which uses xcode).
This works for me
pip install git+git://github.com/pandas-dev/pandas.git
$pip install pandas
For installing pip in Mac :
$curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
OR
Install python with homebrew:
brew install python
Make sure of correct path:
$which python
Now install pandas:
$pip install pandas
Im trying to install lxml within a virtualenv with sudo pip install lxml and also sudo pip install --upgrade lxml but getting the following in both cases:
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,
relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat
-Werror=format-security build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt
-lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools,
tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';
exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'),
__file__, 'exec'))" install --record /tmp/pip-nmFOYf-record/install-record.txt
--single-version-externally-managed --compile failed with error code 1 in
/tmp/pip_build_root/lxml
Storing debug log for failure in /root/.pip/pip.log
I have tried all the posted solutions here, which implies that I have libxml2-dev, libxslt-dev and python-dev installed and I also installed build-essential
I'm currently running Linux Mint 17 Debian Based which uses apt-get as package manager.
python-lxml was already preinstalled.
lxml depends on various C libraries, and you have to have those C libraries installed—including their development files (headers, .so or .a libraries, etc.)—to build lxml. The installation docs explain what prerequisites you need to build on your particular platform.
This error:
/usr/bin/ld: cannot find -lz
… means that the prerequisite you're missing is libz, aka zlib.
Since you're not on Windows, it's incredibly unlikely that you actually don't have zlib at all… but pretty plausible that you don't have the development files for zlib. On some platforms, most notably many linux distros, packages are typically split into separate pieces. For example, the parts of zlib needed at runtime may be in a package named zlib, while the parts needed for building other programs that need zlib in a package named zlib-dev or zlib-devel. The exact details depend on your exact platform.
That being said, if you don't have the zlib development files, you probably don't have the libxml2 or libxslt development files either, because I don't know of any platform where installing those wouldn't have pulled in the zlib files as well.
At any rate, since you haven't told us what platform you're one (and distro, if linux), I don't know what package manager to use, what the packages are named, etc., but do whatever's appropriate for your platform.
Also:
I already have python-lxml installed
You really shouldn't install the same package both with your distro's package manager and with pip; that's a great way to confuse yourself.
But at any rate, most likely you installed python-lxml from a binary package, not from source, which means you didn't need the build requirements. Now you're trying to build it from source, which means you do.
On an ubuntu based linux distro you can use:
sudo apt-get install zlib1g-dev
I am using openpyxl python package in my application. I am getting the following message when using the same.
/usr/local/lib/python2.7/dist-packages/openpyxl/init.py:31:
UserWarning: The installed version of lxml is too old to be used with
openpyxl
warnings.warn("The installed version of lxml is too old to be used with openpyxl")
Openpyxl requires lxml version 3.2.5 or above, and the version in my machine is 3.2.0. When I try to upgrade lxml to the latest version ie
3.3.5, it is getting interrupted with the following message.
i686-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-i686-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-i686-2.7/lxml/etree.so
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
error: command 'i686-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Rolling back uninstall of lxml
Replacing /usr/lib/python2.7/dist-packages/lxml
Replacing /usr/lib/python2.7/dist-packages/lxml-3.2.0.egg-info
Cleaning up...
Removing temporary dir /tmp/pip_build_root...
Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-ov0PUy-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_root/lxml
I need help in resolving the issue.
PS:- I have already installed lxml dependent packages python-dev, libxml2-dev and libxslt1-dev in my machine.
sudo pip install --upgrade lxml
On Ubuntu (and other Debian derivatives), before you build some piece of software, ensure that its build dependencies are installed. Using your example, run this:
sudo apt-get build-dep python-lxml
That is likely going to require a whole bunch of packages, one of which is zlib1g-dev, as mentioned.
I think what you're missing is zlib1g-dev.
sudo apt-get install zlib1g-dev
Running above command will solve the issue.
I have successfully installed MPICH on Ubuntu 12.04 and can get the simple test C programs to compile and run as expected.
Now I am trying to use pip to install mpi4py, simply via pip install mpi4py. I first executed sudo apt-get build-dep python-mpi4py which worked fine.
I'm working with virtualenv, so this mpi4py installation is not with sudo (and should not be).
Here's the command and the error that I hit after a lengthy set of console output that looks like normal compile stuff (I can print all of the console output if needed, but I think most of it is standard compiler warnings that indicate no problems).
(local-dev)espears#espears-w:~$ pip install mpi4py
src/ext/mpi_swig.c: In function ‘SWIG_Python_AddErrorMsg’:
src/ext/mpi_swig.c:859:5: error: format not a string literal and no format arguments [-Werror=format-security]
cc1: some warnings being treated as errors
error: command '/usr/bin/mpicc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /home/espears/.virtualenvs/local-dev/bin/python -c "import setuptools, tokenize;__file__='/home/espears/.virtualenvs/local-dev/build/mpi4py/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-5o6OwN-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/espears/.virtualenvs/local-dev/include/site/python2.7 failed with error code 1 in /home/espears/.virtualenvs/local-dev/build/mpi4py
Traceback (most recent call last):
File "/home/espears/.virtualenvs/local-dev/bin/pip", line 11, in <module>
sys.exit(main())
File "/home/espears/.virtualenvs/local-dev/local/lib/python2.7/site-packages/pip/__init__.py", line 185, in main
return command.main(cmd_args)
File "/home/espears/.virtualenvs/local-dev/local/lib/python2.7/site-packages/pip/basecommand.py", line 161, in main
text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 31: ordinal not in range(128)
But, which mpicc gives the expected result: /usr/bin/mpicc and as mentioned above, if I use this mpicc to compile simple .c test programs, they work fine.
I've seen some links describing a flag option fPIC for compiling MPI, but, I merely used apt-get to install MPI and specifically want to only rely on native Ubuntu packages for these tools, as opposed to maintaining separate installation instructions for manually building them.
How can I determine why mpicc is failing within the mpi4py installation process?
Added
The following from the console output seems to be the compilation command that causes the trouble.
building 'mpi4py._mpi_swig' extension
/usr/bin/mpicc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC -I/usr/include/python2.7 -c src/ext/mpi_swig.c -o build/temp.linux-x86_64-2.7/src/ext/mpi_swig.o
This gives the following error when I try from the command line myself:
(local-dev)espears#espears-w:~$ /usr/bin/mpicc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC -I/usr/include/python2.7 -c src/ext/mpi_swig.c -o build/temp.linux-x86_64-2.7/src/ext/mpi_swig.o
gcc: error: src/ext/mpi_swig.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
I was able to successfully work around this with no changes to anything in my system simply by pointing pip at the MPI4PY package url, rather than the package name:
pip install https://bitbucket.org/mpi4py/mpi4py/downloads/mpi4py-1.3.1.tar.gz
A similar problem was encountered on Ubuntu 18.04 with openmpi-4.0.0.
Solved by installing older version of openmpi-2.1.6.
apt-get install openmpi=2.1.6
I'm having a ton of trouble installing modules. At first I thought I had messed with my python installation on mac os x but I installed a virtual machine and ubuntu 11.04 and have similar troubles. Why are both os x and ubuntu failing with the same error?
For example I can't install tkinter with it failing:
Installing collected packages: tkinter-pypy
Running setup.py install for tkinter-pypy
building '_tkinter' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DWITH_APPINIT -I/usr/include/tcl -I/usr/include/tk -I/usr/include/python2.7 -c src/_tkinter.c -o build/temp.linux-i686-2.7/src/_tkinter.o
src/_tkinter.c:74:17: fatal error: tcl.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Complete output from command /usr/bin/python -c "import setuptools;__file__='/home/pfm/build/tkinter-pypy/setup.py';execfile(__file__)" install --single-version-externally-managed --record /tmp/pip-sMB5Wi-record/install-record.txt:
running install
running build
running build_ext
building '_tkinter' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DWITH_APPINIT -I/usr/include/tcl -I/usr/include/tk -I/usr/include/python2.7 -c src/_tkinter.c -o build/temp.linux-i686-2.7/src/_tkinter.o
src/_tkinter.c:74:17: fatal error: tcl.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command /usr/bin/python -c "import setuptools;__file__='/home/pfm/build/tkinter-pypy/setup.py';execfile(__file__)" install --single-version-externally-managed --record /tmp/pip-sMB5Wi-record/install-record.txt failed with error code 1
Storing complete log in /home/pfm/.pip/pip.log
I encountered exactly the same problem when trying to install tkinter-pypy on Ubuntu 11.04. The error message shows that it's looking for tcl.h file in /usr/include/tcl, but it's not there. I have to install the dev version of the tcl library (I installed tcl8.4-dev).
sudo apt-get install tcl8.4-dev
This installs the header files to /usr/include/tcl8.4. I then created a symlink /usr/include/tcl pointing to that. I also installed the dev version of the tk library (e.g. tk8.4-dev) which installed the tk.h header (needed too by tkinter-pypy) in the /usr/include/tcl directory.
After these steps, tkinter-pypy can be installed successfully.
Not sure about the error, but Tkinter should be available with your Python install. Have you tried to import Tkinter. On a related note I'd definitely recommend using setuptools (aka. easy_install) or one of the other similar installation tools.
EDIT
If Tkinter is still not available, then, on Linux, try locate lib-tk and adding it to your python path
import sys;
sys.path.append(PATH_TO_TK)
Then check out the Wiki to get the setup to stick: http://wiki.python.org/moin/TkInter
Another EDIT
A simple work around might be to install IDLE, which depends on Tkinter (noted by the OP).